c++ - OpenCV 3.0: Calibration not fitting as expected -

i'm getting results don't expect when use opencv 3.0 calibratecamera. here algorithm:

1. load in 30 image points
2. load in 30 corresponding world points (coplanar in case)
3. use points calibrate camera, un-distorting
4. un-distort image points, don't use intrinsics (coplanar world points, intrinsics dodgy)
5. use undistorted points find homography, transforming world points (can because coplanar)
6. use homography , perspective transform map undistorted points world space
7. compare original world points mapped points

the points have noisy , small section of image. there 30 coplanar points single view can't camera intrinsics, should able distortion coefficients , homography create fronto-parallel view.

as expected, error varies depending on calibration flags. however, varies opposite expected. if allow variables adjust, expect error come down. not saying expect better model; expect over-fitting, should still reduce error. see though fewer variables use, lower error. best result straight homography.

i have 2 suspected causes, seem unlikely , i'd hear unadulterated answer before air them. have pulled out code i'm talking about. it's bit long, includes loading points.

the code doesn't appear have bugs; i've used "better" points , works perfectly. want emphasize solution here can't use better points or perform better calibration; whole point of exercise see how various calibration models respond different qualities of calibration data.

any ideas?

added

to clear, know results bad , expect that. understand may learn bad distortion parameters leads worse results when testing points have not been used train model. don't understand how distortion model has more error when using training set test set. is, if cv::calibratecamera supposed choose parameters reduce error on training set of points provided, yet producing more error if had selected 0s k!, k2, ... k6, p1, p2. bad data or not, should @ least better on training set. before can data not appropriate model, have sure i'm doing best can data available, , can't @ stage.

here example image

the points green pins marked. test image.

here more example stuff

in following image cropped big 1 above. centre has not changed. happens when undistort points marked manually green pins , allowing k1 (only k1) vary 0:

before

after

i put down bug, when use larger set of points covers more of screen, single plane, works reasonably well. looks terrible. however, error not bad might think looking @ picture.

// load image points     std::vector<cv::point2f> im_points;     im_points.push_back(cv::point2f(1206, 1454));     im_points.push_back(cv::point2f(1245, 1443));     im_points.push_back(cv::point2f(1284, 1429));     im_points.push_back(cv::point2f(1315, 1456));     im_points.push_back(cv::point2f(1352, 1443));     im_points.push_back(cv::point2f(1383, 1431));     im_points.push_back(cv::point2f(1431, 1458));     im_points.push_back(cv::point2f(1463, 1445));     im_points.push_back(cv::point2f(1489, 1432));     im_points.push_back(cv::point2f(1550, 1461));     im_points.push_back(cv::point2f(1574, 1447));     im_points.push_back(cv::point2f(1597, 1434));     im_points.push_back(cv::point2f(1673, 1463));     im_points.push_back(cv::point2f(1691, 1449));     im_points.push_back(cv::point2f(1708, 1436));     im_points.push_back(cv::point2f(1798, 1464));     im_points.push_back(cv::point2f(1809, 1451));     im_points.push_back(cv::point2f(1819, 1438));     im_points.push_back(cv::point2f(1925, 1467));     im_points.push_back(cv::point2f(1929, 1454));     im_points.push_back(cv::point2f(1935, 1440));     im_points.push_back(cv::point2f(2054, 1470));     im_points.push_back(cv::point2f(2052, 1456));     im_points.push_back(cv::point2f(2051, 1443));     im_points.push_back(cv::point2f(2182, 1474));     im_points.push_back(cv::point2f(2171, 1459));     im_points.push_back(cv::point2f(2164, 1446));     im_points.push_back(cv::point2f(2306, 1474));     im_points.push_back(cv::point2f(2292, 1462));     im_points.push_back(cv::point2f(2278, 1449));      // create corresponding world / object points     std::vector<cv::point3f> world_points;     (int = 0; < 30; i++) {         world_points.push_back(cv::point3f(5 * (i / 3), 4 * (i % 3), 0.0f));     }      // perform calibration     // flags set out can commented out , "freed"     int calibration_flags = 0         | cv::calib_fix_k1         | cv::calib_fix_k2         | cv::calib_fix_k3         | cv::calib_fix_k4         | cv::calib_fix_k5         | cv::calib_fix_k6         | cv::calib_zero_tangent_dist         | 0;      // initialise matrix     cv::mat intrinsic_matrix = cv::mat(3, 3, cv_64f);     intrinsic_matrix.ptr<float>(0)[0] = 1;     intrinsic_matrix.ptr<float>(1)[1] = 1;     cv::mat distortion_coeffs = cv::mat::zeros(5, 1, cv_64f);      // rotation , translation vectors     std::vector<cv::mat> undistort_rvecs;     std::vector<cv::mat> undistort_tvecs;      // wrap in outer vector calibration     std::vector<std::vector<cv::point2f>>im_points_v(1, im_points);     std::vector<std::vector<cv::point3f>>w_points_v(1, world_points);      // calibrate; 1 plane, intrinsics can't trusted     cv::size image_size(4000, 3000);     calibratecamera(w_points_v, im_points_v,         image_size, intrinsic_matrix, distortion_coeffs,          undistort_rvecs, undistort_tvecs, calibration_flags);      // undistort im_points     std::vector<cv::point2f> ud_points;     cv::undistortpoints(im_points, ud_points, intrinsic_matrix, distortion_coeffs);      // ud_points have been "unintrinsiced", don't know intrinsics, reverse        double fx = intrinsic_matrix.at<double>(0, 0);     double fy = intrinsic_matrix.at<double>(1, 1);     double cx = intrinsic_matrix.at<double>(0, 2);     double cy = intrinsic_matrix.at<double>(1, 2);      (std::vector<cv::point2f>::iterator iter = ud_points.begin(); iter != ud_points.end(); iter++) {         iter->x = iter->x * fx + cx;         iter->y = iter->y * fy + cy;     }      // find homography mapping undistorted points known world points, ground plane     cv::mat homography = cv::findhomography(ud_points, world_points);      // transform undistorted image points world points (2d only, z constant)     std::vector<cv::point2f> estimated_world_points;         std::cout << "homography" << homography << std::endl;     cv::perspectivetransform(ud_points, estimated_world_points, homography);      // work out error     double sum_sq_error = 0;     (int = 0; < 30; i++) {         double err_x = estimated_world_points.at(i).x - world_points.at(i).x;         double err_y = estimated_world_points.at(i).y - world_points.at(i).y;          sum_sq_error += err_x*err_x + err_y*err_y;     }     std::cout << "sum squared error is: " << sum_sq_error << std::endl; 

i take random samples of 30 input points , compute homography in each case along errors under estimated homographies, ransac scheme, , verify consensus between error levels , homography parameters, can verification of global optimisation process. know might seem unnecessary, sanity check how sensitive procedure input (noise levels, location)

also, seems logical fixing of variables gets least errors, degrees of freedom in minimization process less. try fixing different ones establish consensus. @ least let know variables sensitive noise levels of input.

hopefully, such small section of image close image centre incur least amount of lens distortion. using different distortion model possible in case? more viable way adapt number of distortion parameters given position of pattern respect image centre.

without knowing constraints of algorithm, might have misunderstood question, that's option too, in such case can roll back.

i have comment rather, not have enough points.