Camera Calibration

The spatial interpretation of image features is substantially simplified if both the internal parameters of the camera mapping and the camera pose with respect to a task-specific reference frame are known. If you are looking for more information on camera calibration and software tools, I recommend checking out Jean-Yves Bouguet's calibration site with a Matlab toolbox for and many useful links. There is also a nice book available on Calibration and Orientation of Cameras in Computer Vision. For those who can read German, have a look at the classic Photogrammetrie by my fellow countryman Karl Kraus.

Stereo Camera Calibration Tool (1995)

At the Vision Group of the Institute of Robotics and System Dynamics of the German Aerospace (DLR) Research Center in Oberpfaffenhofen, I participated in the development of a calibration tool that automatically identifies both internal and external parameters of a stereo camera mounted on robot gripper. For such systems, not only the relative camera orientation has to be determined, but also the pose of the cameras with respect to the gripper-tip (hand-eye calibration). The method does not only account for a variety of typical lens distortions, but also for inaccuracies in robot kinematics. The calibration software was translated to C++ and integrated into a user-friendly Motif interface that provides a variety of tools for data extraction, parameter estimation and results analysis.

The figure below shows a typical stereo image pair of a rectangular calibration grid taken from a miniature stereo hand camera. Note the significant barrel distortion. A distortion model based on bicubic polynomials was chosen.

calibration grid with distortions

Calibration is achieved in three steps:

  1. observe the calibration grid from different perspectives;
  2. extract the intersection points with subpixel accuracy;
  3. fit the model to the grid pattern by means of iterative non-linear minimization so as to determine model parameters and camera poses.

Calibration Grid Extraction

The intersections of the perpendicular lines of the calibration grid pattern (green crosses) are extracted from the input images. Subsequently, third order polynomials (red lines) are fit through these points. The three circular or rectangular landmarks in the center of the calibration grid are used to determine the orientation quadrant.

calibration grid extraction

Model Parameter Estimation

The image formation process taking place in the cameras is modelled by a function that includes

The following image displays residual vectors (yellow) for the initial parameter set. The initial parameters are obviously far from being accurate.

Levenberg/Marquardt least-square minimization is used to solve for the currently 60 model parameters. Usually, convergence to sub-pixel approximation accuracy is achieved in a few iterations, as local minima do not occur. Computing time using 15 input images is about five seconds on an SGI Indy workstation (10 iterations). The following image shows the residual vectors (magnified by a factor of 10) after parameter estimation.

initial parameter set

final parameter set

Results Analysis

A variety of tools to evaluate the quality of the resulting parameter set have been integrated into the toolbox. Among these are

The following image shows a pictorial representation of parameter cross-correlation. The three noticeable blocks of that matrix contain the parameters for calibration grid pose, left camera parameters and right camera parameters, respectively.

parameter cross-correlation matrix

The following image shows the histogram distribution of grid point residuals. Several distribution characteristics are given numerically and graphically.

histogram of residuals

Normalization

Once the camera parameters have been determined, image frames can be corrected by rectification or normalization. This can be done in real-time (25 to 30 frames/sec) on a Datacube Maxvideo 200 image processor equipped with a Mini-Warper. The figures below show both rectification and normalization of the rectangular grid image.

rectified calibration grid

normalized calibration grid