Binocular stereo calibration and stereo correction

Human beings can see the three-dimensional world because of the parallax of the images in two eyes, which let us see the three-dimensional world. Similarly, in order for the computer to "see" the 3D world, it needs to use two cameras to form a binocular stereo vision system.

To let the binocular vision system know the parallax, we should first know the relative position relationship between the two cameras in the binocular vision system.. The relative position relationship between any two coordinate systems can be described by two matrices: rotation matrix R and translation matrix t.

We can calibrate the two cameras at the same time, and get the rotation matrix and translation matrix of the two relative to the same coordinate system, and then we can get the relative position relationship between the two cameras, which is binocular calibration!

I. binocular stereo calibration

Parameters to be calibrated for binocular camera: camera internal parameter matrix, distortion coefficient matrix, eigenmatrix, basic matrix, rotation matrix and translation matrix (in which camera internal parameter matrix and distortion coefficient matrix can be calibrated by single target determination method)

The main difference between binocular camera calibration and monocular camera calibration is that binocular camera needs to calibrate the relative relationship between the left and right camera coordinate systems

We use rotation matrix R and translation matrix t to describe the relative relationship between the left and right camera coordinate systems, specifically: the coordinates under the left camera are converted to the coordinates under the right camera.

If there is a point P in the space, its coordinate in the world coordinate system is pwpw, and its coordinate in the left and right camera coordinate system can be expressed as:

Pl=RlPW+Tl

Pr=RrPW+Tr

The relationship between plpl and prpr is as follows:

Pr=RPl+T

Note: in the analysis of binocular cameras, the left camera is often the main coordinate system, but R and T are the left camera to the right camera conversion, so TX is a negative number

From the above formula, we can get:

R=RrRTl

T=Tr * RTl

LRL, TL are the rotation matrix and translation vector of the relative calibrator obtained by the left camera through single target determination, RR, TR are the rotation matrix and translation vector of the relative calibrator obtained by the right camera through single target determination

When the left and right cameras are calibrated individually, RL, Tl, RR and tr can be obtained respectively. When the above formula is carried in, the rotation matrix R and translation t between the left and right cameras can be calculated.

The R and t obtained are the results of stereo calibration.

The monocular camera needs to be calibrated, and the binocular camera needs to be calibrated. The binocular camera has more parameters than the monocular camera: R and T, which mainly describe the relative position relationship between the two cameras. These parameters are very useful in stereo correction and polar geometry

So we get the result of stereo calibration. What should we do next?

The answer is: stereo correction.

II. Stereo correction

Before introducing the specific methods of stereo correction, let's see why stereo correction should be carried out?

The main task of binocular camera system is ranging, and the parallax distance formula is derived in the ideal situation of binocular system, but in the real binocular stereo vision system, there is no complete coplanar alignment of two camera image planes. So we need to do stereo correction. The aim of stereo correction is to correct the two non coplanar alignment images into coplanar alignment. (coplanar alignment: when two camera image planes are on the same plane and the same point is projected to two camera image planes, the actual binocular system should be corrected to an ideal binocular system in the same row of two pixel coordinate systems).

Ideal binocular system: the image plane of two cameras is parallel, the optical axis is vertical to the image plane, and the pole is in the wireless distance. At this time, the corresponding level line of point (x0, Y0) is y = Y0

Before stereo correction:

After stereo correction:

Bougueret correction principle

In the correction process, both image planes rotate half of R, which can minimize the re projection distortion. At this time, the two camera image planes are coplanar (after distortion correction, the optical axis is parallel), but the lines are not aligned

The pole is the intersection of the line connecting the origin of two camera coordinate systems and the image plane. To make the pole at infinity (line alignment), the line connecting the image plane of two cameras and the origin of two camera coordinate systems must be parallel

The rrectrrect matrix can be calculated so that the poles are at infinity:

Rrect=⎡⎣⎢⎢eT1eT2eT3⎤⎦⎥⎥

Because the image plane is finally parallel to the origin of the camera coordinate system, so

E1=T T

Where t = [txtytz] TT = [txtytz] t, E2 and E1 are orthogonal, and the main optical axis direction (0,0,1) and E1 cross product are selected

e2=[−TyTx0]TT2x+T2y−−−−−−−√

When E3 is orthogonal to E1 and E2, we can get:

e3=e1×e2e3=e1×e2

The final stereo correction matrix can be obtained by multiplying the rrectrrect left by the matrix acting on the left and right camera coordinate system after r decomposition.

Copyright notice: This is the original article of the blogger, following CC 4.0 by-sa copyright agreement. Please attach the original source link and this notice for reprint.

Link to this article: https://blog.csdn.net/aim_brown/article/details/80901372