Binocular stereo vision system

A detailed explanation of the geometric framework of binocular vision

            I. image coordinates: I want to talk with world coordinates (a)

            Corn tried to describe some mathematical problems in binocular 3D reconstruction with easy and concrete description. I hope that this way can make you read corn's "computer vision learning notes" binocular vision mathematics Architecture Series blog with a relaxed mind. This blog series aims to smooth out the mathematical thread in the calibrated binocular vision. Mathematical deduction is a bit boring, but the wonderful world of computer vision is built on a rigorous mathematical framework. Therefore, the understanding of mathematical framework is the only way to understand binocular vision. But please rest assured that next, corn will explain its understanding of binocular vision mathematics or projection geometry in a way that is as interesting and grounded as possible.

            Let's start with the 3D reconstruction example in computer vision: Algorithms and applications!

            Well, let's get back to the point and see how the world in geometry is projected into the camera! Let's take a look at the first "fight" of this series of Blogs:

            Image coordinates: I want to talk with world coordinates (a)

            First of all, let me explain the subject. The literal meaning of the topic is: the image coordinate system wants to talk with the world coordinate system. There are two questions about bread:

            A. interviewee: three coordinate systems of vision system: world coordinate system, camera coordinate system and image coordinate system. This is what corn wants to share with you in this article "I want to talk with world coordinates (a)". It mainly includes the location, function and application scene of the three coordinate systems.

            B. conversation mode: how to communicate between two different coordinate systems? Corn will share rigid body transformation and perspective projection transformation with you in I want to talk with world coordinates (b). The "furthest distance in the world" between two coordinate systems that are not in one reference system.

            OK, let's reveal the true features of Lushan in the three coordinate systems.

            The figure above is a schematic diagram of three coordinates, through which you can have an intuitive understanding of the three coordinates. Let's see what's hidden in the bones of the three coordinate systems.            World coordinate system (XW, YW, ZW): it is the reference system for the position of the target object. In addition to infinity, world coordinates can be placed freely according to the convenience of operation. In binocular vision, the world coordinate system has three main uses: 1. Determining the position of the calibration object during the calibration; 2. As the system reference system of binocular vision, the relationship between the two cameras relative to the world coordinate system is given, so as to find out the relative relationship between the cameras; 3. As a container for reconstructing the three-dimensional coordinates, it holds the three-dimensional coordinates of the reconstructed object. The world coordinate system is the first station to include the object in sight into the operation.

            Camera coordinate system (XC, YC, ZC): it is the coordinate system of the object measured by the camera from its own angle. The origin of the camera coordinate system is on the optical center of the camera, and the Z axis is parallel to the optical axis of the camera. It is the bridgehead that connects with the photographed object. The object in the world coordinate system needs to go through the rigid body change to the camera coordinate system (rotation and translation), and then has a relationship with the image coordinate system. It is the link between the image coordinates and the world coordinates, communicating the furthest distance in the world. Ha-ha

            Image coordinate system (x, y) m / (U, V) pixel: it is a coordinate system based on the two-dimensional photos taken by the camera. Used to specify the position of an object in a photo. Corn prefers to call (x, y) as continuous image coordinate or spatial image coordinate, and (U, V) as discrete image coordinate or pixel image coordinate (although such a name is not verified, it can better convey the physical meaning of both).

            The origin of the (x, y) coordinate system is located on the focal point o '(U0, V0) of the camera optical axis and the imaging plane, and the unit is the length unit (m). The origin of the (U, V) coordinate system is in the upper left corner of the picture (actually the first address of the memory), as shown in the figure above, and the unit is quantity unit (piece). (x, y) is mainly used to represent the perspective projection relationship of objects from camera coordinate system to image coordinate system. And (U, V) is real, the real information we can get from the camera.

            (x, y) and (U, V) are transformed as follows:

            DX represents the width of one pixel in the x-axis direction, and Dy represents the width of one pixel in the y-axis direction. DX and Dy are the internal parameters of the camera. (U0, V0) is called the main point of the image plane, and it is also the internal parameter of the camera. In fact, it is equivalent to the discretization of x-axis and y-axis. The above formula can be written in matrix form by using homogeneous coordinates, as follows:

            (1) using homogeneous coordinates, beginners may be confused. You will ask: how to convert ordinary coordinates into homogeneous coordinates? What are the benefits of homogeneous coordinates?

            Here corn aligns the secondary coordinates to make a popular explanation. This article only talks about how to change the ordinary coordinates into homogeneous coordinates and why to introduce homogeneous coordinates. Here is just a popular but not very rigorous statement. Try to be simple and clear. For the precise mathematical derivation of homogeneous coordinates, please refer to "Zhou Xinghe's higher geometry - 1.3 homogeneous coordinates on the extended plane". Corn had read the book "advanced geometry" in detail, but felt that it was a little far away from computer vision. It was about the projection relationship of pure mathematics, which was rather obscure.

            Homogeneous coordinates can be understood as adding a "small tail" after the original coordinates. Convert normal coordinates

Please read the Chinese version for details.