Optical flow is the approximated motion vector at each pixel location. It can tell us about the relative distances of objects, as closer moving objects will have more apparent motion than moving objects that are further away, given equal speed. The explanation on this page follows the treatment in [1].
Optical flow is based on the assumptions that objects in the image at time t
will generally still be in the image at time t+1, only they will be displaced.
This is represented by
This leads via Taylor series expansion to the 2D motion constraint equation
This one equation has two unknowns and so does not have a unique solution.
This is the mathematical consequence of the aperture problem. The aperture
problem is that there is not enough information in a small area to
uniquely determine motion (this is what causes the barber pole illusion).
To add an additional constraint, Horn and Schunck [2] use
a global regularization calculation.
They assume that images consist of
objects undergoing rigid motion, and so over relatively large areas the
optical flow will be smooth. They then minimize the square of the
magnitude of the gradient of optical flow using the equation
In contrast, Lucas and Kanade [3] use a local least squares calculation
to provide the constraint by minimizing in the neighborhood
surrounding the pixel, represented in equation form by
All partial derivatives were calculated as was outlined in [2].
The partial spatial derivative
in the x direction and the partial spatial derivative in the y direction
are shown here
and the partial derivative in time direction is shown here
Using Horn and Schunck [2] the calculated optical flow for the
example images is
and using Lucas and Kanade [3] it is
Both methods were able to identify me running from the left to
the right. My wife's movement was more subtle and that was represented
in the flow, however the movement of my son was not represented due to
his partial spatial derivatives being lost in the clutter of the television set.
It did less well with movement towards the camera. Given the frames
Using Horn and Schunck [2] the calculated optical flow for the
example images is
and using Lucas and Kanade [3] it is
Here is a video (1.4 MB) of the optical flow calculated as in [2] from the movie in the discovering objects section of the project.
[1] J.L. Barron and N.A. Thacker, "Tutorial: Computing 2D and 3D Optical Flow",
Tina Memo No. 2004-012, 2005.
[2] B.K.P. Horn and B.G. Schunck, "Determining Optical Flow", Artificial Intelligence,
17, 1981, 185-204.
[3] B.D. Lucas and T. Kanade, "An Iterative Image Registration Technique with an
Application to Stereo Vision", DARPA Image Understanding Workshop, 1981, 121-130.