Description
I'm filing this as a feature request, but it also serves as a bug report and a proposal to implement the suggested feature myself. Before beginning work on it, I'd like to ask the development team for their input on whether to proceed.
Module: rotate and perspective
Suggested feature: display the principal vanishing point
Bug reported: incorrect perspective in certain configurations
Suggested Feature
In perspective, the principal vanishing point (PVP) is defined as the orthogonal projection of the viewpoint onto the image plane. A key property of the PVP is that the perspective image of a sphere appears as an undistorted circle if and only if, when observed from the viewpoint, the sphere is centered on the PVP. Additionally, there is no convergence of vertical features in a landscape if and only if the PVP is located on the horizon. In essence, if one models perspective correction as virtually pointing the camera in a new direction, the PVP is that direction. More precisely, it is the vanishing point to which lines orthogonal to the sensor (of the virtual camera) converge.
Displaying the PVP in the specific (lens-accurate) mode of the rotate and perspective module would be beneficial.
There are two primary use cases:
- When cropping a picture taken with a wide-angle lens, one may want to undo the wide-angle distortion in the cropped area. Currently, the only way to do this is by eyeballing the correction. Geometrically, one should place the PVP at the center of the crop.
- Correcting vertical convergence where the horizon is visible becomes trivial: simply place the PVP on the horizon.
Why only in the specific mode? Because the PVP is well-defined only for a perspective projection, or perspectivity. The generic mode implements a more general transformation known as a projectivity, which is the sensible thing to do, but does not have a PVP. I'll provide more details on this later.
The Bug
There seems to be confusion about what the perspective correction module does mathematically, making it challenging for me to describe the situation. For instance, the documentation states, "If set to generic, a lens focal length of 28mm on a 35mm full-frame camera is assumed." However, if you set the module to 28mm or generic and observe the image change between the two, it becomes clear that this assumption is not accurate. Due to the lack of documentation, I had to read the code. Let me describe what the two modes appear to be intended for.
The generic mode is meant to implement the most general perspective correction possible, namely a projectivity, which it does.
The specific mode should use lens data to select a more restrictive kind of transformation that always produces a correct perspective representation of the scene. This is desirable and cannot be achieved without lens data. In fact, that's exactly what the current code does, but only for purely vertical or purely horizontal shifts. Unfortunately, these two transformations, because of details in the algebra of the implementation, do not compose well. As a result, when applying both vertical and horizontal corrections, one obtains an incorrect perspective. This appears to be unintentional, as the idea was likely to always make a perspectivity (if not, then what is the specific mode exactly?).
An unfortunate consequence of this bug is that the suggested feature cannot be added to the specific mode because the PVP is only defined for sound perspective projections.
Suggested Implementation
It appears that the bulk of the algorithm is agnostic with respect to the transformation applied. The transformation itself is calculated in a single function in ashift.c called _homography() (the word "homography" means the same as "projectivity"). I've checked the math, and I can confirm that the horizontal and vertical transformations are individually sound, yet their composition is not. (I can also tell that the generic mode being equivalent to a 28mm lens is incorrect.) Correcting the geometry should be uncomplicated, and I am certainly capable of doing it myself. However, changing the semantics of the module will (I assume) cause old sidecar files to behave incorrectly. If I can proceed with the implementation, I will need guidance on how to handle this backward compatibility issue. Once the bug has been fixed, it should not be hard to also display the PVP, and I think I will be able to figure this part out.