Image stabilization in fisheye lenses: a square peg in a round hole?

How image stabilization works

Image stabilization in digital cameras is mainly achieved in two different ways: in-camera, by physically shifting the sensor, and in-lens, by physically shifting an optical group within the lens. In either case, accelerometers determine the rate of movement of the camera and apply a compensation during the exposure. Up to five accelerometers are used in current cameras to detect translation about the X and Y axes and rotation about X, Y and Z axes.

In principle, it is also possible to achieve image stabilization by continuously reading images out of the sensor during a long exposure, and then by superposing the images in this set and applying a correcting shift (by an integer amount of pixels only) to each individual image. In this case, it could be possible to base the detection of camera movement on finding the optimal superposition of these images, instead of reading the output of a set of accelerometers. An interesting consequence is that this system might be able to stabilize the image of a fast-moving subject even when the camera is not panning it during the exposure, which is not possible with other types of image stabilization. I am not aware of any current digital camera using this type of stabilization.

In general, image stabilization allows an exposure increase corresponding to at least 2-3 stops, compared to non-stabilized cameras and lenses. There is no clear evidence of whether in-camera or in-lens stabilization is inherently better. In-camera stabilization is more versatile since it can be used with any lens, including legacy ones. However, this type of stabilization requires the focal length of the lens to be known to the camera, in order to compute the correct amount of sensor shift. In zoom lenses and/or lenses with internal focusing, which effectively change focal length as a function of focus distance, the camera relies on the lens providing a correct value of the actual focal length for any combination of zoom and focus settings, in order to function with a high precision. In-lens stabilization can potentially be better matched to a specific lens, and if the accelerometers are built into the lens, this solution involves only a minimal exchange of data between camera and lens (or no data exchange at all if the switches to set stabilization mode are located on the lens itself).

A square peg in a round hole?

Fisheye lenses pose special problems with respect to image stabilization. Fisheye lenses are not rectilinear, i.e., they produce images with a large amount of geometric distortion. This is unavoidable, since there is no way to project a field of view of 180° or more onto a flat sensor without introducing some type of geometric distortion. This is similar to the problem of projecting a substantial portion of the spherical Earth surface onto a flat map - it is not possible to do this without introducing geometric distortion. Just like different types of geometric projections can be used to draw a geographic map, different types of fisheye lenses introduce different types of distortion. As a result, there is simply no way to stabilize an image projected by a fisheye lens across the entire image surface. With in-camera stabilization, a horizontal camera shift in the X direction causes a shift of the field of view in the opposite direction at the center of the field of view. On the left and right edges of the field of view, the subject instead shifts axially toward the camera or away from the camera. In turn, this means that the image can be reasonably well stabilized in its center, but not simultaneously in the peripheral regions.

Designing in-lens image stabilization of a fisheye lens to solve the above problem, if at all possible, would require solving an extraordinarily complex problem in lens design. Panasonic avoided this problem in its Micro 4/3 8 mm f/3.5 fisheye by not providing any in-lens stabilization. Olympus does not have Micro 4/3 fisheye lenses, except for the 9 mm f/8 "lens cap" fisheye, which requires the user to manually configure the focal length in the camera body (no Olympus lenses have in-lens image stabilization).

Is the above a real problem in practical use of fisheye lenses? It depends on the shooting conditions. Because of the very short focal length and extremely wide field of view, fisheye lenses are least susceptible to image blurring by camera movements. However, fisheye lenses are a natural choice in many types of poorly lighted interiors where the use of electronic flash or a tripod may be forbidden. This includes, for instance, churches, museums, historical buildings, caves and canyons. Hand-held exposure times of 1 s or more are not unheard of in these environments.

The above example shows situations where a fisheye lens is the best choice. The top image shows people assembled around a kiva by a tour guide. The bottom image shows what a fisheye can accomplish in this condition.

In close quarters where the maximum field of view is desirable, there is often no alternative to a fisheye. I often find vertical shots with an angle of view around 120° (only the diagonal approaches 180°) as even more dramatic than horizontal ones (last two examples above). It is now time to assess how image stabilization affects image quality in these cases.

The above examples were shot handheld with an Olympus E-M5 and image stabilization. The first image is fine, and the 1:1 crop near the image corner shows that this lens is capable of good detail all the way to the corners. The second example used a much higher exposure time, as well as a moderately higher ISO (1600 versus 800, which is not a very large difference). Even at the small size of the image at the left, it is possible to see that the center is sharp but the periphery is fuzzy, especially on the left side of the image. Besides the high-ISO "mottling", the detail in the 1:1 corner crop is decidedly poor. An additional characteristic of fisheye images shot with image stabilization and long exposure times is that one corner/side may be relatively good, the other poorer. Depending on the accidental camera movements, and perhaps also on the nature of the subject, the effectiveness of image stabilization varies across the image frame. It is always higher, as expected, in the central area of the image, where fisheye lenses produce an image with lowest distortion.

The obvious solution to this problem is to use a tripod or equivalent support, and to switch off image stabilization. I had a good success rate by laying the camera on handrails, pavements and against nearby walls, and sometimes adjusting the camera inclination by laying a pen or lens cap between camera and support surface, instead of hand-holding it. Minor changes in camera orientation that could spoil pictures with lenses of longer focal length are often unnoticeable with a fisheye.

I am not aware of any fisheye lenses with in-lens image stabilization, most likely because of the problem described above.

Summary. In-camera image stabilization works by moving the sensor as a whole to compensate for accidental camera movements. To work properly, this presupposes that the lens produces an image without geometric distortion. Fisheye lenses produce images with massive amounts of geometric distortion, and with these lenses in-camera image stabilization works only near the center of the image, where geometric distortion is lowest. In-lens image stabilization capable of correcting this problem is probably too difficult to design.