Photomacrography is defined as photography at reproduction ratios above 1:1 (i.e., 1x magnification), taken with a simple (i.e., non-composite or non-compound) optical system. A simple optical system consists of a lens that focuses an image of the subject onto a film or sensor. A composite or compound optical system, instead, uses an objective to focus an image of the subject in the air, within a tube or chamber (this is called the primary image). An ocular or relay lens focuses on the primary image, and projects a magnified secondary image onto film or sensor (or into the eye pupil of an observer). Microscopes (including stereo microscopes) and telescopes are compound optical systems, while camera lenses are simple optical systems.
The shallow depth of field (DOF) characteristic of close-up and macro photography is even more of a problem in photomacrography. However, as a rule, a simple optical system provides a better DOF (compatibly with the loss of definition caused by diffraction) than a compound one. In particular, compound microscopes for high magnifications usually are designed to provide maximum resolution and luminosity at the expense of DOF.
An additional peculiar problem, which rarely arises in other types of photography, is that the DOF in the image plane (i.e., at the back of the lens) is much higher than in the subject plane (i.e., at the front of the lens). This is a consequence of the magnification being higher than 1x, and is just the opposite of the situation encountered in normal photography. In turn, the consequence of this peculiar DOF is that very small grains of dust sitting on the surface of the anti-aliasing and IR filter of a DSLR, normally out-of-focus and therefore invisible, in photomacrographs are sufficiently within the DOF zone, and cast a shadow onto the sensor. Furthermore, these grains often are so small that they diffract light, rather than obscure it, and therefore, together with the fact that they are slightly out-of-focus, they produce a diffuse circular or ring-shaped shadow instead of a well-defined one, as in the example of the above picture (a small 1:1 crop of a 10 megapixel original). This problem is particularly evident at high magnifications (10x or higher), with lenses of very short focal lengths, and/or at extremely narrow diaphragm settings. Unlike larger dust, it is practically impossible to get rid of these minute particles without replacing them with other ones. The most reasonable solution is to retouch the pictures in post-processing. If the pictures are substantially reduced, these defects are usually so small that they disappear during the reduction process.
Photomacrography is extremely sensitive to small movements and vibration of the camera equipment and subject. Very solid supports and a very precise focusing rack are indispensable. Intense illumination of the subject is usually indispensable to focus accurately, and exposure times are typically long without a flash. On the other hand, light sources can be placed close to the subject, and need to concentrate light only onto a small surface. Heating of the subject by light sources may be a concern.
At magnifications exceeding roughly 3x, photomacrography in the field becomes very difficult or impractical, and a studio setup is necessary. In the field, wind is probably the single most common source of movements of both subject and equipment. The bright side of this is that many subjects for studio photomacrography (including the background) can be transported easily to a studio location, and a studio for photomacrography does not require more space than a fraction of a desk-top.
Photomacrography usually requires specialized equipment. At a minimum, extension rings and/or bellows are required to increase the distance between lens and film/sensor. Most general-purpose lenses (including macro lenses designed to reach a reproduction ratio of 1:1) perform better in photomacrography when they are reversed. Some photographers report success by using a wide-angle lens reversed and used as a close-up lens on a second lens mounted normally on the camera. However, this solution is likely to suffer from field curvature and other aberrations. I see it primarily as a way to obtain quick results without committing to the purchase of specialized equipment.
Special-purpose lenses (see also here) are necessary if photomacrography must be carried out frequently, and/or if best results are required. Almost without exception, these lenses must be used on extension rings and/or bellows, because they have no focusing helicoid. The lenses generally regarded as the best for this purpose are the latest Leitz Photar and Zeiss Luminar (also here) series (produced in the 1960's and 1970's). In general, however, all lenses of these series are exceptionally good. Olympus used to produce a 20 mm and a 38 mm of good performance (albeit sensitive to flare). Nikon also produced equivalent, and highly sought after, Macro Nikkors (ironically, Nikon indeed did call these photomacrographic lenses Macro, while Nikkor macro lenses are called Micro). Canon may still produce two similar models, plus a 60 mm lens for photomacrography which is unique in having a built-in extension helicoid. Most of these lenses have a small diameter of the front portion of the lens barrel, in order to allow the placement of light sources (which is a major concern, because the working distance of these lenses is roughly equal to their focal lengths). Most photomacrographic lenses use a threaded RMS attachment (which is the same thread used by standard microscope objectives).
Older series of photomacrographic lenses are also available on the second-hand market (e.g., Leitz Tessar and Milar, Zeiss and Wild Microtessar). In general, they are more sensitive to flare and have a lesser contrast (and, especially at the extremes of their magnification ranges, a slightly lower resolution) than modern ones, mainly because of their simpler lens coatings. They are also somewhat difficult to find in top condition. However, they may constitute a valid alternative if a substantially lower price is desired.
Most photomacrographic lenses do not need to be reversed, since they are already optimized for high magnifications. Almost without exception, they do require special adapters for connecting to bellows and extension rings.
In addition to the above lenses, three very specialized lenses must be mentioned. One is the Zeiss Tessovar, which is a macro photo/photomacrographic zoom. It provides results comparable to the best Zeiss Luminar lenses, and has the added advantages of unusually long working distances and wide range of magnifications. However, it is also expensive and scarce on the second-hand market (usually, more so than a Luminar lens), and very bulky and heavy. The second is a line of Wild photomacrographs, which consists of photomacrographic zooms mounted onto a head and focusing base similar to those of stereo microscopes (however, they do not produce stereo images). The Wild photomacrographs are very rare and far more expensive than stereo microscopes on the second-hand market, and even bulkier than the Tessovar (albeit not equally strongly built, since they contain several lightweight metal and plastic parts). They are also said to provide very good images. The third is a Unitron zoom microscope lens for industrial microscopes.
In short, if you want the best photomacrographic equipment, you need one or more of the following: Zeiss Tessovar, Zeiss Luminar, Macro Nikkor, Leitz Photar, Wild photomacrograph. However, my comparison of a Zeiss Luminar and an older and much cheaper Leitz Milar of similar focal length gave the surprising result that the older lens is almost as good as the newer and more expensive one.
My personal opinion, having used a Wild photomacrograph for work and owning a Tessovar and a 63 mm and 25 mm Luminar (among other photomacrographic lenses), is that, given a range of pictures taken in practical situations with these three systems, it is usually impossible to tell which lens was used (actually, in my experience, the give-away is not resolution but DOF, which is often higher with the Tessovar). However, you should remember that, in photomacrography at magnifications exceeding approximately 3x, resolution as a whole will be lower (regardless of the equipment used) than in pictures taken, for instance, with a top quality macro lens at a 0.3x magnification in the close-up range. In these close-ups, you should be able to see features 1-2 pixels wide (i.e., the lens is out-resolving the sensor and its anti-aliasing filter). This is so, because the lens is performing at its maximum theoretical resolution: the diaphragm can be closed somewhat to provide a sufficient DOF, without degrading the resolution by introducing diffraction. In photomacrographs, instead, the smallest visible features will be several pixels wide, as a result of compromises between DOF and diffraction. Therefore, especially with subjects that do require a significant DOF, and as long as you don't use techniques that extend the DOF (e,g, focus stacking) don't expect to print out a photomacrograph that displays a significant DOF to poster size, and to be able to see the same level of detail that you can expect in a landscape or close-up photograph. See a few practical examples here , here and here.