Update 6-25-19: This article has been updated in light of the formal release of Advanced Frame Averaging. Click here to see latest info.
As seen in the latest IQ4 video released by Phase One IQ4 will feature a beta Frame Averaging tool that can improve shadow quality (reduce shadow noise) and allow the creation of in-camera pseudo-long-exposures without a heavy ND filter. This tool combines Phase One’s focus on image quality, advancements in sensor technology, and improvements to the IQ4 processing platform. Best of all, it’s very simple to use and produces a single standard raw file with shadows that are even cleaner than those of a standard IQ4 raw file.
CAVEAT: Because this is a tool that is still in beta on a system that hasn’t yet shipped, all the details below are subject to change, and unforeseen limitations may arise.
Before we explain the technical underpinnings of this tool, here are some of the practical uses of this tool:
- Improved Shadow Flexibility in Challenging Scene: The Phase One IQ4 150mp full-frame-645 sensor has the most dynamic range of any camera available today, but some scenes are so challenging that even that dynamic range is not enough. By using Frame Averaging the deepest shadows will be significantly improved, allowing even more aggressive shadow recovery without introducing noise or losing highlight detail.
- Replace your Strong ND filters: Many photographers carry a very strong ND filter (e.g. ten stops) to do long exposures in bright light. These nearly opaque filters allow the photographer to drag the shutter speed out to seconds or minutes or even hours long even in broad daylight, creating rivers and ocean that are glassy-smooth (since all waves and turbulence average out) surreal scenes of city streets that appear as a ghosted river (since any cars that flow with traffic average out to a sea of “smoke”), sidewalks that appear empty, and clouded skies that blur with an effect straight out of science fiction.
- Special Effects: Multiple Exposure is a time-honored special effect in still photography. This tool will allow you to generate a single raw file in-camera from multiple exposures.
- Forget Graduated ND Filters: The use of a graduated ND Filter can be helpful in evening out the exposure of a landscape image with a bright top and a dark bottom. For example in a horizon-at-dawn scene a graduated ND filter can help maintain detail in the sky and foreground land. But graduated ND filters are very brute-force instruments that produce their effect in a fixed pattern over the scene. In the horizon-at-dawn example, a tree or mountain that sticks up over the horizon line will be inadvertently darkened alongside the sky. In the digital era many photographers choose to shoot without a graduated ND filter and lift the shadows in post processing, but this has the effect of increasing the noise in the shadows if done too aggressively. At base ISO a standard single-exposure raw file from the IQ4 150mp can be lifted around 4 stops before the increase in shadow noise becomes pronounced; with Frame Averaging we expect a significant improvement to shadow quality during such heavy shadow lifts.
Why is it in Beta?
This is an entirely new way of thinking about exposure, shutter speed, and dynamic range. Phase One is eager to see how users actually use this technique, and what ideas they have for a final production tool that employs this technique. That’s why the technique will first debut as a Phase One Labs beta feature; users can get a preview of this technique, try it in a variety of scenes and scenarios, provide feedback through their dealer, and help shape the many exciting places this tool can go.
The Technical Underpinnings: How does it work?
At its heart, this tool works by averaging two or more (often many more) sequential captures together, before the raw file is generated. This has the effect of evening out noise in the shadows. With four samples the noise should be roughly half as much as a single capture (which is already extraordinarily low), with sixteen samples it should be roughly half as much again. In theory this technique can be used by anyone with any camera by capturing more than one image of the same scene and layering them with a low opacity in Photoshop or via specialized software. However, in practice, the specifics of how the IQ4 150mp does frame averaging make this tool far more useful than doing it manually with another camera. Let’s take a look at the technical components that make the IQ4 150mp frame averaging unique.
Sensor-Level Calculation
When you do this, averaging matters. With traditional/manual/laborious frame averaging the sensor captures light, counts it, and records the count for every exposure, then later those counts are averaged together in software. With the IQ4 Frame Averaging the sensor captures light from the first exposure, captures light from the second exposure, captures light from the third exposure, and so on, then after all frames have been captured it counts and records the total averaged light. In technical terms the averaging is occurring before the A/D conversion which leads to lower noise.
Certain electronics are required inside the sensor to allow this kind of use, which is why the IQ4 150mp and IQ4 150mp Achromatic will have this feature, but the IQ4 100mp Trichromatic will not.
Electronic Shutter (ES)
When doing frame-averaging, the IQ4 uses the sensor-based Electronic Shutter which allows it to frame-average very quickly in succession. In fact it’s possible that the IQ4 Frame Averaging will allow successive captures with no temporal gap. Traditional mechanical shutters (focal plane shutters or leaf shutters) must reset between exposures, so even the cameras with mechanical shutters capable of very high frame rates, cannot have the entire frame exposed all the time, which leads to gaps of time (aka “temporal gaps”) when the scene is not being recorded. For example, in a scene of a car driving across a desert at night a temporal gap leads to the headlights being rendered as a series of dots rather than a continuous long blurred line. The ES of the IQ4 can expose for a given length of time, end that exposure, and begin a new exposure immediately.
The ES is also beneficial to frame averaging because it generates zero vibration. With a traditional mechanical shutter (focal plane shutters or leaf shutters) there is a small amount of vibration created each time the camera captures. When averaging several or many captures together that vibration can reduce sharpness and cause visual artifacts. ES creates no vibration, so many captures can be averaged together and retain the same sharpness as a single capture.
Capture One Inside
The IQ4 platform (dubbed the “Infinity Platform”) is incredibly powerful and easy to extend with Feature Updates. The raw computing power of the IQ4 is roughly equal to that of the iPad Pro, which is incredible given its size. This power makes it easy for Phase One to implement high-end tools like this directly into the back. Moreover, the software stack (the way the different firmware, software, and UI are layered together) has been completely revamped from the IQ1/2/3, bringing it to a hyper-modern architecture that allows different developers to work independently and concurrently on different projects. That means the team working on the Frame Averaging tool can modify the tool and do testing without waiting for the UI team to finalize whatever improvements they are working on.
Pro-Oriented Focus
It’s a fact in the camera industry that most of all the cameras sold in the world are sold to the general public who have an interest in taking high-quality pictures and having a good camera, but who do not have any particular training or expertise in the art or science of photography. As a result the interfaces, feature sets, and overall design of most cameras is made with the general public in mind. This is true not only of the remaining point-and-shift cameras, but of the seemingly “high-end” cameras that many pros use. Believe it or not, even the 5Ds R, a very capable small-format camera used by many pros, is mostly bought by the general public. Phase One is the only camera companies laser focused exclusively on professional users and enthusiasts willing and excited to invest the time to learn the ins and outs of the art and science of photography. This means Phase One can assume its users understand photographic topics like exposure, noise, and dynamic range, and focus all their efforts on advanced tools that require that foundational knowledge. This is a big reason why Phase One is the only camera company with a proper dedicated focus stacking tool (try to explain focus stacking to Uncle Bob), and it’s one of the major reasons why Phase One is the only camera company with in-camera raw-based ES frame averaging.
Summary
Frame averaging techniques are possible with any camera. But frame averaging often call for many frames, even dozens or more for some use cases. So having the IQ4’s ES do the heavy lifting in an automated and user-friendly way, in camera, is transformative to the use of this technique. Doing frame-averaging manually is possible, but highly tedious and time consuming. All indications are the IQ4 tool will make it just another technique that can be easily applied to specific kinds of scenes. In this way it’s reminiscent of the automated Focus Stacking Tool in the XF; focus stacking is possible with nearly any camera, but after you’ve done a full day of focus stacking with the XF it’s unimaginable to do it on another platform.
WHERE ARE THE DANG PHOTOS IN THIS ARTICLE??
Trust me, we understand. This article has a notable lack of example files. We were so excited to share news about this tool that we couldn’t wait! We expect that this feature will be enabled in the pre-production unit we show off the week of the PhotoPlus Expo in NYC. After that very busy week we expect to be able to generate some real-world test raw files using this new tool.
