AI is rapidly changing the way we think about photography. Just a couple of years from now, most advances in the photo space will be AI-centric, and not optics or sensor-centric as before. The advancement in photography technology will, for the first time ever, be untethered from physics, and will create a whole new way of thinking about photography. This is how it’s going to happen.
Processing power
Just six months ago we saw the first glimpse of AI entering our consumer world when Apple introduced A11 Bionic neural engine chip, which powers current generation of iPhones. The A11 is important because the chip is specifically designed for tasks such as image–and face–recognition, AR applications, etc. In applications like this, it’s way more effective.
I then wrote that the Google Pixel line would introduce it’s own hardware chips, designed for specific tasks, and that indeed happened sooner than anyone—including me—expected, as the Pixel 2 featured dedicated image enhancement chip to help with image processing on the fly. What made it intriguing is that when Pixels were announced and shipped, there was no mention of the feature, and only sometime later did Google admit that the Pixels had a dedicated chip which would be “enabled†sometime in the future (if you own Pixel 2 today, this hardware feature is already enabled).
Then came Chinese smartphone maker Huawei with the P20 Pro, featuring 4 cameras — 1 in front and 3 in the back. In addition to achieving the highest DxO Mark score to date, the Huawei P20 Pro is packed with AI features, such as real-time image scene recognition, meaning it can discern 500 scenarios in 19 categories, such as animals, landscapes, as well as an advanced night mode, where the AI assists in processing noisy photos, making them almost perfect. The Verge has great coverage with image samples to provide a good overview of this photo powerhouse.
As the next generation of smartphone products are developed, many manufacturers are focused on image capture and real-time processing, partially because it’s a great marketing differentiator, but also because advances in this area are clearly visible to the consumer.
Catering to the pros
But in professional and semi-pro setting, there are several other developments that are key to image quality. First of all, is the processing part, that has to happen right after the photo has been taken. Advances in RAW processing have been steady and predictable (but yet, very welcome by everyone), but AI is ready to supercharge this process. Recently PetaPixel featured a research paper named “Learning to See in the Dark†by Chen Chen, Qifeng Chen, Jia Xu, and Vladlen Koltun, that covers techniques of recovery of extremely underexposed RAW files.
For the consumer it means that AI-assisted software can create high-quality images way beyond the current physical limit — allowing smaller sensors (such as found in drones or mirrorless cameras) to leapfrog current top-end DSLR’s.
In other applications, it might allow tiny security cameras to yield high-quality imagery, increasing overall surveillance.
Photo optimization
One intriguing technology I had a chance to see recently is AI-powered upscaling, far beyond in quality than what is currently available to the public. A team of AI developers at Skylum is putting finishing touches on technology that will allow smartphone images to be upscaled and printed at an incredibly high resolution and sharpness. As I’ve previously pointed out, not everyone has an iPhone X in their pocket — hundreds of millions of people today are buying brand new phones that use 4-year-old technology, so having sharper, crispier photos from outdated smartphone sensors will allow millions of people future-proof their precious moments.
Thousands of kilometers from Skylum AI research lab is another startup, that stealthily applying quantum mechanics research to RAW files, is promising to compress your photos up to 10x without loss of data.
A year ago Apple introduced HEIF, High Efficiency Image Format. If you use iPhone with iOS11 you are likely using HEIF without even knowing it. HEIF allows for higher quality images (compared to JPEG) at about half the size, allowing to keep twice as many photos as before. Dotphoton, a small startup from Switzerland, is aiming to up HEIF format and is focusing on the professional applications, from aerial footage to professional photographers.
After a long technological hiatus in image tech, we are yet again seeing an explosion of interest in the space. Photography plays an important role in every tech company, but nowhere it is more important than in the smartphone race. And as September edges closer, Google and Apple will both be aiming to announce cutting-edge photography advances. Yet, an influx of smaller players are innovating at a rapid rate and raising the stakes for everyone.
