Posted by zdw 22 hours ago
I think the headline is problematic because it suggests the raw photos aren't very good and thus need processing, however the raw data isn't something the camera makers intend to be put forth as a photo, and the data is intended to be processed right from the start. The data of course can be presented in as images but that serves as visualizations of the data rather than the source image or photo. Wikipedia does it a lot more justice. https://en.wikipedia.org/wiki/Raw_image_format If articles like OP's catch on, camera makers might be incentivized to game the sensors so their output makes more sense to the general public, and that would be inefficient, so the proper context should be given, which this "unprocessed photo" article doesn't do in my opinion.
That’s not how I read either the headline or the article at all. I read it as “this is a ‘raw photo’ fresh off your camera sensor, and this is everything your camera does behind the scenes to make that into something that we as humans recognize as a photo of something.” No judgements or implications that the raw photo is somehow wrong and something manufacturers should eliminate or “game”
Something that surprised me is that very little of the computation photography magic that has been developed for mobile phones has been applied to larger DSLRs. Perhaps it's because it's not as desperately needed, or because prior to the current AI madness nobody had sufficient GPU power lying around for such a purpose.
For example, it's a relatively straightforward exercise to feed in "dark" and "flat" frames as extra per-pixel embeddings, which lets the model learn about the specifics of each individual sensor and its associated amplifier. In principle, this could allow not only better denoising, but also stretch the dynamic range a tiny bit by leveraging the less sensitive photosites in highlights and the more senstive ones in the dark areas.
Similarly, few if any photo editing products do simultaneous debayering and denoising, most do the latter as a step in normal RGB space.
Not to mention multi-frame stacking that compensates for camera motion, etc...
The whole area is "untapped" for full-frame cameras, someone just needs to throw a few server grade GPUs at the problem for a while!
Sony Alpha 6000 had face detection in 2014.
I've seen only a couple of papers from Google talking about stacking multiple frames from a DSLR, but that was only research for improving mobile phone cameras.
Ironically, some mobile phones now have more megapixels than my flagship full-frame camera, yet they manage to stack and digitally process multiple frames using battery power!
This whole thing reminds me of the Silicon Graphics era, where the sales person would tell you with a straight face that it's worth spending $60K on a workstation and GPU combo that can't even texture map when I just got a Radeon for $250 that runs circles around it.
One industry's "impossible" is a long-since overcome minor hurdle for another.
Mobile phone camera's are severely handicapped by the optics & sensor size. Therefore to create a acceptable picture (to share on social media) they need to do a lot of processing.
DSLR and professional camera's feature much greater hardware. Here the optics and sensor size/type are important it optimize the actual light being captured. Additionally in a professional setting the image is usually captured in a raw format and adjusted/balanced afterwards to allow for certain artistic styles.
Ultimately the quality of a picture is not bound to it's resolution size but to the amount and quality of light captured.
You sound exactly like the sales guy trying to explain why that Indigo workstation is “different” even though it was performing the exact same vector and matrix algebra as my gaming GPU. The. Exact. Same. Thing.
Everything else you’ve said is irrelevant to computational photography. If anything, it helps matters because there’s better raw data to work with.
The real reason is that one group had to solve these problems, the other could keep making excuses for why it was “impossible” while the problem clearly wasn’t.
And anyway, what I’m after isn’t even in-body processing! I’m happy to take the RAW images and grind them through an AI that barely fits into a 5090 and warms my room appreciably for each photo processed.