Top
Best
New

Posted by zdw 5 days ago

It's all a blur(lcamtuf.substack.com)
338 points | 62 comments
siofra 45 minutes ago|
Beautiful walkthrough. The key insight people miss is that "looks unreadable to humans" and "is information-theoretically destroyed" are very different bars. The blur looks opaque because our visual system is bad at detecting small per-pixel differences, but the math does not care about our perception.

Same principle applies to other "looks safe" redactions — pixelation with small block sizes, partial masking of credentials, etc. If you can describe the transform as a linear operation, there is probably a pseudoinverse waiting to undo it.

jeremyscanvic 9 hours ago||
Blur is perhaps surprisingly one of the degradations we know best how to undo. It's been studied extensively because there's just so many applications, for microscopes, telescopes, digital cameras. The usual tricks revolve around inverting blur kernels, and making educated guesses about what the blur kernel and underlying image might look like. My advisors and I were even able to train deep neural networks using only blurry images using a really mild assumption of approximate scale-invariance at the training dataset level [1].

[1] https://ieeexplore.ieee.org/document/11370202

criddell 6 hours ago||
Isn't that roughly (ok, very roughly) how generative diffusion AIs work when you ask them to make an image?
jeremyscanvic 5 hours ago|||
You're absolutely right! Diffusion models basically invert noise (random Gaussian samples that you add independently to every pixel) but they can also work with blur instead of noise.

Generally when you're dealing with a blurry image you're gonna be able to reduce the strength of the blur up to a point but there's always some amount of information that's impossible to recover. At this point you have two choices, either you leave it a bit blurry and call it a day or you can introduce (hallucinate) information that's not there in the image. Diffusion models generate images by hallucinating information at every stage to have crisp images at the end but in many deblurring applications you prefer to stay faithful to what's actually there and you leave the tiny amount of blur left at the end.

dangond 6 hours ago|||
I believe diffusion image models learn to model a reverse-noising function, rather than reverse-blurring.
jeremyscanvic 5 hours ago||
Most of them do but it's not mandatory and deblurring can be used [1]

[1] Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise, Bansal et al., NeurIPS 2023

deaddodo 5 hours ago|||
Just to add to this: intentional/digital blur is even easier to undo as the source image is still mostly there. You just have to find the inverse metric.

This is how one of the more notorious pedophiles[1] was caught[2].

1 - https://en.wikipedia.org/wiki/Christopher_Paul_Neil

2 - https://www.bbc.com/news/world-us-canada-39411025

dekhn 7 hours ago||
I didn't learn about this trick (deconvolution) until grad school and even then it seemed like spooky mystery to me.
swiftcoder 9 hours ago||
One salient point not touched on here, is that an awful lot of the time, the things folks are blurring out specifically is text. And since we know an awful lot about what text ought to look like, we have a lot more information to guide the reconstruction...
jlokier 7 hours ago||
Good point, though you have to beware that text-aware image enhancement sometimes replaces characters with what it thinks is a more likely character from context.

I've seen my phone camera's real-time viewfinder show text on a sign with one letter different from the real sign. If I wasn't looking at the sign at the same time, I might not have noticed the synthetic replacement.

wffurr 7 hours ago||
>> sometimes replaces characters with what it thinks is a more likely character from context

Like the JBIG2 algorithm used in a zero click PDF-as-GIF exploit in iMessage a while back: https://projectzero.google/2021/12/a-deep-dive-into-nso-zero...

The vulnerability of that algorithm to character-swapping caused incorrect invoices, incorrect measurements in blueprints, incorrect metering of medicine, etc. https://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres...

gwbas1c 6 hours ago||
And older people are very good at reading blurry text.

(My grandmother always told me to "never get old." I wish I followed her advice.)

coldtea 8 hours ago||
>But then, it’s not wrong to scratch your head. Blurring amounts to averaging the underlying pixel values. If you average two numbers, there’s no way of knowing if you’ve started with 1 + 5 or 3 + 3. In both cases, the arithmetic mean is the same and the original information appears to be lost. So, is the advice wrong?

Well, if you have a large enough averaging window (like is the case with bluring letters) they have constraints (a fixed number of shapes) information for which is partly retained.

Not very different from the information retained in minesweeper games.

derektank 9 hours ago||
Captain Disillusion recently covered this subject in a more popular science format as well

https://youtu.be/xDLxFGXuPEc

lupire 8 hours ago|
8 months ago, for those of us who got excited by the idea of a "recent" new video from CD.
derektank 6 hours ago||
In my defense, that is quite literally the most recent full video the Captain has uploaded!
bmandale 6 hours ago||
> This nets us another original pixel value, img(8).

This makes it all seem really too pat. In fact, this probably doesn't get us the original pixel value, because of quantizing deleting information when the blur was applied, which can never be recovered afterwards. We can at best get an approximation of the original value, which is rather obvious given that we can vaguely make out figures in a blurred image already.

> Nevertheless, even with a large averaging window, fine detail — including individual strands of hair — could be recovered and is easy to discern.

The reason for this is that he's demonstrating a box blur. A box blur is roughly equivalent to taking the frequency transform of the image, then multiplying it by a sort of decaying sin wave. This achieves a "blur" in that the lowest frequency is multiplied by 1 and hence is retained, and higher frequencies are attenuated. However, visually we can see that a box blur doesn't look very good, and importantly it doesn't necessarily attenuate the very highest frequencies by much more than far lower frequencies. Hence it isn't surprising that the highest frequencies can be recovered in good fidelity. Compare a gaussian blur, which is usually considered to look better, and whose frequency transform focuses all the attenuation at the highest frequencies. You would be far less able to recover individual strands of hair in an image that was gaussian blurred.

> Remarkably, the information “hidden” in the blurred images survives being saved in a lossy image format.

Remarkable, maybe, but unsurprising if you understand that jpeg operates on basically the same frequency logic as described above. Specifically, it will be further attenuating and quantizing the highest frequencies of the image. Since the box blur has barely attenuated them already, this doesn't affect our ability to recover the image.

mananaysiempre 6 hours ago|
> You would be far less able to recover individual strands of hair in an image that was gaussian blurred.

Frequency-domain deconvolution is frequency-domain deconvolution, right? It doesn’t really matter what your kernel is.

cornhole 12 hours ago||
reminds me of the guy who used the photoshop swirl effect to mask his face in csam he produced, who was found out when someone just undid the swirl
jszymborski 7 hours ago||
This is the case I always think of when it comes to reversing image filters.
lupire 8 hours ago|||
Action Lab just did a video on physical swirling vs mixing. Swirling is reversible.
Traubenfuchs 11 hours ago||
https://www.bbc.com/news/world-us-canada-39411025
dsego 11 hours ago||
Can this be applied to camera shutter/motion blur, at low speeds the slight shake of the camera produces this type of blur. This is usually resolved with IBIS to stabilize the sensor.
alphazard 10 hours ago||
The ability to reverse is very dependent on the transformation being well known, in this case it is deterministic and known with certainty. Any algorithm to reverse motion blur will depend on the translation and rotation of the camera in physical space, and the best the algorithm could do will be limited by the uncertainty in estimating those values.

If you apply a fake motion blur like in photoshop or after effects then that could probably be reversed pretty well.

crazygringo 7 hours ago|||
> and the best the algorithm could do will be limited by the uncertainty in estimating those values

That's relatively easy if you're assuming simple translation and rotation (simple camera movement), as opposed to a squiggle movement or something (e.g. from vibration or being knocked). Because you can simply detect how much sharper the image gets, and hone in on the right values.

dizzant 9 hours ago||||
I recall a paper from many years ago (early 2010s) describing methods to estimate the camera motion and remove motion blur from blurry image contents only. I think they used a quality metric on the resulting “unblurred” image as a loss function for learning the effective motion estimate. This was before deep learning took off; certainly today’s image models could do much better at assessing the quality of the unblurred image than a hand-crafted metric.
yorwba 1 hour ago||
Probably not the exact paper you have in mind, but... https://jspan.github.io/projects/text-deblurring/index.html
johnmaguire 8 hours ago|||
Record gyro motion at time of shutter?
jeremyscanvic 8 hours ago|||
The missing piece of the puzzle is how to determine the blur kernel from the blurry image. There's a whole body of literature on that that's called blind deblurring.

For instance: https://deepinv.github.io/deepinv/auto_examples/blind-invers...

crazygringo 7 hours ago|||
Absolutely, Photoshop has it:

https://helpx.adobe.com/photoshop/using/reduce-camera-shake-...

Or... from the note at the top, had it? Very strange, features are almost never removed. I really wonder what the architectural reason was here.

tracker1 6 hours ago||
Just guessing, patent troll.
crazygringo 1 hour ago||
Oof, I hope not. I wonder if the architecture for GPU filters migrated, and this feature didn't get enough usage to warrant being rewritten from scratch?
tonymillion 7 hours ago||
I believe Microsoft of all people solved this a while ago by using the gyroscope in a phone to produce a de-blur kernel that cleaned up the image.

Its somewhere here: https://www.microsoft.com/en-us/research/product/computation...

ryukoposting 7 hours ago||
I wonder if the "night mode" on newer phone cameras is doing something similar. Take a long exposure, use the IMU to produce a kernel that tidies up the image post facto. The night mode on my S24 actually produces some fuzzy, noisy artifacts that aren't terribly different from the artifacts in the OP's deblurs.
tflinton 6 hours ago||
I did my thesis on using medioni's tensor voting framework to reconstruct noisy, blurry, low-res and the like images. It was sponsored by USGS on a data set that I thought was a bit of a bizarre use case. The approach worked pretty well, with some reasonable success at doing "COMPUTER ENHANCE" type computer vision magic. Later on talking with my advisor about the bizarrely mundane and uninteresting data sets we were working on from the grant he quipped that "You built a reasonable way of unblurring and enhancing unreadable images, the military doesn't care about this mundane use case." It then occurred that i'd been wildly ignorant to what I just spent 2 years of my life on.
srean 11 hours ago|
Encode the image as a boundary condition of a laminar flow and you can recover the original image from an observation.

If, however, you observe after turbulence has set in, then some of the information has been lost, it's in the entropy now. How much, that depends on the turbulent flow.

Don't miss out on this video by smarter every day

https://youtu.be/j2_dJY_mIys?si=ArMd0C5UzbA8pmzI

Treat the dynamics and time of evolution as your private key, laminar flow is a form of encryption.

lupire 8 hours ago|
If you encode code your data directly in the fluid, then turbulence becomes the statistical TTL on the data.
More comments...