Posted by rbanffy 8 hours ago
I wish there was some explanation how this is a vulnerability and not just a bug generating erroneous data.
Vulnerability for me sounds like there’s a reasonable way to create an exploit from the bug, and I don’t see one here as someone who’s not very familiar with the topic.
This is pretty situational, though, isn't it? You still have to be dealing with IDN names.
DNS names are a thing where Sales is going to tell the Engineer that they can't issue the customers randomized ASCII names like abxuewrf.my-thing.example because real customers want to write our-brand-name.my-thing.example instead - even though you already know bad guys will choose billing.my-thing.example and name-of-bank.my-thing.example and every other unintended bad choice even before we realise about likelihood of these confusion bugs in software like Python.
I also like how sites like github use githubusercontent.com or something like that when linking to UGC assets directly, to avoid someone direct linking to something with the implication that it's coming from GitHub.
Server-side Request Forgery (SSRF) is an example of such an exploit targeting a differential in implementations of URL parsers, which is similar to this implementation difference.
"Safe strings" is an example of that idea. Not always possible or practical, but always worth considering if you're doing "validation" as a function.
In particular, if my corporate security team started just mass-flagging all instances of "str.lower" as "security bugs" I would be having a talk with their manager about their threshold for what constitutes a "security bug". Their job is arguably to be more sensitive to that than most engineers, but not that sensitive. It would be like flagging all instances of string concatenation as a vulnerability... and I say that as the guy who would like to eliminate simple string concatenation from programming languages, already a very extreme position on that operation, because of it being at the root cause of so many vulnerabilities... but simply flagging every use as a "vulnerability" is way too sensitive. A demonstration of the ability to use it to bypass some sort of security barrier is necessary to call any specific instance a "vulnerability".
And string concatenation has caused orders of magnitude more actual, verified vulnerabilities than incorrect case folding has.
This is a rhetorical question because there isn’t a generally accepted way of doing so. Automatically patch everything is a silly way to do vulnerability management but software is cheap to change, so it’s often easier at scale to just force engineering teams to patch even if it doesn’t make sense in context.
I’m not a fan of this approach, but I can understand why it’s so popular.
Of course that can then lead to warnings fatigue so it’s not necessarily a big improvement, or an improvement at all, in the long run, depends a lot on the org philosophy and habits.
Is that a real thing though? Is someone doing that?
For example you might use a ready-made WAF written in a non-Python language in front of a Python app.
> We have a working exploit (OOB access in the V8 heap), our security folks put one together based on the example I posted above (and they're cleaning it up to post it here). In general, we find that correctness issues like this are pretty much always exploitable with a bit of effort (not even that much effort normally, just gluing together a few gadgets), so we treat correctness issues as security issues until they are proven not to be, rather than the other way around.
The floating-point-to-heap-corruption chain here is... uniquely JavaScript, but in general getting two different implementations to disagree is the start of lots of interesting inconsistent behaviour.
The rule for how SAN DnsNames match againt like names, from the DNS is very, very simple so that you don't screw it up. You handle a single wildcard (ASCII * code 42 matches any single DNS label) and beyond that it's literally byte comparison. You don't care what these bytes mean, either the bytes are all identical or that's not a match and we're done.
This sounds like a really hacky solution compared to implementing a separate frozen Unicode 3.2.0 lower.
https://github.com/python/cpython/commit/7e109d084d55e7eb
The important part is:
# B.3 is mostly Python's .lower, except for a number
# of special cases, e.g. considering canonical forms.
+# To enforce Unicode 3.2.0 behavior of .lower instead of
+# whatever Unicode version is included with Python we
+# add unassigned or newly case-folding codepoints to
+# the exception map, too.
b3_exceptions = {}
for k,v in table_b2.items():
if list(map(ord, chr(k).lower())) != v:
b3_exceptions[k] = "".join(map(chr,v))
+for cp in range(0x110000):
+ ch = chr(cp)
+ # Assigned in current Unicode version
+ # and supports case folding, but not
+ # explicitly in B.2 or B.3 tables.
+ if (unicodedata_current.category(ch) != "Cn"
+ and ch.lower() != ch
+ and cp not in table_b2
+ and cp not in table_b3):
+ b3_exceptions[cp] = ch # Identity.Impressive to have found such a vulnerability!
[SpecialCasing] Contains additional case mappings that map to more than one character, such as "ß" to "SS".