Seth Michael Larson: When str.lower() is a security vulnerability in Python
Internet standards often lack support for non-ASCII characters, necessitating a mapping for domain names. NamePrep, defined in RFC 3491, was designed for this purpose as part of IDNA 2003. This older IDNA standard has since been superseded by IDNA 2008, specified in RFCs 5890-5893. Python offers support for both versions, with IDNA 2003 accessible via the idna codec and IDNA 2008 via the external idna package. The stringprep module in Python's standard library implements StringPrep. A crucial step in StringPrep is "case folding," which involves mapping characters to lowercase or uppercase for case-insensitive comparisons. This process utilizes mapping tables B.2 and B.3 from RFC 3454, which are based on Unicode 3.2.0 rules. A vulnerability was identified in Python's implementation where the standard str.lower() function, using newer Unicode versions, deviated from the required Unicode 3.2.0 case-folding rules. This inconsistency resulted in different IDNA encodings for certain characters. The fix involved creating specific exceptions to ensure Python's str.lower() behavior aligns with Unicode 3.2.0 for StringPrep and IDNA calculations. This remediation ensures IDNA 2003's consistency with its specification.
str.lower()function, using newer Unicode versions, deviated from the required Unicode 3.2.0 case-folding rules. This inconsistency resulted in different IDNA encodings for certain characters. The fix involved creating specific exceptions to ensure Python'sstr.lower()behavior aligns with Unicode 3.2.0 for StringPrep and IDNA calculations. This remediation ensures IDNA 2003's consistency with its specification.