Security personnel around the world are on high alert Tuesday following the discovery of eight TLS certificates with private keys so weak they could allow anyone to mint trusted browser and app credentials for any site on the Internet.

As of late morning Tuesday, browser makers and some major apps and programming languages had blocked all eight of the intermediate certificates. There are no reports of certificate transparency logs, which track the issuance of all browser-trusted certificates, showing the compromised keys being used maliciously. The threat remains far from over, however, because it will be hard and time-consuming to update other key parts of the Internet, such as Linux distributions and embedded and Internet of Things devices.

One word: bonkers

The eight intermediate certificates belong to India-based certificate authority eMudhra Limited. An informal advisory published to emudhra.we-have-a-problem.com over on Tuesday posted the message “We have a problem here – Wayne.” The words were digitally signed using the private key of one of the eMudhra certificates. With proof that the reporting party had control of the key, defenders scrambled because the same key could be used to sign unauthorized certificates that would be trusted by millions of servers worldwide.

“This will be a massive cleanup job, bigger than Heartbleed,” Filippo Valsorda, a cryptography engineer and a cryptography maintainer for the open source Go programming language, told Ars. “There will be a very, very long tail, IoT devices, Docker containers, forgotten Linux machines, unmaintained CLI tools that shipped their own copy of the roots (which is common in some ecosystems, like Python). Also, Linux distributions are still rolling out updates, some of them are not ready yet, and then system administrators will need to apply them to a lot of machines.”

Heartbleed is the name of a critical vulnerability in the OpenSSL cryptography library. In 2014, it came under mass exploitation. Exploits let many unaffiliated hackers decrypt TLS-protected exchanges between parties. They also recovered private TLS certificate keys, session cookies, and passwords. The attacks compromised thousands of servers, some belonging to large companies and government agencies around the world.

Recovery required a long series of steps that took years. Steps included patching OpenSSL on all affected devices, reissuing millions of TLS keys, revoking old certificates, rotating passwords and session tokens and updating embedded devices and other forgotten systems—something that was extremely time- and resource-intensive, especially at Internet scale.

“Every time you need a lot of different people in different contexts to do something, it will take a long time,” Valsorda explained. “This time it’s worse because there’s more contexts, more people, and more heterogeneity.”

The compromised eMudhra keys were defective at a level that’s challenging the imagination of many of the engineers studying the incident. A normal size for a NIST P-384 elliptic curve cryptography private key contains roughly 384 bits of entropy, a size allowing almost 2384, or a number with 116 digits, different possibilities. The private key sizes of the compromised keys ranged from 3 to 8 bits. Astonishingly, the private keys were 4, 56, 66, 115, 155, 156, 230, 237.

“That is extremely in the ‘this should not be possible’ category,” Valdorda said. “Maybe some horribly dangerous key generation interface has a ‘size’ parameter and they set it to 1 by mistake???”

Ben Cartwright-Cox, director of BGP.tools, said: “My guess is that it’s example code that accidentally leaked to production, but hard to know. But it’s bonkers that no one noticed.”

At this early stage, the most immediate threat appears to be for those using VPNs, especially in countries with governments likely to maliciously intercept traffic. There are likely many other cases for concern. End users largely have no recourse other than to hold off using many non-browser forms of TLS communication until they receive assurances updates are in place.

As noted, most browsers have already blocked the intermediate certificates. The exception appears to be Edge, which at the time this article went live was still connecting to 12 vulnerable hosts. In response to questions asking why and if the company has plans to do so, a Microsoft representative the company will be in touch if it has anything to “share.” Meanwhile, Signal, WhatsApp, Wireguard, and other apps that don’t rely on TLS for confidentiality aren’t affected. This page and the emudhra.we-have-a-problem.com site show apps known to be patched so far.

The incident is the latest to expose a fatal flaw in the TLS regimen. It’s only as strong as its weakest link, which in this case was eMudhra. Fortunately, transparency logs account for much of that failure, but this measure requires parties to actively monitor massive feeds and works only after unauthorized certificates have been issued. eMudhra representatives weren’t immediately available to answer questions.

One caveat: exploiting compromised certificates requires an active adversary-in-the-middle attack, meaning the attacker must act as a proxy between two TLS parties and receive encrypted traffic, decrypt it, reencrypt it, and send it to the other party. Such capabilities, in some cases, are usually limited to only the most well-resourced attackers. People with control of a local network—at a coffee shop, university, or other organization—face much less of a challenge.

  • SocialistVibes01@lemmy.mlOP
    link
    fedilink
    arrow-up
    13
    ·
    edit-2
    18 hours ago

    Rollback your recent updates.

    And check the news on your distro of choice. Maybe it’s already doing the needed security patches.

    • vas@lemmy.ml
      link
      fedilink
      arrow-up
      8
      ·
      edit-2
      2 hours ago

      What good would rolling back updates do? Also, most if not all distros use package signing. It’s by far safer to update forward, not backwards.

      The advise to “rollback” seems actively harmful, to be honest.

      UPDATE: I think I understood the source of the confusion. If we’re speaking about rolling back the distro itself, then that would be actively harmful. On the contrary, *do* upgrade ASAP and don’t roll back. However, if we’re speaking about stuff that is derived from the system, such as pip/npm/etc registries, then latest artifacts from those could be compromised, so you could roll back those to instead get fresh-er upgrades from an upgraded distro where the signatures are already removed. I think that explains our contradictory-at-first-glance views. If the above is true, we’re actually in agreement on what to do.

      • ffhein@lemmy.world
        link
        fedilink
        arrow-up
        3
        ·
        4 hours ago

        From my amateur investigation, it seems like pip/npm/etc. packages are potential attack vectors, BUT afaik this issue only allows a malicious actor to impersonate a HTTPS server, it doesn’t give them any new means to redirect your connect to their fake server. So if I understand this right, to be directly affected you would have had to:

        • Accidentally connected to a fake server, like steamcomnunity instead of steamcommunity.
        • Used a network controlled by the malicious actor, for example a public wifi.
        • Be targeted by your ISP, a government agency or some other corporation capable of intercepting internet traffic from your own network.

        So to me it seems like the risk of being affected is quite low for the average user, BUT (again) I assume that through sheer numbers, there’s a risk that some other developer gets infected by malware or gets their credentials stolen, which in turn could to supply chain attacks in the future.

        I guess one thing Pythong developers have to be mindful of is that they might have venvs containing old versions of pip, which aren’t automatically fixed when you upgrade pip on system level

        • vas@lemmy.ml
          link
          fedilink
          arrow-up
          1
          ·
          2 hours ago

          Oh, your comment made me understand the source of the confusion, I think. I was speaking about the distro itself, hinted by the author above me using the word “distros” as well. But if we’re speaking about pip/npm/etc, then reverting those does make sense, even if a bit paranoid (but being paranoid is no problem). I’ve edited my message to help resolve this misunderstanding.

          • ffhein@lemmy.world
            link
            fedilink
            arrow-up
            2
            ·
            2 hours ago

            I’m not sure what the previous poster was thinking of, but I also thought they were talking about distro packages. I meant my post to be a complement to yours, not a counter-argument.

            So far the recommendations I’ve seen have been to update OS ca-certificates, or to manually remove the compromised ones. And the Python devs are saying to update pip and uv. I guess the same goes for npm, but I uninstalled that a long time ago due to the abundance of supply chain attacks. I haven’t seen anyone trustworthy source recommend rolling anything back.

            • vas@lemmy.ml
              link
              fedilink
              arrow-up
              1
              ·
              1 hour ago

              👍 Makes sense. Also, thanks for the additional info. (“the Python devs are saying to update pip and uv”)