Category: Security

  • Timing side channels: why == on a secret is a bug, and how constant-time comparison works

    Timing side channels: why == on a secret is a bug, and how constant-time comparison works

    Here’s a line of code I still find in reviews every few months:

    if request.headers["X-Signature"] == hmac.new(key, body, "sha256").hexdigest():
        process(body)

    It looks fine. The HMAC is correct, the key is secret, the comparison is a comparison. The bug is that == on strings returns as soon as it finds the first byte that differs, so the time it takes leaks how many leading bytes of the attacker’s guess were right. Given enough measurements, an attacker can recover a valid signature one byte at a time without ever knowing the key.

    How the attack works

    Illustrative chart: time of an early-exit == comparison rises in steps with each correct leading byte of a 32-byte MAC guess, while a constant-time comparison stays flat

    A short circuiting comparison does roughly one unit of work per matching byte before it bails out. If the attacker submits a signature starting with a, then b, then c and so on, the guess with the correct first character takes very slightly longer than the other fifteen hex digits. Fix that character, move to the second one, repeat. For a 64 character hex digest that’s at most 16 x 64 = 1,024 candidate positions instead of 16^64 possible values. The brute force problem goes from impossible to linear.

    The obvious objection is that a few nanoseconds per byte can’t survive network jitter. That objection is weaker than it sounds. Jitter is noise, and noise averages out with repetition. Crosby, Wallach and Riedi’s 2009 paper “Opportunities and Limits of Remote Timing Attacks” showed that with careful statistics an attacker could distinguish differences in the tens of microseconds across the internet and around 100 nanoseconds on a LAN. In 2020, “Timeless Timing Attacks” by Van Goethem and colleagues removed much of the network jitter entirely by sending two requests in the same HTTP/2 packet and only looking at which response came back first. And an attacker who can get a VM in the same cloud region as your service is not measuring across the internet at all.

    I treat any comparison against a secret value as attackable. Whether a particular one is exploitable this week is not the question I want to be answering during an incident.

    A real example: Rails basic auth

    CVE-2015-7576 is a clean case. Action Controller’s http_basic_authenticate_with compared the supplied username and password with a normal string comparison, which, in the words of the advisory, made it easier for remote attackers to bypass authentication by measuring timing differences. The fix, released in January 2016 across the supported Rails branches, switched to a constant-time comparison.

    TLS has a much more famous version of the same class of problem. Lucky Thirteen (CVE-2013-0169) recovered plaintext from CBC mode TLS because the time taken to check the MAC depended on how much padding was removed. That one took years of careful patching across every TLS library, and it’s a big part of why TLS 1.3 dropped CBC cipher suites entirely.

    What constant-time comparison does

    The idea is to do the same amount of work regardless of where, or whether, the inputs differ. Instead of returning at the first mismatch, you accumulate the differences and only look at the result at the end:

    int ct_equal(const unsigned char *a, const unsigned char *b, size_t len)
    {
        unsigned char diff = 0;
        for (size_t i = 0; i < len; i++)
            diff |= a[i] ^ b[i];
        return diff == 0;
    }

    XOR is zero only when two bytes match, and OR-ing every result together means a single mismatch anywhere leaves diff non-zero. There’s no branch that depends on the data inside the loop, so the loop takes the same time for a guess that’s wrong in the first byte as for one that’s wrong in the last.

    Please don’t write this yourself in production, though. Optimising compilers are allowed to notice that once diff is non-zero the answer is decided, and to turn your careful loop back into an early exit. Library implementations use compiler barriers or assembly to stop that, and they get tested for it. Use them:

    import hmac
    if hmac.compare_digest(received_sig, expected_sig):
        process(body)
    import "crypto/subtle"
    
    if subtle.ConstantTimeCompare(receivedMAC, expectedMAC) == 1 {
        process(body)
    }
    const crypto = require("crypto");
    const a = Buffer.from(receivedSig, "hex");
    const b = Buffer.from(expectedSig, "hex");
    if (a.length === b.length && crypto.timingSafeEqual(a, b)) {
      process(body);
    }

    Other ecosystems have the same thing: hash_equals in PHP, MessageDigest.isEqual in Java, ActiveSupport::SecurityUtils.secure_compare in Rails, CRYPTO_memcmp in OpenSSL and sodium_memcmp in libsodium.

    Length is a leak too

    Most constant-time functions are only constant-time for inputs of equal length. Go’s ConstantTimeCompare documents that it returns 0 immediately when lengths differ, and Node’s timingSafeEqual throws, which is why the JavaScript example checks length first. For HMACs and hashes that’s fine, since the length is public anyway.

    For things like API tokens where you’d rather not reveal the length, a common trick is to hash both values first (SHA-256 or an HMAC with a per-process key) and compare the fixed length digests.

    The comparisons people forget

    Token lookups are the subtle one. If your API looks up a key with SELECT ... WHERE api_key = ?, the database’s index search is itself a comparison that can leak timing about the stored values. The pattern I prefer is a split token: the first part is a non-secret identifier you look up by, and the second part is a secret you verify against a stored hash with a constant-time comparison. A leaked database then also doesn’t contain usable tokens, which is a nice side benefit.

    Login endpoints leak in a different way. If you return immediately for an unknown username but spend 200 ms running Argon2 for a known one, the response time tells an attacker which accounts exist. Run a dummy hash against a fixed value when the user isn’t found, so both paths cost the same. This ties back to the password storage post from earlier this week: a slow hash makes the timing difference larger, not smaller.

    Error paths count as well. Returning “invalid padding” and “invalid MAC” as different errors, or failing at different points, turns a timing oracle into an even easier one. Verify the MAC first, over the whole message, and give one error for everything.

    What I check in review

    Any ==, !=, equals(), strcmp or memcmp that touches a signature, MAC, token, password reset code, session identifier or API key gets flagged. A Semgrep rule for this takes a few minutes to write and is worth running in CI, because the bug is easy to reintroduce. It costs nothing to fix and doesn’t change behaviour. It removes a whole class of attack that needs nothing more than a stopwatch and patience.

  • Supply chain attacks on package registries: typosquatting, dependency confusion and what SLSA actually asks for

    Supply chain attacks on package registries: typosquatting, dependency confusion and what SLSA actually asks for

    Every npm install or pip install is a decision to run someone else’s code with your build’s permissions. Most of the time that’s a good trade. The attacks I want to cover here don’t exploit a bug in your code at all. They exploit how package managers decide which package to fetch, and how little most organisations know about where their artifacts came from.

    Typosquatting is boring and still works

    Pipeline from source repo to build platform, publish, package registry and install, with the attack that lands at each stage above it and the matching control, such as SLSA provenance, trusted publishing, scoped names and lockfile hashes, below it

    The oldest trick is publishing a package with a name one keystroke away from a popular one. In 2017 an npm package called crossenv (the real one is cross-env) copied the legitimate package’s behaviour and also sent the contents of the environment to a remote server. Environment variables on a CI runner are often the most valuable thing on the box: registry tokens, cloud credentials, signing keys. The same year Slovakia’s national security authority reported a batch of typosquats on PyPI with names like urlib3, and PyPI has been removing waves of them ever since.

    The registry operators catch a lot of these now, but not before someone installs them. On your side, the defences are mundane. Don’t let people add dependencies by typing names into a terminal on a machine that holds credentials. Review dependency additions in pull requests the same way you review code, and look at the package’s age, download count and repository link before approving it. A two week old package with a name almost matching a library with millions of weekly downloads should not get through review.

    Dependency confusion

    Alex Birsan’s February 2021 research turned a quirk of package resolution into a way to run code inside Apple, Microsoft, PayPal and over thirty other companies, earning more than $130,000 in bug bounties. The idea is simple. Companies use internal package names that never exist on the public registry. Those names leak through public manifests, JavaScript bundles and error messages. If an attacker publishes a public package with the same name and a higher version number, many resolvers happily prefer it.

    The classic pip mistake looks like this:

    pip install --index-url https://pypi.internal.example/simple \
                --extra-index-url https://pypi.org/simple \
                acme-billing-utils

    --extra-index-url doesn’t mean “fall back to PyPI if the internal index doesn’t have it”. pip collects candidates from every index and picks the best version across all of them. If someone publishes acme-billing-utils 99.0.0 on PyPI, that’s what gets installed.

    This wasn’t a one-off. In December 2022 the PyTorch project disclosed that its nightly builds installed between December 25 and 30 had pulled a malicious torchtriton package from PyPI, because the legitimate one lived on PyTorch’s own index and PyPI took precedence. The malicious version collected system information and files from the machines that installed it.

    The structural fix is to have exactly one place your builds resolve from. Run a proxying repository (Artifactory, Nexus, a pull-through cache) that serves internal packages and mirrors public ones, and configure it so an internal name can never be satisfied from upstream. For npm, put internal packages under a scope you own and pin that scope to your registry:

    # .npmrc
    @acme:registry=https://npm.internal.example/
    registry=https://registry.npmjs.org/

    Registering your internal names (or scope) on the public registry as empty placeholders is a reasonable extra layer, but I wouldn’t rely on it alone.

    Lockfiles and hashes

    A lockfile pins exact versions, and the integrity hashes in it pin exact bytes. That turns “install whatever matches ^4.2” into “install this tarball and fail if it’s different”. Use npm ci rather than npm install in CI, since ci refuses to proceed if package-lock.json and package.json disagree. For Python, pip’s hash checking mode does the same thing:

    pip install --require-hashes -r requirements.txt

    with each line carrying --hash=sha256:..., which tools like pip-compile --generate-hashes produce. Hashes don’t protect you on the day you add or upgrade a dependency. They do stop a later build from silently picking up something different.

    Compromised maintainers

    Some attacks don’t need a lookalike name. In 2018 the maintainer of event-stream handed the project to a volunteer, who added a dependency called flatmap-stream carrying code aimed at stealing from a specific cryptocurrency wallet app. In October 2021 the npm account behind ua-parser-js was hijacked and versions shipping a cryptominer and credential stealer were published. And in 2024, CVE-2024-3094 showed a backdoor in xz utils that had been inserted after a multi-year effort to gain maintainer trust, caught largely because one engineer noticed SSH logins getting slower.

    Nothing on the consumer side fully prevents this, but you can shrink the window. Delay adopting brand new versions by a few days unless they fix something you need, because most malicious releases are spotted and pulled quickly. Watch for install scripts (preinstall, postinstall) appearing in packages that never had them, and consider npm config set ignore-scripts true for builds that don’t need them.

    What SLSA requires

    SLSA (Supply-chain Levels for Software Artifacts) is easy to misread as a scanning standard. It’s not. Version 1.0’s Build track is about provenance: a verifiable statement of which source, which build platform and which process produced an artifact.

    The levels build on each other. Build L1 means provenance exists and describes how the artifact was built. Build L2 means the build ran on a hosted platform that generates and signs that provenance itself, so a developer laptop can’t produce it. Build L3 means the platform is hardened: builds are isolated from each other, and the signing material is out of reach of the user-defined build steps, so even a malicious build script can’t forge provenance.

    What SLSA does not do is tell you whether a dependency is trustworthy. It lets a consumer check that a package came from the repository and workflow it claims to, which is exactly what you want to know after an account takeover. On the publishing side, PyPI’s Trusted Publishing and npm’s provenance support both bind packages to a specific CI workflow using short lived OIDC credentials instead of long lived API tokens. On the consuming side you verify:

    npm audit signatures
    
    slsa-verifier verify-artifact ./mytool-linux-amd64 \
      --provenance-path ./mytool-linux-amd64.intoto.jsonl \
      --source-uri github.com/example/mytool \
      --source-tag v1.4.2

    Here is the order I’d tackle it in. Get to one resolution source and committed lockfiles with hashes first, because that kills dependency confusion and drift. Move your own publishing to trusted publishing so there’s no static token to steal. Then start verifying provenance for the dependencies that matter most, beginning with anything that runs in your build or release pipeline.

    The part people skip

    Your build environment is part of the supply chain. A CI job that installs packages while holding production deploy credentials gives every one of those packages production access. Split dependency installation from credentialed steps where you can, give build jobs only the secrets they need, and treat a compromised dependency the same as a compromised server: rotate what it could reach, don’t just bump the version.

  • MFA that survives phishing: TOTP, push and FIDO2 passkeys compared

    MFA that survives phishing: TOTP, push and FIDO2 passkeys compared

    Part 2 of 4 in the Authentication done right series. Part 1: Password storage in 2026.

    In the first part of this series I covered how to store passwords so a database leak doesn’t hand attackers working credentials. This part assumes the password is already gone, because sooner or later it will be. Phished, reused, scraped off an infostealer log. The question is what your second factor does when someone is actively trying to get past it, and the honest answer for most deployed MFA is “less than you’d think”.

    Not all second factors are the same control

    Two sequence diagrams side by side: a phishing proxy relays a password and TOTP code to the real site and steals the session cookie, while with WebAuthn the origin-bound credential is not offered on the fake domain and the login fails

    People talk about MFA as if it were one thing. In practice the methods fall into two groups. Shared secret and approval based factors (SMS codes, TOTP apps, push notifications) prove the user has a device, but the proof can be relayed by whoever the user is talking to. Public key factors bound to an origin (FIDO2/WebAuthn security keys and passkeys) produce proof that only works for the site that asked for it. That distinction is what CISA means by phishing resistant MFA in its 2022 guidance, and it’s what OMB memo M-22-09 required US federal agencies to adopt.

    SMS is the weakest of the first group. SIM swaps and number porting fraud move the phone number to an attacker, and NIST SP 800-63B has long classed SMS and voice as a restricted authenticator. I’d still take SMS over no second factor, but I wouldn’t let it protect anything I cared about.

    TOTP and the relay problem

    TOTP (RFC 6238, built on HOTP from RFC 4226) derives a six digit code from a shared secret and the current 30 second time step. It’s offline, cheap and works everywhere. It also has nothing in it that identifies which website the user is typing it into.

    That’s the core weakness. An adversary-in-the-middle kit like Evilginx sits between the victim and the real login page as a reverse proxy. The victim sees what looks like the real site, types their password and their TOTP code, and the proxy forwards both in real time and keeps the resulting session cookie. The code was valid. It just went to the wrong party.

    The 2022 “0ktapus” campaign worked this way at scale. SMS messages pointed employees of Twilio and many other companies to lookalike Okta pages that captured credentials and one time codes as they were typed. Cloudflare was targeted by the same campaign, and some of its employees did enter their credentials. The attack still failed there, because Cloudflare required FIDO2 hardware keys and the fake domain couldn’t get a valid assertion from them. Cloudflare’s own write-up of that incident is one of the clearest arguments for WebAuthn I know of.

    If you do run TOTP, at least implement it properly. Accept a small clock skew window and reject codes that have already been used.

    from datetime import datetime, timezone
    import pyotp
    
    def check_totp(user, code):
        totp = pyotp.TOTP(user.totp_secret)
        # accept the current step and one on either side for clock drift
        if not totp.verify(code, valid_window=1):
            return False
        step = totp.timecode(datetime.now(timezone.utc))
        if user.last_totp_step is not None and step <= user.last_totp_step:
            return False  # replayed code
        user.last_totp_step = step
        user.save()
        return True

    Rate limit verification attempts too. A six digit code has a million values, which is not a lot if an endpoint lets someone try thousands of codes per account.

    Push approval and MFA fatigue

    Push MFA looked like an upgrade because the user doesn’t type anything. It created a different failure. If an attacker has the password, they can trigger approval prompts over and over until the user taps Approve to make it stop, or until someone calls pretending to be IT and asks them to accept.

    That’s what happened at Uber in September 2022. According to Uber’s own statement, the attacker had a contractor’s password, sent repeated push requests, and the contractor eventually accepted one. Cisco’s breach that year, disclosed in August, combined voice phishing with the same kind of repeated push prompts. In both cases the MFA product behaved exactly as designed.

    The fixes are real but partial. Number matching shows a code on the login screen that the user has to enter in the authenticator app, so a blind tap no longer works; Microsoft made it mandatory for Authenticator push in May 2023. Showing the location and application in the prompt helps a little. Limiting how many prompts can be sent per hour and alerting on bursts helps more. None of it stops a real-time phishing proxy, because the user is looking at a page that displays the number too.

    Why WebAuthn is different

    With WebAuthn, the browser builds a clientDataJSON structure that includes the challenge and the origin the user is actually on, and the authenticator signs it along with the relying party ID hash. Credentials are scoped to the relying party ID, so an authenticator won’t even offer a credential for example.com to a page on examp1e-login.com. There’s no code for the user to copy, so there is nothing to relay.

    The server side still has to do its part. Every serious library makes you pass the expected origin and RP ID, and you should never loosen those to make a test environment work.

    from webauthn import verify_authentication_response
    
    verification = verify_authentication_response(
        credential=assertion_from_browser,
        expected_challenge=session.pop("webauthn_challenge"),
        expected_rp_id="example.com",
        expected_origin="https://example.com",
        credential_public_key=stored.public_key,
        credential_current_sign_count=stored.sign_count,
        require_user_verification=True,
    )
    stored.sign_count = verification.new_sign_count

    The challenge must be random, single use and tied to the session that requested it. I’ve reviewed implementations that reused a static challenge, which turns a signature scheme back into something replayable.

    Passkeys, synced and device bound

    Passkeys are WebAuthn credentials, usually discoverable ones, and most are synced through a platform account such as iCloud Keychain or Google Password Manager. That’s great for adoption, because users who lose a phone don’t lose their credentials. It also means the security of the passkey now depends on the security of that cloud account and its recovery process.

    For most consumer apps, synced passkeys are a large improvement over passwords plus TOTP. For administrators and production access I still want device bound credentials on hardware keys, and I want attestation checked so I know what kind of authenticator registered.

    Recovery is part of your MFA

    Attackers go where the controls are weakest, and that’s usually account recovery. A strong factor with a “lost your key? we’ll send an SMS” fallback is an SMS factor with extra steps. Help desk resets are the same problem with a human in the loop. The MGM Resorts intrusion in 2023 was widely reported to have started with a call to the help desk.

    So register at least two strong authenticators per user, make recovery require something stronger than knowledge of public facts, and log and alert on every factor reset. Treat a new factor enrolled right after a password reset as a high priority event.

    The next part moves from authenticating a person to delegating access between systems, which means OAuth 2.0, OpenID Connect and the token mistakes that come with them.

    Other parts: Part 1: Password storage in 2026, Part 3: OAuth 2.0 vs OpenID Connect, Part 4: Session management.