We Tested 96 Authentication Scenarios Against Real AWS Cognito Tokens. Here's What We Learned.

Mocked JWT tests are useful. They also make it easy to test the token you wish your identity provider issued.

While building Gentlity, I wanted to check the authentication boundary with real provider tokens and signing keys, real HTTP requests and a real database. Would a valid identity still be blocked without organization membership? Would a machine credential reach another tenant's deployment? What happens to an existing token when its user is disabled?

We ran 96 checks against a local platform connected to AWS Cognito. The original run recorded 95 passes. One check expected the wrong rejection status. The work also exposed an access-token compatibility gap that the test suite had correctly anticipated. A passing test can confirm a limitation; it does not make the limitation disappear.

The setup

The platform ran locally over HTTPS, backed by PostgreSQL 17. Cognito was real: the platform fetched its public signing keys over HTTPS and verified provider-issued tokens. The local certificate came from a test CA, not a public certificate.

We used two synthetic organizations with separate deployments, human users with different roles, and organization-owned machine credentials. We included a valid Cognito user with no platform membership. Nothing in this setup involved customer data or a staging deployment.

These results describe this specific local test configuration. That distinction matters. Real provider keys exercise an integration boundary. They do not turn a laptop test into a production security assessment.

What we tested

The checks covered valid and invalid machine credentials, expiry and revocation, scope restrictions, human permissions, tenant isolation, malformed tokens, duplicate headers, request bounds, browser sessions and CSRF.

We tested replay and conflicting evidence submissions as well as authentication. Successful authentication should not let a caller overwrite an existing fact under the same identity. We also checked that machine credentials could not read evidence or use human administration routes.

At the token boundary, tampered signatures, unsigned tokens, an asymmetric/symmetric algorithm-confusion attempt, unknown signing keys, wrong issuer, wrong audience and expired credentials were rejected in the tested scenarios. An unknown-key rejection is useful evidence, but it is not the same as testing a real provider key rotation.

For sessions, we checked cookie attributes, missing and incorrect CSRF tokens, logout, organization selection and what happens when a request presents both a session and a machine credential. The machine path did not fall back to the cookie when its credential failed.

What 95 out of 96 actually means

The remaining discrepancy was a session created for one organization and presented on a read request selecting the other. The test expected HTTP 403. The platform returned HTTP 401 because the session lookup was bound to its organization and found no valid session for the selected organization.

The request was denied and no cross-organization data was exposed. The expected status was wrong for that implementation path.

We preserved the original 95/96 result. Reviewing the assertion explained the discrepancy; it was not another execution and should not be reported as a new 96/96 run. We found no demonstrated authentication bypass in these checks, but the compatibility and token-purpose limitations below remain important.

The access-token compatibility lesson

The Cognito access tokens from our administrator-password flow identified the application client with client_id and did not contain aud. The platform verifier required aud, so those valid provider tokens returned 401. The corresponding ID tokens carried the expected audience and were accepted.

That last observation is a limitation, not a recommendation. ID tokens describe authentication to a client; they should not be substituted for API access tokens. The tested verifier checked signature, issuer, audience and lifetime, but did not enforce token purpose. A matching set of claims is not sufficient evidence that a token was intended for a particular API.

AWS documents the distinction between access-token and ID-token validation (opens in a new tab). Cognito can also issue an access-token audience for a requested resource binding; that flow was outside this test. The finding is specific to the tokens and configuration we exercised, not a claim that Cognito access tokens never contain an audience.

An integration needs an explicit token-purpose and audience/client policy, followed by negative tests for token substitution. Dropping audience checks or accepting either claim without understanding its meaning would be the wrong fix. This run does not establish complete Cognito compatibility.

Tenant isolation needs database evidence too

We checked both directions between the organizations. A foreign deployment and a nonexistent deployment produced the same not-found response. Human access still depended on membership, and a machine credential's organization came from the verified credential rather than an asserted header.

Then we read the database. Events existed only under their owning organizations; runs did not reference another organization's deployment, and events were not attached to another organization's run.

API denial and storage integrity answer different questions. A response can look correct while a write path still attaches data to the wrong parent. Checking both made the isolation result more useful, within the scenarios exercised.

Disabling a user and invalidating a JWT are different operations

After disabling a Cognito user, new sign-in was refused. An already-issued JWT still authorized a platform request while it remained unexpired and the platform membership was active. The test tokens had five-minute lifetimes.

The platform verified the JWT locally; it did not ask Cognito for the user's current state on each request. AWS explains that signature-and-expiry verification alone does not detect provider-side token revocation (opens in a new tab).

Disabling platform membership blocked a still-valid bearer in our test. Revoked and expired machine credentials were also rejected. Those are separate controls with separate responsibilities.

Browser sessions deserve their own review as well. They have a local lifetime and revocation state; the tested implementation does not automatically tie their expiry to the JWT that established them. Provider-side disable should not be treated as a complete platform logout strategy.

Logs are part of the boundary

We checked that the tested tokens and machine credentials did not appear in the application log. Verified actor and organization metadata were distinct from client-asserted identity headers, and mutation audit rows were present for human and machine actors.

That evidence does not guarantee secret-free logging under every future configuration. Proxies, debug clients and incident tooling need their own checks. The mutation-audit middleware is also best-effort: a failed audit write is logged rather than guaranteed to succeed with every request.

What remains to test

We did not exercise a real reverse proxy, trusted-forwarded-header behavior, public certificates, actual Cognito signing-key rotation, refresh-token flows, MFA or staging deployment. We also did not test provider tier limits under load.

Those are separate validation tasks. The access-token compatibility gap and token-purpose handling need implementation work and review before we can describe that integration as supported. Local validation does not certify production readiness.

What I would take into the next integration

Use real provider tokens early, and inspect their intended purpose as carefully as their signature. Treat current organization authorization as a separate boundary from identity verification. Check storage relationships alongside HTTP denials. Preserve original test outcomes when an assertion turns out to be wrong.

And test revocation as behavior, not as a checkbox. “The user is disabled” and “this API will reject every existing credential” are different claims. An integration test should tell you which one your system actually implements.

For the broader engineering context, read What Is Agent Reliability Engineering? and Gentlity’s security boundaries.