Skip to content

Threat model

What Aftershock protects, from whom, and how. Every release is reviewed against the checklist at the end.

Assets

AssetWhere it livesWhy it matters
Customer source codeGitHub; copied into a sandbox for one runConfidentiality of the customer's product
GitHub App private key, webhook secretVercel environmentFull access to every installation
Installation tokensMinted per run, one hour, contents: read, one repositoryRead access to one repository
Session secret, encryption keyVercel environmentForging sessions, decrypting stored secrets
Model keys, integration signing secretsPostgres, AES-256-GCM encryptedSpending the customer's model budget; forging alerts
API tokensPostgres, SHA-256 hash onlyWorkspace-wide API access
Incidents, proofs, evidencePostgresIntegrity of the record auditors rely on
Sandbox minutesVercel accountCost and availability

Actors

  • Workspace members: authenticated through GitHub; roles owner, admin, member.
  • Other tenants: authenticated users of a different workspace.
  • Anonymous internet: hits public pages, webhooks, badge, MCP and API endpoints.
  • Untrusted content authors: anyone who can write an issue, alert, postmortem, PR description, code comment or commit message that Aftershock reads. This includes prompt-injection attempts against the model.
  • Untrusted code: the customer's repository and its dependencies, plus the model's draft. All of it executes during runs.

Trust boundaries and controls

  1. Internet → webhooks (/api/webhooks/*)
    • HMAC signatures (or, for sources without a signing scheme, a per-workspace token) are verified in constant time before parsing.
    • Per-source rate limits.
    • A payload size cap and a ±5 minute replay window wherever the source sends a timestamp.
    • Incidents are deduplicated by source fingerprint.
  2. Internet → API, MCP and badge
    • API tokens are compared by hash; sessions are HS256 JWTs.
    • Per-caller rate limits.
    • Tokens carry scopes (read, write).
    • The badge answers 404 for private or unknown repositories, so it can't be used to find out whether a repository exists.
  3. Tenant ↔ tenant
    • Every query goes through scoped(workspaceId).
    • Resource ids from the client are always re-checked against the session's workspace.
  4. App → model
    • Untrusted text is wrapped in random-nonce fences (lib/model/fence.ts).
    • Model output is parsed as JSON and validated: one file, inside the allowed path, of the right framework, defining a test, with balanced brackets.
    • A model never decides anything is proven or safe; only test runs do.
    • Custom model endpoints must be public https:// hosts (no private, loopback or link-local addresses).
  5. App → runner (untrusted code)
    • One fresh microVM (Vercel Sandbox) or container (Docker) per run.
    • The network is allowed only while dependencies install, then set to deny-all before any test runs. A test fails the build if the network is still open after a snapshot is restored.
    • No secrets enter a run. The clone token is read-only, scoped to one repository, and not left in the working tree.
    • Run time and output size are capped. Paths written into the run are checked by safeTestPath.
  6. App → GitHub
    • Writes go only to aftershock/* branches, check runs and comments; nothing is ever pushed to a default branch.
    • Clone tokens are minted with the minimum permission.
  7. App → outbound webhooks and Slack
    • URLs are resolved before connecting, and private, loopback, link-local and metadata addresses are refused (SSRF guard).
    • Bodies are signed with Standard Webhooks (HMAC-SHA256 over id, timestamp and body).
    • Retries use back-off; a target that keeps failing is disabled automatically.
  8. Operators
    • Kill switch AFTERSHOCK_RUNS_DISABLED.
    • Secrets are redacted from structured logs.
    • Migrations run only on production builds and are backward-compatible, so rollback stays safe.

Threats and mitigations

ThreatMitigationStatus
Forged webhook creates incidents or starts runs (cost)Signatures or tokens, rate limits, ±5 minute replay window on signed timestamps, 256 KiB cap, dedupe, one 404 for unknown or disabled sourcesShipped (milestone 8)
Leaked intake token (token sources)Header delivery preferred over ?token=; rotate from Integrations; tokens are encrypted and shown onceShipped (milestone 8)
Prompt injection in issue or alert text steers the model to exfiltrate dataFencing, no tools for the model, output validation, sandbox without network or secretsShipped
Ask Aftershock is steered by incident text, or invents incidentsLocal retrieval; question and incidents fenced; JSON output; citations must be among the incidents it was given, otherwise no answer is shownShipped (milestone 13)
Public reliability report leaks detailsAggregate numbers only (no titles, files or code), 24-byte secret link compared in constant time, revocable, noindexShipped (milestone 13)
Draft test reads environment secretsNo secrets in the sandbox; network deniedShipped
Malicious repository code escapes its runFirecracker microVM isolation; network denied during tests; no shared state between runsShipped; shared dependency caches are per-repository and read-only during tests (milestone 4)
Install cache poisoning: a pull request's install scripts tamper with a cached environment that later runs reuseSnapshots are only saved from trusted default-branch commits; pull requests may restore only an identical-lockfile snapshot and never save; keys are per repository; network denied after every restoreMilestone 4
Cross-tenant data accessscoped() everywhere; integration tests for isolationShipped
Stolen API tokenHash-only storage, revocation, last-used time, read and write scopes (read-only by default in the form)Shipped
SSRF via custom model URL or outbound webhookPublic-https check; connect-time DNS check that refuses private, loopback, link-local and metadata answers; no redirects; Slack targets pinned to hooks.slack.comShipped (webhooks in milestone 9)
Notification flood or stuck retriesOutbox with claim-once delivery, 5 attempts with back-off, auto-disable after 10 consecutive failures, 30-day retentionShipped (milestone 9)
Badge or showcase leaks private dataBadge only for repositories GitHub reports as public (unknown counts as private), same 404 for private, unknown and disabled; showcase reads only the public sample workspaceBadge shipped (milestone 11); showcase milestone 14
Tampered evidenceEd25519 signatures over RFC 8785 canonical JSON, with a JWKS endpoint that lists rotated keys; the default key is derived from AUTH_SECRET with HKDFShipped (milestone 11)
Runaway cost from runsPlan quotas, kill switch, per-source rate limits, metering of every runKill switch shipped; metering in milestone 6
Account deletion leaves data behindCascading deletes, snapshot removal, 30-day uninstall purge, audit logMilestone 5
Supply-chain vulnerability in dependenciesnpm audit in CI, pinned versions, overrides for patched transitive packagesShipped

Release security checklist

For every milestone:

  • New endpoints: authentication, scoped() tenancy, rate limit, size limits, and input validation with zod or explicit checks.
  • New outbound calls: SSRF guard, timeouts, no secrets in URLs or logs.
  • New stored secrets: encrypted with lib/crypto.ts, never returned after creation.
  • New untrusted text reaching a model: fenced, with output validated.
  • New run types: network denied during tests, no secrets, metered, covered by the kill switch.
  • Migrations: additive and backward-compatible.
  • npm audit reports 0 vulnerabilities; tests cover the abuse case, not only the happy path.
Edit this page on GitHub