Trust boundaries and threat model

A study moves your app's screen through several parties. This page names them, says what each receives, and lists the threats the design does and does not cover. Where a behaviour is enforced by code, the sentence links the command or document that proves it; where it depends on a provider's terms, it says so.

Who sees whatCopy Anchor Link

No command publishes a bundle, and feedback issue renders a draft and makes no GitHub API call.

Provider termsCopy Anchor Link

humanish does not control what a provider keeps. These are the providers' own statements, read on 2026-09-29, with the humanish behaviour that decides what reaches them.

• OpenAI (participant model and analyst). Each step sends the current screenshot, the persona, the mission and the actions so far; humanish analyze sends selected text and captures from a finished bundle. OpenAI's data controls page states that data sent to the API is not used to train or improve OpenAI models unless you opt in, that abuse-monitoring logs are retained for up to 30 days, that approved customers can exclude content from those logs under Zero Data Retention, and that regional processing is available on request. humanish doctor cannot tell whether your organization has those controls; check your OpenAI organization settings.
• Google (Gemini analyst). Only a study or command that selects provider: gemini sends anything to Google: the same selected text and captures the OpenAI analyst gets, with store: false. Google's Gemini API terms, read 2026-10-21, separate paid from unpaid use. On a key whose Cloud project has an active billing account, Google does not use prompts or responses to improve its products and logs them for a limited period for abuse detection. On unpaid quota, Google uses the content to improve its products and human reviewers may read it, and the terms say not to submit sensitive, confidential or personal information. In the European Economic Area, Switzerland and the United Kingdom the paid terms apply to both. humanish cannot tell which kind of key it holds; check the key's Cloud project before analyzing private evidence on Gemini.
• E2B (hosted desktop). Each participant gets its own sandbox, created with a timeout and killed by id when the participant ends; humanish never pauses a sandbox, and E2B's docs say only paused sandboxes are kept. E2B's privacy policy governs what it retains about your account and traffic; it does not state a retention period for a killed sandbox's filesystem, so treat anything placed in a desktop as visible to E2B for as long as their policy allows.
• Codex or Claude Code (actor.type: local-agent). The same screenshots and prompts flow through your signed-in agent under that plan's terms. The bundle records that the usage is unpriced and nothing else about the account.
• humanish telemetry. TELEMETRY.md lists every field sent and every field excluded; npx humanish telemetry disable or DO_NOT_TRACK=1 stops it.
• npm registry (update check). At most once a day, a command run at a terminal asks registry.npmjs.org for humanish's latest version with a plain GET that carries no machine id, version, command or path. TELEMETRY.md describes it; HUMANISH_NO_UPDATE_CHECK=1 or DO_NOT_TRACK=1 stops it.

Store credentialsCopy Anchor Link

npx humanish keys set e2b and npx humanish keys set openai store the two keys, and npx humanish keys lists each provider key with the source that supplies it, never its value. E2B_API_KEY and OPENAI_API_KEY in the environment work too, and --dotenv <path> loads an ignored env file for one command. Add two keys shows the setup. npx humanish keys set anthropic stores ANTHROPIC_API_KEY for participants on Claude (actor.type: computer-use with a Claude model); a signed-in Claude Code local-agent participant runs on its own login and never receives it. npx humanish keys set gemini stores GEMINI_API_KEY for Gemini analysis; GOOGLE_API_KEY is read only when GEMINI_API_KEY is unset, and discovery never fills one of the two while the other is already in your environment, as with GH_TOKEN and GITHUB_TOKEN. Keep env files and credentials out of version control. Run bundles record provisioned variable names and never their values, and persisted text has known secret values scrubbed.

humanish keys set <vendor> writes to $XDG_CONFIG_HOME/humanish/keys.env, by default ~/.config/humanish/keys.env. The directory is created with mode 0700 and the file with 0600, opened without following symlinks, and only allowlisted provider names are accepted. The values are plain text; file permissions are the whole protection, and nothing is encrypted at rest. keys list prints names only, keys unset <vendor> removes an entry, and the E2B CLI's own ~/.e2b/config.json is read when present. To rotate a key, revoke it at the provider, then run keys set again; every persisted humanish text has known secret values scrubbed, so a rotated key does not need a bundle sweep.

The desktop receives actions and never the computer-use model's API key. In a local run that key stays on the host. A remote run, the default for a computer-use study on e2b-desktop or modal-desktop with a public app-url subject and for a live terminal study, runs the study's own process in a runner sandbox in your own E2B or Modal account. The run uses only the keys the study's plan reads: the model key, the analysis key and the provider's own key or token pair. The command names them before it creates the runner, and the run's JSON result lists them under remote.keys. On E2B, each key that humanish sends as a request header to a fixed host stays in E2B's egress proxy: the OpenAI key for api.openai.com, the Anthropic key for api.anthropic.com (as its x-api-key header), the Gemini key for generativelanguage.googleapis.com and the E2B key for the E2B API. A study with an email catch you host adds the catch token its authTokenEnv names, held as an Authorization header for the catch's host when catchBaseUrl is https:// at a domain name on the default port; otherwise the runner gets its value. E2B adds the key to each request the runner sends to that host, and the runner's run gets a placeholder in its place. This keeps the keys out of the runner's environment, files and logs. It does not keep them from code running in the runner: E2B's sandbox info endpoint returns a sandbox's header transforms as they were set, and the runner can call it with the E2B key the proxy adds. On Modal, which has no such proxy, the keys live in the runner's process environment for the run's length, where anyone with access to that provider account can read them, and are never written to a file there. A terminal study's runtime key, CODEX_API_KEY or OPENAI_API_KEY, enters the runner's environment on E2B too: the run in the runner puts that value into the shell sandbox's own egress rule, or into the Codex command under openai-env, and a placeholder there would reach OpenAI and be refused. The runner installs this humanish from a tarball packed on your machine, and npm resolves its dependencies at install time without a lockfile. --local or execution.remote: false keeps the run, and its keys, on this machine; see keep a long study going. Terminal studies run Codex inside the sandbox. Their default, execution.runtimeAuth: openai-egress, keeps the raw OpenAI key outside the sandbox: E2B's proxy adds it to requests for api.openai.com, and Codex gets a placeholder. Sandbox processes can still make paid requests through that proxy, so the default imposes no provider spending limit. execution.runtimeAuth: openai-env instead supplies the key to the Codex command, where the participant and its child processes can read and use it. The terminal authentication setup and limits describe both. Any credentials you declare for the subject app also enter its sandbox.

A URL in a study (subject.appUrl, subject.serve.url, subject.product.publicSurfaces, or a participant's target or entry) may not carry a credential, because the run records it and the participant's address bar shows it in every screenshot. The study is refused when it is parsed if such a URL, or a URL nested in it, has a user name or password, or if its path, query or fragment holds a credential the share gate below flags, such as token=, access_token=, x-vercel-protection-bypass= or _vercel_share= with a value of 16 characters or more. A URL with a tab, line break or other control character, which the URL parser drops, is refused too. Ordinary parameters such as ?page=2 are accepted, and so is an E2B app URL in a parameter, such as a sign-in return address: its host names a sandbox, as the target's own host may, and the share gate keeps it out of a shared run. A library caller's config skips the parser, so the computer-use, shared-world and terminal planners refuse the same URLs, and the scripted route drops a loopback URL's user info, query and fragment before it records it.

Knowing that cleanup happenedCopy Anchor Link

Three records say what happened to each desktop, and a command checks them against E2B:

• run.json lists every sandbox the run created under providerResources, with status killed, running (kept on purpose for debugging) or unknown (its release was not confirmed) and a cleanup.reason that gives the release warning. Each names its sandbox as [redacted-sandbox-id] with an idDigest: from 0.110 the raw ids are only in the run's sandbox-receipts.ndjson, and run records, results and command output name a sandbox by digest. Runs recorded earlier keep raw ids in run.json.
• status.json adds a sandbox-cleanup entry to outcome.execution.warnings for each sandbox whose release was not confirmed, naming the participant (or subject) and the humanish reclaim --run <id> command. A local VM has no receipt, so its entry names the container and the docker rm --force --volumes <container> command instead, and automatic analysis records skipped with AUTOMATIC_ANALYSIS_CLEANUP_UNCONFIRMED. On computer-use, shared-world and scripted runs the run stays ok, because the evidence stands and the sandbox has its create-time timeout; a terminal run fails instead.
• Every sandbox carries the owner tags tool: humanish, runId and runKey in its E2B metadata, and before the run's first create, sandbox-receipts.ndjson records those tags in an owner line that names no sandbox. runKey is a digest of the run directory's inode and creation time, so two projects that reuse a run id never match each other's sandboxes.
• humanish reclaim --run <id> kills the sandboxes an interrupted run journaled in sandbox-receipts.ndjson, and any sandbox E2B lists with all three of the run's tags, which covers one whose id never reached a receipt. It searches with the tags the owner line recorded as well as the directory's own, so a copied or moved run directory still finds its sandboxes. An owner line counts only with exactly tool: humanish, the run's own id and a 16-hex runKey. The list is filtered by E2B and each result is checked against every tag again, so it never returns another run's or another tool's sandbox. Kills run concurrently. reclaim-receipt.json is written before the first kill, with each sandbox pending, and again as each answers, so an exit at any point leaves a record of every sandbox not yet confirmed gone. Each outcome is killed, already-gone, kill-failed, unsupported-provider or pending, names the sandbox by sandboxIdDigest, and says where reclaim found it (receipt, create or tag). The result's state is clean only when every sandbox found is gone, the tag search finished and the run recorded its owner tags; it is unconfirmed when a kill did not confirm, and unknown when E2B could not be searched or the run has no owner line. A run from humanish 0.110.0 or earlier has none, so it is never more than unknown. A dry run creates no sandboxes: when its run.json and status.json record it as one and it journaled nothing, reclaim reports it clean with reason: dry-run, exits 0 and does not contact E2B. A scripted run against an app-url subject drives a local browser and creates no sandbox; its status.json records sandboxes: none at start, and reclaim reports it clean with reason: no-sandbox the same way when its run.json agrees: the scripted route wrote it, its subject is no clone, it lists no provider resource or desktop minutes, and its outcome marks no cleanup unconfirmed. A missing or contradicting run.json, a journal line or an earlier reclaim receipt that names a sandbox sends the run through the normal E2B search. already-gone means Sandbox.kill(id) answered false or threw a SandboxNotFoundError; any other error is kill-failed, whatever its message says. Each receipt and owner line names its provider (e2b or modal; older receipts have none and count as e2b). Reclaim kills and searches through that provider, so a run on Modal needs MODAL_TOKEN_ID and MODAL_TOKEN_SECRET and not the E2B SDK; on Modal, already-gone means the sandbox had finished or Modal answered NOT_FOUND for its id. A receipt from a provider this version cannot reach is reported as unsupported-provider and nothing is attempted for it. humanish reclaim --preflight does the same for the sandboxes interrupted humanish study check probes journaled under .humanish/preflight/. It removes a journal only once its sandboxes are confirmed gone; a journal from before owner lines stays, whatever time has passed, and the warning names the directory to delete after checking the E2B dashboard. It leaves a journal whose probe may still be running, including one opened on another host or in another pid namespace, until the probe marks it abandoned or its lease elapses, and the result's warnings say why. With E2B_DEBUG=true both forms refuse with HUMANISH_RECLAIM_E2B_DEBUG and kill nothing, because the SDK's debug mode answers Sandbox.kill with true without contacting E2B.
• humanish reclaim --check --run <id> asks E2B whether each journaled sandbox still exists, by exact id, and lists the run's tagged sandboxes, then reports clean, running or unknown. It kills nothing and writes nothing, and exits 0 only for clean. humanish verify still reads a cleanup.json an earlier version wrote, and a failed one still fails verify.
• When Ctrl-C (or SIGTERM or SIGHUP) stops a live run, humanish prints that it is stopping, refuses any further sandbox create, kills each sandbox whose id arrives during the stop, reclaims as above, then rewrites any run file that quotes a raw sandbox id, such as a desktop startup error, with the marker and digest, all within 10 seconds. A desktop sandbox's receipt is written when the SDK reports its id, before desktop startup, so verify and this rewrite know an id that a startup error quotes. A second signal prints the humanish reclaim command for the run and exits at once.

The create-time timeout on every sandbox is the backstop when a kill cannot be confirmed: E2B stops the sandbox, and its metering, when that timeout elapses.

A preflight sandbox also stops on its own. A public-preview probe gets each target's readiness budget plus five minutes. A sandbox-loopback probe gets the longest its clone and serve steps can take with your study's serve and seed-step budgets, including the one retry of the runtime and install steps, plus five minutes, so a preflight does not time out where the run would finish provisioning. A declared execution.desktop.sandboxTimeoutMs, or E2B's 60-minute maximum, caps both. Without a declared timeout, a clone probe gets 13 minutes (a clone served as-is), 53 minutes for an npm app that installs and builds, and the 60-minute cap once pnpm or yarn adds its step, where earlier versions gave 10; the receipt is what lets you stop it sooner.

Recover after an interruptionCopy Anchor Link

After a crash, run npx humanish runs, then npx humanish reclaim --run YOUR_RUN_ID, which kills what the run journaled and anything E2B lists with the run's tags. npx humanish reclaim --check --run YOUR_RUN_ID asks the same questions and kills nothing. For an interrupted humanish study check probe, run npx humanish reclaim --preflight --json. Knowing that cleanup happened explains each receipt and how long a preflight sandbox lives.

ReproducibilityCopy Anchor Link

A rerun is a new sample, not a replay. What is pinned: the model (actor.model, default gpt-6.1-sol for computer use and gpt-5.6-sol for the terminal route), the reasoning effort (reasoningEffort per actor or per participant, recorded in the bundle), the humanish version, and the rate sheet date behind every cost figure. A cloned repository's commit is recorded, but a study cannot choose it: the clone takes the head of the default branch. What is not pinned: sampling (humanish sets no temperature, so the provider's default applies), the desktop's timing, and the page under test. Transport retries are automatic and bounded: a model request is retried up to three times on 408, 409, 429 and 5xx responses, honouring Retry-After up to a cap, and a sandbox create is retried once on a transient E2B error. Every retry stays inside the run's spend cap. The only variance data published so far is the persona-contrast study on the home page: five or six runs per cell, and four runs on the control app. Keyboard-first participants reported drawDB's database modal 5 of 5 times and TodoMVC's double-click rename 6 of 6 times, and were blocked by them 3 of 5 and 2 of 6 times. Mouse newcomers reported them 0 of 5 and 0 of 6 times. Every participant on the control app reached the goal, 12 of 12.

Support and compatibilityCopy Anchor Link

The CLI requires Node 22.19 or newer (engines in package.json). The project has one maintainer, no support contract and no SLA; bugs go to GitHub issues and vulnerabilities follow SECURITY.md. The compatibility policy says what a release may change: a deprecated name lives at least 30 days and through one minor release, and breaking changes ship together with a migration table. Pin the version in package.json and read the release notes before upgrading. Every bundle records the version that produced it, and published studies name theirs.

What the design protectsCopy Anchor Link

• Keys. The computer-use model key stays off the desktop, which receives actions. A local run keeps it on the host. A remote run names the keys its plan reads first. On E2B they stay in E2B's egress proxy and the runner gets placeholders; on Modal they enter the runner's environment. A terminal study's runtime key enters the runner's environment on both. A terminal study's runtime key stays on E2B's proxy unless the study declares openai-env. Provisioned variable names are recorded, values are not, and known secret values are scrubbed from persisted text. A study URL with a credential in it is refused at parse.
• Evidence at rest. Bundles land in gitignored .humanish/. Screenshots are full fidelity by default; policies.redactScreenshots: true blurs frames at capture, and humanish export --format bundle --redact-screenshots produces a separately verified redacted copy.
• Sharing. humanish verify grades a run folder share_ready, local_only or blocked and fails closed: a run that cannot pass every gate is never share_ready. The gate matches secret and path shapes as written and inside escapes, percent-encoding, base64 and hex: the key formats of OpenAI, Anthropic, E2B, GitHub, GitLab, npm, Vercel, AWS, Google, Stripe, Slack, SendGrid and Hugging Face; JWTs; bearer and basic Authorization values; private key headers; the password in any URL's user info; credential query and fragment parameters, including Vercel's protection bypass and share parameters; an upper-case variable named like *_TOKEN, *_SECRET, *_PASSWORD, *_API_KEY or *_ACCESS_KEY set to a value of 16 characters or more with a digit and a letter in it, quoted or not; an AWS secret access key next to its name; E2B URLs; and local paths on Linux, macOS and Windows. A file it cannot read as text, or a screenshot that is not a blurred thumbnail, keeps the run local_only. A share_ready folder still holds sandbox-receipts.ndjson, with the raw sandbox ids reclaim needs, and status.json, with the recording process id. No Observer server serves either file in any mode, observe --all --safe included, and humanish export --format bundle leaves both out of its copy, so share that copy or the safe library.
• Spend. Per-participant and per-study caps stop runs on the estimate; desktops stop at their timeout; analysis is refused above its admission limit. See what a study costs.
• Resources. Every desktop is journaled by id and tagged with its run; humanish reclaim --run <id> stops the ones a crashed run left, and humanish reclaim --check reports any still running.
• The repository. Every release runs a public-surface scan that fails on secret shapes, private names and unapproved binaries, and the package is MIT under danielgwilson/humanish.

What it does not protectCopy Anchor Link

• Free-form sensitive content in pixels or text. verify finds patterns, not meaning. A secret with no known shape passes it: a short password, or a token that has no known prefix and is not next to a credential name. A screenshot can contain anything your app displayed; review it before sharing.
• Provider retention and training use. Screenshots and prompts reach OpenAI, E2B or your agent's provider under their terms. Retention, region and training use are outside humanish's control; provider terms above lists what each states.
• Egress from the hosted desktop. The desktop has internet access; the participant can reach any site the mission or the page leads it to.
• Billing limits. Caps act on estimates after usage is reported. Your provider dashboard is the only hard limit.
• The safety of the released code. Every npm release from 0.15.0 on carries two registry attestations, and npm audit signatures verifies them. The npm publish attestation covers the package name, version and registry. The SLSA provenance v1 attestation binds the tarball's sha512 to a run of .github/workflows/publish.yml in danielgwilson/humanish at the release tag, on a GitHub-hosted runner. Together they show which commit and workflow built the tarball; they say nothing about whether that code is safe.

Threats worth planning forCopy Anchor Link

1. Prompt injection from the page under test. The participant reads the screen and follows it. A hostile page can steer it. Nothing in humanish detects injection. Do not place credentials in a desktop that will visit pages you do not control; on terminal studies keep the default openai-egress. What a steered participant can reach depends on which model decides its actions:

   • computer-use and openai-computer-use: the desktop. The model key stays off the desktop (on the host, or in the runner sandbox of a remote run), the desktop holds no credentials by default, a timeout and a spend cap bound the run, and the desktop is destroyed afterwards.
   • Claude Code (localAgent: claude): a process on your machine, started with --restricted, --tools Read, --strict-mcp-config, --permission-mode dontAsk and --no-session-persistence in a scratch folder that holds only its screenshots. Read is its only tool and cannot open a file outside that folder. Your user, project and local settings files, their hooks, your MCP servers and your CLAUDE.md do not load; managed (administrator) settings still apply. Its environment keeps PATH, HOME, USER, LOGNAME, locale, CLAUDE_CONFIG_DIR and proxy variables, and no provider key or --dotenv name. humanish reads every message it streams and stops the run with HUMANISH_CLAUDE_PARTICIPANT_TOOL_REFUSED on any other tool call. This needs Claude Code 2.1.248 or newer; local agents has the details.
   • Codex (localAgent: codex): an app-server process on your machine, in a scratch folder, with a read-only sandbox, approval policy never, web search, skills, agents and configured MCP servers turned off, and one humanish desktop tool; any other tool item stops the request. It keeps your Codex home and configuration, and its environment is the one humanish runs with, which can include provider keys.

2. Sensitive data captured into evidence. Use synthetic accounts; enable redactScreenshots for studies meant to be shared as-is; keep private studies in ignored files; verify and read before sharing.

3. A leaked key. Keys come from the environment and --dotenv first. A name still unset is then filled from the project's .humanish/local/provider.env, the E2B CLI's ~/.e2b/config.json, gh auth token (as GH_TOKEN) and the 0600 store, in that order, unless HUMANISH_STRICT_KEYS=1 is set. The study file never supplies one. In a checkout you did not write, read its .humanish/local/provider.env first: it chooses the account when your environment names none. The OpenAI, E2B and Codex keys and the declared subject.env values a run reads are scrubbed from persisted text by value; the Gemini key reaches only the analysis request's x-goog-api-key header, and an analysis keeps no provider error text that could echo it; any other secret, GH_TOKEN included, is caught only by its shape. Rotate at the provider if a bundle ever shows one.

4. Runaway spend. Set both caps and a timeout; they stop on the estimate, so budget the panel with the worksheet and watch the first live run.

5. An orphaned desktop. After a crash, runs, then reclaim --run <id>. If reclaim says unconfirmed or unknown, run it again once E2B is reachable, or check the E2B console for sandboxes tagged with the run id.

6. A study that is not what it claims. Every number in a bundle carries its denominator and its rates' date; a cloned subject's commit is recorded; public shared-world targets carry an owner attestation. Studies published on this site name the humanish version they ran on.

Reporting a vulnerabilityCopy Anchor Link

Report it privately through GitHub private vulnerability reporting, as SECURITY.md describes. Include the version, the command, safe reproduction steps and a redacted evidence path.