Study your own app

You have a build nobody outside the team has used, and you want to know where a newcomer gets stuck. Give one participant one concrete job, let them try it, and keep what came back for the next change.

Use an app you own or have permission to study, with synthetic accounts and data. All three setups below are route: computer-use studies; they differ in where the app runs. How a study works explains the fields they use.

• A preview or staging URL: a hosted desktop opens it, below. This is the common case.
• A repository, not yet deployed: a clone subject builds it inside each desktop.
• Running on your machine, and you use Codex: a local browser study needs no E2B or OpenAI key.

Create a study for your preview URLCopy Anchor Link

Finish the install and keys, including @e2b/desktop. Then create .humanish/studies/your-app.yaml. That directory is ignored by git, which makes it the right place for preview URLs and experiments. Replace the URL and the task with yours:

schema: humanish.study.v3
id: your-app
title: First project, a newcomer
route: computer-use
mode: live
subject:
  source: app-url
  appUrl: https://preview.example.com/
actor:
  type: openai-computer-use
  persona: synthetic-new-user
  maxOutputTokens: 8192
  mission: >-
    You are trying this project-management app for the first time.
    Create a project named Weekend plan, then find it again and
    rename it Saturday plan. Stop when you finish or get stuck.
    Say what happened and what confused you.
participants: 1
caps:
  maxTotalUsd: 4
  maxUsd: 2
execution:
  target: e2b-desktop
  timeoutMs: 600000
policies:
  allowPublicTargets: true
defaults:
  open: true

Write the task the way you would brief a person. "Create a project and find it again" leaves room for the participant to look around. "Click the plus icon in the top right" hands them the answer and hides the navigation problem you wanted to see.

The two caps are checked on every run and do not carry across runs: maxTotalUsd is the study's model budget, maxUsd the backstop per participant. Desktop time is billed on top, and a supported live run also requests analysis when it finishes, which is refused when its estimate exceeds $10 (a Codex participant's run is analyzed by the same Codex account instead). Set a study budget has the exact rules and the opt-outs.

To use your signed-in Codex or Claude Code as the participant instead of the OpenAI key, swap the actor for the local-agent fragment; a Codex participant's run is analyzed by the same Codex account, and a Claude Code participant's needs an OpenAI key for its analysis.

One public URL takes one participant per run. For a second observation, reset the test data or use another test account and run the study again, and keep both run ids so the first is not lost behind latest. For several newcomers at once, each on their own copy of the app, use a clone subject, below.

Check, then runCopy Anchor Link

These two commands create no desktop and call no model:

npx humanish study show .humanish/studies/your-app.yaml
npx humanish doctor --study .humanish/studies/your-app.yaml

study show prints the resolved study, including the persona text the participant will get. doctor tells the participant's authentication apart from desktop access and analysis; it does not check keys against the provider or confirm quota. With --dotenv <path>, pass it to both doctor and the run command.

Next, check that a hosted desktop can reach the preview. This needs E2B_API_KEY, creates one desktop for the probe and shuts it down afterwards, and makes no model call:

npx humanish study check .humanish/studies/your-app.yaml --reachability public-preview

The probe only confirms reachability from E2B; it does not log in through preview protection. Make the deployment reachable without a credential in the URL: a subject.appUrl with a user name, a password, or a bypass or share token in its query is refused when the study is parsed, because the participant's address bar would show it in every screenshot. Then run:

npx humanish run .humanish/studies/your-app.yaml
npx humanish observe --run latest --open

To watch the desktop during the study, use watch in place of run. For an agent or a CI job, add --json. Read results before changing the app or adding participants.

Build the app inside the sandboxCopy Anchor Link

A deployment is optional. The starter humanish/studies/cua-browser.yaml shows the pattern: clone a repository, install, build and start the app inside each desktop. Replace the subject block with your repository and its real commands:

subject:
  source: clone
  repos: [your-org/your-app]
  serve:
    install: npm ci
    build: npm run build
    start: npm run start -- --hostname 127.0.0.1 --port 3000
    url: http://127.0.0.1:3000/

The start command shown fits a Next.js app; use yours, with a readiness URL on the port it serves. Remove policies.allowPublicTargets: the app is served inside the desktop, and a clone study that keeps it is refused at parse when participants is above 1. For two participants, set participants to 2: each gets its own clone and desktop under the same $4 study budget. A shared external database still needs separate test accounts or data.

If you edit the scaffolded cua-browser.yaml directly, change mode from dry-run to live and add a caps budget; a dry run writes a synthetic bundle and does not study the app.

For uncommitted code, subject.source: local-tree packages a selected working tree with a content pin. Environment allowlists, state seeding and the other subject sources are in the computer-use reference and the study file reference.

Study your running local appCopy Anchor Link

A local browser study drives a browser on your own machine through your signed-in Codex account, so it needs neither E2B nor an OpenAI API key. It needs a supported Codex CLI with a file-backed ChatGPT login, plus either Linux x64 with rootful Docker Engine, KVM and TUN, or an M3-or-newer Mac with native ARM64 Node and Lima 2.2 or later. Docker Desktop is not supported. Check the account profile and platform setup first.

Start your app, then replace the URL and the task:

npm install --save-dev humanish
npx humanish init --yes \
  --local-browser http://127.0.0.1:3000 \
  --local-mission "Create a project named Weekend plan, find it again, and rename it Saturday plan. Stop when you finish or get stuck, and explain anything confusing."
npx humanish doctor --study local-browser
npx humanish run local-browser

If you already ran init, edit subject.appUrl and the actor's mission in humanish/studies/local-browser.yaml instead; init keeps existing files and warns that these options were skipped. Use an explicit loopback port above 1023.

The starter gives one participant two minutes. For a longer task, raise execution.timeoutMs in the study before running. doctor checks setup without launching a participant or downloading the runtime; the first live run prepares the pinned browser image and later runs reuse it. The participant and the separate analyst both use your Codex plan and remote inference, so the dollar cost is unknown and never reported as zero. A local browser study does not fall back to a hosted desktop if setup fails.

npx humanish observe --run latest --open
# In another terminal while Observer is open:
npx humanish verify --run latest --json

For signup flows, point your test app's outgoing mail at the local captured inbox. It accepts HTTP mail or optional SMTP and never receives mail sent to a real mailbox.

Test phone, tablet and desktop in one studyCopy Anchor Link

To see how one flow works at several screen sizes, give the study one participant per size. They run at the same time, the Observer shows them side by side, and the findings compare them. Separate studies per size cannot be compared that way, and a device frame drawn inside the page still renders at desktop width.

schema: humanish.study.v3
id: devices
title: First project on phone, tablet and desktop
route: computer-use
mode: live
subject:
  source: app-url
  appUrl: "http://127.0.0.1:3000"
actor:
  type: local-agent
  localAgent: codex
  persona: synthetic-new-user
  mission: >-
    Create a project named Weekend plan, then find it again. Say what was hard to
    read or reach on your screen.
participants:
  - id: phone
    device: mobile
  - id: tablet
    device: tablet
  - id: desktop
    device: desktop
execution:
  target: local
  timeoutMs: 600000
  desktop:
    fidelity:
      mobileEmulation: true

device sizes each participant's screen and browser window: mobile is 414×896, tablet 820×1180 and desktop 1440×950. Device presets lists the rest. fidelity.mobileEmulation gives phone participants a mobile viewport, pixel ratio, touch input and a mobile user agent in Chromium; tablet and desktop participants are unchanged.

For a preview URL on hosted desktops, set execution.target: e2b-desktop, put the URL in subject.appUrl, and add policies.allowPublicTargets: true. Participants on one public app share its data, so each entry also needs a target: the same URL when each participant creates its own account, or a separate test account or seeded URL per participant.

Give the participant a backgroundCopy Anchor Link

humanish init writes editable profiles under humanish/personas/. Set the actor's persona to a profile's id. A profile you keep only on your machine can live in the ignored .humanish/local/personas/; a committed profile with the same id wins. A profile can be a few sentences or a longer account of who this person is:

schema: humanish.persona.v1
id: volunteer-organizer
name: Morgan
summary: A volunteer organizer trying an unfamiliar planning app.
background: |
  Morgan coordinates twelve volunteers using spreadsheets and email.
  They know those tools well but have never used this product.
  They spend Sunday evening reconciling replies and worry about
  making the roster public or losing their own copy.

The background is who they are; the mission is what they are doing today. Give enough context for the participant to make choices, and keep the expected defect and the hidden completion rule out of both.

Backgrounds keep their paragraphs up to 32 KiB of UTF-8; a larger or invalid background stops the run before it starts. summary is limited to 280 characters and is shortened with a warning. Missing files and unsupported fields warn; backstory is not an alias. Trait directives such as keyboard_first (a preference that allows pointer fallback) and keyboard_only (a condition that does not) are instructions to the participant, not enforced restrictions, and do not certify accessibility.

Inspect the resolved persona text before spending on a study:

npx humanish study show .humanish/studies/your-app.yaml --json

The personas field shows the authored context, separate from the task, route instructions and runtime additions such as inbox access. The bundle records what was supplied; it cannot prove the participant followed it. Keep profiles synthetic or de-identified and review them before committing or sharing; pattern redaction cannot certify that a personal story is safe to publish.

Make a useful comparisonCopy Anchor Link

Keep the task, personas, app state and model settings the same between runs, and record the app revision before and after. Look for the specific friction to disappear, then check whether something new replaced it.

Report denominators: "2 of 2 completed; 1 reported friction." A run finishing, a participant saying it succeeded, and independent task evidence are three different observations; Read results explains how to read them together. Two participants can show repeatable friction; they cannot establish population-wide usability.

Move a study into committed humanish/studies/ only when its URLs, personas, tasks and examples are safe to publish. Generated evidence stays in .humanish/.