
Harness benchmark / 2026-11-04
GPT-5.6 Terra Custom skills supplied: None
Main prompt
Build a polished, responsive grocery list web app. Work only in /home/user/grocery-trial (or the backend project directory if that absolute path is unavailable). The sole starting file is README.md with these exact contents: # Grocery list trial Starting point: empty app. Implement according to the trial prompt. If the backend does not have that file, create it verbatim before implementing. Do not use unrelated existing project files. Requirements: add grocery items with name, quantity and category; edit and delete items; mark purchased/unpurchased; filter all/remaining/purchased; search; clear purchased; show remaining count; persist everything in localStorage. Include useful initial sample groceries only on first load. Make an attractive, accessible mobile and desktop UI with keyboard-friendly controls and empty states. Use a simple self-contained frontend stack of your choice, no external paid services or API keys. Test core flows and run build/tests as appropriate. Do not delegate to other agents or invoke additional AI. Launch an app preview as a detached process (nohup setsid), listening on 0.0.0.0. On Boat use host <port> --public, adding --localhost only if loopback-bound. Verify the public token-free HTTPS URL responds. On hosted backends use their available preview mechanism; if Boat hosting is unavailable explain this and return the native public preview if possible. Never expose unrelated services. Return a concise summary with implemented features, tests and their outcomes, preview URL, project path, and limitations. Do not stop the preview server.
Time taken
Execution time, seconds
Cost
Priced token subtotal, USD · * incomplete · — unknown
| Harness | Run | Total | Input tokens | Output tokens | Cache read | Reasoning | Original preview | Publication check | ||
|---|---|---|---|---|---|---|---|---|---|---|
| Codex | Complete | 49.9s | 59.4s | 16,820 | 5,415 | 53,737 | $0.1094 | Medium | Published | Add + persistence passed |
| OpenCode | Complete | 59.5s | 188.6s | 29,578 | 5,949 | 224,185 | $0.1734 | Not exposed | Publishing failed | Add + persistence passed |
| Hermes | Complete | 80.8s | 234.1s | 25,053 | 5,735 | 228,921 | $0.1647* | Not exposed | Published | Startup error |
| Prime Agent | Complete | 84.9s | 280.7s | 35,656 | 6,788 | 287,166 | $0.2102 | Medium | Published | Add + persistence passed |
| Pi | Complete | 94.9s | 105.2s | 57,594 | 8,294 | 460,836 | $0.3069* | Medium | Published | Add + persistence passed |
| JetBrains Junie | Complete | 100.4s | 124.8s | 49,026 | 10,532 | 296,871 | $0.2838 | Not exposed | Publishing failed | Add + persistence passed |
| Cursor Cloud | Complete | 153.4s | 216.0s | Unknown | Unknown | Unknown | Unknown | Not exposed | No public URL | Add + persistence passed |
| Factory Droid | Complete | 168.4s | 184.9s | 37,097 | 26,000 | 439,953 | $0.4742* | Not exposed | Publishing failed | Add + persistence passed |
Sorted by time taken, fastest first.


OpenCode
59.5s · $0.1734
Add + persistence passed
Agent reported 2 passing tests. Usage includes GPT-5.4 Nano: 1,037 input and 13 output tokens. Public preview needed separate recovery.

Hermes
80.8s · $0.1647*
Startup error
Agent reported static core-flow and HTTP checks. Publication check: fresh storage triggers “Cannot access save before initialization”. Original app preserved.

Prime Agent
84.9s · $0.2102
Add + persistence passed
Agent reported syntax and local checks. Usage was provisional in the first summary and has since settled.

Pi
94.9s · $0.3069*
Add + persistence passed
Agent reported 3 passing tests. Editing uses native prompt dialogs.

JetBrains Junie
100.4s · $0.2838
Add + persistence passed
Agent reported 4 passing tests. Public preview needed separate recovery.

Cursor Cloud
153.4s · Unknown
Add + persistence passed
Agent reported syntax and local HTTP checks. Used Cursor’s native cloud environment. Usage returned zeros and is treated as unknown. Public preview needed separate recovery.

Factory Droid
168.4s · $0.4742*
Add + persistence passed
Agent reported syntax and local HTTP checks. Public preview needed separate recovery.