
GPT-5.6 Sol
140.7s · $0.6407
Add + persistence passed
Agent reported 5 passing Node tests and a Chrome interaction check. Google Fonts has a system-font fallback.
Multiple models Custom skills supplied: None
Build a polished, responsive grocery list web app from scratch in a new directory named grocery-app. Starting application files: none. Before building, inspect the environment for automatically cloned repositories or inherited app files; do not use their code. Report any inherited files and provider/environment differences. Use only your own new grocery-app directory. Do not consult other trial results or delegate to other agents. Requirements: add groceries with name, quantity and category; edit and delete items; mark purchased/unpurchased; filter All/To buy/Purchased; show progress and remaining count; clear purchased items; persist the list in localStorage across reloads. Include a useful initial sample list only on first visit, clear empty states, input validation, accessible labels, keyboard support, and a thoughtful mobile/desktop visual design. No account or external paid services. Stack is your choice. Test the main interactions and persistence, run an appropriate build/check, and report exact checks and any failures honestly. Provide a README with run instructions. Make a working public preview following your execution-specific Artos preview instructions. On Boat expose only this app with host <port> --public (use --localhost only for loopback binding), verify the public URL responds, and report a token-free clickable URL. Hosted providers should use their native preview facilities if Boat hosting is unavailable, and report that difference. Keep the preview running according to your execution-specific service instructions. Finish with: app directory, starting-files inspection, implementation summary, test results, preview URL and HTTP verification, limitations, and any available native timing/usage information. Do not change the selected model or reasoning configuration.
Execution time, seconds
Priced token subtotal, USD · * incomplete · — unknown
| Model | Harness | Run | Total | Input tokens | Output tokens | Cache read | Reasoning | Original preview | Publication check | ||
|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-5.6 Sol | Codex | Complete | 140.7s | 146.9s | 44,379 | 13,368 | 489,581 | $0.6407 | Medium | Published | Add + persistence passed |
| DeepSeek V4.1 Flash | DeepSeek Harness | Complete | 152.4s | 238.9s | 25,545 | 45,778 | 1,345,024 | $0.0482 | Medium | Published | Add + persistence passed |
| GPT-6.1 Sol | Codex | Complete | 244.4s | 253.4s | 42,051 | 13,120 | 464,485 | $0.2618 | Medium | Published | Add + persistence passed |
| Muse Wonder | Muse Code | Complete | 256.9s | 292.6s | 186,812 | 31,768 | 2,312,290 | $1.2537 | Medium | Published | Add + persistence passed |
| GPT-6 Astra | Codex | Complete | 275.7s | 282.5s | 43,304 | 11,981 | 409,904 | $1.4420 | Medium | Published | Add + persistence passed |
| Claude Opus 5.5 | Claude Code | Complete | 451.1s | 462.0s | 2,022 | 36,040 | 1,349,592 | $1.3333 | Medium | Published | Add + persistence passed |
| Devin Normal | Devin Cloud | Complete | 451.6s | 493.3s | Unknown | Unknown | Unknown | Unknown | Normal mode | Published | Add + persistence passed |
| Claude Fable 5.1 | Claude Code | Complete | 543.0s | 551.7s | 2,675 | 30,288 | 993,243 | $2.5855 | Medium | Published | Add + persistence passed |
| Gemini 4 Argon | Gemini CLI | Complete | 740.2s | 771.3s | 159,597 | 36,433 | 2,056,456 | $0.8892 | Native default | Published | Add + persistence passed |
| Kimi K3.1 Pro | Kimi Code | Complete | 909.7s | 936.1s | 95,849 | 16,561 | 983,808 | $0.5541 | Low | Published | Add + persistence passed |
| Grok 4.7 | Cursor Cloud | Complete | 1261.7s | 1273.5s | Unknown | Unknown | Unknown | Unknown | Native default | Published | Add + persistence passed |
| GLM 5.3 | ZCode | Complete | 1795.2s | 1849.5s | 388,479 | 186,693 | 10,088,064 | $1.4699 | Native default | Published | Add + persistence passed |
Sorted by time taken, fastest first.

140.7s · $0.6407
Add + persistence passed
Agent reported 5 passing Node tests and a Chrome interaction check. Google Fonts has a system-font fallback.

152.4s · $0.0482
Add + persistence passed
Agent reported 37 passing unit/DOM tests, syntax checks and a successful build. The DOM tests used happy-dom, not a real browser.

244.4s · $0.2618
Add + persistence passed
Agent reported 27 passing browser checks and syntax checks.

256.9s · $1.2537
Add + persistence passed
Agent reported 19 passing DOM-harness tests and syntax checks; no real-browser test. Usage includes Spark helpers.

275.7s · $1.4420
Add + persistence passed
Agent reported 6 passing Chromium tests and syntax checks.

451.1s · $1.3333
Add + persistence passed
Agent reported 8 passing unit tests, 12 passing browser tests and a successful typecheck/build. Recorded usage includes Claude Haiku 5.5.

451.6s · Unknown
Add + persistence passed
Agent reported 9 passing unit tests, build/lint and 15 passing browser tests. Devin uses its own VM and agent mode. Token usage is unavailable.

543.0s · $2.5855
Add + persistence passed
Agent reported 6 passing Playwright tests and a successful typecheck/build. Recorded usage also includes claude-haiku-5-5.

740.2s · $0.8892
Add + persistence passed
Agent reported 6 passing unit/integration tests and a successful typecheck/build. Native reasoning default.

909.7s · $0.5541
Add + persistence passed
Agent reported 21 passing Chrome checks and syntax checks. Kimi used low effort. Optional Google Fonts has a system fallback.

1261.7s · Unknown
Add + persistence passed
Agent reported 10 passing unit tests, typecheck/build and browser checks. Cursor native token cost is unavailable.

1795.2s · $1.4699
Add + persistence passed
Agent reported completed app checks. Native reasoning default; generic Artos effort does not control ZCode. The agent used a visual-judge subagent despite the task's no-delegation instruction.