# Design Assay Bench v0 (draft)

An open evaluation of how well models produce production-grade design. The bench reuses the fitness functions from `02-FITNESS-FUNCTIONS.md`, so a model's score means the same thing as an assay score.

## Why

Model progress follows what can be measured. Code has strong benchmarks. Design has screenshots and opinions. The bench gives labs and builders a reproducible way to measure design ability.

## Task suites

| Suite | Input | Output | Count (v0) |
|---|---|---|---|
| `S1 Screen` | Target profile plus genome | One production screen in Next.js and Tailwind | 30 |
| `S2 Extend` | Existing design system plus a new component brief | Component that matches the system | 30 |
| `S3 Repair` | A screen with seeded defects | Fixed screen | 30 |
| `S4 System` | Brand brief | Token set plus 5 core components | 10 |

## Scoring

- Every output runs through the same gates, measured and judged functions.
- Bench score per suite is the mean composite across tasks.
- Headline score is the mean across suites, with gate pass rate reported alongside.

## Integrity

- A public dev set and a held-out test set. The test set is never published.
- Seeded defects in `S3` are rotated each release.
- Judge models are calibrated per release and listed in the results.
- Every submission includes model name, version, date, prompt and harness version.

## Release rules

- The bench opens publicly only after at least two published assays have validated the fitness functions.
- Results page lists every model run, including poor results. No cherry-picking.

## Harness

- Runs locally or in CI with Playwright and axe-core.
- One command per suite: `assay bench run --suite S1 --model <id>`.
- Outputs a run record in the same format as `01-PROTOCOL.md`.