# Render gate: exactness and performance, build against build

The render gate measures a **candidate** viewer build against a **baseline** build
on the machine it runs on, and fails when the candidate renders differently than
its commit says it should, or renders slower. It is an opt-in tool for changes to
the render path (shaders, materials, the frame loop, view-dependent CPU logic),
not a CI gate: CI has no GPU, and the answer it gives is only meaningful on real
hardware.

```bash
make render-gate BASE=origin/main CAND=HEAD                   # exactness, IDENTICAL class
make render-gate BASE=HEAD~1 CAND=HEAD CLASS=ULP              # a math-moving commit
make render-gate BASE=HEAD~1 CAND=HEAD INTENDED=env_splats    # declared intended change
make render-gate BASE=origin/main CAND=HEAD SUITE=perf        # performance only
make render-gate BASE=HEAD CAND=WORKTREE ONLY=mixed           # uncommitted work, one case
```

The script behind it is `packages/luxar-viewer/scripts/render-gate/run-gate.mjs`
(pass extra flags with `GATE_ARGS=`, e.g. `GATE_ARGS="--backends webgl --dsf 1"`).
Reports land in `delme/gate/<label>/`: `report.md`, `report.json` and, for every
view that differed, a heatmap PNG (black: identical; blue: drift; red: flip).

## How a run works

1. **Builds.** Each ref is extracted with `git archive` into
   `delme/gate-builds/<sha>/`, installed from the lockfile and built with
   `vite build`, then cached by commit SHA. When a ref's Rust sources match the
   current checkout, its WASM module is copied rather than recompiled. `WORKTREE`
   builds the working tree as it is.
2. **Serving.** Each build's `dist/` is served by its own small static server
   with the checkout's `datasets/` mounted at `/datasets/`: same origin, no CORS.
3. **Driving.** System Chrome (`--channel chrome`), headless. On Linux the
   harness adds `--use-angle=vulkan --enable-features=Vulkan,WebGPU`, without
   which headless Chrome renders on SwiftShader. A page reporting a software
   renderer is refused rather than measured. The page is driven only through
   debug surfaces present on main (`scripts/render-gate/page-ops.mjs`), so the
   baseline may predate the gate.
4. **Scenes** (`scripts/render-gate/gate-scenes.json`). Seeded synthetic points,
   lines (quad and capsule) and splats in every blending mode, plus stores written
   by `scripts/render-gate/generate_gate_scenes.py` into `datasets/gate/`:
   `mixed` (all four geometry types, pickable), `env_splats` (splats reflected in a
   scene-captured environment), `partition_normal`, `glass` (`refract_data`),
   `lod_ladder` (one copy under a rotated parent) and `tiny_units_ortho`. Every
   exact case runs per backend (WebGL, WebGPU), per device scale factor (1 and 2;
   an odd 1277×719 viewport at 1.5 for two cases), per projection and per pose.
   The device scale factor comes from Playwright, not `?dpr=`, which is clamped to
   the native ratio.

## Exactness

For every view the harness captures the raw scene HDR target
(`captureHDRPixels('raw-scene-hdr')`), the visible LDR image before sRGB
encoding (`'visible-ldr'`) and, for pickable stores, the pick-ID buffer. Post
effects that vary with time or add nothing (detector noise, vignette, lens
distortion, bloom) are switched off first.

**Units.** The HDR and LDR targets are HalfFloat, so differences are measured in
half-float steps of each pixel's own value (**ULP16**). A differing pixel is
either **drift** (at most 2 ULP16: the arithmetic producing it moved) or a
**flip** (more: a discrete decision such as a discard at a Gaussian cutoff or a
coverage edge went the other way). Pick IDs admit no drift: any change is a
mismatch.

**Verdict classes.** Each commit declares one:

| class | allowed |
|---|---|
| `IDENTICAL` | nothing: 0 ULP16 in every buffer, 0 pick mismatches, equal drawn element counts |
| `ULP` | frame energy change within ±1e-5; p99.9 of the 4×4 box-filtered tile change ≤ 1.3e-4 of the brightest tile; pick NODE mismatches ≤ 3e-3; equal drawn element counts |
| `INTENDED` (per case, `INTENDED=a,b`) | any change in the named cases; reported, never failed. Each must be backed by its own targeted test. Every other case keeps the commit's class. |

**Why `ULP` is judged on light, not on pixels.** A per-pixel count cannot tell a
real error from rounding jitter. Take a line shader that reads
`projectionMatrix[1][1]` where it used to read a CPU uniform of the same value.
On ANGLE/Metal that changes how the surrounding matrix products compile, and a
dense, far-away line scene moves about one float32 ulp per vertex. That flips
2.7e-3 of the frame's pixels, yet the frame's energy moves 1e-7, exactly like a
1-ulp nudge of the uniform. A width that is really 1e-4 too large moves the
energy by 1e-4. So `ULP` compares what a viewer could see:

- the signed relative change of the frame's total energy;
- the change of each 4×4 box-filtered tile relative to the brightest tile. The
  filter cancels light moved between neighbouring pixels and keeps light added
  or removed.

Per-pixel drift, flips and p99.99 stay in the report for reading.

Pick buffers are judged on NODE identity. Where one node's elements overlap, the
winning element is a near-tie that any rounding flips. A 1-ulp nudge of the line
width changed the element at up to 35% of a dense line scene's pick pixels, and
changed the node at none.

**How the `ULP` limits were set.** They are calibrated, not guessed. Calibration
arms scale one derived quantity by (1 + ε): the point size factor, the line
width scale or the splat focal length. The real commits of the
projection-in-shader work were measured next to them. Every case covered
points, lines and splats in every blending mode, on both backends (Apple M4 Max,
2026-09-25). Worst value per view:

| arm | abs(energy change) | tile p99.9 |
|---|---|---|
| rounding level: 1 or 8 float32 ulps, the real commits | ≤ 2.2e-6 | ≤ 9.4e-5 |
| real error: ε = 1e-4 | ≥ 4.4e-6 (see below) | ≥ 1.7e-4 |

Max- and normal-blended splats at a close pose barely change energy under a
1e-4 focal error, because those modes do not add light. There the tile metric
catches the error, at ≥ 2.2e-3. The tile maximum overlaps between the two rows
of the table and is only reported. The margins are about 1.4× on the tile metric
and about 5× on energy wherever energy separates. Re-run the calibration arms
(`--cand-dist` accepts a hand-patched build) after changing the scenes or the
scorer.

A drawn-element-count difference between the builds fails a case outright (it
means LOD selection or residency diverged, which would otherwise surface as a
baffling pixel difference). The report lists the largest flip relative to the
frame's peak for review; there is no generic cutoff for it, so look at the
heatmap.

**Control arm.** The baseline is captured twice, from two page loads, and each
capture is taken twice within its page. A view whose baseline does not agree with
itself is **excluded**, and any exclusion makes the run **INCOMPLETE** (exit 3),
never a pass: a gate that could not see a view has not certified it.

**First-load effect (ANGLE/Metal).** On WebGL the first page load of a scene in a
fresh browser renders a few ULP16 differently from every later load (measured:
loads 2..N agree bit-for-bit, load 1 does not). Warming up with an unrelated
scene, or `--disable-gpu-program-cache`, does not remove it; WebGPU does not show
it. The harness therefore runs one discarded warm-up pass per build through the
same views before the measured arms.

## Performance

Per case and backend the harness runs R rounds (7 by default); each round loads
the baseline, the candidate and the baseline again, in an order that rotates
between rounds. Each arm records:

- **`gpuMs`**, GPU cost at a settled pose. Five repetitions of 20 back-to-back
  full-pipeline renders, each followed by a GPU sync (a 1-pixel `readPixels` on
  WebGL, `queue.onSubmittedWorkDone()` on WebGPU). The fastest repetition is the
  least-interfered estimate and is the value kept. It depends on neither vsync
  nor timer queries, which ANGLE/Metal reports unreliably.
- **`frameMs`, `frameP95Ms`, `cpuMs`, `rendersPerFrame`**: a deterministic orbit
  of 180 frames, one step per animation frame, with
  `--disable-gpu-vsync --disable-frame-rate-limit`. Motion matters: LOD
  selection, depth-sort scheduling, the density guard and dynamic clipping only
  work while the view changes, so a static camera measures almost none of the
  frame loop. The four figures are:
  - `frameMs`: wall time per presented frame;
  - `frameP95Ms`: p95 frame interval;
  - `cpuMs`: script time per frame, from CDP `Performance.getMetrics`
    `ScriptDuration`;
  - `rendersPerFrame`: `postProcessing.render` calls per frame. A frame that
    renders twice costs twice.

A metric fails when its point ratio `median(candidate) / median(baseline)`
exceeds `1 + floor`, and is reported as a win below `1 − floor`. `floor` is the
band a no-change comparison spans: the bootstrap 95% CI of the two baseline arms'
ratio, never below 1%. The candidate's own CI is reported but does not decide,
because testing one CI edge against another CI's half-width counts the sampling
noise twice and fails unchanged builds at small round counts.

Run the perf suite on the machine whose numbers you care about, with nothing else
heavy on its GPU. In headless Chrome with vsync off, rAF does not wait for the
GPU, so `frameMs` is effectively CPU-bound frame cost and `gpuMs` carries the GPU
cost. Keep the two apart when reading a result.

## When a run is not a pass

- **FAIL** (exit 1): a case broke its class, element counts diverged, a page
  errored or did not settle, or a perf metric regressed beyond its floor.
- **INCOMPLETE** (exit 3): no failure, but some views were excluded as
  nondeterministic. Fix the nondeterminism or remove the case; do not ignore it.
- exit 2: the harness itself crashed (build failure, server port in use).