# Testing

> grade a meaning against labeled fixtures, and run expressions against a fake transport.

There are two things to test: whether a meaning means what you think, and whether the code around it does the right thing with each answer. The first needs the model; the second should not.

## Grading a meaning

`grade(meaning, fixtures, options)` judges the meaning against labeled examples and reports how it did. `expected` on each fixture is what a careful person would answer.

`grade` accepts a condition—a string, code predicate, or meaning—not a branch, query, scale, or Choice. Test the application code around those expressions with a fake transport.

```ts
import { grade, type Fixture } from "jevlish";
import { tickets, type Ticket } from "./support.js";
import { blocked } from "./vocabulary.js";


// `expected` is what a careful person would answer.
const fixtures: Fixture<Ticket>[] = tickets.map((t) => ({
  subject: t,
  expected: ["T-1", "T-3"].includes(t.id),
  note: t.id,
}));


const report = await grade(blocked, fixtures, { seenAs: (t) => ({ subject: t.subject, body: t.body }) });


report.accuracy; // correct over decided; null when nothing was decided
report.coverage; // decided over total
report.falsePositives; // decided true, expected false
report.falseNegatives; // decided false, expected true
report.abstentions; // uncertain: a policy outcome, not a wrong answer
report.misjudged.map((m) => m.fixture.note);
report.abstained.map((a) => a.fixture.note);
```

The report keeps abstentions apart from wrong answers, because they are different failures with different fixes:

* `accuracy` is correct decisions over decided fixtures, `null` when nothing was decided. A low accuracy means the boundary is wrong: revise `including` and `excluding`.
* `coverage` is decided fixtures over all fixtures. Low coverage with high accuracy means the policy is stricter than the model’s calibration needs: consider loosening `yesAbove` and `noBelow`.
* `falsePositives`, `falseNegatives`, `abstentions`, and the per-fixture `misjudged` and `abstained` lists say which examples to look at. Each carries the judgment, so the probability behind a miss is right there.

`options.seenAs` projects fixtures the way production does; `options.policy` grades under different thresholds without changing the runtime.

The sample apps run this as `npm run measure`.

## A fake transport

For everything else, give the runtime a transport that answers without a network. The `jevlish/testing` entry point provides a scripted `fake` transport and answer builders for Noul, Choice, and Score questions.

```ts
import { createSense } from "jevlish";
import { fake, yes } from "jevlish/testing";
import { ticket } from "./support.js";
import { blocked } from "./vocabulary.js";


// This transport answers every question locally and records each request.
const alwaysYes = fake(() => yes(0.95));


const sense = createSense({ client: alwaysYes });


const judgment = await sense.given(ticket).when(blocked);
judgment.status; // "decided"
alwaysYes.calls.length; // 1
alwaysYes.calls[0]?.state; // what the library would have sent
```

With a fake you can assert on `client.calls`, drive each branch of a decision with `yes(0.95)`, `yes(0.05)`, or `yes(0.5)`, and test that `uncertain` reaches the code that handles it. `pick(...)` builds Choice responses and `rate(...)` builds Score responses.

`createSense({ client })` keeps the fake out of the shared runtime, so tests do not leak into each other or into production code that uses the module-level `given`.