Compass · for Natalie

Compass check-in · agenda

Here's the whole picture.

For NatalieFrom MaconWhen Friday, June 5 · 11:00
1What's improvedOpen →
The planning brain — recognize widely, resolve strictlyCleaning houseKnowing when not to answer
2Data syncOpen →
Production as the source of truthNew York enrollment fixDirectionality — nothing gets wiped
3Pearl's selection scenariosOpen →
My read of the scenariosAgree → into the sheetsubstance to fill in
4What's nextOpen →
Three options to prioritizesubstance to fill in

Tap any part to jump in. We'll go in order, then take anything else after.

Item 1 · What's improved

1

How Compass got better.

Three changes that made Compass smarter — each a small before-and-after. The first is the big one: a single reframe — recognize widely, resolve strictly — that's why so much moved at once.

The big one · ends vs. means

Compass learned to recognize widely, and resolve strictly.

One reframe explains why so much changed at once. The old planner — the part that turns your sentence into a data lookup — was a rigid recipe binder, and it worked blind. The redesign splits two kinds of strictness that used to be fused together.

Before

Every question had to fit a fixed menu of pre-approved shapes, or Compass gave up and asked you to rephrase — the clarify spiral (someone in Denver asked four times and got nothing, though the data was right there). And it had to guess a district or metric before it was allowed to check the thing was real — a student marked wrong on the exam, but not allowed to open the textbook.

Now

Recognizing what you mean is generous and flexible: the planner can search the real catalog first, then commit. Turning that into numbers stays unforgivingly strict: every value traces to a real row, nothing invented. The freedom moved to understanding you; the discipline stayed on the data.

What that looks like in practice — each link opens the real case live on staging
“Give me the highest-paying districts for a teacher with a bachelor's, just starting out” open in debug ↗
Before could rank by the wrong thing — district size, or the master's pay lane — and still hand back a clean, authoritative table.
Now locks onto the exact measure — bachelor's, starting — and ranks by pay.
“What are evaluation policies in Cleveland County?” open in debug ↗
Before grabbed one “Cleveland”-ish district and answered confidently for the wrong one.
Now recognizes the name is ambiguous and asks which district you mean.
“Which districts cover the most of teachers' health insurance premiums?” open in debug ↗
Before loose wording like this didn't match a metric label, so it could give up.
Now maps it to the right approved measure — and offers close matches when the wording is off.
“What is NCTQ's position on teacher evaluations?” open in debug ↗
Before tried to force it into a district data table.
Now routes it to NCTQ's published position — the right shelf, not a spreadsheet.

Why it matters: selection mistakes are the dangerous kind — they look authoritative. A clean table of real numbers that quietly answers the wrong question can land in a board memo or a contract before anyone notices. That's the error class this work attacks.

Honest status: this is a reframe still underway by design (RFC #1248). The most painful failure — the clarify spiral — is already fixed and live; the strict half is deliberately left untouched.

Rule of thumb: define the destination strictly, let it find the road, check the odometer before anyone gets out of the car — and never drive into the lake. This one reframe is why so much landed in Selection — and filter, sort, and citation inherit the cleaner foundation.

Cleaning house

Less code, fewer surprises.

Most of the week was the unglamorous foundation work — the kind you mentioned in your text. It doesn't show on screen, but it's what makes everything above trustworthy and quick to change.

  • Removed a lot of old, dead code and broke the single biggest, tangled file into clear pieces.
  • One source of truth for configuration, and the database connections now recycle cleanly instead of going stale.
  • Made the automated test runs reliable — so the quality scoreboard we steer by stops tripping over itself.
  • Re-wrote all 15 milestone docs in plain language, so anyone can see what "done" means without reading code.

Knowing when not to answer

Compass now says "we take no position" — on purpose.

Before

Ask about a topic NCTQ takes no stance on, and Compass forced it into a data lookup that couldn't succeed — a confident answer that was really a dead end.

Now

It recognizes those questions and gives an honest "no position" response instead of pretending.

Why it's better: knowing what not to answer is a big part of being trustworthy. Fixes #1218

Item 2 · Data sync

2

Fresh data, pointed the right way.

Compass answers from its own daily copy of NCTQ's data. Two fixes make that copy trustworthy: run the refresh where it actually counts, and make sure syncing never clobbers the wrong table.

The shift · source of truth

Production should be the freshest copy — today it's the stalest.

Before

The daily refresh runs only on staging. Production — the data customers actually see — isn't on the daily cadence, so it quietly drifts. We have it backwards: the environment that matters most is the one not getting the guaranteed refresh.

The fix

Make production the source of truth: the daily pull runs in prod, and staging becomes a downstream mirror that copies from prod instead of racing the same upstream sources. One canonical copy — not two pullers that can drift apart.

Honest status: this is the plan in flight (RFC #1317, PR #1319) — drafted and reconciled with main; the open item is confirming the Azure schedule.

A win already shipped

New York finally ranks where it should.

Before

Big districts that the data splits into smaller component records were under-counted — so New York sank in any "largest by enrollment" list, looking far smaller than it is.

Now

An enrollment roll-up recombines those component districts into the parent. New York ranks #1 at 847,030 students — correct, and verified on staging.

Why it belongs here: it's recomputed straight from the synced data — exactly the kind of correctness the new sync model is built to protect. PR #1275 · #1270

The careful part · directionality

A sync must never wipe the wrong table.

Not every table should flow the same way — so an explicit "directionality matrix" governs each one, and the staging pull only ever touches the tables production owns.

  • Sourced from production → the NCTQ data: metrics, publications, Navigator answers and citations, national NCES data.
  • Authored on staging (never overwritten) → our evaluation ledger — the scenarios, cases, criteria, and verdicts behind the scorecard — plus curated catalog work.
  • Schema stays in lockstep across both environments via append-only migrations, no matter which way the data flows.

Item 3

3

Pearl's selection scenarios — where I land

Substance to fill in once I've read it — we'll do this section next.

Item 4

4

What I'd work on next

Three options + my lean — we'll do this section after #2.