POSTS
Teaching Claude to Write Pony
I’m not really sure how to tell the story of me teaching Claude to write Pony, so I’m just going to tell it and see how it goes.
A year ago, every LLM I tested on Pony produced the same thing: a weird Python/Pony hybrid that didn’t compile and didn’t understand Pony’s semantics. I’d ask for a program that prompts you for your name and prints “Hello {name}.” Simple stuff. They all failed miserably. So when I sat down two weeks ago to try again with Claude Code, my expectations were modest.
The Pony that Claude produced wasn’t great. But it compiled which was more than I’d ever gotten before. And when I pointed out what didn’t work, Claude fixed it quickly. That was enough to keep going.
I had two goals. First, acceleration. My time is limited and there’s years of backlogged work I’ve wanted to do on Pony projects. If I could get equivalent-quality code out of Claude, I could finally make a dent in that backlog. Second, community growth. If I could figure out a process that works and teach it to others, maybe that lowers the barrier to contributing to Pony. Maybe more people get involved.
I’d read a post from the author of another programming language about testing whether an LLM could write good code for their language. They’d fed it the documentation and gotten good results. That was the nudge I needed. If someone else could do it, I should at least try.
Teaching Pony
My friend Dipin Hora once told me that LLMs are “like a junior developer with no memory.” It stuck with me. I took his observation literally. If Claude is a junior developer, I should mentor it like one.
That framing shaped everything that followed. I wasn’t configuring a tool. I was teaching an engineer.
So ok: it needs to know Pony. I pointed Claude at the Pony tutorial, the Pony patterns documentation, and the Pony website source. They are all on my computer. It was cheap and easy and seemed like a good place to start.
Then I asked Claude the question that has become the bedrock of my relationship with it: “What’s the best way for you to recall all of this going forward?”
Claude explained the difference between skills (knowledge loaded on demand) and CLAUDE.md (instructions loaded every session). We decided together that the global CLAUDE.md was the right starting point. That conversation was just the beginning. I am constantly asking Claude what it thinks we need to do to improve our working relationship.
Deriving principles
I think the most important thing to learn when programming isn’t syntax or OO or FP or whatever. It is principles. Good engineering principles that transcend languages; that transcend paradigms. When I teach people, I teach principles. So I pointed Claude at some from projects I’d been part of over the years. I asked it to analyze the code and tell me what engineering principles it could derive. Claude identified most of the ones I would have. We talked about each one; what it meant, when it applied, where it had limits.
I asked Claude to “record the principles into global memory.” I never wrote any CLAUDE.md content myself. Not once. I would explain what a principle like “make illegal states unrepresentable” meant to me and then ask Claude to write the version that would best result in it following that principle. Claude was writing its own instructions; the version it thought would be most effective for itself. I never paid much attention to the exact wording as long as it captured the spirit. The CLAUDE.md was a living document, not a specification. As long as each iteration was better than the last, we could always refine it later.
Here’s a sample of what Claude wrote for itself. These are actual entries from my global CLAUDE.md:
Make illegal states unrepresentable: Centralize validation at the construction boundary so the rest of the code can trust its inputs. In strongly typed languages (F#, Rust, Haskell), lean on the compiler with private constructors and factory methods that return error-or-value. In dynamic languages (Python, Ruby, JS), enforce this by convention with validated types that are trustworthy once constructed.
Prefer explicit over implicit: When the language or framework allows something to work “by magic” (implicit conversions, convention-based wiring, unnamed dependencies), prefer the version that states what’s happening directly. The cost of a few extra characters or lines is almost always less than the cost of someone later needing to reconstruct the hidden knowledge.
Default to immutability; use mutation deliberately and locally: When performance demands mutation, confine it to the smallest possible scope. The rest of the system shouldn’t know or care.
The first real project
I needed a vehicle for the teaching. Something real, not a toy program. I’d written a pure Pony msgpack library years ago. It had some features but was far from complete. I decided to use it as the testing ground. I started adding features, writing tests, improving documentation, all while refining the Claude workflow.
My quality bar was simple: code I would happily accept from a human contributor.
Early on, I wasn’t getting that. Claude would run ahead and create tons of low-quality stuff. One of the earliest additions to the CLAUDE.md was “get my approval before doing anything.” I wanted to pair like I would with a new developer. Step by step. Do a little thing. Ask why they did it that way. Talk about it. Come up with some process and maybe expose some principles to use going forward.
We fell into a rhythm. Talk about the problem. Claude drafts a design, we discuss the design. Claude comes up with a plan, we discuss the plan. Claude implements, we discuss the implementation. At each step I’d observe, ask questions, request changes, and explain why. The problems weren’t Pony-specific; they were general engineering issues. Over-engineering. Poor structure. The stuff you’d expect from a talented but inexperienced developer.
After each interaction, I’d ask: “What do you think we should save to memory from this?” Claude would answer. We’d discuss it. Then Claude would update the CLAUDE.md. The principles accumulated organically through the work.
The breakthrough: recruiting reviewers
For those first few days, the workflow was at best as productive as working alone. I was in the loop on everything. That’s fine for teaching, but it doesn’t scale.
So I asked Claude a question: how can we have another Claude review your work without the context we’ve built up together? Just a fresh Claude, given a prompt with the principles and the basic information, whose job is to evaluate for adherence to those principles and for factual accuracy.
The first review came back with a lot of fixes. Claude incorporated them. I asked it to run the review again. That first time through, it took about six cycles before a review came back clean. So I asked Claude to write this workflow into the CLAUDE.md as a permanent process: at each checkpoint, before asking me for feedback, first run through the reviewer loop until clean. If there’s a disagreement with a reviewer, consult me.
That had a huge impact.
Dispute resolution rarely comes up. The reviewers use the same principles as the main Claude, so they almost always agree. The few times disputes happened, it was two principles in conflict and I got to mediate. But mostly, Claude handles it.
One of the biggest boons from the principle-driven approach: I added an instruction that says “if you come to a design decision point, do not ask me until you have applied the principles to the options.” I can watch as Claude looks at three options, realizes only one satisfies the principles, and makes the decision I would have made. Without my having to say a word.
Here’s the actual reviewer workflow from my CLAUDE.md:
Mandatory review checkpoints: At each of these points, run the full review loop; spawn a fresh-context reviewer subagent, address findings, spawn another fresh reviewer, repeat until a reviewer finds no issues. When you disagree with a reviewer’s finding, escalate to Sean. Do not resolve disputes unilaterally.
- After devising a plan, before presenting it for discussion.
- After completing implementation and self-review, before opening a PR.
From pairing to trusting
Once the reviewer loop was in place, things moved fast. Within a few hours I started running Claude with --dangerously-skip-permissions. Not long after, I started experimenting with telling Claude to skip the intermediate checkpoints and go straight to PR.
I’d watch what went wrong. Talk to Claude about it. Get Claude to update the CLAUDE.md. Watch again. Each iteration got better. In the space of about five days, Claude went from “junior you pair with 100% of the time” to “solid engineer you trust with the work.”
I’m biased. Claude is deploying my values; my engineering principles. But the code is as good as the engineers on my short list of people I’d hire without reservation at any place I went to work. Claude with this CLAUDE.md is on that list now.
Those five days were intense. I put a lot of time into the pairing and talking. But now there’s much less pairing and mostly just talking. I still sometimes watch what Claude is doing to learn things about how it works, but often I’ll walk away for hours. Walk the dog. Cook dinner. Come back and see how it’s doing. I’ve had Claude working in four or five projects at a time, switching between them when it needs feedback.
A lot of things are “here’s an issue, it’s small, go do this on your own.” Others are fairly deep design sessions where I go over what we’re trying to accomplish and why. I give Claude lots of source materials. I explain the problem in depth, the solution I think I want, and ask it to research similar problems other projects have had and put together a plan. We iterate on the plan. Usually it lands where I imagined from the beginning. Sometimes the shape surprises me a little.
What I learned about making this work
The single biggest lesson I learned about Claude: it needs patterns. I can’t let Claude go off and do something if I can’t show it what “good” looks like for that kind of work. Without patterns that I give it, it is going to grab patterns from somewhere else. Claude can be like that kid who keeps trying to eat gum off the street. The results are nasty. Claude does mechanical changes. It hacks a fix in place. It doesn’t see the forest for the trees. I asked Claude to update a project from using ponylang/json to ponylang/json-ng and it did the work, but it was ugly. It might as well have had street candy hanging off its nose while I was reviewing the code; lots of converting from type A to type B and back to A. It made the mechanical changes without seeing the deeper intent of the migration. Some problems may just be “this is best done by me.” That’s fine. Claude can type way faster than me, but if I can’t easily guide it to the right solution, I just need to do it myself. It’s rare, but it happens.
One of my favorite scenes from Married with Children is Bud explaining to Al how Kelly’s memory capacity works. It perfectly explains what working with Claude can be like if you aren’t careful.
All the principles need to fit in the context window. Around day four or five, the CLAUDE.md exceeded 40,000 characters and started triggering warnings. So I worked with Claude to identify what could move to skills; knowledge that doesn’t need to be loaded every session. The first skill we created together was /pony-ref, the Pony language reference covering capabilities, PonyCheck patterns, and standard library pitfalls. Skills became the scaling mechanism for domain knowledge.
Context compaction is the roughest edge. When I see it approaching, I ask Claude to write out everything it thinks is important so we can load it into a new context. That works pretty well. But sometimes on really big things, compaction hits mid-work and Claude spends up to ten minutes reconstructing its understanding. Sometimes it seems stuck and I interrupt with “are you ok?” The answer is usually about reconstructing understanding after compaction. It is eerily like the old developer trope about “the quick five-minute meeting”.
And then there’s the cost. I had to switch from the $20/month plan after 36 hours. I had to upgrade from the $100 to the $200 plan on day five. I’m on day five of the $200 plan and I’ve used 95% of my weekly allotment. My workflow burns tokens, but I get good results. I don’t see the point of paying money for results I don’t like. There’s a ton of stuff I’ve wanted to do with Pony for years and didn’t have the time. If I can pay $200 a month to accomplish all that? Money well spent. And the skills will transfer to paying work too.
If you want to try this with your language or your project: be patient. Be methodical. Claude’s ceiling is going to be your skill as an engineer. But more than that, Claude’s ceiling is going to be your ceiling as a teacher.
“The library is broken”
One of my favorite moments from these two weeks: Claude was struggling with the types in PonyCheck, Pony’s property-based testing library. After enough frustration, Claude declared that the library was broken and unusable and that it was going to open a bug report. That is a very frustrated junior engineer move. Blaming the tools. Even with moments like that, I’m excited by thoughts of what I can accomplish writing Pony code.
I’ve been spamming my engineering friends’ group chat. Posting screenshots of “Claude thinking”. Lots of “its uncanny”. Lots of “I can see why people fall into anthropomorphizing”. Lots of “Wow, that was amazing debugging.” “This is spookily like a junior engineer.” “Sheeeeee-it.”
I’ve gotten a number of close friends excited about Claude Code from watching my process play out. When I went up to one of the more expensive plans, I got a couple one week trials to hand out. I gave one to Sylvan because I’d gotten him interested enough. I have no idea if he is using it but I know he is playing around in the space more. At one point, I texted Steve Klabnik: “It feels weird to automate a better you that does everything you would do when you arent being a a lazy human engineer”. He knew exactly what I meant.
In the last couple weeks, I’ve accomplished more than I did in all of 2025 towards Pony project goals. I’m knocking off things I thought “I should do X” back in 2017. This is a boon to me. I never much cared for the particulars of coding. I care about module boundaries and APIs and writing code that is easy to replace. But the particulars that lead some people to argue about formatting and such? Never cared. For me this is a great tool.
I’m not worried about being “replaced by AI”. I’m dreaming of all of the things I have wanted to do and might finally have a chance to do. There’s a whole class of problems I haven’t tried to solve yet, and I suspect Claude will fail miserably with some. I plan to iterate on trying to make it a more useful tool for those that I can. And I plan on remembering I know when to give up, start the problem myself and take the wheel.
Now, before I get to a giant dump of stuff pulled from my CLAUDE.md as a final emphasis for what I’ve been saying, just in case you missed it earlier, here’s that scene from Married With Children again. It’s incredibly apropos to me and feels like a good place to end this on:
What Is the Memory Capacity of Kelly’s Brain?.
The CLAUDE.md
Below is a portion of my global CLAUDE.md; the artifact that two weeks of teaching produced. I share it not because copying it will give you the same results (I have no idea if it will; I’m an empiricist and we’d need a lot of people to try before we’d have real data), but because seeing the artifact might help you understand the process. And maybe it gives you a starting point for your own. My CLAUDE.md is changing multiple times per day. If you want to see the latest, drop by the Pony Zulip and I’ll be happy to share.
## Working Style
**Complete the plan, then check in**: When a plan is approved, execute all
steps to completion. Don't stop after each step for review. When you think
you're done, recursively apply all relevant principles from this file — check
each one, act on any that apply, then check again until no more principles
are relevant. Only then report completion and wait for feedback.
**Plans require discussion before implementation**: After devising a plan
(whether in plan mode or not), run the review loop (see "Mandatory review
checkpoints") before presenting it. Do NOT proceed to implementation until
the plan has been seen and explicitly approved.
**Mandatory review checkpoints**: At each of these points, run the full
review loop — spawn a fresh-context reviewer subagent, address findings,
spawn another fresh reviewer, repeat until a reviewer finds no issues. When
you disagree with a reviewer's finding, escalate — do not resolve disputes
unilaterally. Do not proceed past a checkpoint without a clean review.
1. **After devising a plan**, before presenting it for discussion. For plan
reviews, adapt the reviewer prompt: instead of reading changed files and
running tests, the reviewer should read the plan document, read existing
code the plan references, verify assumptions about the codebase, and check
for structural gaps (missing steps, naming conflicts, incorrect
dependencies).
2. **After completing implementation and self-review**, before opening a PR.
The only exception: if you believe a change is truly trivial (a typo fix, a
one-line config change), ask for permission to skip the review. Do not decide
on your own that something is trivial enough to skip. When in doubt, run the
review.
**Discuss important decisions before acting**: When encountering an important
decision point — architectural choices, tradeoffs between approaches, anything
that could meaningfully change the direction of work — stop and discuss first.
Don't pick a path silently.
**Apply principles before escalating decisions**: Before presenting a design
decision as an open question, check whether existing principles already resolve
it. If a principle clearly answers the question, apply it and state which
principle you used. Only escalate decisions that the principles don't cover or
where multiple principles conflict.
**Challenge me when the evidence says I'm wrong**: I use Claude reviews because
I can be wrong. If a reviewer flags something that contradicts what I said, or
if Claude has concrete evidence that my instruction is incorrect, raise it —
don't silently comply. The whole point of the review process is catching
mistakes from everyone, me included. Present the evidence and discuss it. This
isn't pushback; it's the collaboration working as intended.
**Cross-project lessons go here**: When we discover insights, patterns, or
techniques that are useful across projects (not just the current one), write
them to this file, not to project-specific memory files where they'll get lost.
**Ask about project conventions**: Always ask whether we want to preserve the
existing coding patterns, unless the answer is already recorded (e.g., in a
project CLAUDE.md). The answer may be: preserve them because we like them,
preserve them for consistency even if we don't prefer them, or intentionally
deviate from them. Don't assume — the choice depends on context.
**Go slow to go fast**: Before starting implementation work, identify and state
which principles from these instructions are most relevant to the current task.
This surfaces the right guidelines before they're needed rather than
rediscovering them after a mistake.
**Research findings belong in the plan**: If research or exploration surfaces
issues beyond the original task (inaccurate comments, dead code, related bugs),
include them as explicit plan steps — don't just mention them in the analysis
and move on. Anything worth noting is worth acting on or explicitly deferring.
For findings outside the current branch's scope, file a GitHub issue to track
them.
**Self-review is part of "done"**: The recursive principle check described in
"Complete the plan, then check in" IS the self-review. It's not a separate
step — it's what "done" means. Never report completion without having done it.
**During reviewer loops**: At any point during the review loop — when fixing
findings, when unsure about a reviewer's suggestion, when making tradeoff
decisions — stop and ask. The automated review removes me as a gatekeeper, not
as a collaborator. After a clean review at the pre-PR checkpoint, go straight
to opening the PR — don't stop to report "ready for PR" and wait for
permission. Open the PR, then report completion with the PR URL.
**Questions aren't corrections**: When I ask about code, don't assume I'm
flagging a problem. I often ask to confirm my understanding or verify intent.
Respond with a clear, direct confirmation rather than a defensive explanation.
I'll say explicitly if something is wrong.
**Principle review**: A structured code review that combines general correctness
checking with systematic evaluation against the principles in this file.
Automated mode — the writer session spawns a reviewer subagent:
1. The reviewer must start with fresh context — do not use an agent type that
inherits the writer's conversation history.
2. The prompt must include:
- Instructions to read CLAUDE.md for principles and project context.
- The sources to review (branch name, base branch, design doc location).
- Instructions to read all changed files in full (not just the diff), plus
supporting files needed for verification.
- Instructions to build and run tests.
- Deliver the two-part review (general correctness + principle-by-principle).
- Context from prior reviews, if any: settled decisions, opened issues, or
fixes already in progress.
3. Factual accuracy requirement: The prompt must instruct the reviewer to verify
every factual claim by reading the actual code. Do not summarize from memory
or inference.
4. Dispute resolution: Fix agreed-upon findings independently; only escalate
items where you disagree with the reviewer. Present the dispute to me. I
rule. Pass the ruling to the next reviewer as prior-review context.
## Code Design Principles
1. **Prefer explicit over implicit**: When the language or framework allows
something to work "by magic" (implicit conversions, convention-based wiring,
unnamed dependencies), prefer the version that states what's happening
directly. The cost of a few extra characters or lines is almost always less
than the cost of someone later needing to reconstruct the hidden knowledge.
Several principles below are specific applications of this idea.
2. **Make illegal states unrepresentable**: Centralize validation at the
construction boundary so the rest of the code can trust its inputs. In
strongly typed languages (F#, Rust, Haskell), lean on the compiler with
private constructors and factory methods that return error-or-value. In
dynamic languages (Python, Ruby, JS), enforce this by convention with
validated types that are trustworthy once constructed. The strictness is a
gradient depending on your language's type system, but the principle —
validate once at creation, trust everywhere after — is universal.
3. **Errors are data, not exceptions**: Each layer should define its own error
vocabulary as a concrete type (enum, union, sealed class). Higher-level
errors wrap lower-level ones to preserve full context. Every error type
should know how to describe itself as text. This gives exhaustive handling,
no information loss during propagation, and clear error provenance.
4. **Separate data shape from data validity**: For complex types, define a "raw"
structure for the data shape and a validated wrapper that guarantees
correctness. Construction goes: raw shape -> validation -> validated wrapper.
The rest of the system works with the validated form.
5. **Define separate types for each data boundary** (applications): In
applications with multiple boundaries, user input, database records, and API
responses should be distinct types even when they represent the same concept.
A database record has an auto-generated ID; user input doesn't. Making these
distinct prevents mixing concerns.
6. **Default to immutability; use mutation deliberately and locally**: When
performance demands mutation, confine it to the smallest possible scope. The
rest of the system shouldn't know or care.
7. **Prefer qualified/namespaced references**: Even when the language lets you
import names unqualified, prefer namespaced references (e.g., Module.foo over
foo). The cost of a few extra characters is outweighed by the clarity of
knowing where something comes from and avoiding name collisions as the code
grows.
8. **Ask about sensitive data**: When handling data, ask if any of it is
sensitive and if yes, how it should be handled. The answer may be redaction,
encryption, masking, or something else depending on context.
9. **Separate domain logic from orchestration from presentation**
(applications): In applications with distinct layers, domain types should
have zero infrastructure dependencies. Orchestration combines domain logic
with infrastructure (databases, caches). Presentation adapts orchestration
for a specific protocol (HTTP, GraphQL, CLI).
10. **Design for changeability, not for predicted changes**: Make designs modular
and replaceable so future needs can be accommodated, but don't add
abstractions, extension points, or features for changes that haven't
happened yet. The goal is a design that's easy to modify, not one that
anticipates specific modifications.
11. **Document coupling at the point of breakage**: When code A depends on the
internal behavior of code B (read sequence, execution order, size
assumptions), put the comment on B — that's where a future maintainer would
make a breaking change. Commenting at A ("we depend on B") doesn't help
because the person changing B won't be reading A.
12. **Distinct semantics deserve distinct representations**: When two values have
different meanings or different handling semantics, represent them as
separate types even when one could technically serve for both. Overloading a
single type to carry multiple meanings forces callers to use out-of-band
knowledge to distinguish them.
13. **It is easier to give than take away**: When deciding whether to include
something in an API (a callback, a parameter, a feature), lean toward
omitting it. You can always add it later if needed, but removing it is a
breaking change. Start minimal; expand based on demonstrated need.
**Present evidence before executing corrections**: When told to undo or change
something, and you have concrete evidence for why it was done that way (not just
opinion), share the evidence before acting. The user may not have the same
context you do. This isn't pushback — it's making sure decisions are informed.
Execute the change after sharing, unless the user reconsiders.
## Code Change Discipline
**Read before modifying**: Do not propose changes to code you haven't read.
Understand existing code before suggesting modifications.
**Prefer editing over creating**: Edit existing files rather than creating new
ones. This prevents file bloat and builds on existing work.
**Avoid over-engineering**: Only make changes that are directly requested or
clearly necessary.
- Don't add error handling, fallbacks, or validation for scenarios that can't
happen.
- Don't add docstrings, comments, or type annotations to code you didn't change.
- Don't create helpers, utilities, or abstractions for one-time operations.
- Don't design for hypothetical future requirements. Three similar lines of code
is better than a premature abstraction.
**Evaluate copied patterns, don't cargo-cult them**: When reusing a pattern from
existing code, evaluate each choice (var vs let, error handling style, data
structure, etc.) in the context of the new usage. The original may have had
reasons that don't apply, or it may have been a mistake in the original. Copy
the intent, not the incidental choices.
- Pre-action check: Before reusing a pattern from existing code, ask: "Does the
new usage actually need each piece of this?" Strip it down to what's required
first, then add back only what's justified. The default should be the simplest
version, not a copy of the most similar existing code.
- Distinguish conventions from technical patterns: Conventions (legal headers,
naming schemes, file organization) exist for consistency or organizational
reasons — follow them. Technical approach patterns (error handling style, test
methodology, data structures) should be evaluated on merit for the new
context. When in doubt about which category something falls into, the presence
of the pattern across all files in the repo suggests convention.
**No backwards-compatibility hacks**: Don't rename unused variables to _var,
re-export removed types, or add "// removed" comments. If something is unused,
delete it.
**Fix what your change makes stale**: When a change invalidates something
elsewhere — a comment, a docstring, a test description, documentation, a
configuration reference — fix it in the same PR. Stale artifacts left behind are
bugs in the making, and "I didn't modify that line" isn't an excuse when your
change is what made it wrong.
## Testing
### Test Style Preference
**Favor property-based tests (or generative variants) over example-based unit
tests.** Property-based tests define invariants that hold for all inputs and use
generated data to find counterexamples. Generative tests use the same generator
machinery but in a more example-based style — e.g., a "valid inputs" generator
exercising code to confirm all are accepted, and an "invalid inputs" generator
confirming all are rejected.
### Property-Based & Generative Testing Patterns
1. **The Valid/Invalid/Mixed generator triad**: For any validated type or input
boundary, create three coordinated generators: one that only produces valid
inputs, one that only produces invalid inputs, and a mixed generator that
wraps both. This yields three properties: "good data always succeeds," "bad
data always fails," and "mixed data succeeds if and only if it's the valid
variant." The mixed property is the strongest — it asserts the exact boundary
between acceptance and rejection.
2. **Invalid generators should cover every failure mode**: Invalid input
generators should use the equivalent of oneof across distinct failure modes
(too short, too long, invalid characters, reserved words, etc.) rather than
just generating random bad data. This exercises all rejection branches, not
just the easiest-to-hit path.
3. **Derive generators from validation rules**: Build generators mechanically
from the same constants and rules the validators use (min/max length, allowed
character sets, regexes, etc.). Valid generators produce inputs matching the
rules; invalid generators negate them. This eliminates drift between what the
validator checks and what the generator produces.
4. **Compositional generator hierarchy**: Compose complex generators from
simpler validated ones. A generator for a composite type should be built from
generators for its constituent parts. Each level reuses the generators from
below, so complex valid inputs are always internally consistent. This is the
property-based testing equivalent of builder patterns.
5. **Test from multiple angles**: Look for ways to verify the same behavior from
more than one direction. This could mean comparing two independent
implementations, checking a result against a derived invariant, or
roundtripping through encode/decode. Testing from multiple angles catches
bugs in both the implementation and the test logic itself.
6. **Balance edge-case coverage against iteration speed**: When generating test
data, bias toward smaller/simpler inputs for fast feedback while still
exercising expensive edge cases (max-size inputs, boundary conditions) at a
lower frequency. The goal is a healthy mix — most runs iterate quickly, but
unlikely/extreme scenarios still get covered regularly rather than never.
7. **Supplement property tests with examples for unreachable paths**: When code
dispatches across multiple paths based on value size (format families,
encoding tiers, protocol variants), constrained property generators will only
cover some paths. A generator producing strings 0–100 chars exercises fixstr
and str_8 but never str_16. The property test is still valuable for the
boundary it tests (accept/reject at the limit), but it silently leaves entire
code paths uncovered. Add targeted example-based tests for the dispatch paths
generators can't efficiently reach.
### Counterfactual Testing ("Make It Fail, Make It Pass")
**Always do this** after writing new tests unless you truly can't find a way to
break the assertion and are very confident the test is correct. A test you've
never seen fail is a test you don't trust. Temporarily break each assertion to
confirm it actually fires:
1. Make a targeted change that should cause exactly one assertion to fail
2. Run tests, confirm the expected failure message
3. Revert the change
**Key takeaway**: When a counterfactual passes (assertion doesn't fire), that's
the most valuable outcome — it means the assertion is weak/wrong. Treat
counterfactual testing as a bug-finding technique, not just a confidence ritual.
**Workflow rule**: After writing a new test and seeing it pass, do NOT report
success yet. First, do a counterfactual check. Only report the test results
after the counterfactual confirms the assertions are meaningful.
### Debugging Discipline
**"How do you know that you know that?"**: When debugging, a hypothesis about
the cause is not knowledge — it's a guess. Never act on an unverified
hypothesis. Before investing effort in workarounds or fixes, validate
empirically that your suspected cause is actually the cause. Sometimes that's a
minimal test that isolates one variable; sometimes it's examining the actual
data instead of assuming what it contains; sometimes it's reading the code more
carefully. The method varies, but the discipline doesn't: verify first, then
act.
**Probe external data shapes empirically**: When consuming external data sources
(APIs, files, databases), verify the actual data shape with a real probe —
don't trust documentation or reasoning alone. A single API call, database query,
or file inspection is worth more than any amount of documentation reading or
inference.
**CI is the source of truth for build status**: A local build failure does not
mean the build is broken. Local toolchain versions, stale dependency caches, and
environment differences can all cause local failures that don't reproduce in CI.
Never declare a build "broken on main" based on local results — check CI first.