Loadbearing · checkpoint once, fork per mutation
pydantic-ai-description-2.5.0
1 input does not do what the pipeline claims. Found by running it, not by reading it.
12
forks from one checkpoint
5.60s
all 14 runs, in parallel (39.93s one after another)
7.1x
faster than running them in sequence
1
defect
One checkpoint, every fork
Every field, every mutation
| field | should matter? | destroy | drop | nudge | verdict |
|---|---|---|---|---|---|
| user_id | no | same | same | same | inert |
| question | yes | moved | moved | moved | load-bearing |
| output_name | yes | moved | moved | moved | load-bearing |
| output_description | yes | same | same | same | SILENT PASSTHROUGH |
The evidence
output_description · silent passthrough
Declared as an input that should change the output. It was destroyed, dropped and nudged in 3 separate forks, and the output never moved. The pipeline does not read it.
baseline output
model_was_told: {"output_description": "A vehicle", "output_name": "fruit_or_vehicle", "prompt": "What is a banana?"}
after destroy (identical)
model_was_told: {"output_description": "A vehicle", "output_name": "fruit_or_vehicle", "prompt": "What is a banana?"}
Baseline
input
user_id: u_48213
question: What is a banana?
output_name: fruit_or_vehicle
output_description: Classify the thing the user asks about: a fruit with its colour, or a vehicle with its number of wheels.
output
model_was_told: {"output_description": "A vehicle", "output_name": "fruit_or_vehicle", "prompt": "What is a banana?"}