Small models have a way of exposing assumptions that larger ones can hide. When an MSc student working with BrickNet, a system that represents LEGO models as plain text and fine-tunes Qwen3 0.6B with LoRA, found that short captions caused the model to skip the root line entirely, they hit a problem that feels narrow but speaks to something broader. The model's first token was "b" instead of "a" every time the caption ran under about 15 tokens. With longer descriptions, it worked flawlessly. This is the kind of brittle behavior that makes us wonder whether our training pipelines are as robust as we think.
We have written before about how Making math faster: an open-source path to cheaper AI discovery can reveal hidden constraints in model architecture, and about how Why raw web data broke your fine-tuning and how DOM targeting fixed it shows that data formatting decisions ripple through performance. This LEGO caption problem is another example of the same pattern: what looks like a model failure is often a data distribution mismatch. The student suspects the training captions were all long descriptions. If that is true, the SFT adapter never learned to condition on short prompts. The model defaults to generating what it knows, the internal structure of a path, without anchoring to the sparse caption.
The workaround of expanding short captions is practical, but it sidesteps the deeper question. Why does the model land on "b" every time? That specific token suggests the model is jumping straight to a node line, bypassing the root. It behaves as if the root had already been written, which means the positional or structural cue from the caption is missing. Forcing "a" as a prefix did not help either, the model then wrote a node and immediately an edge, confirming it had lost the structural context. This is not a tokenization bug or a saving error; it is a learned behavior from the training data.
The practical takeaway is direct: if you fine-tune with LoRA on structured generation tasks, your conditioning text length distribution matters as much as your content. Augmenting with short captions during training would likely fix this, and it is a simpler fix than constrained decoding or prompt engineering. The student's hunch about bf16 precision merging two adapters is worth testing, but the evidence points to data, not arithmetic. For anyone building similar pipelines, this is a concrete reminder to audit your training captions for length diversity. The model is always telling you what it learned, you just have to listen to the first token.