From Fixed Labels to Open Worlds: Defining the Learning Problem

In your project, you encountered a learning challenge where your model needed to classify more target classes than those in the training dataset.

3 min readMachine Learning

The user who posted this question stumbled onto something bigger than a naming exercise. They built a system that learned to group the unseen, not by memorizing thirty fixed labels, but by understanding what makes things similar and different in a continuous space. That is not a niche trick. It is a direct answer to one of the most persistent limitations of traditional machine learning: the assumption that the world stays neatly divided into the categories you had at training time.

This approach, often called open-set recognition or metric learning for open-world classification, matters because real data never respects your original label sheet. A fraud detection model trained on last year's schemes will face new patterns. A product classifier built for a catalog of 10,000 SKUs will encounter items that belong in categories it has never seen. The standard classifier, softmax and all, can only force everything into one of its known bins, creating confident mistakes. The user here sidestepped that trap by teaching the model to measure distances and then letting the data define its own clusters at inference. The threshold becomes a flexible boundary, not a prison wall.

For practitioners, this is a practical shift in how you think about the problem. You stop asking "Which of my known classes is this?" and start asking "Is this close enough to something I have seen before, and if not, what new group does it form?" The trade-off is that you must design a good embedding space and tune that similarity threshold carefully. But the payoff is a model that can adapt to new conditions without retraining, which is often the difference between a prototype and a production system that survives its first week in the wild.

So call it open-world recognition, metric learning with dynamic clustering, or simply a smarter way to handle the unknown. The label matters less than the principle: your model should be ready for what it has not been taught. Build embedding spaces that preserve relationships, set thresholds that reflect your tolerance for novelty, and let the data tell you when it has found something new. That is not a footnote to classification. It is the foundation for systems that actually work outside the lab.

From Machine Learning

Recently I did a project where I initially had around 30 target classes. But at inference, the model had to be able to handle a lot more classes than these 30 targets i had in my training data. Therefore, I couldn’t just make a ”normal” classifier that predicts one of the 30 target classes.

I instead went with a metric learning approach where i adapted different flavors of arcface/cosface etc. to create an embedding space that tried to maximize inter cosine distance, and minimize intra cosine distance.

Read the original at Machine Learning