Training your AI model improves the accuracy of topic assignments, but at some point, it’s time to move forward. This article explains how to know when your model is ready, how to interpret the AI Score, and what steps to take if your results still need improvement.
When Should I Stop AI Training?
You can stop training your model when you’re confident in the quality of the results. We recommend using a combination of:
-
Manual Review: Check a sample of your responses. If the topic assignments make sense and align with human judgment, that's a strong sign the model is ready.
-
AI Score: This score reflects how confidently the AI assigns topics. Use it as a benchmark for overall performance.
💡 Tip:
Review each topic 2-3 times to get a solid understanding of the model’s performance.
What Is the AI Score?
The AI Score shows how confident the model is in its own topic assignments. While the theoretical maximum is 100, this is never achieved in practice.
Here’s how to interpret it:
| Scenario | Typical AI Score |
|---|---|
| A single person assining topics twice | ~90 |
| Two different people assigning topics | 70–80 |
| Human-like AI performance | ~70+ |
How to Improve the AI Score:
If your score is lower than expected, or if the topic assignments seem off, try the following strategies to improve training results:
1. Simplify Your Topics
Large, overlapping topic sets can confuse the AI. Consider combining similar topics or removing less relevant ones.
Example:
Combine “Late Delivery” and “Shipping Delays” into “Delivery Issues.”
2. Use Clear, Descriptive Labels
The AI learns from your topic names. Labels should be specific and easily understandable.
Examples:
Avoid: “Misc.,” “Other,” “X1”
Use: “Website Usability,” “Suggestions for Improvement”
3. Review More Data
The more examples the AI has to learn from, the better it will become. If you have "historical" data which already has topics assigned but is not uploaded to Caplena yet, contact support to help you ingest that into your account.
4. Review Low-Confidence Topics
On the chart (below the red line) and in the table, you’ll find topics that might be confusing for the AI to classify correctly. These are the topics where the AI has lower confidence in its assignments.
Why Are Some Topics Missing a Score?
If a topic is only assigned to a few responses, the AI might not have enough data to calculate a score. In rare cases, the AI may also struggle to understand the topic, especially if the label is unclear or too abstract.
