Getting the best model

• Your own categories win. The model beats Jev by far on decisions with your company's own answers (your queues, teams, products, intents): BANKING77 0.889 against 0.797 macro-F1, a customer-support set 0.998 against 0.771. On generic questions with little data (the topic of a news text, the type of a contract clause with a few hundred examples) Jev already knows a lot and the margin is thin. Frame the decision around your own answers.
• Examples per answer matter more than the total. Aim for hundreds of labeled examples per answer when you have them. Our base models are pretrained on decisions, which helps most with very few examples; with many, your data decides.
• The weakest answer sets the bar. A model is as good as its worst answer: one answer with few examples drags the whole model down. Merge answers that are rare and close, or collect more of them, before training.
• Answers that do not overlap, with a description. Each answer's description is part of what the model reads. Write them so a person would never hesitate between two.
• The input of production. Train on the text that exists when the decision is made, nothing that is only known later (a "resolved by" field when predicting the team that resolves it).
• Consistent labels. When you label the test examples, the same case gets the same answer every time: the model is measured against those answers.

Before sendingCopy Anchor Link

Send one clean table (CSV or JSON lines), not the raw export. Your AI assistant can do this for you with the skills:

1. One row per example, with a text column holding everything the decision is made on (subject and body, or the fields a person would read) and nothing else.
2. One spelling per answer: Billing, billing and Faturamento are one answer. Leave out empty rows.
3. Rare answers fixed now: merged with their closest answer, or with more examples.
4. No leaks: columns or text that only exist after the decision.
5. No exact duplicates, and no example longer than about 30,000 characters (they are refused, never cut).
6. A precise goal: the question, and one line per answer on when it applies. The closer it is, the better the first proposal.

Checking the proposalCopy Anchor Link

As soon as the proposal is ready, before labeling, go through it (the Review screen, typestrict review, or structure.stats in the API) and send what to change as feedback:

1. Examples per answer. Hundreds per answer is where the model gets good, and an answer needs at least 5 test examples to be measured. The test examples are 20% of your data (up to 2,000) when it comes with answers, or a random 150 you label when it does not: then an answer under about 3% of the data gets fewer than 5. Merge a rare answer with its closest one, or add examples of it.
2. Answers that overlap. Read the samples of each answer. If a person could swap two answers, merge them or rewrite their descriptions until the line between them is clear. When the data had no answers, low agreement between the pre-labelers points to the same problem.
3. Labels that look wrong. A sample whose answer does not match its text means the answer column is noisy: fix where the answer comes from.
4. Every warning. Each one names what to change.
5. Duplicates and leaks. The text must be what exists when the decision is made.
6. The question. Decide with your company's own answers. Large models already answer generic questions (topic, sentiment) well; your queues, teams and products are what only your data knows.

When it does not beat JevCopy Anchor Link

The comparison says why (model.jev.superiority.reason): a gain too small to be sure of, or one answer that trails Jev. More examples of the weak answers, or another base model on the same data, are the next tries. Every version is measured on the same held-out examples, so versions compare fairly.

In productionCopy Anchor Link

Use the model's confidence to decide what to automate and what to send to a person. The result reports exactly this, accuracy over the 80% most confident answers: start by automating above a threshold and move it with what you see. Ask the model the question and options it was trained on, as they are in decision_ai.json.