Glossary

Accuracy. The share of answers that are right. Simple, but it hides rare answers: a model that always answers the most common one can score high.

Anonymization (masking). Replacing personal data in your files (names, emails, phone numbers, documents, card numbers) with markers such as [EMAIL], before anything else reads them.

API key. A secret that lets your code call the API as your account. Ours start with ts_.

Base model (encoder). The model your model starts from. It already reads language; training teaches it your decision. See Base models.

Bootstrap. Measuring again on many resamples of the same test examples, to see how much a result could move by chance. The refund calculation uses 2,000 of them.

Brier score and Brier skill. How far the probabilities are from what happened. Brier skill turns it into a score from 0 to 1 where higher is better: a model that says 90% and is right scores more than one that says 51% and is right.

Calibration. Whether "80% sure" is right about 80% of the time. A calibrated model's confidence can be trusted to decide what to automate.

Confidence. How sure the model is of the answer it chose, from 0 to 1.

Context. How much text one example can hold, in tokens (pieces of words). Depends on the base model.

Credit. What a training costs. It comes back if the model does not beat Jev or the training fails.

Decision API. An API that answers questions about a text with a typed result instead of free text: a choice among options, a noul (true or false, as a probability) or a score on a scale, each with probabilities. Jev's API, which your model's container also speaks, is one.

Encoder. A model that reads a text and represents it, without writing text. Small, fast and cheap to run on a CPU, which is why it fits decisions.

Held-out (test) examples. Examples set aside before training and never seen by the model, so the measure is honest.

Jev. The market reference model for decisions, a large general model. We compare every model against it on your own examples.

Macro-F1. The F1 (a balance of precision and recall) of each answer, averaged with every answer weighing the same. Rare answers count as much as common ones.

Multi-task model. One model that makes several decisions on the same input. Measured per task against its base model.

ONNX. An open file format for models. Runs on CPUs in any language through ONNX Runtime, in servers, browsers and phones.

Quantization (8-bit). Storing the model's numbers with fewer bits. Smaller and faster, with a minimal loss.

Selective accuracy. Accuracy over only the most confident answers (80% of them in the refund calculation): what you get when you automate those and send the rest to a person.

Structuring agent. Our agent that reads your masked data and turns it into training examples for the decision you want.

Worst-class F1. The F1 of the answer the model handles worst. Keeps an average from hiding an answer that collapsed.