Run it anywhere
Your model as a container that answers like Jev, on Docker, Cloud Run, AWS Lambda, Firebase and Kubernetes.
Coming soon
The container is being built; today a model downloads as ONNX, weights and package. This page is its specification. The serving runtime is open source (Apache-2.0).
Each model also ships as a container image: the model, a CPU runtime (ONNX Runtime) and an HTTP server, nothing to configure. It speaks the same decision API as Jev, so switching from Jev to your own model is changing a base URL.
| Endpoint | What it does |
|---|---|
POST /v1/systemone | Answers questions about a text, in Jev's request and response format. |
GET /v1/models | The model inside, by name. |
GET /healthz | ok when ready, for health checks. |
| Variable | Default | What it does |
|---|---|---|
PORT | 8080 | The port to listen on (Cloud Run and Lambda set it). |
TYPESTRICT_TOKEN | none | When set, every request must carry Authorization: Bearer <token>. Prefer your platform's own auth (Cloud Run IAM, Lambda IAM) when you have it. |
No GPU, no internet access and no calls to us: requests and data stay in your infrastructure.
Docker
typestrict pull <model> --container # or: docker load -i <model>.tar (download link on the model page)
docker run -p 8080:8080 typestrict/<model>curl http://localhost:8080/v1/systemone -H "Content-Type: application/json" -d '{
"model": "ticket-router",
"state": "I was charged twice. Please help.",
"questions": {
"team": {
"type": "choice",
"instructions": "Which team handles the ticket?",
"criteria": { "billing": "Payments and invoices", "support": "Everything else" }
}
}
}'
# an answer looks like:
# { "model": "ticket-router", "answers": { "team": { "type": "choice", "choice": "billing", "confidence": 0.97,
# "probabilities": { "billing": 0.97, "support": 0.03 } } }, "usage": { "input_tokens": 41, "output_tokens": 0 } }Google Cloud Run
Cloud Run runs images from Artifact Registry. Copy the image there, then deploy:
typestrict pull <model> --to us-docker.pkg.dev/<project>/<repo>/ticket-router
gcloud run deploy ticket-router \
--image us-docker.pkg.dev/<project>/<repo>/ticket-router \
--region us-central1 --cpu 1 --memory 1Gi \
--no-allow-unauthenticated--no-allow-unauthenticated keeps it private: callers need an identity token from a service account with the
roles/run.invoker role. Set --min-instances 1 if the first request after a quiet period must be fast.
AWS Lambda
The image carries the AWS Lambda Web Adapter, so the same image runs on Lambda unchanged. Lambda runs images from ECR in the same account and region:
typestrict pull <model> --to <account>.dkr.ecr.us-east-1.amazonaws.com/ticket-router
aws lambda create-function --function-name ticket-router \
--package-type Image \
--code ImageUri=<account>.dkr.ecr.us-east-1.amazonaws.com/ticket-router:latest \
--role arn:aws:iam::<account>:role/<lambda-role> \
--memory-size 2048 --timeout 30
aws lambda create-function-url-config --function-name ticket-router --auth-type AWS_IAMOn Lambda, CPU grows with memory: 2 GB is a good start. The first request after a cold start loads the model.
Firebase
Firebase Functions do not run your own images, so the model runs on Cloud Run (above) and Firebase reaches it:
-
Hosting: route a path straight to the Cloud Run service with a rewrite in
firebase.json:{ "hosting": { "rewrites": [{ "source": "/api/decide", "run": { "serviceId": "ticket-router", "region": "us-central1" } }] } } -
Functions: call the Cloud Run service from your function with an identity token, as any other HTTP API.
Kubernetes
apiVersion: apps/v1
kind: Deployment
metadata: { name: ticket-router }
spec:
replicas: 2
selector: { matchLabels: { app: ticket-router } }
template:
metadata: { labels: { app: ticket-router } }
spec:
containers:
- name: model
image: <your-registry>/ticket-router
ports: [{ containerPort: 8080 }]
resources: { requests: { cpu: "1", memory: 1Gi }, limits: { memory: 2Gi } }
readinessProbe: { httpGet: { path: /healthz, port: 8080 } }
livenessProbe: { httpGet: { path: /healthz, port: 8080 }, periodSeconds: 30 }
---
apiVersion: v1
kind: Service
metadata: { name: ticket-router }
spec:
selector: { app: ticket-router }
ports: [{ port: 80, targetPort: 8080 }]The model holds no state, so it scales out with replicas (or a HorizontalPodAutoscaler on CPU).