sysone/README.md

# sysone

A local, self-hosted typed-decision classifier. Instead of generating text,
it takes a `state` (some text or data) and a set of typed `questions`, and
returns calibrated probabilities:

- **Noul** — a yes/no question, answered as P(true).
- **Choice** — pick one of several labeled options, answered as a probability
  per option.
- **Score** — place the input on an ordered scale, answered as a continuous
  position on that scale plus a probability per level.

No chat model, no text generation, no hallucination risk — the model can
only ever return a number or a label from a set you defined. Runs entirely
on this machine: no API key, no external service, no per-call cost.

## Architecture

```
sysone/ (this repo, stdlib-only, no dependencies)
    types.py    Choice / Score / Noul question specs + result types
    client.py   Classifier.invoke(state, questions) -> ClassifierResponse

        │  HTTP POST /v1/systemone
        ▼

von serve (external process, ~/work/von/.venv; not part of this repo)
    von-sdk's own built-in HTTP server -- speaks /v1/systemone natively,
    no wrapper needed.
    → localhost:8011

von-1.0.0 (395M-param ModernBERT encoder, Apache 2.0, runs on MPS)
```

## Setup

**If the model server is already running**, skip to Usage. Check:

```bash
curl -s localhost:8011/v1/systemone -o /dev/null -w '%{http_code}\n' \
  -H 'content-type: application/json' \
  -d '{"state":"x","model":"von","questions":{"q":{"type":"noul","instructions":"x"}}}'
```

**From scratch:**

```bash
mkdir -p ~/work/von && cd ~/work/von
uv venv --python 3.12   # von-sdk requires Python >=3.12
uv pip install von-sdk
```

Run the server:

```bash
~/work/von/.venv/bin/von serve --port 8011 --backend modernbert --device mps
```

First run downloads the `von-1.0.0` checkpoint from Hugging Face (~1.5GB)
and serves on MPS (Apple Silicon GPU). Leave it running in its own terminal,
or background it. It's a plain background process, not a managed service —
if the machine restarts, re-run the command above. First startup after a
restart takes a few seconds to load the model; each request after that is
20-50ms.

## Usage

```python
from sysone import Classifier, Choice, Noul, Score

classifier = Classifier()  # defaults to http://localhost:8011

response = classifier.invoke(
    state="Our production database is down and customers can't check out.",
    questions={
        "urgent": Noul(instructions="Does this need attention right now?"),
        "team": Choice(instructions="Which team should handle this?",
                        options=["billing", "technical", "sales"]),
        "severity": Score(instructions="How severe is this?",
                           levels=["minor", "moderate", "major", "critical"]),
    },
)

response.nouls["urgent"].noul          # -> probability, e.g. 0.94
response.choices["team"].best          # -> "technical"
response.choices["team"].probs         # -> {"billing": 0.03, "technical": 0.92, "sales": 0.05}
response.scores["severity"].score      # -> continuous index into `levels`, e.g. 2.6
```

Run the example and smoke test:

```bash
python3 examples/support_ticket.py   # a worked example
python3 smoke_test.py                # noul/choice/score sanity checks
```

## Wire format (`/v1/systemone`)

| Type | Request `criteria` | Response fields |
|---|---|---|
| `noul` | none | `noul`: P(yes) |
| `choice` | `{option: description}` | `choice`, `probabilities` (by option key), `confidence` |
| `score` | `[level_0, level_1, ...]` (low → high) | `score` (0-indexed position into levels), `legend`, `probabilities` (by index), `confidence` |

## Known limitations

- **Noul and Choice are not equally strong.** This client's own testing found
  yes/no (Noul) judgments to be sharp and well-directed even on pragmatic,
  non-factual questions (e.g. urgency). Multi-way Choice judgments on
  fine-grained, domain-specific categories (tested: classifying software
  tasks by scope) were noticeably weaker, with a real tendency to
  under-predict how much a task actually requires. If your use case leans
  heavily on Choice with many closely-related options, verify against your
  own labeled examples before trusting it.
- **Raw confidence isn't necessarily calibrated for your task.** The
  `confidence` field this client returns is the model's raw output for
  whatever domain it was trained on. Fitting your own temperature scalar on
  a held-out set is worth doing before using confidence for anything
  threshold-based (e.g. "auto-act above 0.85, else escalate").
- **High-cardinality choice questions are untested here.** All examples in
  this repo use ≤5 options; large option sets are unverified.
- **English-only.** Not exercised with non-English input.

## Possible next steps

- Fine-tune or otherwise adapt the Choice head for domain-specific
  multi-category tasks if that's your primary use case.
- Fit per-question-type temperature scaling on a held-out set before relying
  on `confidence` for threshold-based decisions.
- A LangChain-style wrapper and middleware (model routing, tool-call
  guardrails) built on top of this classifier are both natural extensions,
  not implemented here — this repo is the core classifier only.