SOPHIA XT
Open · Apache 2.0 · v0.1.0a1

lens-xtDeclarative spec language for token-level deterministic generation in masked-diffusion LMs

A .lensx document declares position-locked content, retrieval sources, adapter selection, and validation rules. The runtime resolves the spec into a forced-anchor decode against the chosen backend. Locked positions are excluded from the unmasking loop — they cannot be overwritten across denoising steps. Guaranteed by construction on masked-diffusion backends.

256 of 259
Tests passing
2
Backends shipped
DETERMINISTIC
Guarantee level
Apache 2.0
License

The suite collects 259 tests, and what it reports depends on the install. On a default clone without the optional PyTorch extra it runs 253 passed, 5 skipped, 1 deselected and nothing fails. With that extra present, at commit 01e5eed, ten more tests run and it reports 256 passed, 2 skipped, 1 failed. The one failure is test_variant_factory_set, which expects the variant registry to hold exactly the seven base variants when five width-sweep entries have since been added beside them. That is a stale assertion rather than a fault in the library, and it is written here because you will meet it.

60-second tour

YAML in. Token-locked output out.

1. Write a spec

medical_basic.lensx
version: "0.1"
base:
  model: "cassandra-t1.5"
locks:
  - range: [0, auto]
    source: locus("medical:cardiology:nitroglycerin:standard_dose")
generation:
  total_length: 192

2. Run it

bash
lensx validate examples/medical_basic.lensx
lensx run examples/medical_basic.lensx --backend cassandra-t1.5 --show-provenance

The locked content from the locus appears at exactly the positions you specified — guaranteed by construction on masked-diffusion backends, best-effort with logit-bias on autoregressive APIs.

Why this exists

Existing constrained-generation tools work at the wrong level.

Every other approach constrains the shape of the output. lens-xt constrains the content at specific token positions. That distinction matters when you need a literal disclaimer to appear early, or a retrieved fact to appear unchanged inside a freely-generated answer.

approachwhat it constrainsguarantee level
Prompt engineeringinformal NL hintsnone
OpenAI structured outputsschema fieldstype-level
Outlines / LMQL / Guidanceregex / CFG over outputtype-level
JSON Schema validationpost-hoc checkretry on failure
LENS-XTtoken positions in the answer slotDETERMINISTIC on MDLM, BEST_EFFORT on API

Backends + guarantees

The same spec, four guarantee levels.

The runtime picks the strongest available backend for your spec. Switch from local Cassandra to OpenAI without rewriting.

backendruns whereguarantee
Local MDLM (Cassandra T1.5, LLaDA)self-hostedDETERMINISTIC
API-compatible (OpenAI / Anthropic)remote APIBEST_EFFORT
API-native (future Mercury 2)remote APIDETERMINISTIC
Hybrid (API + local)mixedDETERMINISTIC

SDKs

Three-line drop-in. Two languages.

Python pip install lens-xt
python
from lens_xt import LensX

lens = LensX("medical_basic.lensx")
print(lens.run(user_input="What's the standard dose?"))
Node / TypeScript npm install @sophiaxt/lens-xt
typescript
import { LensX } from "@sophiaxt/lens-xt";

const lens = new LensX("medical_basic.lensx");
console.log(await lens.run({ user_input: "What's the dose?" }));

Or skip the SDK and run the CLI directly:lensx run examples/medical_basic.lensx

v0.1 status

What's shipped, what's next.

v0.1.0a1 alpha — APIs may shift before v0.1.0. End-to-end verified against Cassandra T1.5 with anchor_preservation_rate: 1.000 and DETERMINISTIC guarantee preserved.

YAML parser + AST + validatorshipped
Lock resolver (literal / locus / retrieval / compose)shipped
LTMi-XT keyword retrieval (v0.1)shipped
LTMi-XT lattice-walk retrieval (v0.2 — BLAKE2b §2.4)shipped
Reasoning scaffold runtime (multi-stage)shipped
Local MDLM backend (Cassandra T1.5)shipped
OpenAI backend (logit-bias + retry)shipped
HTTP API server (FastAPI)shipped
Python SDK (LensX class)shipped
TypeScript SDK (@sophiaxt/lens-xt)shipped
Anthropic backendnext
Mercury 2 native backendplanned

Lock the tokens that matter.

Apache 2.0. Free for any use. Hosted LENS API for production workloads.