LTMi-XT

Layered Topological Memory Indexing. A file format that stores knowledge as single-claim records, indexes them by hashing their own hierarchy, and serves retrieval and fine-tuning from the same artifact.

Each record, a locus, is one self-contained statement with a four-level breadcrumb, a coordinate on a 64-cell lattice derived by hashing the breadcrumb prefix, byte offsets back to its source, a horizon that decays or consolidates, and a confidence score. Because the coordinate is a hash of the hierarchy, related loci cluster without an embedding model, the position can be re-derived and checked by anyone holding the file, and retrieval needs no GPU. The page below runs the whole pipeline in your browser on text you paste.

Retrieval-augmented systems must commit to a representation of retrieved knowledge. Embedding chunks and retrieving by cosine similarity fragments claims at chunk boundaries, gates retrieval behind a learned similarity function, and leaves the system with no explicit handle on atomicity, provenance or temporal validity. Atomic-fact decomposition fixes the first problem and not the others. LTMi-XT is the format SOPHIA XT converged on after building several retrieval pipelines that needed to support both breadcrumb-anchored retrieval at inference time and anchor-token-masked fine-tuning from the same data.

The format prioritises atomicity, reproducible indexing with coordinates derived from content, provenance by byte offset, an explicit short-term and long-term horizon with consolidation rules, JSONL portability with no service dependency, and dual consumption: the same bundle serves retrieval and training rows.

A bundle is a manifest and an array of loci, serialised as one JSON document or as JSONL with the manifest on the first line. The file extension is .ltmi, the recommended MIME type application/x-ltmi+json, and the JSONL form may use .ltmi.jsonl. Identifiers are content-addressed: a locus id is the BLAKE2b-128 digest of its canonical JSON, a source id the digest of the source bytes, a corpus id the digest of the sorted locus ids.

The lattice field is a pure function of the breadcrumb. Each axis hashes one more level of the path, so loci sharing k breadcrumb levels share k coordinates by construction.

def lattice_for(breadcrumb: list[str]) -> tuple[int, int, int]:
    assert len(breadcrumb) == 4
    coords = []
    for axis_idx, prefix_levels in [(0, 1), (1, 2), (2, 3)]:
        prefix = "/".join(breadcrumb[:prefix_levels]).lower()
        h = blake2b(prefix.encode("utf-8"), digest_size=16).digest()
        coord = int.from_bytes(h[:4], "big") % 64
        coords.append(coord)
    return tuple(coords)

Hash-derived coordinates are a third option beside vector embeddings and graph edges: embedding-free positioning that respects hierarchical structure. The same input gives the same position, always; nothing learned sits in the retrieval path; a position can be re-derived from the data and verified; neighbourhoods compose through Chebyshev distance without recomputing similarities. The trade-off is semantic flexibility. Two loci about one idea under different breadcrumbs land apart, so for retrieval that needs cross-hierarchy similarity the lattice acts as a pre-filter ahead of a finer semantic step.

Explicit horizon and decay give a retrieval layer a principled way to down-weight stale claims without deleting them, promote claims that keep being referenced to a longer half-life, and retire claims while keeping the audit trail. The model is drawn from accounts of episodic and semantic memory consolidation; the claim made for it is mechanical usefulness for agentic retrieval, not biological plausibility.

The TypeScript reference implementation is released under Apache 2.0 with a command-line tool for crystallisation, indexing, retrieval and training export, Python and TypeScript SDKs, an HTTP server with retrieval endpoints, a web interface for inspecting bundles, and the benchmark scripts. This site runs the same implementation behind its LTMi-XT routes, and the hosted tool crystallises with a model of your choice.

npm install @sophiaxt/ltmi-xt
sophiaxt-cli crystallize --input ./docs/ --output ./bundle.ltmi
sophiaxt-cli retrieve --bundle ./bundle.ltmi --query "What is X?"

git clone https://github.com/Chorozion/LTMi-XT.git
cd LTMi-XT && npm install && npm run build
node examples/benchmarks/micro_bench.mjs   # 10,000-trial invariant suite
python3 examples/benchmarks/run_bench.py   # live API, about 5 minutes
python3 examples/benchmarks/compare.py     # BM25 and keyword baselines

This is LTMi-XT v0.1. Backward-incompatible changes increment the major or minor version; forward-compatible additions of optional fields may occur within a minor version. The manifest's v field carries the format version and consumers should reject bundles with an unknown major version. The specification document is CC BY 4.0; the reference implementation is Apache 2.0.

@techreport{garren2026ltmixt,
  author      = {Garren, Thomas},
  title       = {LTMi-XT v0.1: A Retrieval Format with Hash-Derived
                 Topological Indexing},
  institution = {SOPHIA XT LLC},
  year        = {2026},
  month       = {May},
  url         = {https://sophiaxt.com/research/ltmi-xt}
}