Ari models · XerisOS Docs

XerisOS ships with Ari, an on-device large language model (XerisGPT 6.6B) that runs entirely on the CPU, inside the OS, with no cloud and no telemetry. This file explains where the weights live, how the build turns a checkpoint into a bootable disk, and the exporter↔︎loader format contract you must not break.

The weights are not in this repository. They are multi-gigabyte binaries and are deliberately git-ignored (*.pt, ari_*.bin, storage.img). XerisOS boots and runs fine without them — the Ari app simply shows "no model" and everything else (desktop, browser, editor, terminal, games) works. You only need this file if you want Ari itself to answer.

What you need#

Artifact	What it is	Where the build expects it
`ari_6.6b_eval_v3.pt` (the current checkpoint; any `.pt` from `merge_lora.py`/training)	The trained checkpoint — the source of truth for the model (~13 GB)	`MODEL_CHECKPOINT=` to `make export_model` (USB/NVMe) or `make model` (QEMU)
`xeris_tokenizer.json`	Ari's 30K BPE tokenizer	Passed via `MODEL_TOKENIZER=`
`torch` + `numpy` (host Python)	Needed only for the one-time export step	Your build machine

If you do not have a checkpoint, you cannot run Ari — training it is out of scope for a build (the training stack is xeris_gpt_v2.py and the tools/gen_*_sft.py data generators, but a fresh train is a multi-GPU, multi-day effort). Obtain the published checkpoint from wherever the maintainer distributes it, or build with NOARI=1 and skip Ari entirely.

Real hardware (USB stick / installed NVMe) — the normal path#

On a bootable stick or an installed disk there is no storage.img. The UEFI loader (boot/boot_uefi.c) reads ari_0.bin, ari_1.bin, … from the boot volume straight into RAM before the kernel starts, and the kernel's Ari app parses them from RAM when you click ARI. Three steps, run from the project root:

:: 1. checkpoint -> ari_0.bin, ari_1.bin (int4, ~4.0 GB; verifies itself)
mingw32-make export_model MODEL_CHECKPOINT=../ari_6.6b_eval_v3.pt MODEL_TOKENIZER=../xeris_tokenizer.json

:: 2. stage kernel + loader + chunks into build\usb_run  (or installer-usb)
mingw32-make run-usb

:: 3. copy the CONTENTS of build\usb_run to the root of a FAT32 stick

Optional sanity check of whatever is on the stick, no torch required:

python tools\export_split.py --verify E:\ari

Facts that matter on this path:

Export int4 (the default). The bootable OS is sized for the int4 export: ~4.0 GB of weights + ~0.8 GB of KV cache must sit in one free region below the kernel's 18 GB identity map, and the NVIDIA offload only takes int4. The int8 export (MODEL_INT8=1, ~7 GB) loads on the CPU path only. Loaders built before 2026-09-03 reserved a fixed 5 GB window, so an int8 export was silently truncated and rejected — "desktop boots, ARI says error".
Chunk names are ari_N.bin (lowercase, single digit). The old ARILLMn.BIN names are also accepted for the storage.img case.
A re-export must be re-staged. run-usb/installer-usb now re-copy a chunk when its size or mtime differs; they used to skip any file that already existed, which left the old model on the stick after a new export.
Failures now say why. The loader logs [model] files/header/best base/need pages on COM1 and puts a one-line reason on the splash; the kernel posts ari: model rejected - <reason> on the HUD instead of a bare "Error loading model".
The machine needs ~5 GB of contiguous free RAM below 18 GB for int4 (~7.8 GB for int8) on top of everything else; 16 GB total is fine for int4.

Hot-swapping Ari (no reboot, no rebuild)#

Since 2026-09-03 the model is a runtime choice. Three things work together (src/core/ari_models.c, boot/boot_uefi.c, src/apps/ai_engine.c):

Drop-in models. Put an export in a folder on the XerisOS volume — USB copy is fine — as \MODELS\<NAME>\ari_0.bin, ari_1.bin, … (NAME ≤ 8 letters/ digits; it is a FAT 8.3 name). The root ari_N.bin set is the model called default. \ARIMODEL.TXT (one line: the folder name) tells the loader which one to preload at boot, so switching the boot model is editing one line, not rebuilding the stick.

Runtime swap. In the Terminal:

model                  what is live, what is on the volume
model use CODER        stream /MODELS/CODER in and switch to it (writes ARIMODEL.TXT)
model use default      back to the root set

The loader reserves one RAM window at boot (weights + KV cache + a 256 MB adapter arena) and stashes it for the kernel (0x5A0 "ARIWINDW", base at 0x5A8, size at 0x5B0); the swap streams the new chunks into that window through the kernel's own FAT32 driver and re-parses. The desktop pauses while the bytes stream (seconds from NVMe, a minute or two from USB 2) — the read path cannot pump the UI. A new model must fit the window: model prints the window size, and the swap refuses with the numbers if it does not. The old model is overwritten in place, so a swap that fails part-way leaves no model until model use … succeeds again. NVIDIA offload survives a swap between same-shape models (the int4 linears are re-uploaded over the old VRAM allocations); a different shape turns the offload off for the session.

LoRA adapters. Export a train_lora.py adapter once, then swap personas in under a second:

mingw32-make export_lora LORA=../lora_tool_v1.pt NAME=TOOL     -> TOOL.XLA (~71 MB for r=16)
:: copy TOOL.XLA to \LORAS\ on the volume, then in the Terminal:
model lora TOOL        attach (writes ARILORA.TXT: re-attached after every boot/swap)
model lora off         base model only

The base weights are never modified: the engine computes y = Wx + scale·B(Ax) after each of the five linears per layer (lora_apply in ai_engine.c), ~1 M extra MACs per layer against ~200 M for the base, bf16 A/B. Attaching is a pointer flip; two arena slots alternate so a failed read never disturbs the live adapter. Rank ≤ 64 (LORA_MAX_R), adapter ≤ 128 MB.

QEMU only: building the model into storage.img#

Exporting is a one-time step that converts the PyTorch checkpoint into the five FAT32-safe chunks the kernel loads, and writes them into storage.img. It takes roughly an hour and only has to be run once; storage.img then persists.

Windows (default Makefile):

:: delete any stale artifacts first
del /q storage.img ari_0.bin ari_1.bin ari_2.bin ari_3.bin ari_4.bin

mingw32-make model ^
  MODEL_CHECKPOINT=C:/path/to/ari_6.6b_base_sft_v13.pt ^
  MODEL_TOKENIZER=C:/path/to/xeris_tokenizer.json ^
  PY=python

macOS / Linux (Makefile.universal):

rm -f storage.img ari_0.bin ari_1.bin ari_2.bin ari_3.bin ari_4.bin
make -f Makefile.universal model \
  MODEL_CHECKPOINT=/path/to/ari_6.6b_base_sft_v13.pt \
  MODEL_TOKENIZER=/path/to/xeris_tokenizer.json

This creates a 14 GB sparse FAT32 disk (instant — a few KB physical until written), exports five chunks with tools/export_split.py, and writes each into the image with scripts/fat32_write.py, deleting each chunk after a successful write so you never need the full weights twice on disk at once.

fat32_write.py can take several minutes per chunk (it rescans the FAT) — that is not a hang.

Running with the model#

# Windows, hardware-accelerated (Windows Hypervisor Platform must be enabled)
mingw32-make run ACCEL=whpx

# portable / always-boots fallback (software TCG — much slower)
make run

You want a machine with ≥ 32 GB RAM: the kernel maps the model into RAM at physical 0x100000000 (4 GB), above the PCI MMIO hole, and QEMU is launched with -m 18G -mem-prealloc. On a successful load the HUD posts "ari: model loaded" and the Ari app answers. Inference is CPU-only and slow, especially under emulation — see ARI_ROADMAP.md for the speed roadmap (SMP bring-up, dual-channel RAM, int8 head, prefill batching).

The file-format contract (do not let the exporter and loader drift)#

Both exporters — tools/export_split.py (chunked, required for FAT32) and tools/export_xerisgpt.py (single file, too big for FAT32; reference only) — MUST emit exactly the header that src/apps/ai_engine.c (ai_load_llm) reads:

offset 0  : u32 magic   = 0x58474C4D   ("XGLM")
offset 4  : u32 version = 2
offset 8  : u32 num_layers
offset 12 : u32 embed_dim
offset 16 : u32 num_heads
offset 20 : u32 vocab_size
offset 24 : u32 max_seq
offset 28 : u32 ff_dim
offset 32 : u32 vocab_len      <- length of JSON(vocab)
offset 36 : u32 merge_len      <- length of JSON(merges)
offset 40 : vocab_data (vocab_len bytes)
          : merges_data (merge_len bytes)
          : weights start here = 40 + vocab_len + merge_len

weights order: tok_emb, pos_emb, then per layer
   [ln1, attn.qkv, attn.out_proj, ln2, ff.w1, ff.w2, ff.w3], then ln_f

The chunks are written as FAT32 8.3 names ARILLM0 BIN … ARILLM4 BIN (exactly 11 bytes each, note the spaces).

The one rule that has bitten this project before: the loader validates only the magic number. A wrong header does not error — it mis-points the weight pointer and produces garbage output or a crash while still reporting "Ari online." If you change the format, change all three sides (export_split.py, export_xerisgpt.py, ai_engine.c) and verify by simulating the exporter and loader together so the computed weight offset matches the actual one.

Quantization#

The layer weights are int8 (Q8 blocks); the SwiGLU FFN and attention run through the AVX2 matmul in ai_accel.asm. The weight-tied output head can optionally be int8-quantized as well (roadmap item S3) — re-export to activate it, and the loader auto-detects it from the header. Do not use int16_t in ai_engine.c; the project typedef is i16 (signed short).

If you just want to try XerisOS without Ari#

make run NOARI=1 MEM=4G

This is the fastest boot and needs no checkpoint, no export, and only 4 GB of RAM. Everything except Ari works.