lenslapse package¶

Submodules¶

lenslapse.add_model module¶

One-command model onboarding for LensLapse.

Runs the full pipeline for a new model — export ONNX pairs, precompute lens shards, install the tokenizer for the web app, and register the model in web/public/data/models.json — so that adding a model is a data change, not a code change.

Supported sources (see sources.py):

Hub suite    : python add_model.py --model EleutherAI/pythia-31m --id pythia-31m --label "Pythia 31M"                      --steps 0,512,8000,143000 --models-root /path/to/models
Hub single   : python add_model.py --model gpt2 --id gpt2 --label "GPT-2 124M" --final-only ...
Local run dir: python add_model.py --model /path/to/trainer_output --id my-run --label "My run" ...
               (Hugging Face Trainer layout: checkpoint-<step>/ subdirectories)
Hub subfolder: python add_model.py --model m-a-p/neo_scalinglaw_250M --id mapneo-250m --label "MAP-Neo"                      --subfolder-map "16780:hf_ckpt/16.78B,33550:hf_ckpt/33.55B" --models-root /path/to/models
               (checkpoints nested as subfolders within one revision; overrides --steps)

After it finishes: rebuild the web app, and upload <models-root>/<id>/ to your static model host.

lenslapse.add_model.run(module, args)[source]¶

Parameters:

module (str)
args (list[str])

Return type:

None

class lenslapse.add_model.AddModelConfig(*, model, id, label, models_root, steps='0,1,2,4,8,16,32,64,128,256,512,1000,2000,4000,8000,16000,32000,64000,128000,143000', final_only=False, skip_export=False, force=False, subfolder_map=None, prompts_file=None, revision_template='step{}', tokenizer_ref=None)[source]¶

Bases: BaseModel

Validated arguments for add_model; see main’s docstring for what each means.

Parameters:

model (str)
id (str)
label (str)
models_root (Path)
steps (str)
final_only (bool)
skip_export (bool)
force (bool)
subfolder_map (str | None)
prompts_file (str | None)
revision_template (str)
tokenizer_ref (str | None)

model: str¶

id: str¶

label: str¶

models_root: Path¶

steps: str¶

final_only: bool¶

skip_export: bool¶

force: bool¶

subfolder_map: str | None¶

prompts_file: str | None¶

revision_template: str¶

tokenizer_ref: str | None¶

model_config = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

lenslapse.add_model.main(model, id, label, models_root, steps='0,1,2,4,8,16,32,64,128,256,512,1000,2000,4000,8000,16000,32000,64000,128000,143000', final_only=False, skip_export=False, force=False, subfolder_map=None, prompts_file=None, revision_template='step{}', tokenizer_ref=None)[source]¶

Onboard a new model: export ONNX pairs, precompute lens shards, install the tokenizer, and register the model in web/public/data/models.json.

Parameters:

model (str) – HF id or local directory.
id (str) – model id used by the app (directory name).
label (str) – display name in the model picker.
models_root (str) – directory that holds <id>/step*/… for the model host.
steps (str) – hub suites only; local Trainer dirs always use every checkpoint-* they contain.
final_only (bool) – single checkpoint (revision “main”) instead of a step suite.
skip_export (bool) – only precompute + register.
force (bool) – overwrite an id that is already registered.
subfolder_map (str | None) – “step:path,step:path,…” for repos that nest checkpoints as subfolders of one revision instead of using git revisions per checkpoint; overrides steps.
prompts_file (str | None) – JSON curated-prompt list; see precompute_lens.py.
revision_template (str) – revision naming for hub suites, e.g. “global_step{}” for bigscience/bloom-*-intermediate.
tokenizer_ref (str | None) – load the tokenizer from a different ref than the checkpoint weights; see export_checkpoints.py.

Return type:

None

lenslapse.arch module¶

Architecture introspection: locate the pieces the lens recipe needs on any HF decoder LM.

A generic attribute-name heuristic resolves most decoder layouts with no configuration at all:

GPT-NeoX / Pythia : model.gpt_neox.layers, final_layer_norm (LayerNorm), embed_out Llama / OLMo-2 / SmolLM / Qwen : model.model.layers, model.model.norm (RMSNorm), lm_head GPT-2 : model.transformer.h, transformer.ln_f (LayerNorm), lm_head (tied)

For a layout the heuristic can’t (or shouldn’t have to) guess — e.g. a decoder stack nested more than one hop deep, or an attribute name outside the common set below — register an explicit ArchSpec keyed by the model’s own config.model_type (a stable identifier every Transformers architecture defines) via register_architecture(). resolve() checks the registry first, so adding a new architecture never requires touching the generic heuristic or its ordering:

from lenslapse.arch import register_architecture register_architecture(“my_new_model”, base_path=”model.decoder”)

The lens identity — lens(last block output) == model logits — holds for pre-LN decoders where the LM applies final_norm then lm_head to the last block’s output. export_checkpoints.py asserts it per checkpoint, so an unsupported layout fails loudly instead of shipping a wrong lens.

class lenslapse.arch.ArchHandles(*, base, layers, final_norm, lm_head, norm_type, eps)[source]¶

Bases: BaseModel

Parameters:

base (Module)
layers (ModuleList)
final_norm (Module)
lm_head (Module)
norm_type (str)
eps (float)

model_config = {'arbitrary_types_allowed': True}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

base: Module¶

layers: ModuleList¶

final_norm: Module¶

lm_head: Module¶

norm_type: str¶

eps: float¶

class lenslapse.arch.ArchSpec(*, base_path, layer_attrs=('layers', 'h'), norm_attrs=('final_layer_norm', 'norm', 'ln_f'))[source]¶

Bases: BaseModel

Explicit attribute paths for one config.model_type, used instead of (not blended with) the generic heuristic below. base_path is dotted for a stack nested more than one hop deep (e.g. “model.decoder”); layer_attrs/norm_attrs default to the same names the generic heuristic already tries, since only the base path usually needs to be architecture-specific.

Parameters:

base_path (str)
layer_attrs (tuple[str, ...])
norm_attrs (tuple[str, ...])

base_path: str¶

layer_attrs: tuple[str, ...]¶

norm_attrs: tuple[str, ...]¶

model_config = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

lenslapse.arch.register_architecture(model_type, base_path, layer_attrs=None, norm_attrs=None)[source]¶

Register explicit attribute paths for model_type (from the model’s own config.model_type — e.g. “opt”, “gemma3_text”; check AutoConfig.from_pretrained(ref).model_type if unsure).

Call this — from anywhere, before resolve() runs — to add support for a new Transformers decoder architecture without editing this module. Once registered, resolve() uses only these paths for that model_type; it does not also try the generic heuristic, so a wrong override fails loudly (an unresolved path is a bug in the registration, not something to silently paper over by guessing).

Raises ValueError if model_type is already registered — silently overwriting a previous registration would let a copy-paste mistake elsewhere in the process quietly change which architecture an already-working model_type resolves as.

Parameters:

model_type (str)
base_path (str)
layer_attrs (tuple[str, ...] | None)
norm_attrs (tuple[str, ...] | None)

Return type:

None

lenslapse.arch.resolve(model)[source]¶

Parameters:: model (Module)
Return type:: ArchHandles

lenslapse.check_arch_parity module¶

Cross-architecture parity check: does the lens recipe hold for a given model?

Exports the (backbone, lens) pair to a temp dir and asserts:

torch: lens(last block output) == model logits (the identity the whole demo rests on)
ONNX fp32 matches torch
f16-stored weights keep top-1 at every position of the probe

Usage:

python check_arch_parity.py –model gpt2 python check_arch_parity.py –model HuggingFaceTB/SmolLM2-135M python check_arch_parity.py –model EleutherAI/pythia-70m –revision step1000

class lenslapse.check_arch_parity.CheckArchParityConfig(*, model, revision='main')[source]¶

Bases: BaseModel

Validated arguments for check_arch_parity; see main’s docstring for what each means.

Parameters:

model (str)
revision (str)

model: str¶

revision: str¶

model_config = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

lenslapse.check_arch_parity.main(model, revision='main')[source]¶

Check that the lens recipe (lens(last block output) == model logits) holds for a model.

Parameters:

model (str) – HF id or local directory.
revision (str) – HF revision to check, e.g. a training-checkpoint tag.

Return type:

None

lenslapse.cli module¶

lenslapse command-line entry point.

Designed for people who just want to point the hosted web app at their own models:

pip install lenslapse lenslapse server # starts the probe server and opens the app in the browser

Subcommands delegate to the underlying modules, so every flag they document works here too (lenslapse server –port 9000, lenslapse add-model –model gpt2 …).

lenslapse.cli.main()[source]¶

Return type:: None

lenslapse.client module¶

Terminal access to everything the web app’s UI can do.

Every interactive feature has a scriptable counterpart:

lenslapse models add –ref /path/to/trainer_output –id my-run # the “models” dialog lenslapse probe –model my-run –text “The capital of France is” # the “Live probe” button lenslapse trace –model my-run –text “The capital of France is” # the “trace across training” button lenslapse models convert my-run # the “convert to ONNX” button

Commands drive a running lenslapse server when one is reachable (default port 8017), sharing its registry, loaded weights, and probe cache; otherwise the same code runs in-process, writing to the same on-disk state. Either way a result computed here replays instantly in the app’s “live - server” mode, and vice versa. –json prints the exact payload the web app receives.

class lenslapse.client.Backend(*args, **kwargs)[source]¶

Bases: Protocol

What a command needs from either a running server or an in-process fallback.

where: str¶

probe(model, step, text, targets=None)[source]¶

Parameters:

model (str)
step (int)
text (str)
targets (list[int] | None)

Return type:

dict[str, Any]

models()[source]¶

Return type:: list[dict[str, Any]]

add(payload)[source]¶

Parameters:: payload (dict[str, Any])
Return type:: dict[str, Any]

remove(model_id)[source]¶

Parameters:: model_id (str)
Return type:: dict[str, Any]

convert_start(model_id)[source]¶

Parameters:: model_id (str)
Return type:: dict[str, Any]

convert_status(model_id)[source]¶

Parameters:: model_id (str)
Return type:: dict[str, Any]

lenslapse.client.server_alive(base)[source]¶

Parameters:: base (str)
Return type:: bool

class lenslapse.client.HttpBackend(base)[source]¶

Bases: object

Drives a running probe server over HTTP — exactly what the web app does.

Parameters:: base (str)

probe(model, step, text, targets=None)[source]¶

Parameters:

model (str)
step (int)
text (str)
targets (list[int] | None)

Return type:

dict[str, Any]

models()[source]¶

Return type:: list[dict[str, Any]]

add(payload)[source]¶

Parameters:: payload (dict[str, Any])
Return type:: dict[str, Any]

remove(model_id)[source]¶

Parameters:: model_id (str)
Return type:: dict[str, Any]

convert_start(model_id)[source]¶

Parameters:: model_id (str)
Return type:: dict[str, Any]

convert_status(model_id)[source]¶

Parameters:: model_id (str)
Return type:: dict[str, Any]

class lenslapse.client.LocalBackend(device_map=False, dtype='float32')[source]¶

Bases: object

Runs the same code paths as the probe server, in this process (no HTTP, no second port).

Shares the server’s on-disk state at the default locations — registry file and probe cache — so a model registered here appears in the web dialog after the next server start, and a probe computed here replays instantly in the browser (and vice versa). Registry writes are last-writer-wins against a concurrently running server: prefer driving the server over HTTP (the default) for add/remove while one is up.

Parameters:

device_map (bool)
dtype (Literal['float32', 'float16', 'bfloat16', 'auto'])

where = 'in-process'¶

probe(model, step, text, targets=None)[source]¶

Parameters:

model (str)
step (int)
text (str)
targets (list[int] | None)

Return type:

dict[str, Any]

models()[source]¶

Return type:: list[dict[str, Any]]

add(payload)[source]¶

Parameters:: payload (dict[str, Any])
Return type:: dict[str, Any]

remove(model_id)[source]¶

Parameters:: model_id (str)
Return type:: dict[str, Any]

convert_start(model_id)[source]¶

Parameters:: model_id (str)
Return type:: dict[str, Any]

convert_status(model_id)[source]¶

Parameters:: model_id (str)
Return type:: dict[str, Any]

class lenslapse.client.ConnectionConfig(*, server=None, local=False, device_map=False, dtype='float32')[source]¶

Bases: BaseModel

Which backend to drive — shared by every command; see probe’s docstring for each field.

Parameters:

server (str | None)
local (bool)
device_map (bool)
dtype (Literal['float32', 'float16', 'bfloat16', 'auto'])

server: str | None¶

local: bool¶

device_map: bool¶

dtype: Literal['float32', 'float16', 'bfloat16', 'auto']¶

model_config = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

lenslapse.client.choose_backend(conn)[source]¶

Parameters:: conn (ConnectionConfig)
Return type:: Backend

lenslapse.client.parse_targets(spec)[source]¶

Parameters:: spec (str | None)
Return type:: list[int] | None

lenslapse.client.infer_mode(ref, steps)[source]¶

Mirror the registration dialog’s radio buttons without making the user pick one.

Parameters:

ref (str)
steps (str | None)

Return type:

Literal[‘suite’, ‘final’, ‘local’]

lenslapse.client.model_entry(backend, model_id)[source]¶

Parameters:

backend (Backend)
model_id (str)

Return type:

dict[str, Any]

class lenslapse.client.ProbeCliConfig(*, server=None, local=False, device_map=False, dtype='float32', model, text, step=None, pos=None, targets=None, json_output=False)[source]¶

Bases: ConnectionConfig

Validated arguments for probe; see probe’s docstring for what each means.

Parameters:

server (str | None)
local (bool)
device_map (bool)
dtype (Literal['float32', 'float16', 'bfloat16', 'auto'])
model (str)
text (str)
step (int | None)
pos (int | None)
targets (str | None)
json_output (bool)

model: str¶

text: str¶

step: int | None¶

pos: int | None¶

targets: str | None¶

json_output: bool¶

model_config = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

lenslapse.client.cmd_probe(backend, cfg)[source]¶

Parameters:

backend (Backend)
cfg (ProbeCliConfig)

Return type:

None

class lenslapse.client.TraceCliConfig(*, server=None, local=False, device_map=False, dtype='float32', model, text, pos=None, layer=None, top=3, targets=None, json_output=False)[source]¶

Bases: ConnectionConfig

Validated arguments for trace; see trace’s docstring for what each means.

Parameters:

server (str | None)
local (bool)
device_map (bool)
dtype (Literal['float32', 'float16', 'bfloat16', 'auto'])
model (str)
text (str)
pos (int | None)
layer (int | None)
top (int)
targets (str | None)
json_output (bool)

model: str¶

text: str¶

pos: int | None¶

layer: int | None¶

top: int¶

targets: str | None¶

json_output: bool¶

model_config = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

lenslapse.client.cmd_trace(backend, cfg)[source]¶

Parameters:

backend (Backend)
cfg (TraceCliConfig)

Return type:

None

lenslapse.client.probe(model, text, step=None, pos=None, targets=None, json=False, server=None, local=False, device_map=False, dtype='float32')[source]¶

One logit-lens pass — the UI’s “Live probe” button.

Parameters:

model (str) – registered model id (see lenslapse models).
text (str) – prompt text.
step (int | None) – checkpoint step (default: the model’s last).
pos (int | None) – token position to print (default: last).
targets (str | None) – comma-separated token ids to track exactly.
json (bool) – print the full probe payload (what the web app receives).
server (str | None) – probe server to drive (default: auto-detect http://localhost:8017, else run in-process).
local (bool) – run in-process even if a probe server is running.
device_map (bool) – in-process only: load with device_map=’auto’.
dtype (Literal['float32', 'float16', 'bfloat16', 'auto']) – in-process only: compute dtype (float32 matches the shards; lower halves memory).

Return type:

None

lenslapse.client.trace(model, text, pos=None, layer=None, top=3, targets=None, json=False, server=None, local=False, device_map=False, dtype='float32')[source]¶

Probe every checkpoint — the UI’s “trace across training” button.

Parameters:

model (str) – registered model id (see lenslapse models).
text (str) – prompt text.
pos (int | None) – token position to trace (default: last).
layer (int | None) – layer whose probability the table shows (default: final).
top (int) – how many final-checkpoint top tokens to track (default: 3).
targets (str | None) – track these token ids instead of the final-checkpoint top-k.
json (bool) – print the whole trajectory as JSON (table goes away).
server (str | None) – probe server to drive (default: auto-detect http://localhost:8017, else run in-process).
local (bool) – run in-process even if a probe server is running.
device_map (bool) – in-process only: load with device_map=’auto’.
dtype (Literal['float32', 'float16', 'bfloat16', 'auto']) – in-process only: compute dtype (float32 matches the shards; lower halves memory).

Return type:

None

class lenslapse.client.ModelsAddCliConfig(*, server=None, local=False, device_map=False, dtype='float32', ref, id, label=None, mode=None, steps=None)[source]¶

Bases: ConnectionConfig

Validated arguments for models add; see ModelsCommands.add’s docstring for what each means.

Parameters:

server (str | None)
local (bool)
device_map (bool)
dtype (Literal['float32', 'float16', 'bfloat16', 'auto'])
ref (str)
id (str)
label (str | None)
mode (Literal['suite', 'final', 'local'] | None)
steps (str | None)

ref: str¶

id: str¶

label: str | None¶

mode: Literal['suite', 'final', 'local'] | None¶

steps: str | None¶

model_config = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class lenslapse.client.ModelsIdCliConfig(*, server=None, local=False, device_map=False, dtype='float32', id)[source]¶

Bases: ConnectionConfig

Validated arguments for models remove/models convert: a model id + connection flags.

Parameters:

server (str | None)
local (bool)
device_map (bool)
dtype (Literal['float32', 'float16', 'bfloat16', 'auto'])
id (str)

id: str¶

model_config = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class lenslapse.client.ModelsCommands[source]¶

Bases: object

lenslapse models [list|add|remove|convert] — the UI’s “models” dialog.

list(server=None, local=False, device_map=False, dtype='float32')[source]¶

Show every registered model.

Parameters:

server (str | None) – probe server to drive (default: auto-detect http://localhost:8017, else run in-process).
local (bool) – run in-process even if a probe server is running.
device_map (bool) – in-process only: load with device_map=’auto’.
dtype (Literal['float32', 'float16', 'bfloat16', 'auto']) – in-process only: compute dtype (float32 matches the shards; lower halves memory).

Return type:

None

add(ref, id, label=None, mode=None, steps=None, server=None, local=False, device_map=False, dtype='float32')[source]¶

Parameters:

ref (str) – HF id or local checkpoint directory.
id (str) – id the app shows in its model picker.
label (str | None) – display label; defaults to id.
mode (Literal['suite', 'final', 'local'] | None) – default: local if ref is a directory, suite if steps is given, else final.
steps (str | None) – suite only: comma-separated step numbers.
server (str | None) – probe server to drive (default: auto-detect http://localhost:8017, else run in-process).
local (bool) – run in-process even if a probe server is running.
device_map (bool) – in-process only: load with device_map=’auto’.
dtype (Literal['float32', 'float16', 'bfloat16', 'auto']) – in-process only: compute dtype (float32 matches the shards; lower halves memory).

Return type:

None

remove(id, server=None, local=False, device_map=False, dtype='float32')[source]¶

Unregister a model.

Parameters:

id (str) – registered model id.
server (str | None) – probe server to drive (default: auto-detect http://localhost:8017, else run in-process).
local (bool) – run in-process even if a probe server is running.
device_map (bool) – in-process only: load with device_map=’auto’.
dtype (Literal['float32', 'float16', 'bfloat16', 'auto']) – in-process only: compute dtype (float32 matches the shards; lower halves memory).

Return type:

None

convert(id, server=None, local=False, device_map=False, dtype='float32')[source]¶

ONNX-convert a registered model for in-browser use.

Parameters:

id (str) – registered model id.
server (str | None) – probe server to drive (default: auto-detect http://localhost:8017, else run in-process).
local (bool) – run in-process even if a probe server is running.
device_map (bool) – in-process only: load with device_map=’auto’.
dtype (Literal['float32', 'float16', 'bfloat16', 'auto']) – in-process only: compute dtype (float32 matches the shards; lower halves memory).

Return type:

None

lenslapse.export_checkpoints module¶

Export Pythia training checkpoints as browser-runnable ONNX pairs (backbone + lens head).

For each requested training step, produces:

out_dir/step{N}/backbone.f16.onnx  input_ids, attention_mask -> hidden_states [L+1, B, T, H] (pre-ln_f, uniform)
out_dir/step{N}/lens.f16.onnx      hidden [N, H] -> logits [N, V]  (final_layer_norm + unembedding)

and a top-level manifest.json.

Design notes (validated 2026-07-13, see experiments/feasibility-note.md):

HF GPTNeoX applies final_layer_norm to the last entry of output_hidden_states, so a uniform lens head would double-normalize it. Forward hooks capture each block’s raw (pre-ln) output instead; lens(hidden[-1]) then equals the model’s logits exactly.
The lens head is built by hand with onnx.helper (LayerNormalization + MatMul): torch.onnx.export emits a graph that fails ORT shape inference during quantization.
opset >= 18 is required by the torch dynamo exporter (Split num_outputs).
Weights are stored as fp16 + Cast (see onnx_f16.py); compute stays fp32. Dynamic int8 was rejected by fidelity_eval.py: final-layer top-1 agreement vs fp32 drops to 52-76% at late checkpoints, while fp16 weight storage keeps 100.0% agreement at half the fp32 size.

class lenslapse.export_checkpoints.Backbone(m)[source]¶

Bases: Module

Parameters:: m (AutoModelForCausalLM)

forward(input_ids, attention_mask)[source]¶

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Parameters:

input_ids (Tensor)
attention_mask (Tensor)

Return type:

Tensor

lenslapse.export_checkpoints.build_lens_onnx(model, path)[source]¶

Lens head = final norm (LayerNorm or RMSNorm, composed from primitive ops) + unembedding.

Parameters:

model (AutoModelForCausalLM)
path (Path)

Return type:

None

lenslapse.export_checkpoints.export_backbone_onnx(bb, input_ids, attn, path)[source]¶

The one torch.onnx.export call for a backbone — fidelity_eval.py and check_arch_parity.py must export with exactly these axes/opset or their diagnostics stop matching production.

Parameters:

bb (Backbone)
input_ids (Tensor)
attn (Tensor)
path (Path)

Return type:

None

lenslapse.export_checkpoints.export_source(src, out_dir, tok)[source]¶

Parameters:

src (CheckpointSource)
out_dir (Path)
tok (Any)

Return type:

dict[str, Any]

class lenslapse.export_checkpoints.ExportConfig(*, model='EleutherAI/pythia-70m', steps='0,1,2,4,8,16,32,64,128,256,512,1000,2000,4000,8000,16000,32000,64000,128000,143000', out, skip_existing=False, final_only=False, subfolder_map=None, revision_template='step{}', tokenizer_ref=None)[source]¶

Bases: BaseModel

Validated arguments for export_checkpoints; see main’s docstring for what each means.

Parameters:

model (str)
steps (str)
out (Path)
skip_existing (bool)
final_only (bool)
subfolder_map (str | None)
revision_template (str)
tokenizer_ref (str | None)

model: str¶

steps: str¶

out: Path¶

skip_existing: bool¶

final_only: bool¶

subfolder_map: str | None¶

revision_template: str¶

tokenizer_ref: str | None¶

model_config = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

lenslapse.export_checkpoints.main(out, model='EleutherAI/pythia-70m', steps='0,1,2,4,8,16,32,64,128,256,512,1000,2000,4000,8000,16000,32000,64000,128000,143000', skip_existing=False, final_only=False, subfolder_map=None, revision_template='step{}', tokenizer_ref=None)[source]¶

Export a model’s training checkpoints as browser-runnable ONNX pairs.

Parameters:

out (str) – output directory for the ONNX pairs and manifest.json.
model (str) – HF id or local directory.
steps (str) – comma-separated training steps for a hub suite (ignored if subfolder_map is set).
skip_existing (bool) – skip steps whose ONNX files already exist in out.
final_only (bool) – single checkpoint (revision “main”) instead of a step suite.
subfolder_map (str | None) – “step:path,step:path,…” for repos that nest checkpoints as subfolders of one revision instead of using git revisions per checkpoint; overrides steps.
revision_template (str) – revision naming for hub suites, e.g. “global_step{}” for bigscience/bloom-*-intermediate.
tokenizer_ref (str | None) – load the tokenizer from a different ref than the checkpoint weights, as “repo_id” or “repo_id@revision” — for repos where the per-checkpoint tokenizer files do not load cleanly (bigscience/bloom-*-intermediate) or live in a separate repo entirely (m-a-p/neo_scalinglaw_*, whose tokenizer is only published under m-a-p/neo_7b). The tokenizer is identical across checkpoints of the same pretraining run, so this is always safe when it applies.

Return type:

None

lenslapse.fidelity_eval module¶

Fidelity evaluation: which ONNX weight format keeps the browser lens faithful to fp32 torch?

Compares weight formats against fp32 torch ground truth on the curated prompt set:: q8pt : dynamic int8, per-tensor q8pc : dynamic int8, per-channel q8pt+f16lens : per-tensor int8 backbone + fp16-stored lens head q8pc+f16lens : per-channel int8 backbone + fp16-stored lens head f16w : fp16-stored weights for both graphs (the format the demo ships)

Metrics per (config, step): top-1 agreement with fp32 torch over all (layer, position) cells, final-layer-only top-1 agreement, and mean KL(fp32 || onnx) at the final layer. Writes a JSON report for the paper’s fidelity table.

lenslapse.fidelity_eval.build_lens_onnx_f16(model, path)[source]¶

Lens head with fp16-stored weights cast to fp32 at run time (half the size, fp32 math).

LayerNorm-only variant used for the quantization comparison (Pythia); the shipped f16w format (onnx_f16.save_f16 over build_lens_onnx) covers RMSNorm architectures as well.

Parameters:

model (Any)
path (Path)

Return type:

None

lenslapse.fidelity_eval.export_fp32(model, td)[source]¶

Parameters:

model (Any)
td (Path)

Return type:

tuple[Path, Path]

lenslapse.fidelity_eval.make_variants(model, bb_path, lens_path, td)[source]¶

Parameters:

model (Any)
bb_path (Path)
lens_path (Path)
td (Path)

Return type:

dict[str, tuple[Path, Path]]

lenslapse.fidelity_eval.eval_variant(bb_path, lens_path, prompts_enc, ref)[source]¶

Parameters:

bb_path (Path)
lens_path (Path)
prompts_enc (Any)
ref (Any)

Return type:

dict[str, Any]

class lenslapse.fidelity_eval.FidelityEvalConfig(*, model='EleutherAI/pythia-70m', steps='8000,64000,143000', out)[source]¶

Bases: BaseModel

Validated arguments for fidelity_eval; see main’s docstring for what each means.

Parameters:

model (str)
steps (str)
out (Path)

model: str¶

steps: str¶

out: Path¶

model_config = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

lenslapse.fidelity_eval.main(out, model='EleutherAI/pythia-70m', steps='8000,64000,143000')[source]¶

Evaluate ONNX weight-format fidelity against fp32 torch ground truth on the curated prompts.

Parameters:

out (str) – output path for the JSON fidelity report.
model (str) – HF id or local directory.
steps (str) – comma-separated training steps to evaluate.

Return type:

None

lenslapse.logging_utils module¶

Shared logging setup for this package’s CLI entry points.

Every module in this package logs via its own logging.getLogger(__name__) (a child of the lenslapse logger) and never configures handlers or levels itself — that would be a library reconfiguring a caller’s own logging setup as a side effect of being imported or called, which a well-behaved library must not do. configure_cli_logging() is the one place that actually does this, and it must only be called from a true entry point: each module’s own if __name__ == “__main__”: guard, and cli.py’s main() (the installed lenslapse command’s actual entry point, which every other file’s __main__ guard bypasses).

lenslapse.logging_utils.configure_cli_logging()[source]¶

Root logger stays at WARNING (third-party libraries like onnx/onnxscript/transformers default to INFO-or-noisier internal logging that nobody asked to see); only this package’s own loggers are raised to INFO, matching what used to be plain print() calls.

Both the lenslapse namespace and __main__ are raised: whichever of this package’s own files is the actual entry point (python -m lenslapse.export_checkpoints, a subprocess add_model.py shells out to, …) is loaded with __name__ == “__main__”, not its real dotted name — a logging.getLogger(__name__) there is a logger literally named “__main__”, not a child of “lenslapse”, and raising only the latter would silently leave that file’s own messages at the default WARNING threshold.

Return type:: None

lenslapse.onnx_f16 module¶

Halve ONNX file size by storing float32 initializers as float16 + a Cast node.

All computation stays in fp32 (ONNX Runtime constant-folds Cast(initializer) at session load), so outputs differ from fp32 only by fp16 rounding of the weights. This avoids onnxconverter_common.float16, which fails on dynamo-exported graphs containing Cast nodes.

lenslapse.onnx_f16.convert_initializers_to_f16(model)[source]¶

Parameters:: model (ModelProto)
Return type:: ModelProto

lenslapse.onnx_f16.save_f16(src_path, dst_path)[source]¶

Parameters:

src_path (str)
dst_path (str)

Return type:

None

lenslapse.precompute_lens module¶

Precompute logit-lens data for curated prompts across Pythia training checkpoints.

Output layout (static JSON consumed by the web app):

out_dir/index.json   model metadata, step list, prompt catalog (tokens, gold continuation, targets)
out_dir/p{i}.json    per-prompt shard:
                       vocab: {token_id: token_string} for every id referenced
                       steps: {step: {"top": [layer][pos][k] -> [id, prob],
                                      "tgt": {target_id: {"p": [layer][pos], "r": [layer][pos]}}}}

Targets per position = top-3 final-layer predictions of the LAST step in –steps (+ gold next token for the last position), so trajectories are exact for the tokens the model eventually converges to. The last step is processed first to fix targets; every step then stores exact prob/rank for them.

Lens convention: hidden states are the raw (pre-final_layer_norm) residual stream after each block, plus embeddings; lens(h) = embed_out(final_layer_norm(h)). lens at the last layer equals the model’s actual output distribution (validated in export_checkpoints.py).

lenslapse.precompute_lens.lens_all(model, input_ids)[source]¶

Returns per-layer pre-final-norm hidden states passed through the lens head. [L+1, T, V] log-probs.

Parameters:

model (Module)
input_ids (Tensor)

Return type:

Tensor

lenslapse.precompute_lens.target_stats(lp, tid)[source]¶

Exact probability (rounded to 6 decimals) and strictly-greater rank of one target token per (layer, position), from lp [L+1, T, V] log-probs. The precomputed shards and the probe server’s /probe both serialize targets through this one function: the app overlays live and precomputed trajectories, so the two paths must agree to the digit.

Parameters:

lp (Tensor)
tid (int)

Return type:

dict[str, list[list[Any]]]

class lenslapse.precompute_lens.PrecomputeConfig(*, model='EleutherAI/pythia-70m', steps='0,1,2,4,8,16,32,64,128,256,512,1000,2000,4000,8000,16000,32000,64000,128000,143000', out, final_only=False, subfolder_map=None, prompts_file=None, revision_template='step{}', tokenizer_ref=None)[source]¶

Bases: BaseModel

Validated arguments for precompute_lens; see main’s docstring for what each means.

Parameters:

model (str)
steps (str)
out (Path)
final_only (bool)
subfolder_map (str | None)
prompts_file (str | None)
revision_template (str)
tokenizer_ref (str | None)

model: str¶

steps: str¶

out: Path¶

final_only: bool¶

subfolder_map: str | None¶

prompts_file: str | None¶

revision_template: str¶

tokenizer_ref: str | None¶

model_config = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

lenslapse.precompute_lens.main(out, model='EleutherAI/pythia-70m', steps='0,1,2,4,8,16,32,64,128,256,512,1000,2000,4000,8000,16000,32000,64000,128000,143000', final_only=False, subfolder_map=None, prompts_file=None, revision_template='step{}', tokenizer_ref=None)[source]¶

Precompute per-prompt logit-lens shards across a model’s training checkpoints.

Parameters:

out (str) – output directory for index.json and per-prompt p{i}.json shards.
model (str) – HF id or local directory.
steps (str) – comma-separated training steps for a hub suite (ignored if subfolder_map is set).
final_only (bool) – single checkpoint (revision “main”) instead of a step suite.
subfolder_map (str | None) – “step:path,step:path,…” for repos that nest checkpoints as subfolders of one revision instead of using git revisions per checkpoint; overrides steps.
prompts_file (str | None) – JSON file with the same shape as PROMPTS (list of {text, gold, story}); defaults to the built-in English curated set.
revision_template (str) – revision naming for hub suites, e.g. “global_step{}” for bigscience/bloom-*-intermediate.
tokenizer_ref (str | None) – load the tokenizer from a different ref than the checkpoint weights, as “repo_id” or “repo_id@revision”; see export_checkpoints.py.

Return type:

None

lenslapse.server module¶

Optional local probe server for models too heavy for in-browser inference.

The logit lens needs the entire per-layer residual stream of one teacher-forced forward pass — a batch-1 workload with no decoding loop, which generation-oriented serving engines neither expose natively nor accelerate. This server therefore runs plain transformers on CUDA/MPS/CPU, reusing the exact hooked-forward + lens implementation that generates the precomputed shards — the numbers agree by construction.

Every probe result is persisted to –cache-dir keyed by (model ref, revision, prompt), and identical requests are replayed from disk, byte-for-byte: results are reproducible across sessions and auditable as plain JSON files. Note the key is the reference, not the weights: for mutable refs (revision “main”, local run directories) a re-trained model under the same path keeps replaying the old cached results — clear the cache directory after retraining.

Usage:

lenslapse server                  (pip install; serves the bundled app + API on one port)
uv run lenslapse server           (repo checkout; serves the fresh web/dist build)

The registry defaults to the app’s models.json (hub suites with step{N} revisions); extend it with –extra for local runs or heavy hub models — a comma-separated list of specs (fire has no repeated-flag/append mechanism, so unlike the old argparse CLI this is one flag, not one per model), e.g.:

--extra "my-run=/path/to/trainer_output,llama=meta-llama/Llama-3.2-1B:final"

Models can also be registered at runtime from the web app’s “models” dialog (GET/POST/DELETE /models); user-registered entries persist to –registry-file and survive restarts. This is a management API for a localhost tool: anyone who can reach the server can register any Hub model or any directory readable by the server process, so do not expose the port beyond your machine.

async lenslapse.server.allow_private_network(request, call_next)[source]¶

Parameters:

request (Request)
call_next (Callable[[Request], Awaitable[Response]])

Return type:

Response

class lenslapse.server.ServerState[source]¶

Bases: TypedDict

Global server config + mutable registry. total=False: main() and LocalBackend (client.py) each finish populating this before any endpoint can be reached, so every key below is filled in by the time it is read — but neither startup path sets literally all of them (LocalBackend never sets models_root, only main() does), so none are Required.

registry: dict[str, RegistryEntry]¶

cache_dir: Path¶

max_loaded: int¶

device_map: bool¶

registry_file: Path¶

dtype: Literal['float32', 'float16', 'bfloat16', 'auto']¶

models_root: Path¶

class lenslapse.server.HealthResponse(*, ok, models, device)[source]¶

Bases: BaseModel

Parameters:

ok (bool)
models (list[str])
device (str)

ok: bool¶

models: list[str]¶

device: str¶

model_config = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class lenslapse.server.PickFolderResponse(*, path)[source]¶

Bases: BaseModel

Parameters:: path (str)

path: str¶

model_config = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class lenslapse.server.ModelListEntry(*, id, ref, mode, label, steps, origin)[source]¶

Bases: BaseModel

Parameters:

id (str)
ref (str)
mode (Literal['suite', 'final', 'local'])
label (str)
steps (list[int])
origin (Literal['catalog', 'user'])

id: str¶

ref: str¶

mode: Literal['suite', 'final', 'local']¶

label: str¶

steps: list[int]¶

origin: Literal['catalog', 'user']¶

model_config = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class lenslapse.server.RegisterResponse(*, id, ref, mode, label=None, steps)[source]¶

Bases: BaseModel

Parameters:

id (str)
ref (str)
mode (Literal['suite', 'final', 'local'])
label (str | None)
steps (list[int])

id: str¶

ref: str¶

mode: Literal['suite', 'final', 'local']¶

label: str | None¶

steps: list[int]¶

model_config = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class lenslapse.server.RemoveResponse(*, removed)[source]¶

Bases: BaseModel

Parameters:: removed (str)

removed: str¶

model_config = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class lenslapse.server.ConvertStatusResponse(*, id, status, log=None, note=None)[source]¶

Bases: BaseModel

Parameters:

id (str)
status (str)
log (list[str] | None)
note (str | None)

id: str¶

status: str¶

log: list[str] | None¶

note: str | None¶

model_config = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class lenslapse.server.TokenizeResponse(*, ids, tokens)[source]¶

Bases: BaseModel

Parameters:

ids (list[int])
tokens (list[str])

ids: list[int]¶

tokens: list[str]¶

model_config = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class lenslapse.server.ProbeRequest(*, model, step=0, text, targets=None)[source]¶

Bases: BaseModel

Parameters:

model (str)
step (int)
text (str)
targets (list[int] | None)

model: str¶

step: int¶

text: str¶

targets: list[int] | None¶

model_config = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class lenslapse.server.TokenizeRequest(*, model, text)[source]¶

Bases: BaseModel

Parameters:

model (str)
text (str)

model: str¶

text: str¶

model_config = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class lenslapse.server.RegisterRequest(*, id, ref, mode, label=None, steps=None)[source]¶

Bases: BaseModel

Parameters:

id (str)
ref (str)
mode (str)
label (str | None)
steps (list[int] | None)

id: str¶

ref: str¶

mode: str¶

label: str | None¶

steps: list[int] | None¶

model_config = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class lenslapse.server.RegistryEntry(*, ref, mode, label=None, steps=None, origin='user', revision_template='step{}', subfolder_map=None, tokenizer_ref=None)[source]¶

Bases: BaseModel

One probeable model. Serialized (minus origin) to the registry file.

Parameters:

ref (str)
mode (Literal['suite', 'final', 'local'])
label (str | None)
steps (list[int] | None)
origin (Literal['catalog', 'user'])
revision_template (str)
subfolder_map (str | None)
tokenizer_ref (str | None)

ref: str¶

mode: Literal['suite', 'final', 'local']¶

label: str | None¶

steps: list[int] | None¶

origin: Literal['catalog', 'user']¶

revision_template: str¶

subfolder_map: str | None¶

tokenizer_ref: str | None¶

model_config = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

lenslapse.server.build_registry(models_json, registry_file, extras)[source]¶

Parameters:

models_json (Path | None)
registry_file (Path | None)
extras (list[str])

Return type:

dict[str, RegistryEntry]

lenslapse.server.save_user_registry()[source]¶

Persist user-registered entries (dialog/API/–extra) so they survive restarts.

Return type:: None

lenslapse.server.model_steps(entry)[source]¶

Parameters:: entry (RegistryEntry)
Return type:: list[int]

lenslapse.server.source_for(model_id, step)[source]¶

Parameters:

model_id (str)
step (int)

Return type:

CheckpointSource

lenslapse.server.pick_device()[source]¶

Return type:: str

lenslapse.server.load(model_id, src)[source]¶

Parameters:

model_id (str)
src (CheckpointSource)

Return type:

tuple[Any, Any]

lenslapse.server.health()[source]¶

Return type:: HealthResponse

lenslapse.server.pick_folder()[source]¶

Open a native folder dialog on the server machine and return the chosen absolute path.

Lets the dialog’s “browse” button fill in local checkpoint paths for people who do not want to type them. The dialog opens on the machine running this server (they are the same machine in the intended localhost setup).

Return type:: PickFolderResponse

lenslapse.server.list_models()[source]¶

Return type:: list[ModelListEntry]

lenslapse.server.validate_source(req)[source]¶

Fail fast with an actionable message before the entry is persisted.

Parameters:: req (RegisterRequest)
Return type:: None

lenslapse.server.register_model(req)[source]¶

Parameters:: req (RegisterRequest)
Return type:: RegisterResponse

lenslapse.server.unregister_model(model_id)[source]¶

Parameters:: model_id (str)
Return type:: RemoveResponse

lenslapse.server.convert_model(model_id)[source]¶

Run the full ONNX onboarding pipeline (add_model.py) for a registered model, in the background. On success the model graduates into web/public + models.json (browser-runnable after the app is rebuilt or served from source), and its dialog registration is retired.

Parameters:: model_id (str)
Return type:: ConvertStatusResponse

lenslapse.server.convert_status(model_id)[source]¶

Parameters:: model_id (str)
Return type:: ConvertStatusResponse

lenslapse.server.tokenize(req)[source]¶

Tokenize with a registered model’s own tokenizer — the app’s track-a-token feature needs ids for server-backed models, whose tokenizers never ship to the browser.

Parameters:: req (TokenizeRequest)
Return type:: TokenizeResponse

lenslapse.server.probe_cache_key(src, req)[source]¶

Content key for one probe. JSON-encoded fields, not string concatenation: free-form prompt text must never be able to collide with the key of a different (text, targets). The compute dtype is part of the key: fp16 and fp32 forwards genuinely disagree at late checkpoints, so a result computed at one precision must not replay as another. subfolder is part of the key for the same reason it’s part of LOADED’s key (see load()): hub-subfolder suites have no per-checkpoint git revision, so (load_ref, revision) alone collides across every step of the same suite.

Parameters:

src (CheckpointSource)
req (ProbeRequest)

Return type:

str

lenslapse.server.read_cache(cache_file)[source]¶

Parameters:: cache_file (Path)
Return type:: dict | None

lenslapse.server.probe(req)[source]¶

Parameters:: req (ProbeRequest)
Return type:: dict[str, Any]

lenslapse.server.default_models_json()[source]¶

The catalog the app itself uses: the checkout’s copy, or the one bundled in the wheel.

Return type:: Path

lenslapse.server.get_shipped_model_data(model_id, filename)[source]¶

A shipped model’s precomputed index/prompt shard. Registered ahead of the / static mount in main, so it wins for these paths; unrelated 404s there are unaffected.

Parameters:

model_id (str)
filename (str)

Return type:

FileResponse

lenslapse.server.get_shipped_model_tokenizer(model_id, filename)[source]¶

A shipped model’s tokenizer file, same checkout-vs-cache split as the data route.

Parameters:

model_id (str)
filename (str)

Return type:

FileResponse

class lenslapse.server.ServerConfig(*, port=8017, host='127.0.0.1', models_json=PosixPath('/home/runner/work/lenslapse/lenslapse/web/public/data/models.json'), extra='', registry_file=PosixPath('/home/runner/work/lenslapse/lenslapse/server/registry.json'), cache_dir=PosixPath('/home/runner/work/lenslapse/lenslapse/server/probe-cache'), models_root=PosixPath('/home/runner/work/lenslapse/lenslapse/server/exported-models'), max_loaded=1, dtype='float32', device_map=False, open=False)[source]¶

Bases: BaseModel

Validated arguments for server; see main’s docstring for what each means.

Parameters:

port (int)
host (str)
models_json (Path)
extra (str)
registry_file (Path)
cache_dir (Path)
models_root (Path)
max_loaded (int)
dtype (Literal['float32', 'float16', 'bfloat16', 'auto'])
device_map (bool)
open (bool)

port: int¶

host: str¶

models_json: Path¶

extra: str¶

registry_file: Path¶

cache_dir: Path¶

models_root: Path¶

max_loaded: int¶

dtype: Literal['float32', 'float16', 'bfloat16', 'auto']¶

device_map: bool¶

open: bool¶

model_config = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

lenslapse.server.main(port=8017, host='127.0.0.1', models_json='/home/runner/work/lenslapse/lenslapse/web/public/data/models.json', extra='', registry_file='/home/runner/work/lenslapse/lenslapse/server/registry.json', cache_dir='/home/runner/work/lenslapse/lenslapse/server/probe-cache', models_root='/home/runner/work/lenslapse/lenslapse/server/exported-models', max_loaded=1, dtype='float32', device_map=False, open=False)[source]¶

Start the local probe server (+ the hosted web app, when a build is available).

Parameters:

port (int) – TCP port to listen on.
host (str) – bind address.
models_json (str) – the app’s model catalog; defaults to the checkout’s copy or the wheel-bundled one.
extra (str) – comma-separated “id=hf_ref[:final]” or “id=/local/dir” specs to extend the registry beyond models_json (fire has no repeated-flag/append mechanism, so this is one comma-joined flag rather than one –extra per model), e.g. “my-run=/path/to/trainer_output,llama=meta-llama/Llama-3.2-1B:final”.
registry_file (str) – where models registered via the web dialog / POST /models persist.
cache_dir (str) – where probe results are cached, keyed by (model ref, revision, prompt).
models_root (str) – where dialog-triggered ONNX conversions write <id>/step*/…; served back at /models/.
max_loaded (int) – models kept in memory (heavy models: keep 1).
dtype (Literal['float32', 'float16', 'bfloat16', 'auto']) – compute dtype; float32 (default) matches the precomputed shards and the browser path, lower precisions halve memory for heavy models but can flip late-checkpoint top-1s.
device_map (bool) – load with device_map=’auto’ (requires accelerate; multi-device sharding is untested).
open (bool) – open the hosted web app pointed at this server.

Return type:

None

lenslapse.sources module¶

Resolve a model source into (load_ref, revision, step) tuples for export/precompute.

Four source shapes are supported:

1. Hub suite     : --model EleutherAI/pythia-70m --steps 0,1000,...   -> revisions step{N}
2. Hub single    : --model gpt2 --final-only                          -> revision main, step 0
3. Local dir     : --model /path/to/run --local-checkpoints           -> checkpoint-*/ subdirs
                   (Hugging Face Trainer layout; the number suffix is the training step), or a
                   plain single-model directory when no checkpoint-* subdirs exist.
4. Hub subfolder : --model m-a-p/neo_scalinglaw_250M --subfolder-map "16780:hf_ckpt/16.78B,..."
                   for repos that nest each checkpoint as a subfolder within a single revision
                   rather than using git revisions per checkpoint (seen on Megatron-derived HF
                   exports that keep the raw training state and converted HF-format weights in
                   the same repo, e.g. MAP-Neo, BAAI Aquila).

lenslapse.sources.coerce_fire_csv_arg(v)[source]¶

fire always tries to parse a bare CLI value as a Python literal before falling back to str, so an unquoted comma-separated list of numbers (e.g. –steps 0,512,8000) arrives as the tuple (0, 512, 8000) instead of a string, and a single bare number (e.g. –steps 8000) arrives as a plain int — for every CLI parameter in this package that expects a comma-separated string of ids/steps (steps, targets). Use as a pydantic field_validator(mode=”before”) to normalize back to the comma-joined string the rest of the pipeline expects; a value already shaped as a str (the default, or a programmatic caller) passes through unchanged.

Parameters:: v (object)
Return type:: object

class lenslapse.sources.CheckpointSource(load_ref, revision, step, subfolder=None)[source]¶

Bases: object

A pydantic dataclass (not BaseModel): callers throughout this package construct it positionally (CheckpointSource(model, revision, step)), which BaseModel’s keyword-only __init__ does not support — the dataclass shape keeps that calling convention while still validating field types.

Parameters:

load_ref (str)
revision (str | None)
step (int)
subfolder (str | None)

load_ref: str¶

revision: str | None¶

step: int¶

subfolder: str | None = None¶

property name: str¶

lenslapse.sources.resolve_sources(model, steps_arg, final_only, revision_template='step{}')[source]¶

Parameters:

model (str)
steps_arg (str)
final_only (bool)
revision_template (str)

Return type:

list[CheckpointSource]

lenslapse.sources.resolve_subfolder_sources(model, subfolder_map)[source]¶

Hub subfolder suite (shape 4): subfolder_map is “step:path,step:path,…”, e.g. “16780:hf_ckpt/16.78B,33550:hf_ckpt/33.55B”. step is whatever synthetic slider value the caller has already decided on (e.g. tokens in millions when the repo labels checkpoints by tokens rather than optimizer steps) — this function does no unit conversion of its own.

Parameters:

model (str)
subfolder_map (str)

Return type:

list[CheckpointSource]

lenslapse.sources.resolve_all_sources(model, steps, final_only=False, subfolder_map=None, revision_template='step{}')[source]¶

The one precedence rule every CLI shares: a –subfolder-map fully replaces the steps/final-only/revision-template resolution (shape 4 repos label checkpoints by subfolder, not by git revision).

Parameters:

model (str)
steps (str)
final_only (bool)
subfolder_map (str | None)
revision_template (str)

Return type:

list[CheckpointSource]

lenslapse.sources.resolve_tokenizer_ref(tokenizer_ref, fallback)[source]¶

Where to load the tokenizer from: tokenizer_ref (“repo_id” or “repo_id@revision”) when given, else the same ref as fallback (typically sources[0]). Returns (load_ref, revision, subfolder) ready to splat into AutoTokenizer.from_pretrained(…).

A –tokenizer-ref override is for repos where the per-checkpoint tokenizer files don’t load cleanly (bigscience/bloom-*-intermediate, a transformers-version incompatibility) or live in a separate repo entirely (m-a-p/neo_scalinglaw_*, whose tokenizer is only published under m-a-p/neo_7b) — always safe when it applies, since the tokenizer is identical across checkpoints of the same pretraining run.

Parameters:

tokenizer_ref (str | None)
fallback (CheckpointSource)

Return type:

tuple[str, str | None, str]

lenslapse.sources.load_tokenizer(load_ref, revision, subfolder, *, trust_remote_code=True)[source]¶

AutoTokenizer.from_pretrained, with a fallback for a confirmed transformers limitation (checked against transformers 4.57.6): a repo whose tokenizer needs custom code (trust_remote_code=True, e.g. Qwen-family tokenizers) does not pass subfolder through to that code file’s own download — only the tokenizer_config.json/vocab file resolution honors it — so a subfolder-nested custom tokenizer 404s on the code file even though the exact same weights load fine via AutoModelForCausalLM.from_pretrained(subfolder=…). This is generic (not specific to one model): any future subfolder-suite architecture with a custom-code tokenizer can hit it. When it does, download just that subfolder’s files locally (not the whole repo) and retry from there, where subfolder is no longer needed.

Parameters:

load_ref (str)
revision (str | None)
subfolder (str)
trust_remote_code (bool)

Return type:

PreTrainedTokenizerBase

lenslapse.sources.token_display_text(tok, t)[source]¶

convert_ids_to_tokens returns each token in the tokenizer’s internal vocab representation, not display text: raw bytes for tiktoken-based tokenizers (e.g. QWenTokenizer, not always valid standalone UTF-8 and never JSON-serializable as-is), and for byte-level BPE tokenizers (GPT-2/Pythia/BLOOM) a per-byte visible-codepoint encoding that renders non-ASCII text (e.g. Chinese) as mojibake unless reversed. convert_tokens_to_string() undoes both, but called on a single token it also treats that token as the start of a decoded string and drops (rather than converts to a real space) a leading word-start marker (confirmed on SentencePiece; GPT-2-style byte-level BPE already returns a leading space unchanged) — reattach one when the raw token asked for it and the conversion ate it, so every layer’s word-initial predictions still read as “starts a new word” instead of silently looking like mid-word continuations. This is tooltip/grid display text, not round-tripped byte-exactly, so a lossy decode is fine.

None means the id has no vocab entry at all — some checkpoints (e.g. BLOOM) pad the embedding/lm_head matrix past the tokenizer’s real vocab size for hardware alignment, and an undertrained layer can assign a padding id nonzero top-k probability. ‘?’ matches the frontend’s own fallback (web/src/data.ts) for a vocab id it can’t find.

Parameters:

tok (PreTrainedTokenizerBase)
t (str | bytes | None)

Return type:

str

lenslapse.webdata module¶

Lazy download + local cache for shipped models’ precomputed data and tokenizer files.

The wheel only bundles the app shell and the models.json catalog (kept small enough for PyPI’s 100 MB per-file limit); each shipped model’s precomputed shards and tokenizer files are instead fetched from this repo’s own main branch via raw.githubusercontent.com on first use and cached locally, so a pip install lenslapse still ends up fully offline-capable for the shipped models after each one has been opened once. A repo checkout is unaffected: web/public/data/tokenizer are already present on disk there, so nothing here is ever reached (see _webapp_root in server.py, which is checked first).

lenslapse.webdata.safe_segment(segment)[source]¶

Reject anything but a plain filename/id segment — no /, .., or other path escapes, since both come from request paths and end up in local filesystem paths and upstream URLs.

Parameters:: segment (str)
Return type:: bool

lenslapse.webdata.ensure_data_file(cache_root, model_id, filename)[source]¶

Local path to data/<model_id>/<filename> (index.json or one prompt shard p{id}.json), downloading and caching it on first request. None if it doesn’t exist upstream either.

Parameters:

cache_root (Path)
model_id (str)
filename (str)

Return type:

Path | None

lenslapse.webdata.ensure_tokenizer_file(cache_root, model_id, filename)[source]¶

Local path to tokenizer/<model_id>/<filename>, downloading the model’s full tokenizer (every file in _TOKENIZER_CANDIDATES that exists upstream) on first request for it. None if the model has no tokenizer directory upstream at all, or filename isn’t a real member of it.

Parameters:

cache_root (Path)
model_id (str)
filename (str)

Return type:

Path | None

Module contents¶

LensLapse: an in-browser logit lens over training checkpoints.

This package holds the conversion/precompute pipeline and the optional local probe server. Install from git and run lenslapse server to probe models through the hosted web app.