Install, Version, and Remove MoE Expert Packages

What this part does

The registry is the trust and lifecycle boundary for specialization packages. It proves which immutable files make up a package, records which packages are active and in what order, and rejects incompatible or stale transitions before they can alter registry state.

The primary anchored top-1 path is complete from validated package export through installation, explicit model synchronization, local composition save, and exact reload. Compatibility-aware top-2 and registry-specific router versions are validated composable paths with deliberately explicit attachment.

Where it fits

The registry sits between artifact creation and model execution:

1. training exports an expert-only package with explicit payload bindings;
2. calibration derives a new immutable router-bearing package version;
3. strict loading validates its manifest, files, hashes, and compatibility;
4. lifecycle operations change only the active registry snapshot;
5. explicit synchronization materializes that snapshot into routed MLPs;
6. composition export preserves the synchronized snapshot while referencing the separately pinned anchor.

The registry does not train experts, fit routers, choose token routes, execute MLPs, or silently change a live model. That separation is why a package can be inspected, installed, removed, or replaced before a caller deliberately applies the resulting registry.

Implemented approaches

Immutable package identity

ExpertPackageManifest records the pinned base and tokenizer, converted layers, expert representation, safe tensor files, router feature contract, runtime and dataset provenance, and a SHA-256 digest for every tracked file. The manifest itself is serialized as canonical JSON and contributes its own digest to PackageIdentity.

ExpertPackage.from_directory() is the local trust boundary. Strict loading rejects symbolic links, untracked files, unsafe paths, executable payload formats, malformed metadata, and changed bytes. Loading data never imports or executes package code.

Explicit payload ownership

PKG-007 records each complete runtime slot as (layer_index, expert_id, weight_file). The current full-MLP format requires a complete layer-by-expert rectangle, unique slots and files, and the same expert ID set at every converted layer.

Legacy version-one manifests without bindings remain valid for registry-only lifecycle work, but they cannot be attached to a model. Filenames never imply expert ownership.

Stable lifecycle and audit snapshots

ExpertRegistry keeps active packages in stable historical slots. Unloading a package releases its live descriptor but remembers its slot; reinstalling the same immutable identity restores the previous order and registry hash.

An upgrade replaces one active package with a different immutable version of the same specialization without moving unrelated packages. Rollback requires the exact upgrade record, the exact prior package, and an unchanged upgraded snapshot. Every file and compatibility check occurs before the lock-protected state transition.

Explicit model synchronization

Installing, unloading, upgrading, and rolling back packages do not change model routes. AnchoredMoEForCausalLM.synchronize_registry() separately validates the complete intended runtime and then applies it. Synchronizing an empty registry restores forced-anchor execution exactly.

This makes stale execution visible rather than surprising: registry state can advance while the model continues to use the last successfully synchronized snapshot. Composition export rejects that mismatch instead of silently repairing it.

Compatibility-aware mixtures

Packages may declare one mixture_compatibility_group. PKG-003 builds an immutable layer-local index from stable expert IDs. Two distinct experts can mix only when both declare the same non-empty group.

A missing group means “not mixable,” not membership in an implicit default. Candidate order may be realigned by expert ID, but the candidate set must match exactly. The index supports explicit compatibility-constrained top-2 selection and sparse dispatch; automatic package-driven top-2 model attachment remains future work.

Registry-specific routers

Global router recalibration produces behavior tied to one exact active registry. RegistryRouterVersion stores a frozen logical router, complete registry snapshot, feature specification, recalibration report, tensor ownership, and tensor digests in one non-executable safetensors artifact.

Loading reconstructs defensive frozen state. Activation succeeds only for an exactly equal live snapshot and returns fresh storage. It never installs packages, changes a model, or chooses a router version automatically.

Local compositions and Hub transfer

A CORE-006 composition checkpoint contains anchored and generation configuration, byte-identical active packages in registry order, compatibility and registry identities, and an external anchor reference. It deliberately does not copy anchor tensors into every composition.

PKG-006 transfers individual expert packages through familiar Hugging Face methods. Downloads resolve a branch or tag once, pin the immutable commit, and fetch only the manifest-declared closure. Uploads stage and strictly revalidate an exact regular-file copy before the first remote mutation. Credentials, repository permissions, registry installation, and model synchronization stay outside the transport layer.

How it works

The integrated package lifecycle keeps three state boundaries visible:

Immutable package files
  → strict descriptor validation
  → portable PackageIdentity

Active registry
  → install / unload / upgrade / rollback
  → ordered RegistrySnapshot

Explicit model synchronization
  → materialize validated package payloads
  → routed runtime or exact anchor fallback

A registry change is therefore an auditable proposal for execution, not an implicit model mutation. Local composition export is allowed only when the proposal and the live runtime agree.

Mathematical intuition

Minimal usage

Apply lifecycle changes deliberately and save only synchronized state:

package = ExpertPackage.from_directory("./code-experts")
model.install(package)
snapshot = model.synchronize_registry()

model.save_pretrained("./code-composition")
reloaded = AnchoredMoEForCausalLM.from_pretrained("./code-composition")
assert reloaded.synchronized_registry_sha256 == snapshot.registry_sha256

Installation validates and updates the registry. Synchronization is the separate operation that changes routed execution.

Verified implementation details

registry.install(code)
registry.install(math)
installed = registry.snapshot()

registry.unload(code.identity)
registry.install(ExpertPackage.from_directory(code.package_directory))
assert registry.snapshot().active_packages == (code.identity, math.identity)

upgrade = registry.upgrade(code.identity, code_v2)
rollback = registry.rollback(upgrade, code)
assert rollback.after == upgrade.before == installed

model.install(package)
installed = model.synchronize_registry()

model.save_pretrained(composition_checkpoint)
reloaded = AnchoredMoEForCausalLM.from_pretrained(
    composition_checkpoint,
    local_files_only=True,
)
assert reloaded.synchronized_registry_sha256 == installed.registry_sha256

downloaded = ExpertPackage.from_pretrained(
    "organization/code-experts",
    revision="release-1",
    cache_dir=cache_directory,
    local_files_only=True,
)
upload = downloaded.push_to_hub(
    "organization/code-experts-copy",
    commit_message="Offline PKG-006 verification",
)

version = RegistryRouterVersion(
    recalibrated,
    registry_snapshot=snapshot,
    feature_specification=feature_specification,
    recalibration_report=report,
)
version.save_safetensors(path)
loaded = RegistryRouterVersion.from_safetensors(path)
restored = loaded.activate(snapshot)

Current limitations

• Registry transitions are in-memory and process-local. There is no standalone persistent registry history, cross-process lock, or distributed lifecycle.
• Install, unload, upgrade, and rollback remain registry-only until explicit synchronization. Automatic model mutation is intentionally unsupported.
• Package-driven model attachment currently covers complete full-MLP experts with independent affine heads and anchored top-1. Other expert representations and model-level top-2 attachment remain future work.
• Composition checkpoints are local-only, reference an external anchor, and retain active packages rather than inactive slots or rollback records. Remote transfer, optional anchor bundling, and router-history embedding are unsupported.
• Live anchor configuration is checked, but exact live anchor-weight hashing and runtime-version enforcement remain incomplete.
• Registry-specific router versions require explicit selection and activation; they are not embedded in compositions or restored automatically after a lifecycle change.
• Hub support transfers expert packages only. Credentialed service validation, large-package benchmarks, composition transfer, and registry-router artifact transfer remain separate work.

References

Safetensors documentation motivates non-executable tensor payloads. Hugging Face Hub download documentation defines pinned snapshot transfer and file allow-lists, while the Hub API client documentation defines explicit repository creation and folder upload.

FIPS PUB 180-4 defines SHA-256. RFC 8785 motivates stable JSON hashing; Mutations uses documented project-local canonical JSON rules rather than claiming complete RFC 8785 conformance. Python’s threading.RLock documentation defines the in-process synchronization primitive protecting registry transitions.

PKG-001 through PKG-007 and CORE-006 are framework infrastructure, not paper-derived training or optimization algorithms. Their security checks, ordering rules, compatibility boundary, and explicit lifecycle semantics implement the project’s immutable-anchor design. The registry theory README contains the complete algorithms, invariants, validation coverage, and documented format differences. The architecture blueprint defines append-only installation, registry-bound recalibration, removal, and upgrade choices.

Related pages

• Read training for expert-only optimization and package export boundaries.
• Read calibration for router-bearing package derivation and explicit registry-wide recalibration.
• Read configuration and modeling for the façade that owns the registry and applies synchronization.
• Read policies for compatibility-constrained top-2 selection and capacity fallback.
• Read inference for package materialization, sparse dispatch, exact anchor restoration, and route telemetry.