Framework documentation
MoE and LoRA routing
for frozen language models.
Learn Mutations, the Anchored Plug-in MoE Python framework: convert Qwen3 models, train specialists, calibrate expert routing, and deploy Needle branches while preserving a frozen base-model fallback.
All 18 guides are published and validated. Integration labels distinguish complete package lifecycles from composable and research paths.
Search the framework
Find the exact concept.
Search page titles, section headings, strategy names, or tracked feature IDs such as POL-002.
Search 18 documentation pages.
01
Introduction
01OverviewUnderstand Mutations and Anchored Plug-in MoE: frozen LLM backbones, independent expert packages, exact base-model fallback, and supported workflows.02Getting startedInstall the Mutations Python framework, convert a local Qwen3 model to anchored MoE layers, and verify exact fallback without downloading model weights.03Configuration and modelingConfigure AnchoredMoEConfig and AnchoredMoEForCausalLM, choose Qwen3 layers, and manage a frozen language model with explicit expert-package lifecycles.
04
Core components
04BackendsDiscover and convert Qwen3 MLP layers with the Mutations backend contract while preserving exact Hugging Face model behavior and frozen backbone weights.05ExpertsCompare full-MLP, PEFT/LoRA, block-sparse, and micro-expert representations for frozen LLMs, including storage, training, and package-support boundaries.06RoutersLearn MoE expert routing with frozen features, affine heads, hierarchical selection, prompt shortlists, router distillation, and calibrated specialist scores.07PoliciesCompare anchored top-1, anchor-veto top-2, adaptive and dynamic-k MoE routing, expert combination, and capacity-aware fallback in Mutations.
08
Workflows
08TrainingTrain independent MoE experts and LoRA specialists over a frozen LLM backbone, compare local mini-MoE workflows, and inspect isolation and preservation checks.09CalibrationFit and calibrate MoE router heads, build utility labels, control false specialist activation, and measure routing regret while keeping the LLM anchor frozen.10RegistryManage MoE expert packages with manifest validation, Hugging Face Hub transfers, explicit model synchronization, version upgrades, and exact rollback.11InferenceRun sparse anchored top-1 MoE inference, compose expert packages with a frozen Qwen3 base, and inspect dispatch, capacity fallback, and route telemetry.12EvaluationEvaluate MoE correctness, expert routing, model quality, latency, throughput, and memory with bounded experiments and exact base-model fallback checks.16Prompt-routed adaptersTrain independent LoRA adapters for Qwen3 attention and MLP projections, then use an external prompt router to select a specialist or frozen base per response.17Needle branchesBuild frozen-prefix Needle LLM branches, train independent LoRA or copied suffixes, and deploy specialists with cached JAX decoding or native model exports.18Browser deploymentDeploy routed Needle LLMs in the browser with the official WASM runtime, a frozen base download, specialist byte patches, and recent-context tool routing.
13
Learn by example
14
Project
14UtilitiesUse deterministic JSON and file hashing to validate expert-package metadata, verify payload integrity, and reproduce Mutations model compositions.15ReferencesExplore the primary research behind Mutations: Mixture-of-Experts routing, LoRA adapters, sparse experts, calibration, and modular LLM composition.