ai-model-into-rust-mega-fused-hyper-kernel
Made by JeffTurn open-weight model checkpoints into memory-safe pure-Rust mega-fused inference engines that beat PyTorch with proven parity. Use when: frankenize a model, port model weights to Rust, custom inference engine, int8 SIMD kernels (SDOT/VNNI), bit-identical model parity, quantized weight containers, or adding a CUDA backend.
Works With
Quality and Utility come from canonical skill metadata. Community ratings appear separately as stars when subscriber reviews exist.
Subscribe to download
Public previews are available, but downloads, CLI sync, and premium content require an active subscription.
Sign In to Subscribe- Full skill content with deterministic versioning
- CLI sync + local search index
- Access to Jeffrey’s premium library
Save to your account
Saved skills live in your cloud list. Run jsm sync on each machine to pull them locally.
What's New with this Skill
1Updates, enhancements, and release notes for this skill.
New skill: ai-model-into-rust-mega-fused-hyper-kernel, port open-weight models to pure-Rust engines that beat PyTorch, with the evidence to prove it
Distilled from three projects that took open-weight models (OCR, speech recognition, semantic embeddings) from HuggingFace weights to hand-rolled, memory-safe Rust inference engines: roughly 5,000 commits and 1,600 tracker items reduced to a step-by-step porting method. The exemplar runs Baidu's OCR model 3.34x faster end to end than the PyTorch baseline on the same machine, with int8 kernels (NEON SDOT, AVX-512 VNNI) picked at runtime so one binary per OS covers every CPU.
- Distilled from ~5,000 commits and ~1,600 tracker items across three real ports (FrankenSearch, FrankenWhisper, FrankenOCR), keeping what worked and ledgering what measured dead
- FrankenOCR exemplar: 3.34x faster end to end than the pinned PyTorch bf16 baseline; GOT-OCR2 page forward went 8 min -> ~17 s, decode 1.4 -> 64.7 tok/s
- One binary per OS: runtime ISA dispatch picks int8 kernels (NEON SDOT on Apple Silicon, AVX-512 VNNI on x86) per CPU
- The whole safety argument in one sentence: discrete outputs BIT-EXACT where the reference is deterministic; continuous outputs held within the MEASURED tolerance
- Oracle before engine: pin and hash a truth pack, measure the reference's own noise floor before setting any tolerance, then prove the forward seam by seam on the oracle's exact tensors
- The graveyard ships as inheritable priors: every dead lever carries a do-not-retry predicate (SMMLA runs half-rate on M-series; LLVM autovec beat hand SDOT by 4.4x; int4 measured 5.8x slower than int8)
- 46 files, 300+ KB: 21 deep references, a 16-card operator deck, 86 indexed rules, a 12-recipe cookbook, 8 subagent roles, and a bootstrapper that scaffolds a new port
- An anti-ceremony counterweight bounds process work, with honest-credit rules against reward hacking
- The skill corrects its own headline number: the quoted 3.41x was a slip repeated in a commit message; the ledger says 3.34x, and the correction is documented
The skill is 46 files and over 300 KB of method: 21 deep reference files, a 16-card operator deck, 86 indexed rules, a 12-recipe cookbook for when a port goes sideways, 8 spawnable subagent roles, and executable scripts including a bootstrapper that scaffolds a new port. You point it at an open-weight model and it walks the whole path: a fit screen that says whether the model is even a good candidate; a truth pack phase that pins and hashes every input before any code exists; an oracle-before-engine rule that dumps the reference implementation's activations and measures its OWN non