ab-testing
Made by JeffBuild home-rolled A/B testing platform for Next.js 16 SaaS. Server-side variant assignment, GA4/GTM tracking, Bayesian analysis, Rust-WASM stats, contextual bandits. Use when: A/B test, experiment, variant, conversion optimization, or personalization.
Works With
Quality and Utility come from canonical skill metadata. Community ratings appear separately as stars when subscriber reviews exist.
Subscribe to download
Public previews are available, but downloads, CLI sync, and premium content require an active subscription.
Sign In to Subscribe- Full skill content with deterministic versioning
- CLI sync + local search index
- Access to Jeffrey’s premium library
Save to your account
Saved skills live in your cloud list. Run jsm sync on each machine to pull them locally.
What's New with this Skill
1Updates, enhancements, and release notes for this skill.
New: A/B Testing for Next.js SaaS with server-side variants, no flicker, Bayesian stats, and contextual bandits
Google Optimize sunset in September 2023; GA4 has no native A/B testing. This skill is the home-rolled replacement: edge middleware assigns variants on the server before render so users never see the wrong page first, GA4 + GTM dataLayer tracks the assignment, a Beta-Binomial model (with a Rust-WASM module for the heavy Monte Carlo work) computes P(B > A) and credible intervals from raw counts, and Thompson-sampling bandits shift traffic toward the winning variant as evidence accumulates.
- Server-side variant assignment in middleware.ts so the user sees variant A or B with no client-side flicker; a 30-day cookie keeps the assignment stable
- GA4 + GTM dataLayer push pattern: every variant exposure emits an experiment_view event with experiment_name and experiment_variant, registered as a GA4 custom dimension so conversions segment by variant
- Bayesian analysis with a Beta-Binomial model on raw conversion counts: outputs P(B > A) and a credible interval for lift. Rust-WASM handles the 100k+ Monte Carlo draws and bandit inference
- Supabase tables for experiments, experiment events, and personalization rules, with a get_experiment_stats SQL function behind the admin dashboard
- Thompson sampling bandits (in TypeScript and Rust-WASM) shift traffic toward the winning variant as evidence accumulates instead of running a fixed split
- Personalization with contextual bandits: country, device type, and returning vs new visitor feed the choice of variant per segment
- Why DIY: Google Optimize sunset Sept 2023 and GA4 has no native A/B testing. The stack is Next.js 16 + edge middleware + GA4/GTM + Supabase + Rust-WASM
Server-side variant assignment kills the flicker. The middleware checks for a cookie; if missing, assigns A or B with Math.random() < 0.5, sets a 30-day cookie, and the variant is in the request before the first byte of HTML. GA4 + GTM tracks the assignment via window.dataLayer.push({ event: 'experiment_view', experiment_name, experiment_variant }), and experiment_variant is registered as a GA4 custom dimension so conversions can be segmented by variant.