AI-native A/B testing: auto-generate variants, auto-promote winners
THE AUTOMATION ENGINE IS NOW SELF-IMPROVING. ## Core: AI generates challenger variants Click '✨ AI: Test a new approach' → GPT-4o-mini analyzes variant A and creates variant B using a fundamentally DIFFERENT psychological approach. Not a rephrase — a different strategy: - Social proof ('47 others have already paid') - Urgency (deadline framing) - Impact storytelling ('£50 = 3 weeks of food') - Personal connection (heavy name usage) - Brevity (strip to minimum) - Gratitude-first (lead with thanks) - Loss framing ('pledge at risk of being unfulfilled') - Community ('join 23 others who completed this week') AI explains WHY: 'This variant uses social proof instead of a gentle reminder — peer pressure converts better for step 2.' ## Core: Automatic winner promotion Click '🏆 Pick winners' → system evaluates ALL running A/B tests: 1. Checks minimum sample size (20 sends per variant) 2. Runs z-test for statistical significance (90% confidence) 3. Promotes winner to variant A (resets counters) 4. Deletes loser 5. AUTOMATICALLY generates a NEW AI challenger The cycle never stops. Messages continuously evolve. ## Core: AI rewrite toolbar 8 one-click AI rewrites for any template: ✂️ Make shorter · 💛 Make warmer · ⏰ Add urgency 👥 Add social proof · 💚 Add impact story · 🎯 Strip to essentials 🇵🇰 Translate to Urdu · 🇸🇦 Translate to Arabic All rewrites preserve {{variable}} placeholders. All use GPT-4o-mini (~/usr/bin/bash.15/1M tokens). ## UI: A/B Stats Card (below phone mockup) - Side-by-side conversion rates with trophy icon on winner - Progress bar to verdict (% of minimum sample collected) - Lift calculation: 'Variant B converts 63% better' - Real-time during test: 'A: 33% → B: 54% ★' ## UI: Winner Results Banner After 'Pick winners' runs: - Green banner: '🏆 Winners promoted — Gentle reminder · WhatsApp → Variant B wins (54% vs 33%) ✨ New AI challenger created' - Gray banner if not enough data: 'Need 20+ sends per variant' ## API: /api/automations/ai (POST) Actions: - generate_variant: AI creates challenger B with strategy reasoning - rewrite: AI rewrites template with specific instruction - check_winners: evaluate all tests, promote, regenerate ## Architecture The system is a GENETIC ALGORITHM for messaging: 1. Start with default templates (generation 0) 2. AI creates a challenger (mutation) 3. Traffic splits 50/50 (fitness test) 4. Winner survives, loser dies (selection) 5. AI creates new challenger (next generation) 6. Repeat forever → messages get better over time
This commit is contained in:
