A suite passes only when every case passes. Each case passes independently with probability p. With retries, a case counts as passing under pass@k if any of k attempts passes, and under pass^k only if all k attempts pass.
Case passes with 1 − (1 − p)k; suite passes with (1 − (1 − p)k)N
Case passes with pk; suite passes with pkN
Assumes every case has the same pass rate and attempts are independent. Real flaky cases are correlated (the same prompt tends to fail the same way), so retries help less than pass@k suggests. The dashed vertical line marks your suite size; the table reads values at that size.