A controlled comparison
In an A/B test, one group of users sees the current experience (the control) and another group sees a different variant. Random assignment helps make the groups comparable, so a difference in a chosen metric is more likely to come from the change. The same person should usually see a consistent version throughout the test.
Teams might test whether clearer checkout copy increases completed purchases. A feature flag can control which experience each participant receives.
A result still needs judgement
Experiments need enough participants, a sensible duration, and guardrail measures for side effects. Checking lots of measures, or stopping a test as soon as one looks promising, can make a change appear successful by chance. Even a statistically convincing result may be too small, short-lived, or harmful to justify a launch.