P-values and probability

When H0 holds,

On the other hand, when HA holds, p-values are more likely to be near zero and

Examples

p-value = 0.0023
From this, we know that there would be only 0.0023 probability of getting such a small p-value if H0 was true. This is unlikely, so there is strong evidence that H0 does not hold.
p-value = 0.4
There is probability 0.4 of seeing such a low p-value if H0 is true, so there is no evidence against H0.

Of course, we may be wrong. A p-value of 0.0023 could arise when either H0 or HA holds but it is more likely under HA. And a p-value of 0.4 could also arise when either hypothesis is true.

Interpretation of p-values for all tests

p-value Interpretation
over 0.1 no evidence that the null hypothesis does not hold
between 0.05 and 0.1 very weak evidence that the null hypothesis does not hold
between 0.01 and 0.05   moderately strong evidence that the null hypothesis does not hold
under 0.01 strong evidence that the null hypothesis does not hold