Properties of p-value
P-values for testing whether a linear model's slope is zero have the same properties as p-values for other hypothesis tests. In particular,
Simulation of distribution of p-values
The diagram below builds up the distribution of p-values for testing whether the slope is zero.
With β1 = 0, click Take sample several times and verify that the p-values are rectangularly distributed between 0 and 1.
(Click on any p-value to see the data set that gave rise to it.)
Change the linear model slope to β1 = 0.5, then take several more samples. The p-values tend to be closer to zero.
Repeat with β1 = 1.0.
As shown by the above simulation, when Y and X are not related (β1 = 0), it is still possible to get small p-values, suggesting that β1 is not zero. However there is only probability 0.01 of getting a p-value as low as 0.01 — it is unlikely but possible. Such a p-value is more likely if the variables are related so we interpret it as giving strong evidence of a relationship.