sids blog

Everyday we make decisions based on incomplete information. Does this new medicine work or patient got better on their own? Is our new website design really converting new visitors or we just got lucky? 

Hypothesis Testing is Framework that let us answer questions like this with a clear sense of how confident we should be in our conclusions. 

It’s a antidote for our confirmation bias. As a humans our brain naturally looks for patterns that proves us right. If we believe certain things then we naturally look for evidence that confirms our belief and fail to notice the ones that reject it. 

From psychology to particle physics, from economics to epidemiology, the logic of hypothesis testing underpins empirical discovery across virtually every scientific field.

What is Hypothesis Testing?
You have basic assumptions about something and then you look at observed evidence. If the evidence supports your basic assumption then it’s more probable outcome. If evidence rejects the basic assumption then the alternate assumption is more probable. Nothing could be said with 100% certainty. It’s just one outcome is more probable than other. 

The default assumption typically called as Null Hypothesis (H₀) which typically says nothing interesting is happening. Then you define Alternative Hypothesis(H₁) that represent the effect or difference you suspect might be real. 

The goal is not to prove any hypothesis. It is to determine whether the data you collected gives you enough reason to reject null hypothesis. 

Example:
In courtroom. Someone pled to be not guilty.
Null Hypothesis: He is innocent.
Alternative Hypothesis: He is guilty.

Court run a trial. Collect evidence and then ask if the null hypothesis were true how likely is it to get the result like this? That probability is called as P-Value.

• Key Concepts

1. The P-Value
   Assuming null hypothesis were true how likely your results (or something extreme?) it’s a probability. A small p-value means your data is rejecting the null hypothesis.
   Conventionally less than 5% P-value is considered as statistically significant, to reject the null hypothesis. 
2. Significance Level (α)
   It’s a threshold. How much false positive risk you are willing to accept. The most common value is 5%. But it depends on the stakes. In stakes situation the threshold is much lower. 
3. Type I and Type II Errors. 

• When you reject null hypothesis when it’s True that’s Type I error. which is False Positive. You conclude a effect when there is no effect. 
• When you fail to reject null hypothesis, when alternative is true that’s Type II error. it’s failed detection which is False Negative. You failed to detect the effect. 

4. Statistical Power
   If effect is real, how likely our experiment will detect it? Higher power means less Type II errors and less missed discoveries. 

Hypothesis Testing isn’t just statistical technique – it’s philosophy of honest inquiry. It asks us to be explicit about our assumptions, transparent about our methods, and humble about uncertainty.
To make sense of noisy world, understanding Hypothesis Testing gives you superpower. to distinguish between what evidence actually support from your confirmation bias.