4.3 Hypothesis Formulation and Hypothesis Testing

In data analysis and research, a hypothesis is a testable prediction about the relationship between variables. Hypothesis testing is the statistical process used to determine whether the evidence supports or refutes that prediction.

---

What is a Hypothesis?

A hypothesis is a specific, measurable, and falsifiable statement that predicts a relationship between two or more variables.

Example:

A good hypothesis must be:

• Specific — clearly defines the variables involved
• Testable — data can be collected to support or refute it
• Falsifiable — it is possible for the hypothesis to be proven wrong

---

Types of Hypotheses

Null Hypothesis ($H_0$)

The Null Hypothesis states that there is no significant difference or effect between the variables being studied. It is the default assumption.

Alternative Hypothesis ($H_1$)

The Alternative Hypothesis is the statement the researcher aims to support. It claims that a significant relationship or difference does exist.

---

Variables in a Hypothesis

---

Hypothesis Testing Process

Hypothesis testing follows a structured process:

1. Formulate $H_0$ and $H_1$
2. Collect data through experiments or observations (e.g., A/B Testing)
3. Analyze data using statistical methods
4. Calculate the P-value
5. Make a decision — reject or fail to reject $H_0$
6. Communicate findings using data visualizations

---

The P-Value

The P-value is a statistical measure that indicates the probability that the observed results occurred by chance.

$p<\alpha \Rightarrow \text{Reject }{H}_0$

• The standard significance level is $\alpha =0.05$
• If $p<0.05$: the result is statistically significant → reject $H_0$, support $H_1$
• If $p\ge 0.05$: insufficient evidence to reject $H_0$

Example: A P-value of $0.03$ means there is only a 3% chance the result is due to random chance. Since $0.03<0.05$, we reject $H_0$.

---

A/B Testing

A/B Testing (split testing) is one of the most common methods used to validate hypotheses about user interfaces and system designs.

• Version A = the original/control design
• Version B = the modified/experimental design
• Users are randomly assigned to each version
• Data is collected and compared statistically

---

Communicating Findings with Data Visualizations

After hypothesis testing, results must be communicated clearly. Advanced data visualizations are used to:

• Show the distribution of results (box plots, histograms)
• Highlight statistically significant differences (bar charts with error bars)
• Display relationships between variables (scatter plots)
• Tie observed patterns back to the original hypothesis

Choosing the Right Visual

---

Worked Example

Research Question: Does increasing font size improve readability for elderly users?