Based on the “Statistical Consulting Cheatsheet” by Prof. Kris Sankaran
Contingency tables are a useful technique for studying the relationship between categorical variables. Though it’s possible to study K-way contingency tables (relating K categorical variables), we’ll focus on 2 \(\times\) 2 tables, which relate two categorical variables with two levels each. These can be represented like in the table (see example below).
| \(A_1\) | \(A_2\) | total | |
| \(B_1\) | \(n_{11}\) | \(n_{12}\) | \(n_{1\cdot}\) |
| \(B_1\) | \(n_{21}\) | \(n_{22}\) | \(n_{2\cdot}\) |
| total | \(n_{\cdot 1}\) | \(n_{\cdot 2 }\) | \(n_{\cdot\cdot}\) |
We usually imagine a sampling mechanism that leads to this table, where the probability that a sample lands in cell \(ij\) is \(p_{ij}\) . Hypotheses are then formulated in terms of these \(p_{ij}\).
A few summary statistics of 2 \(\times\)2 tables are referred to across a variety of tests:
The most common tests for contingency tables such as this one include:
The \(\chi_2\) test is often used to study whether or not two categorical variables in a contingency table are related. More formally, it assesses the plausibility of the null hypothesis of independence:
\[H_0 : p_{ij} = p_{i\cdot}p_{\cdot j}\]The two most common statistics used to evaluate discrepancies the Pearson and likelihood ratio \(\chi_2\) statistics, which measure the deviation from the expected count under the null:
Under the null hypotheses, and assuming large enough sample sizes, these are both \(\chi_2\) distributed, with degrees of freedom determined by the number of levels in each categorical variable. A useful follow-up step when the null is rejected is to see which cell(s) con- tributed to the most to the \(\chi_2\)-statistic. These are sometimes called Pearson residuals.
Fisher’s Exact test is an alternative to the \(\chi_2\)-test that is useful when the counts within the contingency table are small and the \(\chi_2\)-approximation is not necessarily reliable.
It tests the same null hypothesis of independence as the \(\chi_2\)-test
Under that null, and assuming a binomial sampling mechanism (condition on the row and column totals), the count of the top-left cell can be shown to follow a hypergeometric distribution (and this cell determines counts in all other cells).
This can be used to determine the probability of seeing tables with as much or more extreme departures from independence.
There is a generalization to \(I \times J\) tables, based on the multiple hypergeometric distribution.
The Cochran-Mantel Haenzel test is a variant of the exact test that applies when samples have been stratified across \(K\) groups, yielding K \(2\times 2\) separate contingency tables.
McNemar’s test is used to test symmetry in marginal probabilities across the diagonal in a contingency table. More formally, the null hypothesis asks whether the running marginal probabilities across rows and columns are the same: \(p_{i\cdot} = p_{\cdot i}\) for all i.
This is the so-called ‘‘test of marginal homogeneity,” it it is often used to see whether a treatment had any effect. For example, if the rows of the contingency table measure whether someone is sick before the treatment and the columns measure whether they were still sick afterwards, then if the probability that they are sick has not changed between the timepoints, then the treatment has had no effect (and the null hypothesis of marginal homogeneity holds).
The test statistic used by McNemar’s test is given by
\[\frac{n_{21} - n_{12}}{\sqrt{n_{21} + n_{12}}}\]which measures the discrepancy in off-diagonal elements.