22 Apr How to Calculate P Value using Chi-Square

How to Calculate the P Value

If you can add, subtract, multiply, and divide, you can find success when it comes to statistics. All you need is a little practice. Just like calculating standard deviation, there are different ways of calculating the P Value. But the easiest, most common method—and the one we’ll look at right now—is using the chi-square.

What is the P Value?

The P value is a statistical measurement used in determining whether or not a hypothesis is correct. Depending on the value of P, you’ll be able to say something like, “With 95% confidence, the experiment in question had an effect.”

For example, let’s say every day you go fishing and catch anywhere from 5 to 9 fish a particular hour in the morning. Let’s say you go out an hour earlier for seven days, and each time you get between 4 and 8 fish. Did the hour actually matter? Or did you just have a bad week? That’s what P will let you see.

Got the idea? Great. Let’s get going.

Step One: Calculate Your Expected Results

This is pretty simple. Just jot down whatever it is you consider to be expected when it comes to whatever you’re testing. Let’s say we have 300 marbles in a bag: 200 yellow, 100 blue.

Bob says he has psychic powers, and with his eyes closed, he says he can pull out more blue than yellow than normal.

You want to know whether or not he’s actually psychic or just getting lucky a few times, so you have him pick 60 marbles. If he chose the marbles at random, you’d expect him to get 40 yellow and 20 blue on average.

Step Two: Write Down Observed Results

For the example, we have Bob close his eyes, and he draws 60 marbles. He comes up with 35 yellow and 25 blue. 

Step Three: Degrees of Freedom

The Degree of Freedom (Df) is a number representing how much variation there is involved in the research.

It’s also easy to calculate: 

Df = n-1 

…where n equals how many variables you had. In this case, it’s marbles and n = 2. As such, Df = 1.

If we had four different marbles, Df would be 3.

Step Four: Compare the Results with a Chi-Square

Now we want to work in our chi-square. This might be where some people get a little lost, but if you break down the steps, it’s not that hard at all. First, the formula:

x2  = Σ((o-e) 2/e)

The sigma sign (Σ) means we add up all the values we get from what’s within the parenthesis ((o-e) 2/e). 

“O” is our observed value, and “e” is our expected. This is used for each variation. In this case, 25 blue and 35 yellow marbles.

If we plug those into our formula:

(35-40) 2/40) for yellow and (25-10) 2/10) for blue.

Now remember, we want to add those two results together for a main equation:

x2  = Σ((o-e) 2/e)

x2 = (35-40) 2/40) + (25-10) 2/10)

x2 = (-5 2/40) + (15) 2 / 10)

x2  = 25/40 + 225/10

x2  = 23.125

x = 4.808

Step Five: Significance Level

Now that the major math is done, you can relax. For this step, just choose how certain you want to be of your results. 

Significance levels are written as a decimal.  Researchers are usually happy using 0.05. If they want to be quite certain, they use 0.01. And if they basically want it to be as close to ironclad as they can get without going insane, it’s 0.001 (rarely used). 

Note: the higher the significance level, the harder it is to say your experiment confirms your hypothesis. Or to put it in our scenario’s terms, the higher the significance level, the more blue marbles Bob will need to draw.

Go ahead and assign your own number here between 0 and 1, but try and stay within 0.001 and 0.1. 

Step Six: Check the Chi Table

Another easy step here. The column on the left is how many degrees of freedom you have (one in our case). The numbers across the top will be our P value. What we want to do is start at the farthest left cell for our appropriate degrees of freedom (0.000) in this case, and keep reading to the right until we hit a number that’s GREATER than our result from step four (4.808).

Let’s look:

Description automatically generated

The first column greater than 4.808 is under 0.025 at 5.02. This means that our P value is at MOST 0.05 (3.84 < 4.808) but LESS than 0.025 (5.02 > 4.808). Thus, we assign a P value of 0.05.

Step Seven: Accept or Reject Your Hypothesis

While not technically part of finding P, at this point, you can determine whether or not your experiment was a success. Do this by seeing if:

P <= Significance Level (SL)

For our example:

0.05 <= Significance Level

Let’s say you picked 0.05 as your SL. Since 0.05 <= 0.05, your hypothesis can be accepted: Bob has psychic powers (or at least, he’s drawing marbles well enough beyond chance. Maybe he’s cheating?).

But…

If your SL was 0.01, we get:

0.05 > 0.01

Then we reject the hypothesis, shrug our shoulders, and say, “Sorry, Bob, chance still could’ve helped you get more blue marbles.”