Understanding Factorial Designs
The fastest way to understand a full factorial design is to realize that it is:
- An experimental design that looks at the EFFECTS of 2 Causes on 1 Outcome variable
- An experimental design that tests the effects of AT LEAST 2 levels of each Cause (Cause 1, high amount, low amount, Cause 2, high amount, low amount).
- An experimental design that lets us understand more about multiple causality by allowing us to efficiently test for the effects of each independent variable (cause) ALONE (called a main effect), and for their combined effect, called the interaction.
- To conduct this experiment, we need at least two potential causal factors such as amount of caffeine and amount of sleep and one outcome measurement such as reaction time on a test of your choice. We also need to decide how much caffeine and sleep to test in the experiment. These will be the levels of the independent variables and could be: Caffeine (high = 2 cups coffee, low =1 cup coffee), Sleep (high = 8 hours, low = 6 hours). Out outcome variable could be reaction times as measured in the Sheep Dash Game
To know more, we need to understand the history, definitions, and additional information shown below:
Brief History
The science of experimental design can largely be credited to Ronald Fischer throughout his work from 1912 through the 1960s. (Krishnan, 1997). Fischer believed that multivariate designs were the most efficient way to answer questions and that nature is best understood by asking more than one good question at a time. “We have usually no knowledge that any one factor will exert its effects independently of all others that can be varied, or that its effects are particularly simply related to variations in these other factors.” -R.Fischer
Why bother learning history? Context and depth: “The tendency of modern scientific teaching is to neglect the great books, to lay far too much stress upon relatively unimportant modern work, and to present masses of detail of doubtful truth and questionable weight in such a way as to obscure principles.” -R.Fischer.
General Purpose: Everything has multiple causes, why run an experiment with only one?
Like all experimental designs, a factorial design is a specific variety of experiment meant to answer a question about multiple causality and effect. It requires us to generate ideas about what potential causes, choose the specific variables we feel might be most important, decide how to define them, choose a design, choose how to conduct the experiment and then analyze the numeric results using statistical methods.
The time and expense of experiments is always a consideration. This is why we need to consider the differences in cost-effectiveness, efficiency, and explanatory power of the different designs.
Specific Purpose: There are an infinite number of possible designs, but the simplest factorial design looks only at the effects of 2 potential causes (independent variables) on one outcome (dependent variable) of interest that is worth trying to understand. In order to have a chance to see the effects of a cause or “treatment” we need to vary the amount of that cause using at least two different amounts. These different amounts of the cause or “treatment” are called “levels”.
Therefore, the simplest factorial design has just two factors, two levels of each of those factors, and a single outcome variable. This is called a 2×2 factorial design.
Vocabulary reminder:
- Causes are also called factors, independent variables and/or treatments.
- Levels are the specific sub-categories or amounts of each factor. Experimenters choose these levels (e.g. 1 cup of coffee vs. 2 cups of coffee) when possible or notice these levels when they appear by chance (e.g. male/female are the two natural levels of the variable “sex”.
- Variables, independent variable (cause), dependent variable (effect).
- Main effect (the effect of each independent variable taken separately)
- Interaction effect (the combined effect of all independent variables)
If we want to know the effects of caffeine and sleep on people’s reaction times, a 2×2 factorial experiment would use 2 different amounts (levels) of caffeine [1 cup of coffee vs 2 cups of coffee] and 2 different amounts of sleep [6 hours per night vs 8 hours per night] as the independent variables. We could either test the same people in all conditions, a repeated-measures design, or have different people in each of the possible groups:
Group 1 [High caffeine, high sleep], Group 2 [High caffeine, low sleep]
Group 3 {Low caffeine, high sleep], Group 4 [Low caffeine, low sleep]
As it is possible to use an infinite number of caffeine and sleep amounts for the experiment, the simplest experiment would use only two typical levels of each as shown above.
Required conditions for this experiment: Ideally, everything other than the amount of caffeine and amount of sleep is held constant so that nothing other than those 2 factors can have an influence on people’s reaction time performance. If this is violated and something else influences reaction times, then the experiment cannot make a strong claim about cause and effect.
Definitions of a factorial design:
1) “In a factorial design, each level of each independent variable is paired with each level of each other independent variable. Thus, a 2 x 3 factorial design consists of the 6 possible combinations of the levels of the independent variables.” From the excellent Onlinestatbook.com
2) “A design in which every setting of every factor appears with every setting of every other factor is a full factorial design” From the Engineering and Statistics Handbook (http://www.itl.nist.gov/div898/handbook/pri/section3/pri333.htm):
3) “a full factorial experiment is an experiment whose design consists of two or more factors, each with discrete possible values or “levels”, and whose experimental units take on all possible combinations of these levels across all such factors. A full factorial design may also be called a fully crossed design. Such an experiment allows the investigator to study the effect of each factor on the response variable, as well as the effects of interactions between factors on the response variable.” From the often maligned but extremely useful Wikipedia.com (http://en.wikipedia.org/wiki/Factorial_experiment)
What are the most common designs?
Above 5 factors, the experiments become very time/labor intensive and essentially too expensive to bother with.
Why? “If there are k factors, each at 2 levels, a full factorial design has 2k runs.” NIST
| Number of Factors | Number of Runs |
| 2 | 4 |
| 3 | 8 |
| 4 | 16 |
| 5 | 32 |
| 6 | 64 |
| 7 | 128 |
With so many conditions to test with more than 5 factors, this makes experiments too painful for many groups to conduct. Instead, more efficient designs for larger numbers of factors are listed here http://www.itl.nist.gov/div898/handbook/pri/section3/pri33.htm#Design%20Selection
Paul Greenberg, 2015
Brain Scientist, Entrepreneur, Speaker, Professor - Everybody has more potential than can be acted upon. It is a joy to help people of all ages tap into more of this long-term potential while also getting immediate benefits in daily life. Fast training programs make it simpler for you to use neuroscience and free "thought support" software to simplify & improve life, work, and achievement in the Information Overload Age. Write with your questions.