Once you have a research question, hypothesis, and prediction, the next step is to develop the experimental design. In this section, we talk about the basics of experimental design, participants, and items.
Experimental design is the means by which hypotheses are operationalized. In other words, experimental design is how you collect controlled observations (i.e., data) about the relationship among the variables of interest. Note that our hypotheses often involve predictors that are continuous variables. The predictor variable we've discussed on the previous page—the relative frequency of words—is an example of a continuous predictor. For the purpose of experimental designs, continuous predictors are often binned into a small number of bins (though this is not always desirable, it is a common convention and for the current purpose, it will make it easier to talk about the design options you have for your experiment). Binning turns continuous variables into categorical variables. For example, we might bin words based on their relative frequency into "high" and "low" frequency words.
There are also variables that are naturally considered categorical. For example, if we hypothesized that nouns are read more quickly than verbs, "noun" vs. "verb" would be a categorical predictor.
Just like predictors, outcomes can be continuous or categorical. Reading times are an example of a continous outcome. The accuracy of an answer to a comprehension question (which is either true or false) is an example of a categorical outcome.
The first step in experimental design is to specify the conditions (e.g., test conditions and control conditions) that allow you to test your hypothesis. The conditions describe how we plan to manipulate our predictor variables. The goal of this manipulation is to test whether we will observe the change in the outcome variables predicted by our hypothesis.
To continue the example introduced above, consider that we plan to bin relative frequency into "high" and "low" frequency words. We would then design our experiment to have a high-frequency condition and a low-frequency condition. Our prediction would be that reading times will be faster in the high-frequency condition than in the low-frequency condition.
Without further additions, we would call this a by-2 design, because we have one predictor variable in two conditions and there are no other manipulations in the experiment. We will go through a more detailed example in Materials section, but first we'll establish a little bit more terminology.
1. What is a condition?
2. True or False: Many variables should be simultaneously changed to differentiate between conditions.
3. Researchers want to test whether a time delay between presentation of the first stimuli and the second stimuli will affect how fast and how accurately participants match them. Which type of design should they use?
4. Researchers want to test whether students who participated in sports performed better on different types of reading tasks than students who did not. Which type of design should they use?