Overview
The four goals of science are sometimes defined as description, prediction, explanation and control. The scientific imagination often leads to questions like: "I wonder what would happen if we did this?" or "I wonder what is the state of the world?" If our domain-specific knowledge is good we should be able to make accurate predictions. If the phenomena is unpredictable, we should be able to predict properties of the randomness.Prediction serves several purposes. These include to:
- Encourage a confirmatory orientation to prediction and discourage post hoc reasoning
- Test the predictive power of a theory
- Compare the predictive power of alternative theories
- Require researchers to demonstrate an understanding of the implications of existing theories and research findings
Properties of a Predictions
Predictions have various properties. Predictions answer questions about the state of the world. Scientific predictions should be justified.Rationale
Predictions should have a rationale. The rationale explains why the prediction is made.One type of rationale aims to show why a prediction is accurate. If the prediction is based on theory, evidence may be led about the prior predictive success of the theory and the relevance of the theory to the present circumstances. If the prediction is based on analogy to previous research, evidence may be led about the results of the previous research and the similarities with the present study.
A second type of rationale aims to show that the prediction is consistent with assumptions. In particular, if a theory is used to justify a prediction, the relationship between the theory and the predictions need to be clearly and logically articulated.
The source of the rationale for a prediction can come from many sources. Common sources include: theory; simulation, common sense; personal belief; hunch; and prior empirical findings. In some settings specific quantitative models can be used to make predictions. For example, in cognitive psychology cognitive architectures such as ACT-R can be used to generate quantitative predictions.
Research Question
A prediction answers a question. Answering questions is the basis of expanding knowledge and represents a common aim of empirical reports. For example, a prediction that the relationship between practice and performance follows a power function is a potential answer to several questions. At the simplest level, it could be rephrased as "is the relationship between practice and performance described well by a power function?" More broadly it could be expressed as "what is the relationship between practice and performance?". Thus, the scientific method of reporting results reiterates ideas through the process of aims, questions, predictions, results, and conclusions.Researcher's Belief in Expressed Prediction
The relationship between the prediction and the nature of the belief can vary. Typically, predictions are presented in such a way that the writing suggests that the researcher finds the prediction plausible. However, researchers can present predictions which they do not believe. A researcher can say that a particular theory would make a given prediction, but that they themselves believe something else.Even when researchers make a prediction that they find persuasive, their strength of belief can vary. This should vary based on the strength of the available evidence. Any prediction leading to an experiment suggests uncertainty. Because if the outcome of the experiment does not have the potential to alter your beliefs, then there is no point in doing it. And such potential suggests uncertainty.
Types of predictions
Predictions can be distinguished in various ways.Abstract vs operational
Predictions can be expressed at various levels of generality. Operational predictions refer to predictions made in a specific study when the measurements of particular constructs has been set out. Abstract predictions do not specify one or more of the following: context, design, task, types of participant, or measurement approach. For example, the idea that practice improves performance places no explicit limits on how performance is measured, what the task is, who is learning, or what constitutes practice.In general, theories make abstract predictions. These predictions then need to be operationalised using the specific measurement procedure used in the study.
The benefits of abstraction is that it reflects a claim of generalisation. The negative side of abstraction is that it introduces ambiguity as researchers differ in their interpretation or in what operationalisations are appropriate.
Qualitative versus quantitative
Predictions differ in the degree to which they place constraints on allowable outcomes. In the context of a bivariate relationship, such as when looking at difference between two groups on a numeric dependent variable (e.g., performance in a control versus a training group) or the relationship between two numeric variables (e.g., IQ and grades), the following distinctions apply:- Presence or Absence: Is there a relationship or not? Is a difference between group means expected?
- Direction: Is the correlation positive or negative? Which group has the higher mean?
- Qualitative Degree: What is the size of the relationship? What is the size of the difference between group means? In psychology, this often makes reference to the rules of thumb for small, medium, and large effects proposed by Jacob Cohen. However, many terms can be used that give some sense of the degree or extent.
- Quantitative Degree: What is the predicted correlation? What is the expected difference between group means? This is a precise numeric statement of the expected relationship. Such predictions are often based on meta analyses, prior studies, or quantitative models of the process.
Predictions based on degree also require different approach to formally test. Predictions based on presence or absence are typically tested against using two sided significance tests getting the extremity of the data given assuming a null hypothesis. Similarly directional hypotheses can be evaluated by one-sided tests.
Single versus grouped
A written prediction may apply to a single result or to a grouped set of results. Grouped results are common where variables can be grouped into sets and the same prediction applies to various combinations of the sets.As an example of a grouped hypothesis, Ackerman and Wolman (2007, p.61) wrote "mean self-estimates of abilities will decline from pretest to posttest assessment". Their study included four different types of self-estimates of ability. Because the nature of the prediction was the same for all four variables, the hypotheses could be grouped.
Grouped predictions have the benefit of being concise.
Some specific instances of grouped results include:
- Group differences on a set of dependent variables
- A set of predictor variables and one outcome variable
- A mapping between one set of variables and another set: e.g., when five personality variables are measured both using self-report and other report, the prediction is that self-report and other-report versions of the same variable will correlate more highly than other combinations of the variables.
Tested versus untested
Researchers generally make predictions that the data will either confirm or disconfirm. However, some predictions reflect aspects of what the researcher thinks will happen, but will not be verified. This may sometimes be necessary as a transitional step towards a testable prediction. For example, a researchers might predict that participants will shift strategy on a task and that this would result in improved performance even though only performance is measured and not strategy use.Univariate, bivariate, and multivariate predictions:
Another way of distinguishing predictions is with regard to the nature of the statistic that the data concerns. In general predictions can concern any aspect of the data. In practice in psychology many predictions concern bivariate relationships.Significance testing
The language of hypotheses is closely connected with null hypothesis significance testing (NHST). However, NHST is not the only way to assess whether data are consistent with predictions. It represents a particular inferential procedure.Writing the prediction section
This section discusses some of the issues related to writing predictions in an empirical report.Where to write predictions?
In psychological empirical reports predictions are often presented near the end of the Introduction in a section titled something like "The Current Study". I have previously discussed how to write the introduction. The earlier part of the introduction typically sets out the context, aims, and literature review. These earlier sections provide much of the context and justification for the predictions. The first half of "The Current Study" section of the introduction is typically devoted to a brief overview of the study. The second half then presents the predictions and research questions. Reference is often made back to the literature review to reiterate the rationale for each prediction.An alternative writing model involves presenting predictions throughout the introduction. This involves reviewing the literature and using it to derive predictions. An example of this approach can be seen Keith and Frese's article (2008, Effectiveness of Error Management Training).
Should predictions be recorded?
Some empirical reports do not include predictions. Instead, questions, aims, and research interests are expressed. This may be more appropriate when the domain is novel and the study is exploratory. Word count constraints may also requires that some or all of the potential predictions are left implicit.Which predictions should be included?
The results section of empirical reports typically presents many relationships. It is rare for all of these relationships to have written predictions. For example, a study that presents a correlation matrix of ten variables contains 45 separate correlations. It would be tedious to read a prediction for each correlation.Predictions are more likely to be included in written form if:
- They pertain to the novel contribution of the study
- They have theoretical significance
- They are not obvious
- There are many possible analyses that could be done and the aim is to show that the chosen analyses were selected a priori
In what order should predictions be included?
Once a decision has been made about which predictions will be put in writing, consideration needs to be given to how to sequence these predictions. The sequence often mirrors the presentation in the literature review, results, and discussion. Hypotheses can often be grouped into sets and sets should be presented together.How should predictions be written?
Common words used to express prediction include "predict", "expect", "assume", "hypothesise", "propose", "hope to show", and "suspect".Common words used to describe the object of the investigation include: "explore", "examine", "investigate", "interested in". They are sometimes preceded by words like "aim" or "purpose" (e.g., the aim of the study was to explore ..."). These words are then combined with words in the form of: (a) a question (e.g., "examine what causes performance to improve), (b) a relationship (e.g., "explore the relationship between practice and performance"), (c) or a statement (e.g., "whether practice is related to performance").
Some studies use a personal pronoun (e.g., "we expected", "we predicted"), whereas others use the passive voice (e.g., "it was predicted"). In some contexts the active voice is used without the personal pronoun (e.g., "the study aimed to show", "the purpose of the experiment was").
Enumerated Hypotheses Some studies present hypotheses with enumerated titles (e.g., Hypothesis 1:...; Hypothesis 2:...; etc. or H1, H2, H3). The number scheme can also include a second hierarchical level with letters (e.g., 1a, 1b, 2, 3a, 3b, 3c, etc.). The hierarchy can be used to group related hypotheses. In this form, hypotheses are expressed as present-tense statements of fact (e.g., "intelligence is positively correlated with performance")
Various formatting conventions exist regarding enumerated hypotheses. They are commonly presented indented, aligned vertically and in a separate paragraph. The title of the hypothesis (e.g., "Hypothesis 1a") is often italicised.
There are several benefits to enumerated hypotheses. It forces you be explicit about
Differences in terminology
How are the terms of hypothesis, prediction, thesis, and theory different and similar? In some instances these terms are used interchangeably. They are also used differently across people and across disciplines. However, there are important distinctions to be made in the underlying constructs. Thus, even if these distinctions are not captured in people's use of terms, it is important to be aware of the distinctions.Hypothesis
Robson (2002) defines an hypothesis as "the predicted answer to a research question" (p.65). Chanrasekhar (2002, p.6) states that "the hypothesis defines the aim or objective of an experiment, that if some likely but unproven proposition were indeed true, we would expect to make certain observations or measurements." Chanrasekhar (2002, p.7) cites Holtom and Fisher (1999, p.18) stating that "a hypothesis is an imaginative preconception of what might be true in the form of a declaration with verifiable deductive consequences." Thus, while Key elements of a hypothesis:- declarative statement of possible truth
- proof of its truth is not currently known
- with logical consequences
- consistent with what is known in the literature
- testable
- tested by the empirical study presented
Thesis Statement
The whole document is called a thesis. And the main claim of that document is the thesis. A thesis may also be a hypothesis. However, you are arguing in light of the prior research and your new evidence that the hypothesis is correct. In the narrative of research hypotheses are presented prior to conducting research. Thesis statements integrate prior expectations informed by reasoning and prior research with the empirical results of the thesis.Prediction
Hypotheses are more general than predictionsOne model states that hypotheses are used to generate predictions. Thus, hypotheses tend to be general, whereas predictions tend to be concrete. In the context of writing up an empirical report, hypotheses provide possible answers to the motivating research question, whereas predictions pertain to the expected outcomes of the current studies being presented.
Theory
Theories tend to be more general, include more variables. A set of hypotheses combined might make a theory. Some commentators suggest that theories are what hypotheses become when they achieve a certain amount of support. However, a theory does not need to be supported to be a theory. A theory can be bad and still be a theory. Andre Kukla's writing has many interesting ideas about theorising in psychology.Research Question
Research Aim
Is the aim of a thesis any more than the answering of a research question? Examples of aims that are not directly answering a research question: * refine a methodology (measurement, design, quantitative analysis) * develop a theory * provide additional empirical data Methodological contributions may relate to many research questions and so so indirectly. The development of a theory does relate to answering a research question, but it represents a particular pathway to that contribution. Likewise, empirical data may contribute to more than one research question, and it represents a particular pathway. Thus, an aim may be less directAdditional Resources
References
- Robson, C. (2002). Real World Research. Blackwell.
- D. Holtom and E. Fisher, Enjoy Writing Your Science Thesis or Dissertation!: A step by step guide to planning and writing dissertations and theses for undergraduate and graduate science students. London, UK: Imperial College Press, 1999.
