experimental design statistical model

• Skip to secondary menu
• Skip to main content
• Skip to primary sidebar

Statistics By Jim

Making statistics intuitive

Experimental Design: Definition and Types

By Jim Frost 3 Comments

What is Experimental Design?

An experimental design is a detailed plan for collecting and using data to identify causal relationships. Through careful planning, the design of experiments allows your data collection efforts to have a reasonable chance of detecting effects and testing hypotheses that answer your research questions.

An experiment is a data collection procedure that occurs in controlled conditions to identify and understand causal relationships between variables. Researchers can use many potential designs. The ultimate choice depends on their research question, resources, goals, and constraints. In some fields of study, researchers refer to experimental design as the design of experiments (DOE). Both terms are synonymous.

Ultimately, the design of experiments helps ensure that your procedures and data will evaluate your research question effectively. Without an experimental design, you might waste your efforts in a process that, for many potential reasons, can’t answer your research question. In short, it helps you trust your results.

Learn more about Independent and Dependent Variables .

Design of Experiments: Goals & Settings

Experiments occur in many settings, ranging from psychology, social sciences, medicine, physics, engineering, and industrial and service sectors. Typically, experimental goals are to discover a previously unknown effect , confirm a known effect, or test a hypothesis.

Effects represent causal relationships between variables. For example, in a medical experiment, does the new medicine cause an improvement in health outcomes? If so, the medicine has a causal effect on the outcome.

An experimental design’s focus depends on the subject area and can include the following goals:

• Understanding the relationships between variables.
• Identifying the variables that have the largest impact on the outcomes.
• Finding the input variable settings that produce an optimal result.

For example, psychologists have conducted experiments to understand how conformity affects decision-making. Sociologists have performed experiments to determine whether ethnicity affects the public reaction to staged bike thefts. These experiments map out the causal relationships between variables, and their primary goal is to understand the role of various factors.

Conversely, in a manufacturing environment, the researchers might use an experimental design to find the factors that most effectively improve their product’s strength, identify the optimal manufacturing settings, and do all that while accounting for various constraints. In short, a manufacturer’s goal is often to use experiments to improve their products cost-effectively.

In a medical experiment, the goal might be to quantify the medicine’s effect and find the optimum dosage.

Developing an Experimental Design

Developing an experimental design involves planning that maximizes the potential to collect data that is both trustworthy and able to detect causal relationships. Specifically, these studies aim to see effects when they exist in the population the researchers are studying, preferentially favor causal effects, isolate each factor’s true effect from potential confounders, and produce conclusions that you can generalize to the real world.

To accomplish these goals, experimental designs carefully manage data validity and reliability , and internal and external experimental validity. When your experiment is valid and reliable, you can expect your procedures and data to produce trustworthy results.

An excellent experimental design involves the following:

• Lots of preplanning.
• Developing experimental treatments.
• Determining how to assign subjects to treatment groups.

The remainder of this article focuses on how experimental designs incorporate these essential items to accomplish their research goals.

Learn more about Data Reliability vs. Validity and Internal and External Experimental Validity .

Preplanning, Defining, and Operationalizing for Design of Experiments

This phase of the design of experiments helps you identify critical variables, know how to measure them while ensuring reliability and validity, and understand the relationships between them. The review can also help you find ways to reduce sources of variability, which increases your ability to detect treatment effects. Notably, the literature review allows you to learn how similar studies designed their experiments and the challenges they faced.

Operationalizing a study involves taking your research question, using the background information you gathered, and formulating an actionable plan.

This process should produce a specific and testable hypothesis using data that you can reasonably collect given the resources available to the experiment.

• Null hypothesis : The jumping exercise intervention does not affect bone density.
• Alternative hypothesis : The jumping exercise intervention affects bone density.

To learn more about this early phase, read Five Steps for Conducting Scientific Studies with Statistical Analyses .

Formulating Treatments in Experimental Designs

In an experimental design, treatments are variables that the researchers control. They are the primary independent variables of interest. Researchers administer the treatment to the subjects or items in the experiment and want to know whether it causes changes in the outcome.

As the name implies, a treatment can be medical in nature, such as a new medicine or vaccine. But it’s a general term that applies to other things such as training programs, manufacturing settings, teaching methods, and types of fertilizers. I helped run an experiment where the treatment was a jumping exercise intervention that we hoped would increase bone density. All these treatment examples are things that potentially influence a measurable outcome.

Even when you know your treatment generally, you must carefully consider the amount. How large of a dose? If you’re comparing three different temperatures in a manufacturing process, how far apart are they? For my bone mineral density study, we had to determine how frequently the exercise sessions would occur and how long each lasted.

How you define the treatments in the design of experiments can affect your findings and the generalizability of your results.

Assigning Subjects to Experimental Groups

A crucial decision for all experimental designs is determining how researchers assign subjects to the experimental conditions—the treatment and control groups. The control group is often, but not always, the lack of a treatment. It serves as a basis for comparison by showing outcomes for subjects who don’t receive a treatment. Learn more about Control Groups .

How your experimental design assigns subjects to the groups affects how confident you can be that the findings represent true causal effects rather than mere correlation caused by confounders. Indeed, the assignment method influences how you control for confounding variables. This is the difference between correlation and causation .

Imagine a study finds that vitamin consumption correlates with better health outcomes. As a researcher, you want to be able to say that vitamin consumption causes the improvements. However, with the wrong experimental design, you might only be able to say there is an association. A confounder, and not the vitamins, might actually cause the health benefits.

Let’s explore some of the ways to assign subjects in design of experiments.

Completely Randomized Designs

A completely randomized experimental design randomly assigns all subjects to the treatment and control groups. You simply take each participant and use a random process to determine their group assignment. You can flip coins, roll a die, or use a computer. Randomized experiments must be prospective studies because they need to be able to control group assignment.

Random assignment in the design of experiments helps ensure that the groups are roughly equivalent at the beginning of the study. This equivalence at the start increases your confidence that any differences you see at the end were caused by the treatments. The randomization tends to equalize confounders between the experimental groups and, thereby, cancels out their effects, leaving only the treatment effects.

For example, in a vitamin study, the researchers can randomly assign participants to either the control or vitamin group. Because the groups are approximately equal when the experiment starts, if the health outcomes are different at the end of the study, the researchers can be confident that the vitamins caused those improvements.

Statisticians consider randomized experimental designs to be the best for identifying causal relationships.

If you can’t randomly assign subjects but want to draw causal conclusions about an intervention, consider using a quasi-experimental design .

Learn more about Randomized Controlled Trials and Random Assignment in Experiments .

Randomized Block Designs

Nuisance factors are variables that can affect the outcome, but they are not the researcher’s primary interest. Unfortunately, they can hide or distort the treatment results. When experimenters know about specific nuisance factors, they can use a randomized block design to minimize their impact.

This experimental design takes subjects with a shared “nuisance” characteristic and groups them into blocks. The participants in each block are then randomly assigned to the experimental groups. This process allows the experiment to control for known nuisance factors.

Blocking in the design of experiments reduces the impact of nuisance factors on experimental error. The analysis assesses the effects of the treatment within each block, which removes the variability between blocks. The result is that blocked experimental designs can reduce the impact of nuisance variables, increasing the ability to detect treatment effects accurately.

Suppose you’re testing various teaching methods. Because grade level likely affects educational outcomes, you might use grade level as a blocking factor. To use a randomized block design for this scenario, divide the participants by grade level and then randomly assign the members of each grade level to the experimental groups.

A standard guideline for an experimental design is to “Block what you can, randomize what you cannot.” Use blocking for a few primary nuisance factors. Then use random assignment to distribute the unblocked nuisance factors equally between the experimental conditions.

You can also use covariates to control nuisance factors. Learn about Covariates: Definition and Uses .

Observational Studies

In some experimental designs, randomly assigning subjects to the experimental conditions is impossible or unethical. The researchers simply can’t assign participants to the experimental groups. However, they can observe them in their natural groupings, measure the essential variables, and look for correlations. These observational studies are also known as quasi-experimental designs. Retrospective studies must be observational in nature because they look back at past events.

Imagine you’re studying the effects of depression on an activity. Clearly, you can’t randomly assign participants to the depression and control groups. But you can observe participants with and without depression and see how their task performance differs.

Observational studies let you perform research when you can’t control the treatment. However, quasi-experimental designs increase the problem of confounding variables. For this design of experiments, correlation does not necessarily imply causation. While special procedures can help control confounders in an observational study, you’re ultimately less confident that the results represent causal findings.

Learn more about Observational Studies .

For a good comparison, learn about the differences and tradeoffs between Observational Studies and Randomized Experiments .

Between-Subjects vs. Within-Subjects Experimental Designs

When you think of the design of experiments, you probably picture a treatment and control group. Researchers assign participants to only one of these groups, so each group contains entirely different subjects than the other groups. Analysts compare the groups at the end of the experiment. Statisticians refer to this method as a between-subjects, or independent measures, experimental design.

In a between-subjects design , you can have more than one treatment group, but each subject is exposed to only one condition, the control group or one of the treatment groups.

A potential downside to this approach is that differences between groups at the beginning can affect the results at the end. As you’ve read earlier, random assignment can reduce those differences, but it is imperfect. There will always be some variability between the groups.

In a  within-subjects experimental design , also known as repeated measures, subjects experience all treatment conditions and are measured for each. Each subject acts as their own control, which reduces variability and increases the statistical power to detect effects.

In this experimental design, you minimize pre-existing differences between the experimental conditions because they all contain the same subjects. However, the order of treatments can affect the results. Beware of practice and fatigue effects. Learn more about Repeated Measures Designs .

Design of Experiments Examples

For example, a bone density study has three experimental groups—a control group, a stretching exercise group, and a jumping exercise group.

In a between-subjects experimental design, scientists randomly assign each participant to one of the three groups.

In a within-subjects design, all subjects experience the three conditions sequentially while the researchers measure bone density repeatedly. The procedure can switch the order of treatments for the participants to help reduce order effects.

Matched Pairs Experimental Design

A matched pairs experimental design is a between-subjects study that uses pairs of similar subjects. Researchers use this approach to reduce pre-existing differences between experimental groups. It’s yet another design of experiments method for reducing sources of variability.

Researchers identify variables likely to affect the outcome, such as demographics. When they pick a subject with a set of characteristics, they try to locate another participant with similar attributes to create a matched pair. Scientists randomly assign one member of a pair to the treatment group and the other to the control group.

On the plus side, this process creates two similar groups, and it doesn’t create treatment order effects. While matched pairs do not produce the perfectly matched groups of a within-subjects design (which uses the same subjects in all conditions), it aims to reduce variability between groups relative to a between-subjects study.

On the downside, finding matched pairs is very time-consuming. Additionally, if one member of a matched pair drops out, the other subject must leave the study too.

Learn more about Matched Pairs Design: Uses & Examples .

Another consideration is whether you’ll use a cross-sectional design (one point in time) or use a longitudinal study to track changes over time .

A case study is a research method that often serves as a precursor to a more rigorous experimental design by identifying research questions, variables, and hypotheses to test. Learn more about What is a Case Study? Definition & Examples .

In conclusion, the design of experiments is extremely sensitive to subject area concerns and the time and resources available to the researchers. Developing a suitable experimental design requires balancing a multitude of considerations. A successful design is necessary to obtain trustworthy answers to your research question and to have a reasonable chance of detecting treatment effects when they exist.

Share this:

Reader Interactions

March 23, 2024 at 2:35 pm

Dear Jim You wrote a superb document, I will use it in my Buistatistics course, along with your three books. Thank you very much! Miguel

March 23, 2024 at 5:43 pm

Thanks so much, Miguel! Glad this post was helpful and I trust the books will be as well.

April 10, 2023 at 4:36 am

What are the purpose and uses of experimental research design?

Comments and Questions Cancel reply

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Guide to Experimental Design | Overview, Steps, & Examples

Guide to Experimental Design | Overview, 5 steps & Examples

Published on December 3, 2019 by Rebecca Bevans . Revised on June 21, 2023.

Experiments are used to study causal relationships . You manipulate one or more independent variables and measure their effect on one or more dependent variables.

Experimental design create a set of procedures to systematically test a hypothesis . A good experimental design requires a strong understanding of the system you are studying.

There are five key steps in designing an experiment:

• Consider your variables and how they are related
• Write a specific, testable hypothesis
• Design experimental treatments to manipulate your independent variable
• Assign subjects to groups, either between-subjects or within-subjects
• Plan how you will measure your dependent variable

For valid conclusions, you also need to select a representative sample and control any  extraneous variables that might influence your results. If random assignment of participants to control and treatment groups is impossible, unethical, or highly difficult, consider an observational study instead. This minimizes several types of research bias, particularly sampling bias , survivorship bias , and attrition bias as time passes.

Table of contents

Step 1: define your variables, step 2: write your hypothesis, step 3: design your experimental treatments, step 4: assign your subjects to treatment groups, step 5: measure your dependent variable, other interesting articles, frequently asked questions about experiments.

You should begin with a specific research question . We will work with two research question examples, one from health sciences and one from ecology:

To translate your research question into an experimental hypothesis, you need to define the main variables and make predictions about how they are related.

Start by simply listing the independent and dependent variables .

Then you need to think about possible extraneous and confounding variables and consider how you might control  them in your experiment.

Finally, you can put these variables together into a diagram. Use arrows to show the possible relationships between variables and include signs to show the expected direction of the relationships.

Here we predict that increasing temperature will increase soil respiration and decrease soil moisture, while decreasing soil moisture will lead to decreased soil respiration.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Now that you have a strong conceptual understanding of the system you are studying, you should be able to write a specific, testable hypothesis that addresses your research question.

The next steps will describe how to design a controlled experiment . In a controlled experiment, you must be able to:

• Systematically and precisely manipulate the independent variable(s).
• Precisely measure the dependent variable(s).
• Control any potential confounding variables.

If your study system doesn’t match these criteria, there are other types of research you can use to answer your research question.

How you manipulate the independent variable can affect the experiment’s external validity – that is, the extent to which the results can be generalized and applied to the broader world.

First, you may need to decide how widely to vary your independent variable.

• just slightly above the natural range for your study region.
• over a wider range of temperatures to mimic future warming.
• over an extreme range that is beyond any possible natural variation.

Second, you may need to choose how finely to vary your independent variable. Sometimes this choice is made for you by your experimental system, but often you will need to decide, and this will affect how much you can infer from your results.

• a categorical variable : either as binary (yes/no) or as levels of a factor (no phone use, low phone use, high phone use).
• a continuous variable (minutes of phone use measured every night).

How you apply your experimental treatments to your test subjects is crucial for obtaining valid and reliable results.

First, you need to consider the study size : how many individuals will be included in the experiment? In general, the more subjects you include, the greater your experiment’s statistical power , which determines how much confidence you can have in your results.

Then you need to randomly assign your subjects to treatment groups . Each group receives a different level of the treatment (e.g. no phone use, low phone use, high phone use).

You should also include a control group , which receives no treatment. The control group tells us what would have happened to your test subjects without any experimental intervention.

When assigning your subjects to groups, there are two main choices you need to make:

• A completely randomized design vs a randomized block design .
• A between-subjects design vs a within-subjects design .

Randomization

An experiment can be completely randomized or randomized within blocks (aka strata):

• In a completely randomized design , every subject is assigned to a treatment group at random.
• In a randomized block design (aka stratified random design), subjects are first grouped according to a characteristic they share, and then randomly assigned to treatments within those groups.

Sometimes randomization isn’t practical or ethical , so researchers create partially-random or even non-random designs. An experimental design where treatments aren’t randomly assigned is called a quasi-experimental design .

Between-subjects vs. within-subjects

In a between-subjects design (also known as an independent measures design or classic ANOVA design), individuals receive only one of the possible levels of an experimental treatment.

In medical or social research, you might also use matched pairs within your between-subjects design to make sure that each treatment group contains the same variety of test subjects in the same proportions.

In a within-subjects design (also known as a repeated measures design), every individual receives each of the experimental treatments consecutively, and their responses to each treatment are measured.

Within-subjects or repeated measures can also refer to an experimental design where an effect emerges over time, and individual responses are measured over time in order to measure this effect as it emerges.

Counterbalancing (randomizing or reversing the order of treatments among subjects) is often used in within-subjects designs to ensure that the order of treatment application doesn’t influence the results of the experiment.

Finally, you need to decide how you’ll collect data on your dependent variable outcomes. You should aim for reliable and valid measurements that minimize research bias or error.

Some variables, like temperature, can be objectively measured with scientific instruments. Others may need to be operationalized to turn them into measurable observations.

• Ask participants to record what time they go to sleep and get up each day.
• Ask participants to wear a sleep tracker.

How precisely you measure your dependent variable also affects the kinds of statistical analysis you can use on your data.

Experiments are always context-dependent, and a good experimental design will take into account all of the unique considerations of your study system to produce information that is both valid and relevant to your research question.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Student’s  t -distribution
• Normal distribution
• Null and Alternative Hypotheses
• Chi square tests
• Confidence interval
• Cluster sampling
• Stratified sampling
• Data cleansing
• Reproducibility vs Replicability
• Peer review
• Likert scale

Research bias

• Implicit bias
• Framing effect
• Cognitive bias
• Placebo effect
• Hawthorne effect
• Hindsight bias
• Affect heuristic

Experimental design means planning a set of procedures to investigate a relationship between variables . To design a controlled experiment, you need:

• A testable hypothesis
• At least one independent variable that can be precisely manipulated
• At least one dependent variable that can be precisely measured

When designing the experiment, you decide:

• How you will manipulate the variable(s)
• How you will control for any potential confounding variables
• How many subjects or samples will be included in the study
• How subjects will be assigned to treatment levels

Experimental design is essential to the internal and external validity of your experiment.

The key difference between observational studies and experimental designs is that a well-done observational study does not influence the responses of participants, while experiments do have some sort of treatment condition applied to at least some participants by random assignment .

A confounding variable , also called a confounder or confounding factor, is a third variable in a study examining a potential cause-and-effect relationship.

A confounding variable is related to both the supposed cause and the supposed effect of the study. It can be difficult to separate the true effect of the independent variable from the effect of the confounding variable.

In your research design , it’s important to identify potential confounding variables and plan how you will reduce their impact.

In a between-subjects design , every participant experiences only one condition, and researchers assess group differences between participants in various conditions.

In a within-subjects design , each participant experiences all conditions, and researchers test the same participants repeatedly for differences between conditions.

The word “between” means that you’re comparing different conditions between groups, while the word “within” means you’re comparing different conditions within the same group.

An experimental group, also known as a treatment group, receives the treatment whose effect researchers wish to study, whereas a control group does not. They should be identical in all other ways.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bevans, R. (2023, June 21). Guide to Experimental Design | Overview, 5 steps & Examples. Scribbr. Retrieved July 23, 2024, from https://www.scribbr.com/methodology/experimental-design/

Is this article helpful?

Rebecca Bevans

Other students also liked, random assignment in experiments | introduction & examples, quasi-experimental design | definition, types & examples, how to write a lab report, what is your plagiarism score.

Statistical Design and Analysis of Biological Experiments

Chapter 1 principles of experimental design, 1.1 introduction.

The validity of conclusions drawn from a statistical analysis crucially hinges on the manner in which the data are acquired, and even the most sophisticated analysis will not rescue a flawed experiment. Planning an experiment and thinking about the details of data acquisition is so important for a successful analysis that R. A. Fisher—who single-handedly invented many of the experimental design techniques we are about to discuss—famously wrote

To call in the statistician after the experiment is done may be no more than asking him to perform a post-mortem examination: he may be able to say what the experiment died of. ( Fisher 1938 )

(Statistical) design of experiments provides the principles and methods for planning experiments and tailoring the data acquisition to an intended analysis. Design and analysis of an experiment are best considered as two aspects of the same enterprise: the goals of the analysis strongly inform an appropriate design, and the implemented design determines the possible analyses.

The primary aim of designing experiments is to ensure that valid statistical and scientific conclusions can be drawn that withstand the scrutiny of a determined skeptic. Good experimental design also considers that resources are used efficiently, and that estimates are sufficiently precise and hypothesis tests adequately powered. It protects our conclusions by excluding alternative interpretations or rendering them implausible. Three main pillars of experimental design are randomization , replication , and blocking , and we will flesh out their effects on the subsequent analysis as well as their implementation in an experimental design.

An experimental design is always tailored towards predefined (primary) analyses and an efficient analysis and unambiguous interpretation of the experimental data is often straightforward from a good design. This does not prevent us from doing additional analyses of interesting observations after the data are acquired, but these analyses can be subjected to more severe criticisms and conclusions are more tentative.

In this chapter, we provide the wider context for using experiments in a larger research enterprise and informally introduce the main statistical ideas of experimental design. We use a comparison of two samples as our main example to study how design choices affect an analysis, but postpone a formal quantitative analysis to the next chapters.

1.2 A Cautionary Tale

For illustrating some of the issues arising in the interplay of experimental design and analysis, we consider a simple example. We are interested in comparing the enzyme levels measured in processed blood samples from laboratory mice, when the sample processing is done either with a kit from a vendor A, or a kit from a competitor B. For this, we take 20 mice and randomly select 10 of them for sample preparation with kit A, while the blood samples of the remaining 10 mice are prepared with kit B. The experiment is illustrated in Figure 1.1 A and the resulting data are given in Table 1.1 .

One option for comparing the two kits is to look at the difference in average enzyme levels, and we find an average level of 10.32 for vendor A and 10.66 for vendor B. We would like to interpret their difference of -0.34 as the difference due to the two preparation kits and conclude whether the two kits give equal results or if measurements based on one kit are systematically different from those based on the other kit.

Such interpretation, however, is only valid if the two groups of mice and their measurements are identical in all aspects except the sample preparation kit. If we use one strain of mice for kit A and another strain for kit B, any difference might also be attributed to inherent differences between the strains. Similarly, if the measurements using kit B were conducted much later than those using kit A, any observed difference might be attributed to changes in, e.g., mice selected, batches of chemicals used, device calibration, or any number of other influences. None of these competing explanations for an observed difference can be excluded from the given data alone, but good experimental design allows us to render them (almost) arbitrarily implausible.

A second aspect for our analysis is the inherent uncertainty in our calculated difference: if we repeat the experiment, the observed difference will change each time, and this will be more pronounced for a smaller number of mice, among others. If we do not use a sufficient number of mice in our experiment, the uncertainty associated with the observed difference might be too large, such that random fluctuations become a plausible explanation for the observed difference. Systematic differences between the two kits, of practically relevant magnitude in either direction, might then be compatible with the data, and we can draw no reliable conclusions from our experiment.

In each case, the statistical analysis—no matter how clever—was doomed before the experiment was even started, while simple ideas from statistical design of experiments would have provided correct and robust results with interpretable conclusions.

1.3 The Language of Experimental Design

By an experiment we understand an investigation where the researcher has full control over selecting and altering the experimental conditions of interest, and we only consider investigations of this type. The selected experimental conditions are called treatments . An experiment is comparative if the responses to several treatments are to be compared or contrasted. The experimental units are the smallest subdivision of the experimental material to which a treatment can be assigned. All experimental units given the same treatment constitute a treatment group . Especially in biology, we often compare treatments to a control group to which some standard experimental conditions are applied; a typical example is using a placebo for the control group, and different drugs for the other treatment groups.

The values observed are called responses and are measured on the response units ; these are often identical to the experimental units but need not be. Multiple experimental units are sometimes combined into groupings or blocks , such as mice grouped by litter, or samples grouped by batches of chemicals used for their preparation. More generally, we call any grouping of the experimental material (even with group size one) a unit .

In our example, we selected the mice, used a single sample per mouse, deliberately chose the two specific vendors, and had full control over which kit to assign to which mouse. In other words, the two kits are the treatments and the mice are the experimental units. We took the measured enzyme level of a single sample from a mouse as our response, and samples are therefore the response units. The resulting experiment is comparative, because we contrast the enzyme levels between the two treatment groups.

Figure 1.1: Three designs to determine the difference between two preparation kits A and B based on four mice. A: One sample per mouse. Comparison between averages of samples with same kit. B: Two samples per mouse treated with the same kit. Comparison between averages of mice with same kit requires averaging responses for each mouse first. C: Two samples per mouse each treated with different kit. Comparison between two samples of each mouse, with differences averaged.

In this example, we can coalesce experimental and response units, because we have a single response per mouse and cannot distinguish a sample from a mouse in the analysis, as illustrated in Figure 1.1 A for four mice. Responses from mice with the same kit are averaged, and the kit difference is the difference between these two averages.

By contrast, if we take two samples per mouse and use the same kit for both samples, then the mice are still the experimental units, but each mouse now groups the two response units associated with it. Now, responses from the same mouse are first averaged, and these averages are used to calculate the difference between kits; even though eight measurements are available, this difference is still based on only four mice (Figure 1.1 B).

If we take two samples per mouse, but apply each kit to one of the two samples, then the samples are both the experimental and response units, while the mice are blocks that group the samples. Now, we calculate the difference between kits for each mouse, and then average these differences (Figure 1.1 C).

If we only use one kit and determine the average enzyme level, then this investigation is still an experiment, but is not comparative.

To summarize, the design of an experiment determines the logical structure of the experiment ; it consists of (i) a set of treatments (the two kits); (ii) a specification of the experimental units (animals, cell lines, samples) (the mice in Figure 1.1 A,B and the samples in Figure 1.1 C); (iii) a procedure for assigning treatments to units; and (iv) a specification of the response units and the quantity to be measured as a response (the samples and associated enzyme levels).

1.4 Experiment Validity

Before we embark on the more technical aspects of experimental design, we discuss three components for evaluating an experiment’s validity: construct validity , internal validity , and external validity . These criteria are well-established in areas such as educational and psychological research, and have more recently been discussed for animal research ( Würbel 2017 ) where experiments are increasingly scrutinized for their scientific rationale and their design and intended analyses.

1.4.1 Construct Validity

Construct validity concerns the choice of the experimental system for answering our research question. Is the system even capable of providing a relevant answer to the question?

Studying the mechanisms of a particular disease, for example, might require careful choice of an appropriate animal model that shows a disease phenotype and is accessible to experimental interventions. If the animal model is a proxy for drug development for humans, biological mechanisms must be sufficiently similar between animal and human physiologies.

Another important aspect of the construct is the quantity that we intend to measure (the measurand ), and its relation to the quantity or property we are interested in. For example, we might measure the concentration of the same chemical compound once in a blood sample and once in a highly purified sample, and these constitute two different measurands, whose values might not be comparable. Often, the quantity of interest (e.g., liver function) is not directly measurable (or even quantifiable) and we measure a biomarker instead. For example, pre-clinical and clinical investigations may use concentrations of proteins or counts of specific cell types from blood samples, such as the CD4+ cell count used as a biomarker for immune system function.

1.4.2 Internal Validity

The internal validity of an experiment concerns the soundness of the scientific rationale, statistical properties such as precision of estimates, and the measures taken against risk of bias. It refers to the validity of claims within the context of the experiment. Statistical design of experiments plays a prominent role in ensuring internal validity, and we briefly discuss the main ideas before providing the technical details and an application to our example in the subsequent sections.

Scientific Rationale and Research Question

The scientific rationale of a study is (usually) not immediately a statistical question. Translating a scientific question into a quantitative comparison amenable to statistical analysis is no small task and often requires careful consideration. It is a substantial, if non-statistical, benefit of using experimental design that we are forced to formulate a precise-enough research question and decide on the main analyses required for answering it before we conduct the experiment. For example, the question: is there a difference between placebo and drug? is insufficiently precise for planning a statistical analysis and determine an adequate experimental design. What exactly is the drug treatment? What should the drug’s concentration be and how is it administered? How do we make sure that the placebo group is comparable to the drug group in all other aspects? What do we measure and what do we mean by “difference?” A shift in average response, a fold-change, change in response before and after treatment?

The scientific rationale also enters the choice of a potential control group to which we compare responses. The quote

The deep, fundamental question in statistical analysis is ‘Compared to what?’ ( Tufte 1997 )

highlights the importance of this choice.

There are almost never enough resources to answer all relevant scientific questions. We therefore define a few questions of highest interest, and the main purpose of the experiment is answering these questions in the primary analysis . This intended analysis drives the experimental design to ensure relevant estimates can be calculated and have sufficient precision, and tests are adequately powered. This does not preclude us from conducting additional secondary analyses and exploratory analyses , but we are not willing to enlarge the experiment to ensure that strong conclusions can also be drawn from these analyses.

Risk of Bias

Experimental bias is a systematic difference in response between experimental units in addition to the difference caused by the treatments. The experimental units in the different groups are then not equal in all aspects other than the treatment applied to them. We saw several examples in Section 1.2 .

Minimizing the risk of bias is crucial for internal validity and we look at some common measures to eliminate or reduce different types of bias in Section 1.5 .

Precision and Effect Size

Another aspect of internal validity is the precision of estimates and the expected effect sizes. Is the experimental setup, in principle, able to detect a difference of relevant magnitude? Experimental design offers several methods for answering this question based on the expected heterogeneity of samples, the measurement error, and other sources of variation: power analysis is a technique for determining the number of samples required to reliably detect a relevant effect size and provide estimates of sufficient precision. More samples yield more precision and more power, but we have to be careful that replication is done at the right level: simply measuring a biological sample multiple times as in Figure 1.1 B yields more measured values, but is pseudo-replication for analyses. Replication should also ensure that the statistical uncertainties of estimates can be gauged from the data of the experiment itself, without additional untestable assumptions. Finally, the technique of blocking , shown in Figure 1.1 C, can remove a substantial proportion of the variation and thereby increase power and precision if we find a way to apply it.

1.4.3 External Validity

The external validity of an experiment concerns its replicability and the generalizability of inferences. An experiment is replicable if its results can be confirmed by an independent new experiment, preferably by a different lab and researcher. Experimental conditions in the replicate experiment usually differ from the original experiment, which provides evidence that the observed effects are robust to such changes. A much weaker condition on an experiment is reproducibility , the property that an independent researcher draws equivalent conclusions based on the data from this particular experiment, using the same analysis techniques. Reproducibility requires publishing the raw data, details on the experimental protocol, and a description of the statistical analyses, preferably with accompanying source code. Many scientific journals subscribe to reporting guidelines to ensure reproducibility and these are also helpful for planning an experiment.

A main threat to replicability and generalizability are too tightly controlled experimental conditions, when inferences only hold for a specific lab under the very specific conditions of the original experiment. Introducing systematic heterogeneity and using multi-center studies effectively broadens the experimental conditions and therefore the inferences for which internal validity is available.

For systematic heterogeneity , experimental conditions are systematically altered in addition to the treatments, and treatment differences estimated for each condition. For example, we might split the experimental material into several batches and use a different day of analysis, sample preparation, batch of buffer, measurement device, and lab technician for each batch. A more general inference is then possible if effect size, effect direction, and precision are comparable between the batches, indicating that the treatment differences are stable over the different conditions.

In multi-center experiments , the same experiment is conducted in several different labs and the results compared and merged. Multi-center approaches are very common in clinical trials and often necessary to reach the required number of patient enrollments.

Generalizability of randomized controlled trials in medicine and animal studies can suffer from overly restrictive eligibility criteria. In clinical trials, patients are often included or excluded based on co-medications and co-morbidities, and the resulting sample of eligible patients might no longer be representative of the patient population. For example, Travers et al. ( 2007 ) used the eligibility criteria of 17 random controlled trials of asthma treatments and found that out of 749 patients, only a median of 6% (45 patients) would be eligible for an asthma-related randomized controlled trial. This puts a question mark on the relevance of the trials’ findings for asthma patients in general.

1.5 Reducing the Risk of Bias

1.5.1 randomization of treatment allocation.

If systematic differences other than the treatment exist between our treatment groups, then the effect of the treatment is confounded with these other differences and our estimates of treatment effects might be biased.

We remove such unwanted systematic differences from our treatment comparisons by randomizing the allocation of treatments to experimental units. In a completely randomized design , each experimental unit has the same chance of being subjected to any of the treatments, and any differences between the experimental units other than the treatments are distributed over the treatment groups. Importantly, randomization is the only method that also protects our experiment against unknown sources of bias: we do not need to know all or even any of the potential differences and yet their impact is eliminated from the treatment comparisons by random treatment allocation.

Randomization has two effects: (i) differences unrelated to treatment become part of the ‘statistical noise’ rendering the treatment groups more similar; and (ii) the systematic differences are thereby eliminated as sources of bias from the treatment comparison.

Randomization transforms systematic variation into random variation.

In our example, a proper randomization would select 10 out of our 20 mice fully at random, such that the probability of any one mouse being picked is 1/20. These ten mice are then assigned to kit A, and the remaining mice to kit B. This allocation is entirely independent of the treatments and of any properties of the mice.

To ensure random treatment allocation, some kind of random process needs to be employed. This can be as simple as shuffling a pack of 10 red and 10 black cards or using a software-based random number generator. Randomization is slightly more difficult if the number of experimental units is not known at the start of the experiment, such as when patients are recruited for an ongoing clinical trial (sometimes called rolling recruitment ), and we want to have reasonable balance between the treatment groups at each stage of the trial.

Seemingly random assignments “by hand” are usually no less complicated than fully random assignments, but are always inferior. If surprising results ensue from the experiment, such assignments are subject to unanswerable criticism and suspicion of unwanted bias. Even worse are systematic allocations; they can only remove bias from known causes, and immediately raise red flags under the slightest scrutiny.

The Problem of Undesired Assignments

Even with a fully random treatment allocation procedure, we might end up with an undesirable allocation. For our example, the treatment group of kit A might—just by chance—contain mice that are all bigger or more active than those in the other treatment group. Statistical orthodoxy recommends using the design nevertheless, because only full randomization guarantees valid estimates of residual variance and unbiased estimates of effects. This argument, however, concerns the long-run properties of the procedure and seems of little help in this specific situation. Why should we care if the randomization yields correct estimates under replication of the experiment, if the particular experiment is jeopardized?

Another solution is to create a list of all possible allocations that we would accept and randomly choose one of these allocations for our experiment. The analysis should then reflect this restriction in the possible randomizations, which often renders this approach difficult to implement.

The most pragmatic method is to reject highly undesirable designs and compute a new randomization ( Cox 1958 ) . Undesirable allocations are unlikely to arise for large sample sizes, and we might accept a small bias in estimation for small sample sizes, when uncertainty in the estimated treatment effect is already high. In this approach, whenever we reject a particular outcome, we must also be willing to reject the outcome if we permute the treatment level labels. If we reject eight big and two small mice for kit A, then we must also reject two big and eight small mice. We must also be transparent and report a rejected allocation, so that critics may come to their own conclusions about potential biases and their remedies.

1.5.2 Blinding

Bias in treatment comparisons is also introduced if treatment allocation is random, but responses cannot be measured entirely objectively, or if knowledge of the assigned treatment affects the response. In clinical trials, for example, patients might react differently when they know to be on a placebo treatment, an effect known as cognitive bias . In animal experiments, caretakers might report more abnormal behavior for animals on a more severe treatment. Cognitive bias can be eliminated by concealing the treatment allocation from technicians or participants of a clinical trial, a technique called single-blinding .

If response measures are partially based on professional judgement (such as a clinical scale), patient or physician might unconsciously report lower scores for a placebo treatment, a phenomenon known as observer bias . Its removal requires double blinding , where treatment allocations are additionally concealed from the experimentalist.

Blinding requires randomized treatment allocation to begin with and substantial effort might be needed to implement it. Drug companies, for example, have to go to great lengths to ensure that a placebo looks, tastes, and feels similar enough to the actual drug. Additionally, blinding is often done by coding the treatment conditions and samples, and effect sizes and statistical significance are calculated before the code is revealed.

In clinical trials, double-blinding creates a conflict of interest. The attending physicians do not know which patient received which treatment, and thus accumulation of side-effects cannot be linked to any treatment. For this reason, clinical trials have a data monitoring committee not involved in the final analysis, that performs intermediate analyses of efficacy and safety at predefined intervals. If severe problems are detected, the committee might recommend altering or aborting the trial. The same might happen if one treatment already shows overwhelming evidence of superiority, such that it becomes unethical to withhold this treatment from the other patients.

1.5.3 Analysis Plan and Registration

An often overlooked source of bias has been termed the researcher degrees of freedom or garden of forking paths in the data analysis. For any set of data, there are many different options for its analysis: some results might be considered outliers and discarded, assumptions are made on error distributions and appropriate test statistics, different covariates might be included into a regression model. Often, multiple hypotheses are investigated and tested, and analyses are done separately on various (overlapping) subgroups. Hypotheses formed after looking at the data require additional care in their interpretation; almost never will \(p\) -values for these ad hoc or post hoc hypotheses be statistically justifiable. Many different measured response variables invite fishing expeditions , where patterns in the data are sought without an underlying hypothesis. Only reporting those sub-analyses that gave ‘interesting’ findings invariably leads to biased conclusions and is called cherry-picking or \(p\) -hacking (or much less flattering names).

The statistical analysis is always part of a larger scientific argument and we should consider the necessary computations in relation to building our scientific argument about the interpretation of the data. In addition to the statistical calculations, this interpretation requires substantial subject-matter knowledge and includes (many) non-statistical arguments. Two quotes highlight that experiment and analysis are a means to an end and not the end in itself.

There is a boundary in data interpretation beyond which formulas and quantitative decision procedures do not go, where judgment and style enter. ( Abelson 1995 )

Often, perfectly reasonable people come to perfectly reasonable decisions or conclusions based on nonstatistical evidence. Statistical analysis is a tool with which we support reasoning. It is not a goal in itself. ( Bailar III 1981 )

There is often a grey area between exploiting researcher degrees of freedom to arrive at a desired conclusion, and creative yet informed analyses of data. One way to navigate this area is to distinguish between exploratory studies and confirmatory studies . The former have no clearly stated scientific question, but are used to generate interesting hypotheses by identifying potential associations or effects that are then further investigated. Conclusions from these studies are very tentative and must be reported honestly as such. In contrast, standards are much higher for confirmatory studies, which investigate a specific predefined scientific question. Analysis plans and pre-registration of an experiment are accepted means for demonstrating lack of bias due to researcher degrees of freedom, and separating primary from secondary analyses allows emphasizing the main goals of the study.

Analysis Plan

The analysis plan is written before conducting the experiment and details the measurands and estimands, the hypotheses to be tested together with a power and sample size calculation, a discussion of relevant effect sizes, detection and handling of outliers and missing data, as well as steps for data normalization such as transformations and baseline corrections. If a regression model is required, its factors and covariates are outlined. Particularly in biology, handling measurements below the limit of quantification and saturation effects require careful consideration.

In the context of clinical trials, the problem of estimands has become a recent focus of attention. An estimand is the target of a statistical estimation procedure, for example the true average difference in enzyme levels between the two preparation kits. A main problem in many studies are post-randomization events that can change the estimand, even if the estimation procedure remains the same. For example, if kit B fails to produce usable samples for measurement in five out of ten cases because the enzyme level was too low, while kit A could handle these enzyme levels perfectly fine, then this might severely exaggerate the observed difference between the two kits. Similar problems arise in drug trials, when some patients stop taking one of the drugs due to side-effects or other complications.

Registration

Registration of experiments is an even more severe measure used in conjunction with an analysis plan and is becoming standard in clinical trials. Here, information about the trial, including the analysis plan, procedure to recruit patients, and stopping criteria, are registered in a public database. Publications based on the trial then refer to this registration, such that reviewers and readers can compare what the researchers intended to do and what they actually did. Similar portals for pre-clinical and translational research are also available.

1.6 Notes and Summary

The problem of measurements and measurands is further discussed for statistics in Hand ( 1996 ) and specifically for biological experiments in Coxon, Longstaff, and Burns ( 2019 ) . A general review of methods for handling missing data is Dong and Peng ( 2013 ) . The different roles of randomization are emphasized in Cox ( 2009 ) .

Two well-known reporting guidelines are the ARRIVE guidelines for animal research ( Kilkenny et al. 2010 ) and the CONSORT guidelines for clinical trials ( Moher et al. 2010 ) . Guidelines describing the minimal information required for reproducing experimental results have been developed for many types of experimental techniques, including microarrays (MIAME), RNA sequencing (MINSEQE), metabolomics (MSI) and proteomics (MIAPE) experiments; the FAIRSHARE initiative provides a more comprehensive collection ( Sansone et al. 2019 ) .

The problems of experimental design in animal experiments and particularly translation research are discussed in Couzin-Frankel ( 2013 ) . Multi-center studies are now considered for these investigations, and using a second laboratory already increases reproducibility substantially ( Richter et al. 2010 ; Richter 2017 ; Voelkl et al. 2018 ; Karp 2018 ) and allows standardizing the treatment effects ( Kafkafi et al. 2017 ) . First attempts are reported of using designs similar to clinical trials ( Llovera and Liesz 2016 ) . Exploratory-confirmatory research and external validity for animal studies is discussed in Kimmelman, Mogil, and Dirnagl ( 2014 ) and Pound and Ritskes-Hoitinga ( 2018 ) . Further information on pilot studies is found in Moore et al. ( 2011 ) , Sim ( 2019 ) , and Thabane et al. ( 2010 ) .

The deliberate use of statistical analyses and their interpretation for supporting a larger argument was called statistics as principled argument ( Abelson 1995 ) . Employing useless statistical analysis without reference to the actual scientific question is surrogate science ( Gigerenzer and Marewski 2014 ) and adaptive thinking is integral to meaningful statistical analysis ( Gigerenzer 2002 ) .

In an experiment, the investigator has full control over the experimental conditions applied to the experiment material. The experimental design gives the logical structure of an experiment: the units describing the organization of the experimental material, the treatments and their allocation to units, and the response. Statistical design of experiments includes techniques to ensure internal validity of an experiment, and methods to make inference from experimental data efficient.

• Privacy Policy

Home » Experimental Design – Types, Methods, Guide

Experimental Design – Types, Methods, Guide

Table of Contents

Experimental Design

Experimental design is a process of planning and conducting scientific experiments to investigate a hypothesis or research question. It involves carefully designing an experiment that can test the hypothesis, and controlling for other variables that may influence the results.

Experimental design typically includes identifying the variables that will be manipulated or measured, defining the sample or population to be studied, selecting an appropriate method of sampling, choosing a method for data collection and analysis, and determining the appropriate statistical tests to use.

Types of Experimental Design

Here are the different types of experimental design:

Completely Randomized Design

In this design, participants are randomly assigned to one of two or more groups, and each group is exposed to a different treatment or condition.

Randomized Block Design

This design involves dividing participants into blocks based on a specific characteristic, such as age or gender, and then randomly assigning participants within each block to one of two or more treatment groups.

Factorial Design

In a factorial design, participants are randomly assigned to one of several groups, each of which receives a different combination of two or more independent variables.

Repeated Measures Design

In this design, each participant is exposed to all of the different treatments or conditions, either in a random order or in a predetermined order.

Crossover Design

This design involves randomly assigning participants to one of two or more treatment groups, with each group receiving one treatment during the first phase of the study and then switching to a different treatment during the second phase.

Split-plot Design

In this design, the researcher manipulates one or more variables at different levels and uses a randomized block design to control for other variables.

Nested Design

This design involves grouping participants within larger units, such as schools or households, and then randomly assigning these units to different treatment groups.

Laboratory Experiment

Laboratory experiments are conducted under controlled conditions, which allows for greater precision and accuracy. However, because laboratory conditions are not always representative of real-world conditions, the results of these experiments may not be generalizable to the population at large.

Field Experiment

Field experiments are conducted in naturalistic settings and allow for more realistic observations. However, because field experiments are not as controlled as laboratory experiments, they may be subject to more sources of error.

Experimental Design Methods

Experimental design methods refer to the techniques and procedures used to design and conduct experiments in scientific research. Here are some common experimental design methods:

Randomization

This involves randomly assigning participants to different groups or treatments to ensure that any observed differences between groups are due to the treatment and not to other factors.

Control Group

The use of a control group is an important experimental design method that involves having a group of participants that do not receive the treatment or intervention being studied. The control group is used as a baseline to compare the effects of the treatment group.

Blinding involves keeping participants, researchers, or both unaware of which treatment group participants are in, in order to reduce the risk of bias in the results.

Counterbalancing

This involves systematically varying the order in which participants receive treatments or interventions in order to control for order effects.

Replication

Replication involves conducting the same experiment with different samples or under different conditions to increase the reliability and validity of the results.

This experimental design method involves manipulating multiple independent variables simultaneously to investigate their combined effects on the dependent variable.

This involves dividing participants into subgroups or blocks based on specific characteristics, such as age or gender, in order to reduce the risk of confounding variables.

Data Collection Method

Experimental design data collection methods are techniques and procedures used to collect data in experimental research. Here are some common experimental design data collection methods:

Direct Observation

This method involves observing and recording the behavior or phenomenon of interest in real time. It may involve the use of structured or unstructured observation, and may be conducted in a laboratory or naturalistic setting.

Self-report Measures

Self-report measures involve asking participants to report their thoughts, feelings, or behaviors using questionnaires, surveys, or interviews. These measures may be administered in person or online.

Behavioral Measures

Behavioral measures involve measuring participants’ behavior directly, such as through reaction time tasks or performance tests. These measures may be administered using specialized equipment or software.

Physiological Measures

Physiological measures involve measuring participants’ physiological responses, such as heart rate, blood pressure, or brain activity, using specialized equipment. These measures may be invasive or non-invasive, and may be administered in a laboratory or clinical setting.

Archival Data

Archival data involves using existing records or data, such as medical records, administrative records, or historical documents, as a source of information. These data may be collected from public or private sources.

Computerized Measures

Computerized measures involve using software or computer programs to collect data on participants’ behavior or responses. These measures may include reaction time tasks, cognitive tests, or other types of computer-based assessments.

Video Recording

Video recording involves recording participants’ behavior or interactions using cameras or other recording equipment. This method can be used to capture detailed information about participants’ behavior or to analyze social interactions.

Data Analysis Method

Experimental design data analysis methods refer to the statistical techniques and procedures used to analyze data collected in experimental research. Here are some common experimental design data analysis methods:

Descriptive Statistics

Descriptive statistics are used to summarize and describe the data collected in the study. This includes measures such as mean, median, mode, range, and standard deviation.

Inferential Statistics

Inferential statistics are used to make inferences or generalizations about a larger population based on the data collected in the study. This includes hypothesis testing and estimation.

Analysis of Variance (ANOVA)

ANOVA is a statistical technique used to compare means across two or more groups in order to determine whether there are significant differences between the groups. There are several types of ANOVA, including one-way ANOVA, two-way ANOVA, and repeated measures ANOVA.

Regression Analysis

Regression analysis is used to model the relationship between two or more variables in order to determine the strength and direction of the relationship. There are several types of regression analysis, including linear regression, logistic regression, and multiple regression.

Factor Analysis

Factor analysis is used to identify underlying factors or dimensions in a set of variables. This can be used to reduce the complexity of the data and identify patterns in the data.

Structural Equation Modeling (SEM)

SEM is a statistical technique used to model complex relationships between variables. It can be used to test complex theories and models of causality.

Cluster Analysis

Cluster analysis is used to group similar cases or observations together based on similarities or differences in their characteristics.

Time Series Analysis

Time series analysis is used to analyze data collected over time in order to identify trends, patterns, or changes in the data.

Multilevel Modeling

Multilevel modeling is used to analyze data that is nested within multiple levels, such as students nested within schools or employees nested within companies.

Applications of Experimental Design 

Experimental design is a versatile research methodology that can be applied in many fields. Here are some applications of experimental design:

• Medical Research: Experimental design is commonly used to test new treatments or medications for various medical conditions. This includes clinical trials to evaluate the safety and effectiveness of new drugs or medical devices.
• Agriculture : Experimental design is used to test new crop varieties, fertilizers, and other agricultural practices. This includes randomized field trials to evaluate the effects of different treatments on crop yield, quality, and pest resistance.
• Environmental science: Experimental design is used to study the effects of environmental factors, such as pollution or climate change, on ecosystems and wildlife. This includes controlled experiments to study the effects of pollutants on plant growth or animal behavior.
• Psychology : Experimental design is used to study human behavior and cognitive processes. This includes experiments to test the effects of different interventions, such as therapy or medication, on mental health outcomes.
• Engineering : Experimental design is used to test new materials, designs, and manufacturing processes in engineering applications. This includes laboratory experiments to test the strength and durability of new materials, or field experiments to test the performance of new technologies.
• Education : Experimental design is used to evaluate the effectiveness of teaching methods, educational interventions, and programs. This includes randomized controlled trials to compare different teaching methods or evaluate the impact of educational programs on student outcomes.
• Marketing : Experimental design is used to test the effectiveness of marketing campaigns, pricing strategies, and product designs. This includes experiments to test the impact of different marketing messages or pricing schemes on consumer behavior.

Examples of Experimental Design 

Here are some examples of experimental design in different fields:

• Example in Medical research : A study that investigates the effectiveness of a new drug treatment for a particular condition. Patients are randomly assigned to either a treatment group or a control group, with the treatment group receiving the new drug and the control group receiving a placebo. The outcomes, such as improvement in symptoms or side effects, are measured and compared between the two groups.
• Example in Education research: A study that examines the impact of a new teaching method on student learning outcomes. Students are randomly assigned to either a group that receives the new teaching method or a group that receives the traditional teaching method. Student achievement is measured before and after the intervention, and the results are compared between the two groups.
• Example in Environmental science: A study that tests the effectiveness of a new method for reducing pollution in a river. Two sections of the river are selected, with one section treated with the new method and the other section left untreated. The water quality is measured before and after the intervention, and the results are compared between the two sections.
• Example in Marketing research: A study that investigates the impact of a new advertising campaign on consumer behavior. Participants are randomly assigned to either a group that is exposed to the new campaign or a group that is not. Their behavior, such as purchasing or product awareness, is measured and compared between the two groups.
• Example in Social psychology: A study that examines the effect of a new social intervention on reducing prejudice towards a marginalized group. Participants are randomly assigned to either a group that receives the intervention or a control group that does not. Their attitudes and behavior towards the marginalized group are measured before and after the intervention, and the results are compared between the two groups.

When to use Experimental Research Design 

Experimental research design should be used when a researcher wants to establish a cause-and-effect relationship between variables. It is particularly useful when studying the impact of an intervention or treatment on a particular outcome.

Here are some situations where experimental research design may be appropriate:

• When studying the effects of a new drug or medical treatment: Experimental research design is commonly used in medical research to test the effectiveness and safety of new drugs or medical treatments. By randomly assigning patients to treatment and control groups, researchers can determine whether the treatment is effective in improving health outcomes.
• When evaluating the effectiveness of an educational intervention: An experimental research design can be used to evaluate the impact of a new teaching method or educational program on student learning outcomes. By randomly assigning students to treatment and control groups, researchers can determine whether the intervention is effective in improving academic performance.
• When testing the effectiveness of a marketing campaign: An experimental research design can be used to test the effectiveness of different marketing messages or strategies. By randomly assigning participants to treatment and control groups, researchers can determine whether the marketing campaign is effective in changing consumer behavior.
• When studying the effects of an environmental intervention: Experimental research design can be used to study the impact of environmental interventions, such as pollution reduction programs or conservation efforts. By randomly assigning locations or areas to treatment and control groups, researchers can determine whether the intervention is effective in improving environmental outcomes.
• When testing the effects of a new technology: An experimental research design can be used to test the effectiveness and safety of new technologies or engineering designs. By randomly assigning participants or locations to treatment and control groups, researchers can determine whether the new technology is effective in achieving its intended purpose.

How to Conduct Experimental Research

Here are the steps to conduct Experimental Research:

• Identify a Research Question : Start by identifying a research question that you want to answer through the experiment. The question should be clear, specific, and testable.
• Develop a Hypothesis: Based on your research question, develop a hypothesis that predicts the relationship between the independent and dependent variables. The hypothesis should be clear and testable.
• Design the Experiment : Determine the type of experimental design you will use, such as a between-subjects design or a within-subjects design. Also, decide on the experimental conditions, such as the number of independent variables, the levels of the independent variable, and the dependent variable to be measured.
• Select Participants: Select the participants who will take part in the experiment. They should be representative of the population you are interested in studying.
• Randomly Assign Participants to Groups: If you are using a between-subjects design, randomly assign participants to groups to control for individual differences.
• Conduct the Experiment : Conduct the experiment by manipulating the independent variable(s) and measuring the dependent variable(s) across the different conditions.
• Analyze the Data: Analyze the data using appropriate statistical methods to determine if there is a significant effect of the independent variable(s) on the dependent variable(s).
• Draw Conclusions: Based on the data analysis, draw conclusions about the relationship between the independent and dependent variables. If the results support the hypothesis, then it is accepted. If the results do not support the hypothesis, then it is rejected.
• Communicate the Results: Finally, communicate the results of the experiment through a research report or presentation. Include the purpose of the study, the methods used, the results obtained, and the conclusions drawn.

Purpose of Experimental Design 

The purpose of experimental design is to control and manipulate one or more independent variables to determine their effect on a dependent variable. Experimental design allows researchers to systematically investigate causal relationships between variables, and to establish cause-and-effect relationships between the independent and dependent variables. Through experimental design, researchers can test hypotheses and make inferences about the population from which the sample was drawn.

Experimental design provides a structured approach to designing and conducting experiments, ensuring that the results are reliable and valid. By carefully controlling for extraneous variables that may affect the outcome of the study, experimental design allows researchers to isolate the effect of the independent variable(s) on the dependent variable(s), and to minimize the influence of other factors that may confound the results.

Experimental design also allows researchers to generalize their findings to the larger population from which the sample was drawn. By randomly selecting participants and using statistical techniques to analyze the data, researchers can make inferences about the larger population with a high degree of confidence.

Overall, the purpose of experimental design is to provide a rigorous, systematic, and scientific method for testing hypotheses and establishing cause-and-effect relationships between variables. Experimental design is a powerful tool for advancing scientific knowledge and informing evidence-based practice in various fields, including psychology, biology, medicine, engineering, and social sciences.

Advantages of Experimental Design 

Experimental design offers several advantages in research. Here are some of the main advantages:

• Control over extraneous variables: Experimental design allows researchers to control for extraneous variables that may affect the outcome of the study. By manipulating the independent variable and holding all other variables constant, researchers can isolate the effect of the independent variable on the dependent variable.
• Establishing causality: Experimental design allows researchers to establish causality by manipulating the independent variable and observing its effect on the dependent variable. This allows researchers to determine whether changes in the independent variable cause changes in the dependent variable.
• Replication : Experimental design allows researchers to replicate their experiments to ensure that the findings are consistent and reliable. Replication is important for establishing the validity and generalizability of the findings.
• Random assignment: Experimental design often involves randomly assigning participants to conditions. This helps to ensure that individual differences between participants are evenly distributed across conditions, which increases the internal validity of the study.
• Precision : Experimental design allows researchers to measure variables with precision, which can increase the accuracy and reliability of the data.
• Generalizability : If the study is well-designed, experimental design can increase the generalizability of the findings. By controlling for extraneous variables and using random assignment, researchers can increase the likelihood that the findings will apply to other populations and contexts.

Limitations of Experimental Design

Experimental design has some limitations that researchers should be aware of. Here are some of the main limitations:

• Artificiality : Experimental design often involves creating artificial situations that may not reflect real-world situations. This can limit the external validity of the findings, or the extent to which the findings can be generalized to real-world settings.
• Ethical concerns: Some experimental designs may raise ethical concerns, particularly if they involve manipulating variables that could cause harm to participants or if they involve deception.
• Participant bias : Participants in experimental studies may modify their behavior in response to the experiment, which can lead to participant bias.
• Limited generalizability: The conditions of the experiment may not reflect the complexities of real-world situations. As a result, the findings may not be applicable to all populations and contexts.
• Cost and time : Experimental design can be expensive and time-consuming, particularly if the experiment requires specialized equipment or if the sample size is large.
• Researcher bias : Researchers may unintentionally bias the results of the experiment if they have expectations or preferences for certain outcomes.
• Lack of feasibility : Experimental design may not be feasible in some cases, particularly if the research question involves variables that cannot be manipulated or controlled.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Focus Groups – Steps, Examples and Guide

Questionnaire – Definition, Types, and Examples

Qualitative Research Methods

Descriptive Research Design – Types, Methods and...

Textual Analysis – Types, Examples and Guide

Phenomenology – Methods, Examples and Guide

Experimental Design: Types, Examples & Methods

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

Experimental design refers to how participants are allocated to different groups in an experiment. Types of design include repeated measures, independent groups, and matched pairs designs.

Probably the most common way to design an experiment in psychology is to divide the participants into two groups, the experimental group and the control group, and then introduce a change to the experimental group, not the control group.

The researcher must decide how he/she will allocate their sample to the different experimental groups.  For example, if there are 10 participants, will all 10 participants participate in both groups (e.g., repeated measures), or will the participants be split in half and take part in only one group each?

Three types of experimental designs are commonly used:

1. Independent Measures

Independent measures design, also known as between-groups , is an experimental design where different participants are used in each condition of the independent variable.  This means that each condition of the experiment includes a different group of participants.

This should be done by random allocation, ensuring that each participant has an equal chance of being assigned to one group.

Independent measures involve using two separate groups of participants, one in each condition. For example:

• Con : More people are needed than with the repeated measures design (i.e., more time-consuming).
• Pro : Avoids order effects (such as practice or fatigue) as people participate in one condition only.  If a person is involved in several conditions, they may become bored, tired, and fed up by the time they come to the second condition or become wise to the requirements of the experiment!
• Con : Differences between participants in the groups may affect results, for example, variations in age, gender, or social background.  These differences are known as participant variables (i.e., a type of extraneous variable ).
• Control : After the participants have been recruited, they should be randomly assigned to their groups. This should ensure the groups are similar, on average (reducing participant variables).

2. Repeated Measures Design

Repeated Measures design is an experimental design where the same participants participate in each independent variable condition.  This means that each experiment condition includes the same group of participants.

Repeated Measures design is also known as within-groups or within-subjects design .

• Pro : As the same participants are used in each condition, participant variables (i.e., individual differences) are reduced.
• Con : There may be order effects. Order effects refer to the order of the conditions affecting the participants’ behavior.  Performance in the second condition may be better because the participants know what to do (i.e., practice effect).  Or their performance might be worse in the second condition because they are tired (i.e., fatigue effect). This limitation can be controlled using counterbalancing.
• Pro : Fewer people are needed as they participate in all conditions (i.e., saves time).
• Control : To combat order effects, the researcher counter-balances the order of the conditions for the participants.  Alternating the order in which participants perform in different conditions of an experiment.

Counterbalancing

Suppose we used a repeated measures design in which all of the participants first learned words in “loud noise” and then learned them in “no noise.”

We expect the participants to learn better in “no noise” because of order effects, such as practice. However, a researcher can control for order effects using counterbalancing.

The sample would be split into two groups: experimental (A) and control (B).  For example, group 1 does ‘A’ then ‘B,’ and group 2 does ‘B’ then ‘A.’ This is to eliminate order effects.

Although order effects occur for each participant, they balance each other out in the results because they occur equally in both groups.

3. Matched Pairs Design

A matched pairs design is an experimental design where pairs of participants are matched in terms of key variables, such as age or socioeconomic status. One member of each pair is then placed into the experimental group and the other member into the control group .

One member of each matched pair must be randomly assigned to the experimental group and the other to the control group.

• Con : If one participant drops out, you lose 2 PPs’ data.
• Pro : Reduces participant variables because the researcher has tried to pair up the participants so that each condition has people with similar abilities and characteristics.
• Con : Very time-consuming trying to find closely matched pairs.
• Pro : It avoids order effects, so counterbalancing is not necessary.
• Con : Impossible to match people exactly unless they are identical twins!
• Control : Members of each pair should be randomly assigned to conditions. However, this does not solve all these problems.

Experimental design refers to how participants are allocated to an experiment’s different conditions (or IV levels). There are three types:

1. Independent measures / between-groups : Different participants are used in each condition of the independent variable.

2. Repeated measures /within groups : The same participants take part in each condition of the independent variable.

3. Matched pairs : Each condition uses different participants, but they are matched in terms of important characteristics, e.g., gender, age, intelligence, etc.

Learning Check

Read about each of the experiments below. For each experiment, identify (1) which experimental design was used; and (2) why the researcher might have used that design.

1 . To compare the effectiveness of two different types of therapy for depression, depressed patients were assigned to receive either cognitive therapy or behavior therapy for a 12-week period.

The researchers attempted to ensure that the patients in the two groups had similar severity of depressed symptoms by administering a standardized test of depression to each participant, then pairing them according to the severity of their symptoms.

2 . To assess the difference in reading comprehension between 7 and 9-year-olds, a researcher recruited each group from a local primary school. They were given the same passage of text to read and then asked a series of questions to assess their understanding.

3 . To assess the effectiveness of two different ways of teaching reading, a group of 5-year-olds was recruited from a primary school. Their level of reading ability was assessed, and then they were taught using scheme one for 20 weeks.

At the end of this period, their reading was reassessed, and a reading improvement score was calculated. They were then taught using scheme two for a further 20 weeks, and another reading improvement score for this period was calculated. The reading improvement scores for each child were then compared.

4 . To assess the effect of the organization on recall, a researcher randomly assigned student volunteers to two conditions.

Condition one attempted to recall a list of words that were organized into meaningful categories; condition two attempted to recall the same words, randomly grouped on the page.

Experiment Terminology

Ecological validity.

The degree to which an investigation represents real-life experiences.

Experimenter effects

These are the ways that the experimenter can accidentally influence the participant through their appearance or behavior.

Demand characteristics

The clues in an experiment lead the participants to think they know what the researcher is looking for (e.g., the experimenter’s body language).

Independent variable (IV)

The variable the experimenter manipulates (i.e., changes) is assumed to have a direct effect on the dependent variable.

Dependent variable (DV)

Variable the experimenter measures. This is the outcome (i.e., the result) of a study.

Extraneous variables (EV)

All variables which are not independent variables but could affect the results (DV) of the experiment. Extraneous variables should be controlled where possible.

Confounding variables

Variable(s) that have affected the results (DV), apart from the IV. A confounding variable could be an extraneous variable that has not been controlled.

Random Allocation

Randomly allocating participants to independent variable conditions means that all participants should have an equal chance of taking part in each condition.

The principle of random allocation is to avoid bias in how the experiment is carried out and limit the effects of participant variables.

Order effects

Changes in participants’ performance due to their repeating the same or similar test more than once. Examples of order effects include:

(i) practice effect: an improvement in performance on a task due to repetition, for example, because of familiarity with the task;

(ii) fatigue effect: a decrease in performance of a task due to repetition, for example, because of boredom or tiredness.

Calcworkshop

Experimental Design in Statistics w/ 11 Examples!

// Last Updated: September 20, 2020 - Watch Video //

A proper experimental design is a critical skill in statistics.

Jenn, Founder Calcworkshop ® , 15+ Years Experience (Licensed & Certified Teacher)

Without proper controls and safeguards, unintended consequences can ruin our study and lead to wrong conclusions.

So let’s dive in to see what’s this is all about!

What’s the difference between an observational study and an experimental study?

An observational study is one in which investigators merely measure variables of interest without influencing the subjects.

And an experiment is a study in which investigators administer some form of treatment on one or more groups?

In other words, an observation is hands-off, whereas an experiment is hands-on.

So what’s the purpose of an experiment?

To establish causation (i.e., cause and effect).

All this means is that we wish to determine the effect an independent explanatory variable has on a dependent response variable.

The explanatory variable explains a response, similar to a child falling and skins their knee and starting to cry. The child is crying in response to falling and skinning their knee. So the explanatory variable is the fall, and the response variable is crying.

Explanatory Vs Response Variable In Everyday Life

Let’s look at another example. Suppose a medical journal describes two studies in which subjects who had a seizure were randomly assigned to two different treatments:

• No treatment.
• A high dose of vitamin C.

The subjects were observed for a year, and the number of seizures for each subject was recorded. Identify the explanatory variable (independent variable), response variable (dependent variable), and include the experimental units.

The explanatory variable is whether the subject received either no treatment or a high dose of vitamin C. The response variable is whether the subject had a seizure during the time of the study. The experimental units in this study are the subjects who recently had a seizure.

Okay, so using the example above, notice that one of the groups did not receive treatment. This group is called a control group and acts as a baseline to see how a new treatment differs from those who don’t receive treatment. Typically, the control group is given something called a placebo, a substance designed to resemble medicine but does not contain an active drug component. A placebo is a dummy treatment, and should not have a physical effect on a person.

Before we talk about the characteristics of a well-designed experiment, we need to discuss some things to look out for:

• Confounding
• Lurking variables

Confounding happens when two explanatory variables are both associated with a response variable and also associated with each other, causing the investigator not to be able to identify their effects and the response variable separately.

A lurking variable is usually unobserved at the time of the study, which influences the association between the two variables of interest. In essence, a lurking variable is a third variable that is not measured in the study but may change the response variable.

For example, a study reported a relationship between smoking and health. A study of 1430 women were asked whether they smoked. Ten years later, a follow-up survey observed whether each woman was still alive or deceased. The researchers studied the possible link between whether a woman smoked and whether she survived the 10-year study period. They reported that:

• 21% of the smokers died
• 32% of the nonsmokers died

So, is smoking beneficial to your health, or is there something that could explain how this happened?

Older women are less likely to be smokers, and older women are more likely to die. Because age is a variable that influences the explanatory and response variable, it is considered a confounding variable.

But does smoking cause death?

Notice that the lurking variable, age, can also be a contributing factor. While there is a correlation between smoking and mortality, and also a correlation between smoking and age, we aren’t 100% sure that they are the cause of the mortality rate in women.

Lurking – Confounding – Correlation – Causation Diagram

Now, something important to point out is that a lurking variable is one that is not measured in the study that could influence the results. Using the example above, some other possible lurking variables are:

• Stress Level.

These variables were not measured in the study but could influence smoking habits as well as mortality rates.

What is important to note about the difference between confounding and lurking variables is that a confounding variable is measured in a study, while a lurking variable is not.

Additionally, correlation does not imply causation!

Alright, so now it’s time to talk about blinding: single-blind, double-blind experiments, as well as the placebo effect.

A single-blind experiment is when the subjects are unaware of which treatment they are receiving, but the investigator measuring the responses knows what treatments are going to which subject. In other words, the researcher knows which individual gets the placebo and which ones receive the experimental treatment. One major pitfall for this type of design is that the researcher may consciously or unconsciously influence the subject since they know who is receiving treatment and who isn’t.

A double-blind experiment is when both the subjects and investigator do not know who receives the placebo and who receives the treatment. A double-blind model is considered the best model for clinical trials as it eliminates the possibility of bias on the part of the researcher and the possibility of producing a placebo effect from the subject.

The placebo effect is when a subject has an effect or response to a fake treatment because they “believe” that the result should occur as noted by Yale . For example, a person struggling with insomnia takes a placebo (sugar pill) but instantly falls asleep because they believe they are receiving a sleep aid like Ambien or Lunesta.

Placebo Effect – Real Life Example

So, what are the three primary requirements for a well-designed experiment?

• Randomization

In a controlled experiment , the researchers, or investigators, decide which subjects are assigned to a control group and which subjects are assigned to a treatment group. In doing so, we ensure that the control and treatment groups are as similar as possible, and limit possible confounding influences such as lurking variables. A replicated experiment that is repeated on many different subjects helps reduce the chance of variation on the results. And randomization means we randomly assign subjects into control and treatment groups.

When subjects are divided into control groups and treatment groups randomly, we can use probability to predict the differences we expect to observe. If the differences between the two groups are higher than what we would expect to see naturally (by chance), we say that the results are statistically significant.

For example, if it is surmised that a new medicine reduces the effects of illness from 72 hours to 71 hours, this would not be considered statistically significant. The difference from 72 hours to 71 hours is not substantial enough to support that the observed effect was due to something other than normal random variation.

Now there are two major types of designs:

• Completely-Randomized Design (CRD)
• Block Design

A completely randomized design is the process of assigning subjects to control and treatment groups using probability, as seen in the flow diagram below.

Completely Randomized Design Example

A block design is a research method that places subjects into groups of similar experimental units or conditions, like age or gender, and then assign subjects to control and treatment groups using probability, as shown below.

Randomized Block Design Example

Additionally, a useful and particular case of a blocking strategy is something called a matched-pair design . This is when two variables are paired to control for lurking variables.

For example, imagine we want to study if walking daily improved blood pressure. If the blood pressure for five subjects is measured at the beginning of the study and then again after participating in a walking program for one month, then the observations would be considered dependent samples because the same five subjects are used in the before and after observations; thus, a matched-pair design.

Please note that our video lesson will not focus on quasi-experiments. A quasi experimental design lacks random assignments; therefore, the independent variable can be manipulated prior to measuring the dependent variable, which may lead to confounding. For the sake of our lesson, and all future lessons, we will be using research methods where random sampling and experimental designs are used.

Together we will learn how to identify explanatory variables (independent variable) and response variables (dependent variables), understand and define confounding and lurking variables, see the effects of single-blind and double-blind experiments, and design randomized and block experiments.

Experimental Designs – Lesson & Examples (Video)

1 hr 06 min

• Introduction to Video: Experiments
• 00:00:29 – Observational Study vs Experimental Study and Response and Explanatory Variables (Examples #1-4)
• Exclusive Content for Members Only
• 00:09:15 – Identify the response and explanatory variables and the experimental units and treatment (Examples #5-6)
• 00:14:47 – Introduction of lurking variables and confounding with ice cream and homicide example
• 00:18:57 – Lurking variables, Confounding, Placebo Effect, Single Blind and Double Blind Experiments (Example #7)
• 00:27:20 – What was the placebo effect and was the experiment single or double blind? (Example #8)
• 00:30:36 – Characteristics of a well designed and constructed experiment that is statistically significant
• 00:35:08 – Overview of Complete Randomized Design, Block Design and Matched Pair Design
• 00:44:23 – Design and experiment using complete randomized design or a block design (Examples #9-10)
• 00:56:09 – Identify the response and explanatory variables, experimental units, lurking variables, and design an experiment to test a new drug (Example #11)
• Practice Problems with Step-by-Step Solutions
• Chapter Tests with Video Solutions

Get access to all the courses and over 450 HD videos with your subscription

Monthly and Yearly Plans Available

Get My Subscription Now

Still wondering if CalcWorkshop is right for you? Take a Tour and find out how a membership can take the struggle out of learning math.

Experimental Design

• What is Experimental Design?
• Validity in Experimental Design
• Types of Design
• Related Topics

1. What is Experimental Design?

Experimental design is a way to carefully plan experiments in advance so that your results are both objective and valid . The terms “Experimental Design” and “Design of Experiments” are used interchangeably and mean the same thing. However, the medical and social sciences tend to use the term “Experimental Design” while engineering, industrial and computer sciences favor the term “Design of experiments.”

Design of experiments involves:

• The systematic collection of data
• A focus on the design itself, rather than the results
• Planning changes to independent (input) variables and the effect on dependent variables or response variables
• Ensuring results are valid, easily interpreted, and definitive.

Ideally, your experimental design should:

• Describe how participants are allocated to experimental groups. A common method is completely randomized design, where participants are assigned to groups at random. A second method is randomized block design, where participants are divided into homogeneous blocks (for example, age groups) before being randomly assigned to groups.
• Minimize or eliminate confounding variables , which can offer alternative explanations for the experimental results.
• Allow you to make inferences about the relationship between independent variables and dependent variables .
• Reduce variability , to make it easier for you to find differences in treatment outcomes.

The most important principles 1 are:

• Randomization : the assignment of study components by a completely random method, like simple random sampling . Randomization eliminates bias from the results
• Replication : the experiment must be replicable by other researchers. This is usually achieved with the use of statistics like the standard error of the sample mean or confidence intervals .
• Blocking: controlling sources of variation in the experimental results.

2. Variables in Design of Experiments

• What is a Confounding Variable?
• What is a Control Variable?
• What is a Criterion Variable?
• What are Endogenous Variables?
• What is a Dependent Variable?
• What is an Explanatory Variable?
• What is an Intervening Variable?
• What is a Manipulated Variable?
• What is an Outcome Variable?

Back to Top

3. Validity in Design of Experiments

• What is Concurrent Validity?
• What is Construct Validity?
• What is Consequential Validity?
• What is Convergent Validity?
• What is Criterion Validity?
• What is Ecological validity?
• What is External Validity?
• What is Face Validity?
• What is Internal Validity?
• What is Predictive Validity?

4. Design of Experiments: Types

• Adaptive designs.
• Balanced Latin Square Design.
• Balanced and Unbalanced Designs .
• Between Subjects Design.
• What are Case Studies?
• What is a Case-Control Study?
• What is a Cohort Study?
• Completely Randomized Design.
• Cross Lagged Panel Design .

Cross Sectional Research

• Cross Sequential Design.
• Definite Screening Design

Factorial Design.

• Flexible Design.
• Group sequential Design.
• Longitudinal Research.

Matched-Pairs Design.

• Parallel Design.
• Observational Study .
• Plackett-Burman Design.

Pretest-Posttest Design.

• Prospective Study.

Quasi-Experimental Design.

Randomized block design., randomized controlled trial.

• Repeated Measures Design .
• Retrospective Study.
• Split-Plot Design.
• Strip-Plot Design .
• Stepped Wedge Designs .
• What is Survey Research?

Within subjects Design.

Between subjects design (independent measures)., what is between subjects design.

In between subjects design, separate groups are created for each treatment. This type of experimental design is sometimes called independent measures design because each participant is assigned to only one treatment group.For example, you might be testing a new depression medication: one group receives the actual medication and the other receives a placebo . Participants can only be a member of one of the groups (either the treatment or placebo group). A new group is created for every treatment. For example, if you are testing two depression medications, you would have:

• Group 1 (Medication 1).
• Group 2 (Medication 2).
• Group 3 (Placebo).

Advantages and Disadvantages of Between Subjects Design.

Advantages..

Between subjects design is one of the simplest types of experimental design setup. Other advantages include:

• Multiple treatments and treatment levels can be tested at the same time.
• This type of design can be completed quickly.

Disadvantages.

A major disadvantage in this type of experimental design is that as each participant is only being tested once, the addition of a new treatment requires the formation of another group. The design can become extremely complex if more than a few treatments are being tested. Other disadvantages include:

• Differences in individuals (i.e. age, race, sex) may skew results and are almost impossible to control for in this experimental design.
• Bias can be an issue unless you control for this factor using experimental blinds (either a single blind experiment–where the participant doesn’t know if they are getting a treatment or placebo–or a double blind, where neither the participant nor the researcher know).
• Generalization issues means that you may not be able to extrapolate your results to a wider audience.
• Environmental bias can be a problem with between subjects design. For example, let’s say you were giving one group of college students a standardized test at 8 a.m. and a second group the test at noon. Students who took the 8 a.m. test may perform poorly simply because they weren’t awake yet.

Back to Top.

Completely Randomized Experimental Design.

What is a completely randomized design.

A completely randomized design (CRD) is an experiment where the treatments are assigned at random. Every experimental unit has the same odds of receiving a particular treatment. This design is usually only used in lab experiments, where environmental factors are relatively easy to control for; it is rarely used out in the field, where environmental factors are usually impossible to control. When a CRD has two treatments, it is equivalent to a t-test .

A completely randomized design is generally implemented by:

• Listing the treatment levels or treatment combinations.
• Assigning each level/combination a random number.
• Sorting the random numbers in order, to produce a random application order for treatments.

However, you could use any method that completely randomizes the treatments and experimental units, as long as you take care to ensure that:

• The assignment is truly random.
• You have accounted for extraneous variables .

Completely Randomized Design Example.

Completely Randomized Design with Subsampling.

This subset of CRD is usually used when experimental units are limited. Subsampling might include several branches of a particular tree, or several samples from an individual plot. Back to Top.

What is a Factorial Design?

A factorial experimental design is used to investigate the effect of two or more independent variables on one dependent variable . For example, let’s say a researcher wanted to investigate components for increasing SAT Scores . The three components are:

• SAT intensive class (yes or no).
• SAT Prep book (yes or no).
• Extra homework (yes or no).

The researcher plans to manipulate each of these independent variables. Each of the independent variables is called a factor , and each factor has two levels (yes or no). As this experiment has 3 factors with 2 levels, this is a 2 x 2 x 2 = 2 3 factorial design. An experiment with 3 factors and 3 levels would be a 3 3 factorial design and an experiment with 2 factors and 3 levels would be a 3 2 factorial design.

The vast majority of factorial experiments only have two levels. In some experiments where the number of level/factor combinations are unmanageable, the experiment can be split into parts (for example, by half), creating a fractional experimental design.

Null Outcome.

A null outcome is when the experiment’s outcome is the same regardless of how the levels and factors were combined. In the above example, that would mean no amount of SAT prep (book and class, class and extra homework etc.) could increase the scores of the students being studied.

Main Effect and Interaction Effect.

Two types of effects are considered when analyzing the results from a factorial experiment: main effect and interaction effect . The main effect is the effect of an independent variable (in this case, SAT prep class or SAT book or extra homework) on the dependent variable (SAT Scores). For a main effect to exist, you’d want to see a consistent trend across the different levels. For example, you might conclude that students who took the SAT prep class scored consistently higher than students who did not. An interaction effect occurs between factors. For example, one group of students who took the SAT class and used the SAT prep book showed an increase in SAT scores while the students who took the class but did not use the book didn’t show any increase. You could infer that there is an interaction between the SAT class and use of the SAT prep book. Back to Top.

What is Matched Pairs Design?

Matched pairs design is a special case of randomized block design. In this design, two treatments are assigned to homogeneous groups (blocks) of subjects. The goal is to maximize homogeneity in each pair. In other words, you want the pairs to be as similar as possible. The blocks are composed of matched pairs which are randomly assigned a treatment (commonly the drug or a placebo).

Stacking in Matched Pairs Design.

You can think of matched pair design as a type of stacked randomized block design . With either design, your goal is to control for some variable that’s going to skew your results. In the above experiment, it isn’t just age that could account for differences in how people respond to drugs, several other confounding variables could also affect your experiment. The purpose of the blocks is to minimize a single source of variability (for example, differences due to age). When you create matched pairs, you’re creating blocks within blocks, enabling you to control for multiple sources of potential variability. You should construct your matched pairs carefully, as it’s often impossible to account for all variables without creating a huge and complex experiment. Therefore, you should create your blocks starting with which candidates are most likely to affect your results. Back to Top.

Observational Study

What is an observational study.

An observational study (sometimes called a natural experiment or a quasi-experiment) is where the researcher observes the study participants and measures variables without assigning any treatments. For example, let’s say you wanted to find out the effect of cognitive therapy for ADHD. In an experimental study, you would assign some patients cognitive therapy and other patients some other form of treatment (or no treatment at all). In an observational study you would find patients who are already undergoing the therapy , and some who are already participating in other therapies (or no therapy at all).

Ideally, treatments should be investigated experimentally with random assignment of treatments to participants. This random assignment means that measured and unmeasured characteristics are evenly divided over the groups. In other words, any differences between the groups would be due to chance. Any statistical tests you run on these types of studies would be reliable. However, it isn’t always ethical or feasible to run experimental studies, especially in medical studies involving life-threatening or potentially disabled studies. In these cases, observational studies are used.

Examples of Observational Studies

Selective Serotonin Reuptake Inhibitors and Violent Crime: A Cohort Study A study published in PLOS magazine studied the uncertain relationship between SSRIs (like Prozac and Paxil) and Violent Crime. The researchers “…extracted information on SSRIs prescribed in Sweden between 2006 and 2009 from the Swedish Prescribed Drug Register and information on convictions for violent crimes for the same period from the Swedish national crime register. They then compared the rate of violent crime while individuals were prescribed SSRIs with the rate of violent crime in the same individuals while not receiving medication.” The study findings found an increased association between SSRI use and violent crimes.

Cleaner Air Found to Add 5 Months to Life A Brigham Young University study examined the connected between air quality and life expectancy. The researchers looked at life expectancy data from 51 metropolitan areas and compared the figures to air quality improvements in each region from the 1980s to 1990s. After taking into account factors like smoking and socioeconomic status, the researchers found that an average of about five months life expectancy was attributed to clean air. The New York Times printed a summary of the results here .

Effects of Children of Occupational Exposures to Lead Researchers matched 33 children whose parents were exposed to lead at work with 33 children who were the same age and loved in the same neighborhood. Elevated levels of lead were found in the exposed children. This was attributed to levels of lead that the parents were exposed to at work, and poor hygiene practices of the parent (UPenn).

Longitudinal Research

Longitudinal research is an observational study of the same variables over time. Studies can last weeks, months or even decades. The term “longitudinal” is very broad, but generally means to collect data over more than one period, from the same participants(or very similar participants). According to sociologist Scott Menard, Ph.D. , the research should also involve some comparison of data among or between periods. However, the longitudinal research doesn’t necessarily have to be collected over time. Data could be collected at one point in time but include retrospective data. For example, a participant could be asked about their prior exercise habits up to and including the time of the study.

The purpose of Longitudinal Research is to:

• Record patterns of change. For example, the development of emphysema over time.
• Establish the direction and magnitude of causal relationships. For example, women who smoke are 12 times more likely to die of emphysema than non-smokers.

Cross sectional research involves collecting data at one specific point in time. You can interact with individuals directly, or you could study data in a database or other media. For example, you could study medical databases to see if illegal drug use results in heart disease. If you find a correlation ( what is correlation? ) between illegal drug use and heart disease, that would support the claim that illegal drug use may increase the risk of heart disease.

Cross sectional research is a descriptive study ; you only record what you find and you don’t manipulate variables like in traditional experiments. It is most often used to look at how often a phenomenon occurs in a population .

Advantages and Disadvantages of Cross Sectional Research

• Can be very inexpensive if you already have a database (for example, medical history data in a hospital database).
• Allows you to look at many factors at the same time, like age/weight/height/tobacco use/drug use.

Disadvantages

• Can result in weak evidence, compared to cohort studies (which cost more and take longer).
• Available data may not be suited to your research question. For example, if you wanted to know if sugar consumption leads to obesity, you are unlikely to find data on sugar consumption in a medical database.
• Cross sectional research studies are usually unable to control for confounding variables . One reason for this is that it’s usually difficult to find people who are similar enough. For example, they might be decades apart in age or they might be born in very different geographic regions.

Cross sectional research can give the “big picture” and can be a foundation to suggest other areas for more expensive research. For example, if the data suggests that there may be a relationship between sugar consumption and obesity, this could bolster an application for funding more research in this area.

Cross-Sectional vs Longitudinal Research

Both cross-sectional and longitudinal research studies are observational. They are both conducted without any interference to the study participants. Cross-sectional research is conducted at a single point in time while a longitudinal study can be conducted over many years.

For example, let’s say researchers wanted to find out if older adults who gardened had lower blood pressure than older adults who did not garden. In a cross-sectional study, the researchers might select 100 people from different backgrounds, ask them about their gardening habits and measure their blood pressure. The study would be conducted at approximately the same period of time (say, over a week). In a longitudinal study, the questions and measurements would be the same. But the researchers would follow the participants over time. They may record the answers and measurements every year.

One major advantage of longitudinal research is that over time, researchers are more able to provide a cause-and-effect relationship. With the blood pressure example above, cross-sectional research wouldn’t give researchers information about what blood pressure readings were before the study. For example, participants may have had lower blood pressure before gardening. Longitudinal research can detect changes over time, both at the group and at the individual level.

Types of Longitudinal Design

Longitudinal Panel Design is the “traditional” type of longitudinal design, where the same data is collected from the same participants over a period of time. Repeated cross-sectional studies can be classified as longitudinal. Other types are:

• Total population design, where the total population is surveyed in each study period.
• Revolving panel design, where new participants are selected each period.

What is Pretest Posttest Design?

A pretest posttest design is an experiment where measurements are taken both before and after a treatment . The design means that you are able to see the effects of some type of treatment on a group. Pretest posttest designs may be quasi-experimental, which means that participants are not assigned randomly. However, the most usual method is to randomly assign the participants to groups in order to control for confounding variables. Three main types of pretest post design are commonly used:

• Randomized Control-Group Pretest Posttest Design.
• Randomized Solomon Four-Group Design.
• Nonrandomized Control Group Pretest-Posttest Design.

1. Randomized Control-Group Pretest Posttest Design.

The pre-test post-test control group design is also called the classic controlled experimental design . The design includes both a control and a treatment group. For example, if you wanted to gauge if a new way of teaching math was effective, you could:

• Randomly assign participants to a treatment group or a control group .
• Administer a pre-test to the treatment group and the control group.
• Use the new teaching method on the treatment group and the standard method on the control group, ensuring that the method of treatment is the only condition that is different.
• Administer a post-test to both groups.
• Assess the differences between groups.

Two issues can affect the Randomized Control-Group Pretest Posttest Design:

• Internal validity issues: maturation (i.e. biological changes in participants can affect differences between pre- and post-tests) and history (where participants experience something outside of the treatment that can affect scores).
• External validity issues : Interaction of the pre-test and the treatment can occur if participants are influenced by the tone or content of the question. For example, a question about how many hours a student spends on homework might prompt the student to spend more time on homework.

2. Randomized Solomon Four-Group Design.

In this type of pretest posttest design, four groups are randomly assigned: two experimental groups E1/E2 and two control groups C1/C2. Groups E1 and C1 complete a pre-test and all four groups complete a post-test. This better controls for the interaction of pretesting and posttesting; in the “classic” design, participants may be unduly influenced by the questions on the pretest.

3. Nonrandomized Control Group Pretest-Posttest Design.

This type of test is similar to the “classic” design, but participants are not randomly assigned to groups. Nonrandomization can be more practical in real-life, when you are dealing with groups like students or employees who are already in classes or departments; randomization (i.e. moving people around to form new groups) could prove disruptive. This type of experimental design suffers from problems with internal validity more so than the other two types. Back to Top.

What is a Quasi-Experimental Design?

A quasi-experimental design has much the same components as a regular experiment, but is missing one or more key components. The three key components of a traditional experiment are:

• Pre-post test design.
• Treatment and control groups.
• Random assignment of subjects to groups.

You may want or need to deliberately leave out one of these key components. This could be for ethical or methodological reasons. For example:

• It would be unethical to withhold treatment from a control group. This is usually the case with life-threatening illness, like cancer.
• It would be unethical to treat patients; for example, you might want to find out if a certain drug causes blindness.
• A regular experiment might be expensive and impossible to fund.
• An experiment could technically fail due to loss of participants, but potentially produce useful data.
• It might be logistically impossible to control for all variables in a regular experiment.

These types of issues crop up frequently, leading to the widespread acceptance of quasi-experimental designs — especially in the social sciences. Quasi-experimental designs are generally regarded as unreliable and unscientific in the physical and biological sciences.

Some experiments naturally fall into groups. For example, you might want to compare educational experiences of first, middle and last born children. Random assignment isn’t possible, so these experiments are quasi-experimental by nature.

Quasi-Experimental Design Examples.

The general form of a quasi-experimental design thesis statement is “What effect does (a certain intervention or program) have on a (specific population)”?

Example 1 : Does smoking during pregnancy leads to low birth weight? It would be unethical to randomly assign one group of mothers packs of cigarettes to smoke. The researcher instead asks the mothers if they smoked during pregnancy and assigns them to groups after the fact.

Example 2 : Does thoughtfully designed software improve learning outcomes for students? This study used a pre-post test design and multiple classrooms to show how technology can be successfully implemented in schools.

Example 3 : Can being mentored for your job lead to increased job satisfaction? This study followed 73 employees, some who were mentored and some who were not. Back to Top.

What is Randomized Block Design?

In randomized block design, the researcher divides experimental subjects into homogeneous blocks. Treatments are then randomly assigned to the blocks. The variability within blocks should be greater than the variability between blocks. In other words, you need to make sure that the blocks contain subjects that are very similar. For example, you could put males in one block and females in a second block. This method is practically identical to stratified random sampling (SRS), except the blocks in SRS are called “ strata .” Randomized block design reduces variability in experiments.

Age isn’t the only potential source of variability. Other blocking factors that you could consider for this type of experiment include:

• Consumption of certain foods.
• Use of over the counter food supplements.
• Adherence to dosing regimen.
• Differences in metabolism due to genetic differences, liver or kidney issues, race, or sex.
• Coexistence of other disorders.
• Use of other drugs.

Randomized block experimental design is sometimes called randomized complete block experimental design , because the word “complete” makes it clear that all subjects are included in the experiment, not just a sample. However, the setup of the experiment usually makes it clear that all subjects are included, so most people will drop the word complete . Back to Top.

What is a Randomized Controlled Trial?

A randomized controlled trial is an experiment where the participants are randomly allocated to two or more groups to test a specific treatment or drug. Participants are assigned to either an experimental group or a comparison group. Random allocation means that all participants have the same chance of being placed in either group. The experimental group receives a treatment or intervention, for example:

• Diagnostic Tests.
• Experimental medication.
• Interventional procedures.
• Screening programs.
• Specific types of education.

Participants in the comparison group receive a placebo (a dummy treatment), an alternative treatment, or no treatment at all. There are many randomization methods available. For example, simple random sampling , stratified random sampling or systematic random sampling. The common factor for all methods is that researchers, patients and other parties cannot tell ahead of time who will be placed in which group.

Advantages and Disadvantages of Randomized Controlled Trials

• Random allocation can cancel out population bias ; it ensures that any other possible causes for the experimental results are split equally between groups.
• Blinding is easy to include in this type of experiment.
• Results from the experiment can be analyzed with statistical tests and used to infer other possibilities, like the likelihood of the method working for all populations.
• Participants are readily identifiable as members of a specific population./li>
• Generally more expensive and more time consuming than other methods.
• Very large sample sizes (over 5,000 participants) are often needed.
• Random controlled trials cannot uncover causation/risk factors. For example, ethical concerns would prevent a randomized controlled trial investigating the risk factors for smoking.
• This type of experimental design is unsuitable for outcomes which take a long time to develop. Cohort studies may be a more suitable alternative.
• Some programs, for example cancer screening, are unsuited for random allocation of participants (again, due to ethical concerns).
• Volunteer bias can be an issue.

What is a Within Subjects Experimental Design?

In a within subjects experimental design, participants are assigned more than one treatment: each participant experiences all the levels for any categorical explanatory variable . The levels can be ordered, like height or time. Or they can be un-ordered. For example, let’s say you are testing if blood pressure is raised when watching horror movies vs. romantic comedies. You could have all the participants watch a scary movie, then measure their blood pressure. Later, the same group of people watch a romantic comedy, and their blood pressure is measured.

Within subjects designs are frequently used in pre-test/post-test scenarios. For example, if a teacher wants to find out if a new classroom strategy is effective, they might test children before the strategy is in place and then after the strategy is in place.

Within subjects designs are similar to other analysis of variance designs, in that it’s possible to have a single independent variable, or multiple factorial independent variables. For example, three different depression inventories could be given at one, three, and six month intervals.

Advantages and Disadvantages of Within Subjects Experimental Design.

• It requires fewer participants than the between subjects design. If a between subjects design were used for the blood pressure example above, double the amount of participants would be required. Within subjects design therefore requires fewer resources and is generally cheaper.
• Individual difference between participants are controlled for, as each participant acts as their own control. As the subjects are measured multiple times, this better enables the researcher to hone in on individual differences so that they can be removed from the analysis.
• Effects from one test could carry over to the next, a phenomenon called the “range effect.” In the blood pressure example, if participants were asked to watch the scary movie first, their blood pressure could stay elevated for hours afterwards, skewing the results from the romantic comedy.
• Participants can exhibit “practice effects”, where they improve scores simply by taking the same test multiple times. This is often an issue on pre-test/post-test studies.
• Data is not completely independent, which may effect running hypothesis tests , like ANOVA .

References : Merck Manual. Retrieved Jan 1, 2016 from: http://www.merckmanuals.com/professional/clinical-pharmacology/factors-affecting-response-to-drugs/introduction-to-factors-affecting-response-to-drugs Penn State: Basic Principles of DOE. Retrieved Jan 1, 2016 from: https://onlinecourses.science.psu.edu/stat503/node/67 Image: SUNY Downstate. Retrieved Jan 1, 2016 from: http://library.downstate.edu/EBM2/2200.htm

5. Related Topics

• Accuracy and Precision .
• Block plots .
• Cluster Randomization .
• What is Clustering?
• What is the Cohort Effect?
• What is a Control Group?
• What is Counterbalancing?
• Data Collection Methods
• What is an Effect Size?
• What is a Experimental Group (or Treatment Group)?
• Fixed, Random, and Mixed Effects Models
• What are generalizability and transferability?
• What is Grounded Theory?
• The Hawthorne Effect .
• The Hazard Ratio.
• Inter-rater Reliability.
• Main Effects .
• Order Effects .
• The Placebo Effect
• What is the Practice Effect?
• Primary and Secondary Data .
• What is Qualitative Research?
• What is Quantitative Research?
• What is a Randomized Clinical Trial?
• Random Selection and Assignment.
• Randomization .
• Recall Bias .
• What is Response Bias?
• Research Methods (includes Quantitative and Qualitative).
• Subgroup Analysis .
• What is Survey Sampling?
• Systematic Errors.
• Treatment Diffusion.

Agresti A. (1990) Categorical Data Analysis. John Wiley and Sons, New York. Cook, T. (2005). Introduction to Statistical Methods for Clinical Trials(Chapman & Hall/CRC Texts in Statistical Science) 1st Edition. Chapman and Hall/CRC Friedman (2015). Fundamentals of Clinical Trials 5th ed. Springer.” Dodge, Y. (2008). The Concise Encyclopedia of Statistics . Springer. Everitt, B. S.; Skrondal, A. (2010), The Cambridge Dictionary of Statistics , Cambridge University Press. Gonick, L. (1993). The Cartoon Guide to Statistics . HarperPerennial. Kotz, S.; et al., eds. (2006), Encyclopedia of Statistical Sciences , Wiley. Levine, D. (2014). Even You Can Learn Statistics and Analytics: An Easy to Understand Guide to Statistics and Analytics 3rd Edition. Pearson FT Press UPenn. http://finzi.psych.upenn.edu/library/granovaGG/html/blood_lead.html. Retrieved May 1, 2020.

• History & Society
• Science & Tech
• Biographies
• Animals & Nature
• Geography & Travel
• Arts & Culture
• Games & Quizzes
• On This Day
• One Good Fact
• New Articles
• Lifestyles & Social Issues
• Philosophy & Religion
• Politics, Law & Government
• World History
• Health & Medicine
• Browse Biographies
• Birds, Reptiles & Other Vertebrates
• Bugs, Mollusks & Other Invertebrates
• Environment
• Fossils & Geologic Time
• Entertainment & Pop Culture
• Sports & Recreation
• Visual Arts
• Demystified
• Image Galleries
• Infographics
• Top Questions
• Britannica Kids
• Saving Earth
• Space Next 50
• Student Center
• Introduction
• Tabular methods
• Graphical methods
• Exploratory data analysis
• Events and their probabilities
• Random variables and probability distributions
• The binomial distribution
• The Poisson distribution
• The normal distribution
• Sampling and sampling distributions
• Estimation of a population mean
• Estimation of other parameters
• Estimation procedures for two populations
• Hypothesis testing
• Bayesian methods

Analysis of variance and significance testing

Regression model, least squares method, analysis of variance and goodness of fit, significance testing.

• Residual analysis
• Model building
• Correlation
• Time series and forecasting
• Nonparametric methods
• Acceptance sampling
• Statistical process control
• Sample survey methods
• Decision analysis

Experimental design

Our editors will review what you’ve submitted and determine whether to revise the article.

• Arizona State University - Educational Outreach and Student Services - Basic Statistics
• Princeton University - Probability and Statistics
• Statistics LibreTexts - Introduction to Statistics
• University of North Carolina at Chapel Hill - The Writing Center - Statistics
• Corporate Finance Institute - Statistics
• statistics - Children's Encyclopedia (Ages 8-11)
• statistics - Student Encyclopedia (Ages 11 and up)
• Table Of Contents

Data for statistical studies are obtained by conducting either experiments or surveys. Experimental design is the branch of statistics that deals with the design and analysis of experiments. The methods of experimental design are widely used in the fields of agriculture, medicine , biology , marketing research, and industrial production.

Recent News

In an experimental study, variables of interest are identified. One or more of these variables, referred to as the factors of the study , are controlled so that data may be obtained about how the factors influence another variable referred to as the response variable , or simply the response. As a case in point, consider an experiment designed to determine the effect of three different exercise programs on the cholesterol level of patients with elevated cholesterol. Each patient is referred to as an experimental unit , the response variable is the cholesterol level of the patient at the completion of the program, and the exercise program is the factor whose effect on cholesterol level is being investigated. Each of the three exercise programs is referred to as a treatment .

Three of the more widely used experimental designs are the completely randomized design, the randomized block design, and the factorial design. In a completely randomized experimental design, the treatments are randomly assigned to the experimental units. For instance, applying this design method to the cholesterol-level study, the three types of exercise program (treatment) would be randomly assigned to the experimental units (patients).

The use of a completely randomized design will yield less precise results when factors not accounted for by the experimenter affect the response variable. Consider, for example, an experiment designed to study the effect of two different gasoline additives on the fuel efficiency , measured in miles per gallon (mpg), of full-size automobiles produced by three manufacturers. Suppose that 30 automobiles, 10 from each manufacturer, were available for the experiment. In a completely randomized design the two gasoline additives (treatments) would be randomly assigned to the 30 automobiles, with each additive being assigned to 15 different cars. Suppose that manufacturer 1 has developed an engine that gives its full-size cars a higher fuel efficiency than those produced by manufacturers 2 and 3. A completely randomized design could, by chance , assign gasoline additive 1 to a larger proportion of cars from manufacturer 1. In such a case, gasoline additive 1 might be judged to be more fuel efficient when in fact the difference observed is actually due to the better engine design of automobiles produced by manufacturer 1. To prevent this from occurring, a statistician could design an experiment in which both gasoline additives are tested using five cars produced by each manufacturer; in this way, any effects due to the manufacturer would not affect the test for significant differences due to gasoline additive. In this revised experiment, each of the manufacturers is referred to as a block, and the experiment is called a randomized block design. In general, blocking is used in order to enable comparisons among the treatments to be made within blocks of homogeneous experimental units.

Factorial experiments are designed to draw conclusions about more than one factor, or variable. The term factorial is used to indicate that all possible combinations of the factors are considered. For instance, if there are two factors with a levels for factor 1 and b levels for factor 2, the experiment will involve collecting data on a b treatment combinations. The factorial design can be extended to experiments involving more than two factors and experiments involving partial factorial designs.

A computational procedure frequently used to analyze the data from an experimental study employs a statistical procedure known as the analysis of variance. For a single-factor experiment, this procedure uses a hypothesis test concerning equality of treatment means to determine if the factor has a statistically significant effect on the response variable. For experimental designs involving multiple factors, a test for the significance of each individual factor as well as interaction effects caused by one or more factors acting jointly can be made. Further discussion of the analysis of variance procedure is contained in the subsequent section.

Regression and correlation analysis

Regression analysis involves identifying the relationship between a dependent variable and one or more independent variables . A model of the relationship is hypothesized, and estimates of the parameter values are used to develop an estimated regression equation . Various tests are then employed to determine if the model is satisfactory. If the model is deemed satisfactory, the estimated regression equation can be used to predict the value of the dependent variable given values for the independent variables.

In simple linear regression , the model used to describe the relationship between a single dependent variable y and a single independent variable x is y = β 0 + β 1 x + ε. β 0 and β 1 are referred to as the model parameters, and ε is a probabilistic error term that accounts for the variability in y that cannot be explained by the linear relationship with x . If the error term were not present, the model would be deterministic; in that case, knowledge of the value of x would be sufficient to determine the value of y .

In multiple regression analysis , the model for simple linear regression is extended to account for the relationship between the dependent variable y and p independent variables x 1 , x 2 , . . ., x p . The general form of the multiple regression model is y = β 0 + β 1 x 1 + β 2 x 2 + . . . + β p x p + ε. The parameters of the model are the β 0 , β 1 , . . ., β p , and ε is the error term.

Either a simple or multiple regression model is initially posed as a hypothesis concerning the relationship among the dependent and independent variables. The least squares method is the most widely used procedure for developing estimates of the model parameters. For simple linear regression, the least squares estimates of the model parameters β 0 and β 1 are denoted b 0 and b 1 . Using these estimates, an estimated regression equation is constructed: ŷ = b 0 + b 1 x . The graph of the estimated regression equation for simple linear regression is a straight line approximation to the relationship between y and x .

As an illustration of regression analysis and the least squares method, suppose a university medical centre is investigating the relationship between stress and blood pressure . Assume that both a stress test score and a blood pressure reading have been recorded for a sample of 20 patients. The data are shown graphically in Figure 4 , called a scatter diagram . Values of the independent variable, stress test score, are given on the horizontal axis, and values of the dependent variable, blood pressure, are shown on the vertical axis. The line passing through the data points is the graph of the estimated regression equation: ŷ = 42.3 + 0.49 x . The parameter estimates, b 0 = 42.3 and b 1 = 0.49, were obtained using the least squares method.

A primary use of the estimated regression equation is to predict the value of the dependent variable when values for the independent variables are given. For instance, given a patient with a stress test score of 60, the predicted blood pressure is 42.3 + 0.49(60) = 71.7. The values predicted by the estimated regression equation are the points on the line in Figure 4 , and the actual blood pressure readings are represented by the points scattered about the line. The difference between the observed value of y and the value of y predicted by the estimated regression equation is called a residual . The least squares method chooses the parameter estimates such that the sum of the squared residuals is minimized.

A commonly used measure of the goodness of fit provided by the estimated regression equation is the coefficient of determination . Computation of this coefficient is based on the analysis of variance procedure that partitions the total variation in the dependent variable, denoted SST, into two parts: the part explained by the estimated regression equation, denoted SSR, and the part that remains unexplained, denoted SSE.

The measure of total variation, SST, is the sum of the squared deviations of the dependent variable about its mean: Σ( y − ȳ ) 2 . This quantity is known as the total sum of squares. The measure of unexplained variation, SSE, is referred to as the residual sum of squares. For the data in Figure 4 , SSE is the sum of the squared distances from each point in the scatter diagram (see Figure 4 ) to the estimated regression line: Σ( y − ŷ ) 2 . SSE is also commonly referred to as the error sum of squares. A key result in the analysis of variance is that SSR + SSE = SST.

The ratio r 2 = SSR/SST is called the coefficient of determination. If the data points are clustered closely about the estimated regression line, the value of SSE will be small and SSR/SST will be close to 1. Using r 2 , whose values lie between 0 and 1, provides a measure of goodness of fit; values closer to 1 imply a better fit. A value of r 2 = 0 implies that there is no linear relationship between the dependent and independent variables.

When expressed as a percentage , the coefficient of determination can be interpreted as the percentage of the total sum of squares that can be explained using the estimated regression equation. For the stress-level research study, the value of r 2 is 0.583; thus, 58.3% of the total sum of squares can be explained by the estimated regression equation ŷ = 42.3 + 0.49 x . For typical data found in the social sciences, values of r 2 as low as 0.25 are often considered useful. For data in the physical sciences, r 2 values of 0.60 or greater are frequently found.

In a regression study, hypothesis tests are usually conducted to assess the statistical significance of the overall relationship represented by the regression model and to test for the statistical significance of the individual parameters. The statistical tests used are based on the following assumptions concerning the error term: (1) ε is a random variable with an expected value of 0, (2) the variance of ε is the same for all values of x , (3) the values of ε are independent, and (4) ε is a normally distributed random variable.

The mean square due to regression, denoted MSR, is computed by dividing SSR by a number referred to as its degrees of freedom ; in a similar manner, the mean square due to error, MSE , is computed by dividing SSE by its degrees of freedom. An F-test based on the ratio MSR/MSE can be used to test the statistical significance of the overall relationship between the dependent variable and the set of independent variables. In general, large values of F = MSR/MSE support the conclusion that the overall relationship is statistically significant. If the overall model is deemed statistically significant, statisticians will usually conduct hypothesis tests on the individual parameters to determine if each independent variable makes a significant contribution to the model.

An Introduction to Data Analysis

8 statistical models.

Uninterpreted data is uninformative. We cannot generalize, draw inferences or attempt to make predictions unless we make (however minimal) assumptions about the data at hand: what it represents, how it came into existence, which parts relate to which other parts, etc. One way of explicitly acknowledging these assumptions is to engage in model-based data analysis. A statistical model is a conventionally condensed formal representation of the assumptions we make about what the data is and how it might have been generated. In this way, model-based data analysis is more explicit about the analyst’s assumptions than other approaches, such as test-based approaches, which we will encounter in Chapter 16 .

There is room for divergence in how to think about a statistical model, the assumptions it encodes and the truth. Some will want to reason with models using language like “if we assume that model \(M\) is true, then …” or “this shows convincingly that \(M\) is likely to be the true model”. Others feel very uncomfortable with such language. In times of heavy discomfort they might repeat their soothing mantra:

All models are wrong, but some are useful. — Box ( 1979 )

To become familiar with model-based data analysis, Section 8.1 introduces the concept of a probabilistic statistical model . Section 8.2 expands on the notation, both formulaic and graphical, which we will use in this book to communicate about models. Finally, Section 8.3 enlarges on the crucial aspects of parameters and priors.

The learning goals for this chapter are:

• become familiar with the notion of a (Bayesian) statistical model
• likelihood function, parameters, prior, prior distribution
• formulas & graphs

--> \( Y = \beta_{0} + \beta_{1}X_{1} + \beta_{2}X_{2} + \beta_{3}X_{3} + \beta_{12}X_{1}X_{2} + \\ \beta_{13}X_{1}X_{3} + \beta_{23}X_{2}X_{3} + \beta_{123}X_{1}X_{2}X_{3} + \\ \mbox{experimental error} \)

. --> \( \beta_{11}X_{1}^{2} + \beta_{22}X_{2}^{2} + \beta_{33}X_{3}^{2} \)

Help Center Help Center

• Help Center
• Trial Software
• Product Updates
• Documentation

Design of Experiments (DOE)

Passive data collection leads to a number of problems in statistical modeling. Observed changes in a response variable may be correlated with, but not caused by, observed changes in individual factors (process variables). Simultaneous changes in multiple factors may produce interactions that are difficult to separate into individual effects. Observations may be dependent, while a model of the data considers them to be independent.

Designed experiments address these problems. In a designed experiment, the data-producing process is actively manipulated to improve the quality of information and to eliminate redundant data. A common goal of all experimental designs is to collect data as parsimoniously as possible while providing sufficient information to accurately estimate model parameters.

Full Factorial Designs

Fractional Factorial Designs

Response Surface Designs

D-Optimal Designs

Latin Hypercube Designs

Quasi-Random Designs

Designs for all treatments

Designs for selected treatments

Quadratic polynomial models

This example shows how to improve the performance of an engine cooling fan through a Design for Six Sigma approach using Define, Measure, Analyze, Improve, and Control.

Minimum variance parameter estimates

MATLAB Command

You clicked a link that corresponds to this MATLAB command:

Run the command by entering it in the MATLAB Command Window. Web browsers do not support MATLAB commands.

Select a Web Site

Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .

• Switzerland (English)
• Switzerland (Deutsch)
• Switzerland (Français)
• 中国 (English)

You can also select a web site from the following list:

How to Get Best Site Performance

Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.

• América Latina (Español)
• Canada (English)
• United States (English)
• Belgium (English)
• Denmark (English)
• Deutschland (Deutsch)
• España (Español)
• Finland (English)
• France (Français)
• Ireland (English)
• Italia (Italiano)
• Luxembourg (English)
• Netherlands (English)
• Norway (English)
• Österreich (Deutsch)
• Portugal (English)
• Sweden (English)
• United Kingdom (English)

Asia Pacific

• Australia (English)
• India (English)
• New Zealand (English)

Contact your local office

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Asian-Australas J Anim Sci
• v.31(9); 2018 Sep

Guidelines for experimental design and statistical analyses in animal studies submitted for publication in the Asian-Australasian Journal of Animal Sciences

Seongwon seo.

1 Division of Animal and Dairy Science, Chungnam National University, Daejeon 34134, Korea

Seoyoung Jeon

2 Asian-Australasian Journal of Animal Sciences, Seoul 08776, Korea

3 Department of Agricultural Biotechnology, College of Agriculture and Life Science, Seoul National University, Seoul 08826, Korea

Animal experiments are essential to the study of animal nutrition. Because of the large variations among individual animals and ethical and economic constraints, experimental designs and statistical analyses are particularly important in animal experiments. To increase the scientific validity of the results and maximize the knowledge gained from animal experiments, each experiment should be appropriately designed, and the observations need to be correctly analyzed and transparently reported. There are many experimental designs and statistical methods. This editorial does not aim to review and present particular experimental designs and statistical methods. Instead, we discuss some essential elements when designing an animal experiment and conducting statistical analyses in animal nutritional studies and provide guidelines for submitting a manuscript to the Asian-Australasian Journal of Animal Sciences for consideration for publication.

INTRODUCTION

For scientific, ethical, and economic reasons, experiments involving animals should be appropriately designed, correctly analyzed, and transparently reported. This increases the scientific validity of the results and maximizes the knowledge gained from each experiment. Nonetheless, biologists, on average, feel uncomfortable with mathematics and statistics, and they often design experiments and analyze data in inappropriate ways [ 1 ]. Therefore, in some fields of research where animal experiments are essential, the editorial board regularly reviews the statistical methodologies reported in the papers and presents their suitability [ 2 – 5 ]. Some fields of research have set up consortia and provide guidelines for animal experiments [ 6 , 7 ], and some scientific journals have guidelines for their authors to follow for publication [ 8 , 9 ]. For example, in the animal science field, the Journal of Dairy Science provides detailed guidance on statistical methodology in the instructions to authors [ 10 ]. Animal Feed Science and Technology has published two editorials that discuss proper experimental design and statistical analyses to guide authors who are submitting manuscripts to the journal [ 11 , 12 ].

The Asian-Australasian Journal of Animal Sciences (AJAS) published the first issue in January 1988, and its contribution and influence to the animal science fields have continuously expanded over the past three decades. In particular, a total of 102 nutritional studies were published in AJAS in 2017, which included 84 in vivo trials. In these studies, statistical methods are essential, and authors should strive to employ an appropriate experimental design and statistical analyses to provide the reader with scientifically relevant and valid knowledge.

This editorial will discuss some of the principles of experimental design and statistical analysis and provide guidelines when submitting nutritional studies to AJAS for consideration for publication.

EXPERIMENTAL DESIGN

Authors must provide details regarding the experimental design in a manuscript such that reviewers and readers have sufficient information about how the study was conducted and can evaluate the quality of the experimental design. Details include animal characteristics (e.g., species, breed, gender, weight), number of treatments, number of experimental and sampling units, arrangement of treatments (e.g., factorial, change-over), and consideration for known variation (e.g., blocking, covariates). Only properly designed experiments can yield valid and reliable results that lead to correct and appropriate interpretations and conclusions in the study.

The experimental unit and the number of replicates

Treatments, the set of circumstances created for an experiment based on research hypotheses, are the effects to be measured and compared in the experiment [ 13 ]. The treatment is applied to the experimental unit, and one experimental unit corresponds to a single replication; Kuehl [ 14 ] defines the experimental unit as “the physical entity” or subject exposed to the treatment independently of other units. The number of replicates (i.e., sample size) is the number of experimental units per each treatment. Defining the experimental unit correctly is crucial for proper experimental design and statistical analysis. However, correctly defining the experiment unit is sometimes not easy. This is especially true in the cases where a group of animals are fed together in a pen, there is debate as to the most appropriate experimental unit between statisticians and biologists [ 11 ].

Like most other biostatisticians [ 14 , 15 ], editors of AJAS have a more conservative view regarding the determination of the experimental unit. For many nutritional studies, the purpose of the experiment is to infer population means. For example, in a feeding trial in which different dietary treatments are applied to different groups of animals, the ultimate goal of the experiment is not to observe the treatment effect within the experimental animals but to investigate its effect on independent animals in the real world. The role of replication is to provide measures of how much the results are reliable and reproducible, and thus replicates are to be independent observations and experimental units must be independent of each other. If a treatment is applied to a group of animals in a single pen, the individual animals are not independent; thus, the pen is considered the experimental unit even when measurements are made individually. The treatment effect is confounded by the effect of the pen in this case, and it is obvious that the pen should be the experimental unit because it is unknown whether the results of the experiment were caused by the treatment of the pen. On the other hand, if treatments are randomly assigned to individual animals within a group of animals in a pen, the individual animal can be considered the experimental unit even though they are in the same pen.

A sufficient number of replicates are needed to obtain a reliable outcome from an experiment. Because the number of replicates is related with the power of a test, more experimental replicates can provide greater statistical power to detect a desired difference among treatments. The cost of replicates, however, is high in animal experiments, and the smallest number of replicates is preferred, as long as it is sufficient to detect a difference. For this purpose, power tests are performed prior to initiating an experiment to determine the required sample size based on expected variation in means and the size of the difference between means that needs to be detected.

Power tests are also useful for supporting the validity of an experiment when no significant difference is observed between the treatment means. It is not uncommon to fail to detect a significant difference between treatments, and in this case, one can argue that significance was not observed simply because the sample size was small. The result from the power test can provide supportive evidence that the reason for the failure to detect a difference between treatments was not because the sample size was small, rather the difference between the treatment means was not great enough to be considered significant.

Therefore, AJAS encourages authors to provide the results of power tests. The results of power tests can be used to justify that the experiment was appropriately designed.

Consideration for known variations

To properly test for treatment effects, factors other than the main treatment that may affect the response of the animals should be minimized or at least accounted for. In this regard, the use of a block or covariate is recommended.

Blocking is a practice wherein the experimental units are placed into groups of similar units based on one or more factors that are known or expected to affect the responses to be measured and tested. Physical and physiological characteristics, such as sex, litter, and initial body weight, are commonly used for blocking in the animal science field. Blocking controls the variability of experimental units and reduces experimental error.

Covariates are variables that are known or expected to be related to the response variables of interest. The primary difference between blocks and covariates is that covariates are continuous variables, whereas blocks are categorical variables. For example, animals can be grouped or blocked as high, medium, and low groups according to their body weight. Conversely, individual body weight can be used as a covariate to reduce the estimates of experimental error in the statistical model. Blocking is applied at the experimental design stage, whereas the use of covariates is applied when conducting statistical analysis.

The use of a block and covariate is a sound and logical way to account for known errors and reduce unexplained errors. The AJAS editorial board thus encourages authors to use blocks and covariates if there are known or expected variables that could have a significant effect on the response to be tested for in the experimental treatments.

When a limited number of animals are available or when individual animal variation is to be removed, crossover (i.e., changeover) designs are often used in animal nutritional studies. In this case, it can be an issue if a carryover effect from a treatment given in a previous period influences the response in the following treatment. It should be noted that crossover designs should be avoided when significant carryover effects are expected [ 16 ]. Even if a significant carryover effect is not expected, the potential for a carryover effect should not be ignored in crossover designs. A sufficient rest or wash-out period between two treatment periods is one of the practical ways to minimize carryover effects. More importantly, the order of treatments for each animal should be balanced to avoid confounding of treatment and period effects and to minimize the influence of carryover effects. In a balanced crossover design, each treatment follows each of the other treatments an equal number of times, and each treatment is assigned once for each animal and the same number of times in each period. When a carryover effect is suspected, its significance also needs to be tested by statistical analysis. The AJAS editorial board recommends authors describe the procedure used to minimize possible carryover effects and show that carryover effects are not significant in their study when using a crossover design.

Randomization

Randomization is an essential procedure to ensure the reliability of the experiment and the validity of the statistical analysis. The purpose of an experiment is to make inferences about the population mean and variance, and the statistical analysis assumes the observations are from a random sample from a normally distributed population. This assumption can be valid only through randomization. In animal nutritional studies, two randomization processes are required: random sampling of experimental units and random allocation of treatments to experimental animals.

Theoretically, experimental animals represent the animal population of interest; thus, they need to be randomly selected from the population. However, this is usually not feasible, if not impossible, in the real world and whether experimental animals can be considered a random sample is questionable. Nevertheless, whenever possible, randomization must be practiced in selecting experimental animals to eliminate biases and to obtain valid estimates of experimental error variance. For example, when a deep analysis is performed on selected animals (e.g., blood analysis for selected animals from a group of animals in each treatment), random selection should be conducted.

Random allocation of treatments to experimental units is the most important and critical step to justify and establish the validity of statistical inferences for the parameters of the population and tests of hypothesis. The experimental errors are assumed to be independently and normally distributed. Estimation of parameters and statistical inferences can be possible if and only if this assumption is valid. Random assignment of treatments to experimental animals is the only method that guarantees the independence of observations and permits us to proceed with the analysis as if the observations are independent and normally distributed. The authors are required to describe the randomization procedure used for their animal trials.

STATISTICAL ANALYSIS

Statistical analysis is conducted to test the hypotheses and significance of tests in a study. There are many methods for conducting statistical analysis and various methods yield different results and conclusions. Proper statistical methods should be applied when conducting an experiment, and details of statistical methods should be provided in the statistical methods section of a manuscript to allow reviewers and readers to assess the quality of statistical methods used in the study.

Statistical models

When submitting a manuscript for publication in AJAS, authors should clearly define their statistical models used for the statistical analysis. Statistical models are usually expressed as linear models with the overall mean of the response variable, fixed or random variables that are known to influence the response variable, and unexplained experimental random error. The statistical model should be consistent with the experimental design and be appropriate to analyze the observations from the experiment. A clear description of the statistical model as an equation, as well as in words, is useful to understand the analytical procedure and the meaning of statistical implications and to evaluate the correctness and relevance of the statistical methods used in the study. Thus, the statistical model is often used as a criterion for the recommendation of manuscript rejection by reviewers and editors [ 11 ].

Statistical methods

Various statistical methods are available, and the choice of method depends on the data type of observations, research questions to answer, and the statistical model.

If observations of the response variables are binary (i.e., yes or no) or categorical, the logistic model or other categorical analysis needs to be used. Sometimes research questions are not about means but seek to understand the quantitative relationship between response variables or between the response variable and treatment (e.g., dose-response analysis). The linear or non-linear regression analysis is the method to be used in this case.

When the response variable of interest is a continuous variable and the research question is about means or an interval of the value, either parametric or non-parametric statistical methods can be applied. The most famous parametric statistical methods are the t-test and analysis of variance (ANOVA). A t-test is used for comparing two samples or treatments, whereas the ANOVA is used when there are more than two treatments. Different methods can be used within a t-test and an ANOVA. For example, if two samples are paired (e.g., blood samples collected before and after treatment in the same animal), a paired t-test is most appropriate. Additionally, because different levels of complexity can exist in statistical models (e.g., the existence of both fixed and random effects and their interactions, repeated measures over time), the most appropriate method may vary by the statistical models when conducting an ANOVA. Parametric methods assume that the observations are independent and normally distributed around their mean. This assumption is generally true in animal nutritional studies as long as randomization is practiced. However, it is always a good practice to test this assumption, especially if variables are expected not to follow it. For example, particle size normally has a log-normal distribution [ 17 ], and thus statistical tests need to be performed on transformed values.

If the observations are not normally distributed or the sample size is not large enough, non-parametric analyses (e.g., Mann-Whitney U test instead of a t-test and Kruskal-Wallis H test instead of a one-way ANOVA) would be the methods of choice. Non-parametric methods do not assume a normal distribution of experimental errors and more powerful to detect differences among treatments than parametric methods (e.g., t-test and ANOVA). Because non-parametric methods have more statistical power, they can exaggerate the significance of the difference between treatments. A parametric method is thus preferred when it is applicable.

Comparing the means of interest

When an ANOVA reveals that the probability that treatment means are all equal is sufficiently small enough to conclude that at least one of the treatment means is different from the others, we may ask further questions, such as which ones are different from each other? Before conducting further analyses, two things are to be considered.

First, we need to determine how small is sufficiently small. This is called the level of significance, and it is normally assumed that the probability of less than 5% (i.e., p<0.05) is statistically significant in animal nutritional studies. The level of significance is also called type I error or α, which is the probability of rejecting a null hypothesis when it is true. If α = 0.05, the test can mistakenly find treatment effects in a maximum of one out of 20 trials. When the p-value obtained using an ANOVA test is less than the level of significance, the results may be meaningful and need to be discussed; thus, comparing the means becomes interesting. If the obtained p-value is larger than the predetermined level of significance, we need to conclude that the null hypothesis is plausible, and we do not have enough evidence to reject the null hypothesis and accept the alternative hypothesis. It should be pointed out that we must not accept the null hypothesis. It is logically impossible to test whether the null hypothesis is true and to prove all the means are the same. We cannot ensure that the null hypothesis would remain plausible if the number of replicates was larger. The authors are thus required to state the level of statistical significance in the statistical analysis section.

Next, we need to determine which techniques are most appropriate for the post hoc analysis on the basis that there is a significant difference among the treatments using an ANOVA. One of the most intuitive and simplest methods to compare the means of interest is linear contrasts. If the number of treatments is t, then a set of t – 1 orthogonal contrasts can be tested. Sets of orthogonal contrasts are not unique for a given experiment; there may be many such sets. Finding an appropriate set of orthogonal contrasts lies in the structure of the treatments. For example, suppose there is an experiment of testing two feed additives as alternatives to antibiotics, and it has four treatments: without feed additives (CONT), antibiotics (ANTI), feed additive A (ADTA), and feed additive B (ADTB). A set of 3 (4 – 1) orthogonal contrasts that can be made, and logical and obvious contrasts are i) CONT vs the others, ii) ANTI vs (ADTA and ADTB), and iii) ADTA vs ADTB.

In addition to linear contrasts, there are many methods available for multiple comparisons of means; the most widely used methods include Dunnett’s test [ 18 ], Tukey’s test [ 19 ], Scheffe’s test [ 20 ], the least significant difference (LSD) [ 21 ], and Duncan’s multiple range test [ 22 ]. Among these, Duncan’s test is the most popular method in the animal nutritional studies. Approximately 37% of animal nutrition papers that conducted pair-wise comparisons in 2017 in AJAS used Duncan’s test. The second most used tests were the LSD and Tukey’s test; each accounted for 14% of multiple comparison tests.

The AJAS editorial board does not take a position on which test is more desirable under certain circumstances and leaves the decision to authors as long as the test can properly test logical questions according to the experimental design. For example, for a dose-response experiment with increasing inclusion levels, testing the significance of differences between particular means is inappropriate. Instead, linear and curve-linear regression for testing the dose-response relationship would be a better choice. A pairwise comparison procedure is appropriate to use when there is no structure among a series of treatments.

Statistical software packages and statistical procedures

There are several software packages available for statistical analysis. Even using the data analysis add-in of Microsoft Excel allows for the t-test, analysis for correlation, linear regression analysis, and one-way ANOVA to be performed. More complicated statistical models, however, require software with statistical packages, which include the statistical analysis system (SAS), general statistics (GENSTAT), statistical program for the social sciences (SPSS), Minitab, and R. The most commonly used statistical software in animal nutritional studies is SAS. Fifty-five percent of animal nutrition papers published in 2017 in AJAS used SAS. The second most popular statistical software was SPSS (27.5%), and more than 83% of the papers used one of them. Like other journals, AJAS takes no position on which of these statistical software packages is more desirable in any particular circumstances and leaves that decision to authors. However, it is required for authors to report which software is used for the statistical analysis.

Even within each statistical software package, there are different procedures that can be used for analyzing data. For example, when conducting an ANOVA in SAS, any procedures that can solve a general linear model, such as the ANOVA, GLM, and MIXED procedures, can be used. However, each procedure may have different features and work better for a specific circumstance. For example, in SAS, compared with the GLM procedure (PROC GLM) which is designed to analyze a general linear model with fixed effects, the MIXED procedure can better handle statistical models having random effects. For the analysis of binary or categorical variables with fixed effects, the GENMOD procedure that uses a generalized linear model should be used instead of PROC GLM. A more recent procedure, PROC GLIMMIX, can analyze statistical models with fixed and random effects for both categorical and continuous variables. AJAS does not take a position on which procedures are more desirable under certain circumstances and leaves the decision to authors as long as the procedure can properly handle the data type. However, when the observations are repeatedly measured or random effects are included in the statistical model, PROC MIXED or PROC GLIMMIX in SAS or similar procedures in other statistical packages are preferred.

Reporting all relevant information is important in scientific papers to increase the transparency and validity of the results and provide information for confidence and limitations of scientific knowledge gained from experiments. Not only the probability value (p-value) but also error measures (e.g., standard error of means [SEM]) should be reported in tables. Likewise, error measures should be present as error bars in figures. Error measures can be expressed in several ways: standard deviation (SD), SEM, and the standard error of the difference (SED). AJAS recommends the presentation of the pooled SEM because the objective of animal nutritional studies is usually to provide inferences about the population. If the sample sizes are different among treatments, the sample sizes are to be reported, as well as pooled SEM. However, the use of SD is also allowed when it is used for descriptive statistics.

When there are outliers or missing data, they need to be clearly reported in the Materials and Methods section or the Results section of the manuscript where it is more appropriate. In particular, the methods and their rationale for identifying outliers should be provided, and the results from the statistical analysis of the data with and without outliers should be compared and discussed in the manuscript.

SUMMARY OF RECOMMENDATIONS

The AJAS editorial board takes no position on which experimental designs and statistical methods are more desirable in certain circumstances and leaves that decision to the authors. Nevertheless, a summary of the recommendations of the AJAS editorial board is as follows:

• Provide details of experimental design and statistical methods in the Materials and Methods section.
• Define the experimental unit and report the number of replicates. Replicates are to be independent observations and experimental units must be independent of each other.
• Conduct power tests and provide their results to justify the experiment was appropriately designed.
• Use blocks or covariates whenever applicable to reduce unexplained experimental errors.
• Describe the procedure used to minimize possible carryover effects and to show carryover effects are not significant when using a crossover design.
• Ensure the implementation of randomization when sampling experimental units and allocating treatments to experimental units.
• Describe the statistical models used for the statistical analysis as equations, as well as in words.
• Use appropriate statistical methods depending on the data type of observations, research questions to be answered, and the statistical model.
• Test if the observations are normally distributed around their mean. If not or the sample size is not large enough, use non-parametric analyses instead; otherwise, use parametric methods.
• State the level of statistical significance in the statistical analysis section.
• Conduct post hoc analysis on the basis that there is a significant difference among the treatments and use appropriate methods according to the structure of the treatments.
• Perform pair-wise comparisons (e.g., Duncan’s multiple range test) only when there is no structure among a series of treatments.
• Report which software and procedures are used for the statistical analysis.
• Use appropriate statistical methods and procedures if observations are repeatedly measured or random effects are expected.
• Present both probability value (p-value) and pooled SEM as error measures. The standard deviation can only be used for descriptive statistics.
• Report outliers or missing data in the Materials and Methods section or the Results section where it is more appropriate.

CONFLICT OF INTEREST

We certify that there is no conflict of interest with any financial organization regarding the material discussed in the manuscript.

• Main Navigation
• Main Content

• Current Students
• Give to UNE

Statistical Modelling and Experimental Design (STAT410)

UNE unit code: STAT410

*You are viewing the 2024 version of this unit which may be subject to change in future.

• Trimester 1 - On Campus
• Trimester 1 - Online
• Armidale Campus

Unit information

Are you interested in developing and applying statistical models for the natural or social sciences? Do you want to learn more about the principles of designing a data collection? This unit will help you develop the core skills and knowledge needed for experimental designs and applied statistical models that are used in many scientific fields.

Studying this unit, you will learn to develop and analyse various types of linear regression models which are the foundation of many statistical analyses. You will also explore some common experimental designs such as factorial design and randomised block design.

Focusing on both the theoretical and technical aspects of key statistical concepts, topics include multiple linear regression with quantitative and qualitative explanatory variables, polynomial regression and generalised linear models.

For further information about UNE's teaching periods, please go to Principal Dates .

*Offering is subject to availability

Intensive schools

There are no intensive schools required for this unit.

Enrolment rules

• STAT210 Statistical Modelling and Experimental Design

Three lectures and a one-hour laboratory session per week for on campus students; online students should have access to the statistical package R & RStudio. Students can use Open Source to download R & RStudio. Instructions are given on the unit's Moodle site.

Please refer to the student handbook for current details on this unit.

Unit coordinator(s)

Learning outcomes.

Upon completion of this unit, students will be able to:

• analyse data, and interpret and communicate results and conclusions, from a wide range of experimental designs;
• fit and interpret more complex statistical models, including the linear model;
• build on and broaden their theoretical and technical knowledge of statistical terminology, concepts and methodology to an advanced level, which will enable them to read and critically appraise scientific literature with some confidence; and
• demonstrate a high level of understanding and highly developed communication skills by independently and critically analysing a special topic in advanced regression.

Assessment information

Assessments are subject to change up to 8 weeks prior to the start of the teaching period in which you are undertaking the unit.

Learning resources

Textbooks are subject to change up to 8 weeks prior to the start of the teaching period in which you are undertaking the unit.

Note: Students are expected to purchase prescribed material. Please note that textbook requirements may vary from one teaching period to the next.

A Second Course in Statistics

ISBN: 9781292042909

Mendenhall, W. and Sincich, T.T., Pearson 7th ed. 2013

Text refers to: All offerings

A five-star experience

Five Stars, 18 Years in a Row

UNE is the only public uni in Australia awarded 18 straight years of five stars for Overall Experience

No.1 in NSW for Student Experience

QILT (government-endorsed) ranks UNE as the top public NSW uni for Student Experience

Five Stars for Teaching Quality

UNE rates among the top 20 per cent of universities in Australia for Teaching Quality

Studying online

At UNE we know it takes more than just being online to be a great online university. It takes time and experience. We pioneered distance education for working adults back in the 1950s, so we’ve been doing this longer than any other Australian university.

We understand the challenges faced by busy adults studying at home. We know that a vital part of online study is your engagement with the learning community. Communication with your classmates, teaching staff and university support staff will enhance your study experience and ensure that your skills extend beyond the subject matter. UNE’s teaching staff are experts in their field which is why UNE consistently receives five stars from students for teaching quality, support and overall experience.*

* The Good Universities Guide

Stay connected

Register your interest and we'll keep you updated

Why study with us?

I'm grateful to UNE for the opportunities studying has given me, but above all, for making me realise I could actually do it, and giving me a sense of accomplishment.

What happens next?

Got any questions about a course you would like to study? Don’t hesitate to contact us , our Future Student team is standing by to help.

2024 applications are now open. The application process only takes 20 minutes to complete. Don’t delay, apply now !

Your start date is based on the study period you choose to apply for.

• Study options
• How to apply
• Scholarships
• Study online
• Study on campus
• Regional Study Centres
• International
• Fees and costs
• English Language Requirements
• UNE Armidale
• UNE Accommodation
• UNE Tamworth
• Events Calendar
• Safe Communities
• Research Performance
• Research Integrity & Ethics
• Centres, Institutes, CRCs
• Graduate Research School
• Research Themes and Clusters
• Office of the Deputy Vice-Chancellor (Research)
• High Schools
• Businesses and Community
• Aboriginal and Torres Strait Islander Community
• Alumni Community
• Honorary and Visiting Appointments
• Teachers & Education Students
• Our Values and Culture
• University Structure
• Faculty of Humanities, Arts, Social Sciences and Education
• Faculty of Medicine and Health
• Faculty of Science, Agriculture, Business and Law
• Principal Dates 2024
• Rankings and Ratings
• Annual reports
• Right to Information
• Accessibility
• CRICOS Provider Number 00003G
• TEQSA Provider Code: PRV12054 Australian University
• ABN: 75 792 454 315
• UNE is a member of the Regional Universities Network

© University of New England, 2024

The University of New England respects and acknowledges that its people, courses and facilities are built on land, and surrounded by a sense of belonging, both ancient and contemporary, of the world's oldest living culture. In doing so, UNE values and respects Indigenous knowledge systems as a vital part of the knowledge capital of Australia. We recognise the strength, resilience and capacity of the Aboriginal community and pay our respects to the Elders past, present and future.

ForLion: a new algorithm for D-optimal designs under general parametric statistical models with mixed factors

• Original Paper
• Published: 18 July 2024
• Volume 34 , article number  157 , ( 2024 )

Cite this article

• Yifei Huang 1 ,
• Keren Li 2 ,
• Abhyuday Mandal 3 &
• Jie Yang 1  

47 Accesses

Explore all metrics

In this paper, we address the problem of designing an experimental plan with both discrete and continuous factors under fairly general parametric statistical models. We propose a new algorithm, named ForLion, to search for locally optimal approximate designs under the D-criterion. The algorithm performs an exhaustive search in a design space with mixed factors while keeping high efficiency and reducing the number of distinct experimental settings. Its optimality is guaranteed by the general equivalence theorem. We present the relevant theoretical results for multinomial logit models (MLM) and generalized linear models (GLM), and demonstrate the superiority of our algorithm over state-of-the-art design algorithms using real-life experiments under MLM and GLM. Our simulation studies show that the ForLion algorithm could reduce the number of experimental settings by 25% or improve the relative efficiency of the designs by 17.5% on average. Our algorithm can help the experimenters reduce the time cost, the usage of experimental devices, and thus the total cost of their experiments while preserving high efficiencies of the designs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or Ebook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Optimal design of multifactor experiments via grid exploration

A-ComVar: A Flexible Extension of Common Variance Designs

Approximate and exact optimal designs for $$2^k$$ factorial experiments for generalized linear models via second order cone programming

Data availability.

No datasets were generated or analysed during the current study.

Atkinson, A.C., Donev, A.N., Tobias, R.D.: Optimum Experimental Designs, with SAS. Oxford University Press, Oxford (2007)

Book   Google Scholar  

Ai, M., Ye, Z., Yu, J.: Locally D-optimal designs for hierarchical response experiments. Stat. Sin. 33 , 381–399 (2023)

MathSciNet   Google Scholar  

Byrd, R.H., Lu, P., Nocedal, J., Zhu, C.: A limited memory algorithm for bound constrained optimization. SIAM J. Sci. Comput. 16 , 1190–1208 (1995)

Article   MathSciNet   Google Scholar  

Bu, X., Majumdar, D., Yang, J.: D-optimal designs for multinomial logistic models. Ann. Stat. 48 (2), 983–1000 (2020)

Broyden, C.G.: A class of methods for solving nonlinear simultaneous equations. Math. Comput. 19 (92), 577–593 (1965)

Duarte, B.P.M., Atkinson, A.C., Granjo, J.F.O., Oliveira, N.M.C.: Optimal design of experiments for implicit models. J. Am. Stat. Assoc. 117 (539), 1424–1437 (2022)

Duarte, B.P.M., Granjo, J.F.O., Wong, W.K.: Optimal exact designs of experiments via mixed integer nonlinear programming. Stat. Comput. 30 (1), 93–112 (2020)

Dennis, J.E., Jr., Moré, J.J.: Quasi-newton methods, motivation and theory. SIAM Rev. 19 (1), 46–89 (1977)

Duarte, B.P.M., Sagnol, G., Wong, W.K.: An algorithm based on semidefinite programming for finding minimax optimal designs. Comput. Stat. Data Anal. 119 , 99–117 (2018)

Duarte, B.P.M.: Exact optimal designs of experiments for factorial models via mixed-integer semidefinite programming. Mathematics 11 (4), 854 (2023)

Article   Google Scholar  

Duarte, B.P.M., Wong, W.K.: A semi-infinite programming based algorithm for finding minimax optimal designs for nonlinear models. Stat. Comput. 24 (6), 1063–1080 (2014)

Duarte, B.P.M., Wong, W.K.: Finding Bayesian optimal designs for nonlinear models: a semidefinite programming-based approach. Int. Stat. Rev. 83 (2), 239–262 (2015)

Duarte, B.P.M., Wong, W.K., Dette, H.: Adaptive grid semidefinite programming for finding optimal designs. Stat. Comput. 28 (2), 441–460 (2018)

Fedorov, V.V.: Theory of Optimal Experiments. Academic Press, Cambridge (1972)

Google Scholar  

Fedorov, V.V., Hackl, P.: Model-Oriented Design of Experiments. Springer, New York (1997)

Fedorov, V.V., Leonov, S.L.: Optimal Design for Nonlinear Response Models. Chapman & Hall/CRC, Baton Rouge (2014)

Fletcher, R., Reeves, C.M.: Function minimization by conjugate gradients. Comput. J. 7 (2), 149–154 (1964)

Givens, G.H., Hoeting, J.A.: Computational Statistics, 2nd edn. Wiley, New Jersey (2013)

Glonek, G.F.V., McCullagh, P.: Multivariate logistic models. J. Roy. Stat. Soc. B 57 , 533–546 (1995)

Harman, R., Filová, L.: Computing efficient exact designs of experiments using integer quadratic programming. Comput. Stat. Data Anal. 71 , 1159–1167 (2014)

Harman, R., Filová, L., Richtárik, P.: A randomized exchange algorithm for computing optimal approximate designs of experiments. J. Am. Stat. Assoc. 115 (529), 348–361 (2020)

Harman, R., Jurík, T.: Computing c-optimal experimental designs using the simplex method of linear programming. Comput. Stat. Data Anal. 53 (2), 247–254 (2008)

Harman, R., Rosa, S.: On greedy heuristics for computing D-efficient saturated subsets. Oper. Res. Lett. 48 (2), 122–129 (2020)

Hestenes, M.R., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand. 49 (6), 409–436 (1952)

Huang, Y., Tong, L., Yang, J.: Constrained d-optimal design for paid research study. Stat. Sin. (2023). https://www3.stat.sinica.edu.tw/ss_newpaper/SS-2022-0414_na.pdf

Itepan, N.M.: Aumento do periodo de aceitabilidade de pupas de musca domestica l., 1758 (diptera: muscidae), irradiadas com raios gama, como hospedeira de parasitoides (hymenoptera: pteromalidae). Master’s thesis, Universidade de São Paulo (1995)

Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of ICNN’95-International Conference on Neural Networks, vol. 4, pp. 1942–1948 (1995). IEEE

Kirkpatrick, S., Gelatt, C.D., Jr., Vecchi, M.P.: Optimization by simulated annealing. Science 220 (4598), 671–680 (1983)

Kiefer, J.: General equivalence theory for optimum designs (approximate theory). Ann. Stat. 2 , 849–879 (1974)

Lukemire, J., Mandal, A., Wong, W.K.: d-QPSO: a quantum-behaved particle swarm technique for finding D-optimal designs with discrete and continuous factors and a binary response. Technometrics 61 (1), 77–87 (2019)

Lukemire, J., Mandal, A., Wong, W.K.: Optimal experimental designs for ordinal models with mixed factors for industrial and healthcare applications. J. Qual. Technol. 54 (2), 184–196 (2022)

McCullagh, P., Nelder, J.: Generalized Linear Models, 2nd edn. Chapman and Hall/CRC, Baton Rouge (1989)

Nelder, J.A., Mead, R.: A simplex method for function minimization. Comput. J. 7 (4), 308–313 (1965)

Nocedal, J., Wright, S.J.: Numerical Optimization, 2nd edn. Springer, New York (2006)

Phadke, M.S.: Quality Engineering Using Robust Design. Prentice-Hall, Englewood Cliffs (1989)

Poli, R., Kennedy, J., Blackwell, T.: Particle swarm optimization: an overview. Swarm Intell. 1 , 33–57 (2007)

Pukelsheim, F.: Optimal Design of Experiments. Wiley, New Jersey (1993)

Pukelsheim, F.: Optimal Design of Experiments. SIAM, Philadelphia (2006)

Sagnol, G.: Computing optimal designs of multiresponse experiments reduces to second-order cone programming. J. Stat. Plann. Inference 141 (5), 1684–1708 (2011)

Seber, G.A.F.: A Matrix Handbook for Statisticians. Wiley, New Jersey (2008)

Sagnol, G., Harman, R.: Computing exact -optimal designs by mixed integer second-order cone programming. Ann. Stat. 43 (5), 2198–2224 (2015)

Silvey, S.D., Titterington, D.M., Torsney, B.: An algorithm for optimal designs on a finite design space. Commun. Stat. Theory Methods 14 , 1379–1389 (1978)

Stufken, J., Yang, M.: Optimal designs for generalized linear models. In: Hinkelmann, K. (ed.) Design and Analysis of Experiments, Volume 3: Special Designs and Applications, pp. 137–164. Wiley, New Jersey (2012). Chap. 4

Titterington, D.M.: Algorithms for computing d-optimal design on finite design spaces. In: Proc. of the 1976 Conf. on Information Science and Systems, vol. 3, pp. 213–216. John Hopkins University, Baltimore (1976)

Titterington, D.M.: Estimation of correlation coefficients by ellipsoidal trimming. J. R. Stat. Soc. Ser. C (Appl. Stat.) 27 , 227–234 (1978)

Venables, W.N., Ripley, B.D.: Modern Applied Statistics with S, 4th edn. Springer, New York (2002)

Vo-Thanh, N., Jans, R., Schoen, E.D., Goos, P.: Symmetry breaking in mixed integer linear programming formulations for blocking two-level orthogonal experimental designs. Comput. Oper. Res. 97 , 96–110 (2018)

Whitman, C., Gilbert, T.M., Rahn, A.M., Antonell, J.A.: Determining factors affecting ESD failure voltage using DOE. Microelectron. Reliab. 46 (8), 1228–1237 (2006)

Wu, F.-C.: Simultaneous optimization of robust design with quantitative and ordinal data. Int. J. Ind. Eng. Theory Appl. Pract. 5 , 231–238 (2008)

Wong, W.K., Zhou, J.: Using CVX to construct optimal designs for biomedical studies with multiple objectives. J. Comput. Graph. Stat. 32 (2), 744–753 (2023)

Yang, M., Biedermann, S., Tang, E.: On optimal designs for nonlinear models: a general and efficient algorithm. J. Am. Stat. Assoc. 108 , 1411–1420 (2013)

Yang, J., Mandal, A.: D-optimal factorial designs under generalized linear models. Commun. Stat. Simul. Comput. 44 , 2264–2277 (2015)

Yang, J., Mandal, A., Majumdar, D.: Optimal designs for \(2^k\) factorial experiments with binary response. Stat. Sin. 26 , 385–411 (2016)

Yang, J., Tong, L., Mandal, A.: D-optimal designs with ordered categorical data. Stat. Sin. 27 , 1879–1902 (2017)

Yu, Y.: D-optimal designs via a cocktail algorithm. Stat. Comput. 21 , 475–481 (2011)

Ye, J.J., Zhou, J.: Minimizing the condition number to construct design points for polynomial regression models. SIAM J. Optim. 23 (1), 666–686 (2013)

Zocchi, S.S., Atkinson, A.C.: Optimum experimental designs for multinomial logistic models. Biometrics 55 , 437–444 (1999)

Download references

Acknowledgements

The authors gratefully acknowledge the authors of Ai et al. ( 2023 ), Lukemire et al. ( 2019 ) and Lukemire et al. ( 2022 ) for kindly sharing their source codes, which we used to implement and compare their methods with ours. The authors gratefully acknowledge the support from the U.S. NSF grants DMS-1924859 and DMS-2311186.

Author information

Authors and affiliations.

Department of Mathematics, Statistics, and Computer Science, University of Illinois at Chicago, Chicago, IL, 60607, USA

Yifei Huang & Jie Yang

Department of Mathematics, University of Alabama at Birmingham, Birmingham, AL, 35294, USA

Department of Statistics, University of Georgia, Athens, GA, 30602, USA

Abhyuday Mandal

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization and methodology, all authors; software, Y.H., K.L., and J.Y.; validation, Y.H., A.M., and J.Y.; formal analysis, Y.H. and K.L.; investigation, Y.H. and J.Y.; resources, all authors; data curation, Y.H.; writing-original draft preparation, all authors; writing-review and editing, all authors.; supervision, J.Y. and A.M.; project administration, J.Y. and A.M.; funding acquisition, J.Y. and A.M.. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Jie Yang .

Ethics declarations

Conflict of interest.

The authors declare no conflict of interest.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (pdf 1561 KB)

Computing \(u_{st}^{{\textbf{x}}}\) in fisher information \({\textbf{f}}_{{\textbf{x}}}\).

In this section, we provide more technical details for Sect.  3.1 and Theorem  2 .

For MLM ( 1 ), Corollary 3.1 in Bu et al. ( 2020 ) provided an alternative form \({{\textbf{F}}}_{{{\textbf{x}}}_i} = {\textbf{X}}_i^T {{\textbf{U}}}_i {{\textbf{X}}}_i\) , which we use for computing the Fisher information \({{\textbf{F}}}_{{\textbf{x}}}\) at an arbitrary \({{\textbf{x}}} \in {{\mathcal {X}}}\) . More specifically, first of all, the corresponding model matrix at \({{\textbf{x}}}\) is

where \({{\textbf{h}}}^T_j(\cdot ) = (h_{j1}(\cdot ), \ldots , h_{jp_j}(\cdot ))\) and \({{\textbf{h}}}^T_c(\cdot )\) \( = \) \((h_1(\cdot ), \ldots , h_{p_c}(\cdot ))\) are known predictor functions. We let \(\varvec{\beta }_j\) and \(\varvec{\zeta }\) denote the model parameters associated with \({{\textbf{h}}}^T_j({{\textbf{x}}})\) and \({{\textbf{h}}}^T_c({{\textbf{x}}})\) , respectively, then the model parameter vector \(\varvec{\theta }=(\varvec{\beta }_{1},\varvec{\beta }_{2},\cdots ,\varvec{\beta }_{J-1},\varvec{\zeta })^T \in {\mathbb {R}}^p\) , and the linear predictor \(\varvec{\eta }_{{\textbf{x}}} = {{\textbf{X}}}_{{\textbf{x}}} \varvec{\theta }= (\eta _1^{{\textbf{x}}}, \ldots , \eta _{J-1}^{{\textbf{x}}}, 0)^T \in {\mathbb {R}}^J\) , where \(\eta _j^{{\textbf{x}}} = {{\textbf{h}}}_j^T({{\textbf{x}}}) \varvec{\beta }_j + {{\textbf{h}}}_c^T({{\textbf{x}}}) \varvec{\zeta }\) , \(j=1, \ldots , J-1\) .

According to Lemmas S.10, S.12 and S.13 in the Supplementary Material of Bu et al. ( 2020 ), the categorical probabilities \(\varvec{\pi }_{{\textbf{x}}} = (\pi _1^{{\textbf{x}}}, \ldots , \pi _J^{{\textbf{x}}})^T \in {\mathbb {R}}^J\) at \({{\textbf{x}}}\) for baseline-category, adjacent-categories and continuation-ratio logit models can be expressed as follows:

for \(j=1, \ldots , J-1\) , where \(D_j = \exp \{\eta _{J-1}^{{\textbf{x}}} + \cdots + \eta _1^{{\textbf{x}}}\} + \exp \{\eta _{J-1}^{{\textbf{x}}} + \cdots + \eta _2^{{\textbf{x}}}\} + \cdots +\exp \{\eta _{J-1}^{{\textbf{x}}}\} + 1\) , and

where \(D_J = \exp \{\eta _{J-1}^{{\textbf{x}}} + \cdots + \eta _1^{{\textbf{x}}}\} + \exp \{\eta _{J-1}^{{\textbf{x}}} + \cdots + \eta _2^{{\textbf{x}}}\} + \cdots +\exp \{\eta _{J-1}^{{\textbf{x}}}\} + 1\) . Note that we provide the expression of \(\pi _J^{{\textbf{x}}}\) for completeness while \(\pi _J^{{\textbf{x}}} = 1 - \pi _1^{{\textbf{x}}} - \cdots - \pi _{J-1}^{{\textbf{x}}}\) is an easier way for numerical calculations.

As for cumulative logit models, the candidate \({{\textbf{x}}}\) must satisfy \(-\infty< \eta _1^{{\textbf{x}}}< \eta _2^{{\textbf{x}}}< \cdots< \eta _{J-1}^{{\textbf{x}}} < \infty \) . Otherwise, \(0< \pi _j^{\textbf{x}} < 1\) might be violated for some \(j=1, \ldots , J\) . In other words, the feasible design region should be

which depends on the regression parameter \(\varvec{\theta }\) (see Section S.14 in the Supplementary Material of Bu et al. ( 2020 ) for such an example). For cumulative logit models, if \({{\textbf{x}}} \in {{\mathcal {X}}}_{\varvec{\theta }}\) , then

according to Lemma S.11 of Bu et al. ( 2020 ).

Once \(\varvec{\pi }_{{\textbf{x}}} \in {\mathbb {R}}^J\) is obtained, we can calculate \(u_{st}^{{\textbf{x}}} = u_{st}({\varvec{\pi }}_{\textbf{x}})\) based on Theorem A.2 in Bu et al. ( 2020 ) as follows:

\(u_{st}^{{\textbf{x}}} = u_{ts}^{{\textbf{x}}}\) , \(s,t=1, \ldots , J\) ;

\(u_{sJ}^{{\textbf{x}}} = 0\) for \(s=1, \ldots , J-1\) and \(u_{JJ}^{{\textbf{x}}} = 1\) ;

For \(s=1, \ldots , J-1\) , \(u_{ss}^{{\textbf{x}}}\) is

For \(1\le s < t \le J-1\) , \(u_{st}^{{\textbf{x}}}\) is

where \(\gamma _j^{{\textbf{x}}} = \pi _1^{{\textbf{x}}} + \cdots + \pi _j^{{\textbf{x}}}\) , \(j=1, \ldots , J-1\) ; \(\gamma _0^{{\textbf{x}}}\equiv 0\) and \(\gamma _J^{{\textbf{x}}}\equiv 1\) .

Example that \({{\textbf{F}}}_{{{\textbf{x}}}} = {{\textbf{F}}}_{{{\textbf{x}}}'}\) with \({{\textbf{x}}} \ne {{\textbf{x}}}'\)

Consider a special MLM ( 1 ) with proportional odds (po) (see Section S.7 in the Supplementary Material of Bu et al. ( 2020 ) for more technical details). Suppose \(d=2\) and a feasible design point \({{\textbf{x}}} = (x_1, x_2)^T \in [a, b]\times [-c, c] = \mathcal{X}\) , \(c > 0\) , \(J\ge 2\) , \({{\textbf{h}}}_c({{\textbf{x}}}) = (x_1, x_2^2)^T\) . Then the model matrix at \({{\textbf{x}}} = (x_1, x_2)^T\) is

Then \(p=J+1\) . Let \(\varvec{\theta }= (\beta _1, \ldots , \beta _{J-1}, \zeta _1, \zeta _2)^T \in {\mathbb {R}}^{J+1}\) be the model parameters (since \(\varvec{\theta }\) is fixed, we may assume that \(\mathcal{X} = \mathcal{X}_{\varvec{\theta }}\) if the model is a cumulative logit model). Let \({{\textbf{x}}}' = (x_1, -x_2)^T\) . Then \({{\textbf{X}}}_{{\textbf{x}}} = {{\textbf{X}}}_{{{\textbf{x}}}'}\) and thus \(\varvec{\eta }_{{\textbf{x}}} = \varvec{\eta }_{{{\textbf{x}}}'}\) . According to ( A2 ) (or ( A4 )), we obtain \(\varvec{\pi }_{{\textbf{x}}} = \varvec{\pi }_{{{\textbf{x}}}'}\) and then \({{\textbf{U}}}_{{\textbf{x}}} = {{\textbf{U}}}_{{{\textbf{x}}}'}\)  . The Fisher information matrix at \({{\textbf{x}}}\) is \({\textbf{F}}_{{\textbf{x}}} = {{\textbf{X}}}_{{\textbf{x}}}^T {{\textbf{U}}}_{{\textbf{x}}} {{\textbf{X}}}_{{\textbf{x}}} = {{\textbf{X}}}_{{{\textbf{x}}}'}^T {\textbf{U}}_{{{\textbf{x}}}'} {{\textbf{X}}}_{{{\textbf{x}}}'} = {{\textbf{F}}}_{{\textbf{x}}'}\) . Note that \({{\textbf{x}}} \ne {{\textbf{x}}}'\) if \(x_2 \ne 0\) .

First-order derivative of sensitivity function

As mentioned in Sect.  3.1 , to apply Algorithm 1 for MLM, we need to calculate the first-order derivative of the sensitivity function \(d({\textbf{x}}, \varvec{\xi })\) .

Recall that the first k ( \(1\le k\le d\) ) factors are continuous. Given \({{\textbf{x}}} = (x_1, \ldots , x_d)^T \in \mathcal{X}\) , for each \(i=1, \ldots , k\) , according to Formulae 17.1(a), 17.2(a) and 17.7 in Seber ( 2008 ),

\(\frac{\partial {{\textbf{U}}}_{{\textbf{x}}}}{\partial x_i} = \left( \frac{\partial u^{{\textbf{x}}}_{st}}{\partial x_i}\right) _{s,t=1, \ldots , J}\) with

\({{\textbf{C}}}\) and \({{\textbf{L}}}\) defined as in ( 1 ), and \({{\textbf{D}}}_{{\textbf{x}}} = \textrm{diag}({{\textbf{L}}} \varvec{\pi }_{{\textbf{x}}})\) . Explicit formula of \(({{\textbf{C}}}^T {{\textbf{D}}}_{{\textbf{x}}}^{-1} {{\textbf{L}}})^{-1}\) can be found in Section S.3 in the Supplementary Material of Bu et al. ( 2020 ) with \({{\textbf{x}}}_i\) replaced by \( {{\textbf{x}}}\) . As for \(\frac{\partial u^{{\textbf{x}}}_{st}}{\partial \varvec{\pi }_{{\textbf{x}}}^T}\) , we have the following explicit formulae

\(\frac{\partial u_{st}^{{\textbf{x}}}}{\partial \varvec{\pi }_{{\textbf{x}}}} = \frac{\partial u_{ts}^{{\textbf{x}}}}{\partial \varvec{\pi }_{{\textbf{x}}}}\) , \(s,t=1, \ldots , J\) ;

\(\frac{\partial u_{sJ}^{{\textbf{x}}}}{\partial \varvec{\pi }_{{\textbf{x}}}} = {{\textbf{0}}} \in {\mathbb {R}}^J\) for \(s=1, \ldots , J\) ;

For \(s=1, \ldots , J-1\) , \(\frac{\partial u_{ss}^{{\textbf{x}}}}{\partial \varvec{\pi }_{{\textbf{x}}}}\) is

where \({{\textbf{e}}}_s\) is the \(J\times 1\) vector with the s th coordinate 1 and all others 0, \({{\textbf{1}}}_s\) is the \(s\times 1\) vector of all 1, and \({{\textbf{0}}}_s\) is the \(s\times 1\) vector of all 0.

For \(1\le s < t \le J-1\) , \(\frac{\partial u_{st}^{{\textbf{x}}}}{\partial \varvec{\pi }_{{\textbf{x}}}}\) is

Thus the explicit formulae for \(\frac{\partial d({{\textbf{x}}}, \varvec{\xi })}{\partial x_i}\) , \(i=1, \ldots , k\) can be obtained via ( C5 ). Only \(\frac{\partial {\textbf{X}}_{{\textbf{x}}}}{\partial x_i}\) is related to i , which may speed up the computations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Huang, Y., Li, K., Mandal, A. et al. ForLion: a new algorithm for D-optimal designs under general parametric statistical models with mixed factors. Stat Comput 34 , 157 (2024). https://doi.org/10.1007/s11222-024-10465-x

Download citation

Received : 02 April 2024

Accepted : 26 June 2024

Published : 18 July 2024

DOI : https://doi.org/10.1007/s11222-024-10465-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• ForLion algorithm
• Generalized linear model
• Lift-one algorithm
• Mixed factors
• Multinomial logistic model
• D-optimal design
• Find a journal
• Publish with us
• Track your research

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Experimental and statistical determination of convective heat and mass transfer coefficients for eucalyptus nitens sawn wood drying.

1. Introduction

2. materials and methods, 2.1. origin and sample preparation, 2.2. experimental design, 2.3. drying process, 2.4. measurement of mass flow during drying, 2.5. experimental determination of convective heat and mass transfer coefficients.

• h c = Convective heat transfer coefficient (W m −2 K −1 );
• k m = Convective mass heat transfer coefficient (m s −1 );
• F m a s s = Convective mass flow (kg m −2 s −1 );
• F h e a t = Convective heat flow (kg m −2 );
• λ = Latent heat of vaporization (J kg −1 ) at T wb ;
• T d b = Dry bulb temperature (°C);
• T w b = Wet bulb temperature (°C);
• M e v p A × t = Evaporated water flow (kg s −1 m −2 );
• A H = Absolute humidity at a given T db y p v (kg m −3 );
• A H a i r = AH of the drying (kg m −3 );
• A H 0 = AH in control piece at T wb , equal to saturation absolute humidity (kg m −3 );
• p v a = Partial pressure of water vapor at T db (Pa);
• p v = Partial pressure of vapor at given drying conditions (Pa).

2.6. Calculation of Convective Coefficients by Empirical Correlation

• h c = Convective heat coefficient (W m −2 K −1 );
• k m = Convective mass coefficient (m s −1 );
• k q a = Thermal conductivity of the air–vapor mixture (J s −1 m −1 K −1 );
• D h = Hydraulic diameter (m);
• ρ g = Density of the air–vapor mixture (kg m −3 );
• c p a i r e = Specific heat of the air–vapor mixture (J kg −1 K −1 );
• J a = Chilton-Colburn analogy for heat transfer;
• N u = Dimensionless Nusselt number;
• R e = Dimensionless Reynolds number;
• P r = Dimensionless Prandtl number;
• S h = Dimensionless Sherwood number;
• S c = Dimensionless Schmidt number;
• D v = Diffusivity of water vapor in air (m −2 s −1 );
• V a = Air velocity (m s −1 );
• L = Heat and mass transfer surface area (m);
• μ g = Viscosity of the air–vapor mixture (Pa s);
• v g = Velocity of the air–vapor mixture (m s −1 );
• A = Cross-sectional area (m 2 ) of the control piece;
• P = Cross-sectional perimeter (m) of the control piece.

2.7. Statistical Analysis

• γ k n i j = Multivariable regression statistical model where; i -th convective coefficient obtained at the j -th dry bulb temperature, k -th relative humidity, and n -th air velocity, Ɐi : 1, 2; h c = 1, k m = 2;
• T d b j = j -th dry bulb temperature (°C), Ɐj : 40, …, 55;
• R H k = k -th relative humidity (%); Ɐk : 55, …, 75;
• V n = n -th air velocity (m s −1 ), Ɐn : 2, …, 3;
• ɛ = Residual error;
• β 0 , β 1 , β 2 , β 3 = Model parameters.
• E r e l = Relative error between the experimental value and the empirical correlation (%);
• Z e x p = Convective heat coefficient (W m −2 K −1 ) or experimental mass (m s −1 );
• Z t = Convective heat coefficient (W m −2 K −1 ) or mass (m s −1 ) obtained by the empirical correlation.

3. Results and Discussion

3.1. isothermal drying process, 3.2. convective heat transfer coefficients, 3.3. convective mass transfer coefficient, 3.4. modeling of convective heat and mass transfer coefficients, 4. conclusions, author contributions, data availability statement, acknowledgments, conflicts of interest.

• Zhao, J.; Fu, Z.; Jia, X.; Cai, Y. Modeling Conventional Drying of Wood: Inclusion of a Moving Evaporation Interface. Dry. Technol. 2016 , 34 , 530–538. [ Google Scholar ] [ CrossRef ]
• Salin, J. External Heat and Mass Transfer—Some Remarks. In Proceedings of the 8th International IUFRO Wood Drying Conference, Brasov, Romania, 24–29 August 2003. [ Google Scholar ]
• Sargent, R.; Riley, S.; Schöttle, L. Measurement of Dynamic Sorption Behaviour of Small Specimens of Pinus radiata —Influence of Wood Type and Moisture Content on Difusion Rate. Maderas Cienc. Tecnol. 2010 , 12 , 93–103. [ Google Scholar ] [ CrossRef ]
• Salin, J.G. Inclusion of the Sorption Hysteresis Phenomenon in Future Drying Models. Some Basic Considerations. Maderas Cienc. Tecnol. 2011 , 13 , 173–182. [ Google Scholar ] [ CrossRef ]
• Montero, C.; Rozas, C. Exploratory Study for the Characterization of the Wood Drying Rate of Eucalyptus nitens Timber, Applying Multiple Regression Models. Sci. For. Sci. 2019 , 47 , 105–113. [ Google Scholar ] [ CrossRef ]
• Sanchez, R. Mathematical Correlation to Get Mass Transfer Coefficient in Solids Drying. Maderas Cienc. Tecnol. 2007 , 9 , 245–258. [ Google Scholar ]
• Haque, M.N. Analysis of Heat and Mass Transfer during High Temperature Drying of Pinus radiata . Dry. Technol. 2007 , 25 , 379–389. [ Google Scholar ] [ CrossRef ]
• Rozas, C. Modelo de Transferência de Calor e Massa Na Secagem de Madeira Serrada de Pinus. Ph.D. Thesis, Universidade Federal do Paraná, Curitiba, Brazil, 2007. [ Google Scholar ]
• He, Z.; Zhao, Z.; Zhang, Y.; Lv, H.; Yi, S. Convective Heat and Mass Transfer during Vacuum Drying Process. Wood Res. 2015 , 60 , 929–938. [ Google Scholar ]
• Skuratov, N.; Sapozhnikov, I.; Alexeeva, I. Measurements during Wood Drying Based on X-ray and Slicing Tecnhiques and Computation of Diffusion Coefficients. Pro Ligno 2015 , 11 , 383–388. [ Google Scholar ]
• Ratanawilai, T.; Nuntadusit, C.; Promtong, N. Drying Characteristics of Rubberwood by Impinging Hot-Air and Microwave Heating. Wood Res. 2015 , 60 , 59–70. [ Google Scholar ]
• Montero, C.; Rozas, C.; Picarte, C. Simulación Del Secado En Madera de Eucalyptus nitens Usando Modelos de Difusión Bajo Condiciones Unidimensionales e Isotérmicas. Sci. For. 2020 , 48 , e2957. [ Google Scholar ] [ CrossRef ]
• Siau, J. Wood: Influence of Moisture on Physical Properties ; Department of Wood Science and Forest Products, Virginia Polytechnic Institute and State University: Blacksburg, VA, USA, 1995; 227p. [ Google Scholar ]
• Hrčka, R.; Babiak, M. Wood Thermal Properties. In Wood in Civil Engineering ; IntechOpen: London, UK, 2017. [ Google Scholar ] [ CrossRef ]
• Zadin, V.; Kasemägi, H.; Valdna, V.; Vigonski, S.; Veske, M.; Aabloo, A. Application of Multiphysics and Multiscale Simulations to Optimize Industrial Wood Drying Kilns. Appl. Math. Comput. 2015 , 267 , 465–475. [ Google Scholar ] [ CrossRef ]
• Erriguible, A.; Bernada, P.; Couture, F.; Roques, M.A. Modeling of Heat and Mass Transfer at the Boundary between a Porous Medium and Its Surroundings. Dry. Technol. 2005 , 23 , 455–472. [ Google Scholar ] [ CrossRef ]
• Tremblay, C.; Cloutier, A.; Fortin, Y. Experimental Determination of the Convective Heat and Mass Transfer Coefficients for Wood Drying. Wood Sci. Technol. 2000 , 34 , 253–276. [ Google Scholar ] [ CrossRef ]
• Harris, D. An Investigation of Ambient Drying of Eucalyptus Grandis Wood. Master’s Thesis, University of Hawaii, Honolulu, HI, USA, 2016. [ Google Scholar ]
• Rozas, C.; Tomaselli, I.; Zanoelo, E.F. Internal Mass Transfer Coefficient during Drying of Softwood ( Pinus elliottii Engelm.) Boards. Wood Sci. Technol. 2009 , 43 , 361–373. [ Google Scholar ] [ CrossRef ]
• Sarricolea, P.; Herrera-Ossandon, M.; Meseguer-Ruiz, Ó. Climatic Regionalisation of Continental Chile. J. Maps 2017 , 13 , 66–73. [ Google Scholar ] [ CrossRef ]
• NCh 176/1:2019 ; Madera—Parte 1: Determinación de Contenido de Humedad. Instituto Nacional de Normalización: Santiago, Chile, 2019.
• Vermaas, H.F. Drying Eucalypts for Quality: Material Characteristics, Pre-Drying Treatments, Drying Methods, Schedules and Optimisation of Drying Quality. S. Afr. For. J. 1995 , 174 , 41–49. [ Google Scholar ] [ CrossRef ]
• Jankowsky, I.; Dos Santos, G. Drying Behavior and Permeability of Eucalyptus Grandis Lumber. Maderas Cienc. Tecnol. 2005 , 7 , 17–25. [ Google Scholar ] [ CrossRef ]
• Welty, J.; Wicks, C.; Wilson, R.; Rorrer, G. Fundamentals of Momentum, Heat, and Mass Transfer ; John Wiley & Sons, Inc.: Hoboken, NJ, USA, 2008; 729p. [ Google Scholar ]
• Nabhani, M.; Remblay, C.; Fortin, Y. Experimental Determination of Convective Heat and Mass Transfer Coefficients During Wood Drying. In Proceedings of the 8th International IUFRO Wood Drying Conference, Brasov, Romania, 24–29 August 2003; pp. 225–230. [ Google Scholar ]
• Pang, S. External Heat and Mass Transfer Coefficients for Kiln Drying of Timber. Dry. Technol. 1996 , 14 , 859–871. [ Google Scholar ] [ CrossRef ]
• Hukka, A. The Effective Diffusion Coefficient and Mass Transfer Coefficient of Nordic Softwoods as Calculated from Direct Drying Experiments. Holzforschung 1999 , 53 , 534–540. [ Google Scholar ] [ CrossRef ]
• Simpson, W.; Liu, J. Dependence of the Water Vapor Diffusion Coefficient of Aspen ( Populus Spec .) on Moisture Content. Wood Sci. Technol. 1991 , 26 , 9–21. [ Google Scholar ] [ CrossRef ]
• Simpson, W.; Liu, J. An Optimization Technique to Determine Red Oak Surface and Internal Moisture Transfer Coefficients during Drying. Wood Fiber Sci. 1997 , 29 , 312–318. [ Google Scholar ]
• Daza-Gómez, M.; Gómez, C.; Gómez, J.; Ratkovich, N. 3D Computational Fluid Dynamics Analysis of a Convective Drying Chamber. Processes 2022 , 10 , 2721. [ Google Scholar ] [ CrossRef ]
• Maier, L.; Brosch, S.; Gaehr, M.; Linkhorst, J. Convective Drying of Porous Media: Comparison of Phase—Field Simulations with Microfluidic Experiments. Transp. Porous Media 2024 , 151 , 559–583. [ Google Scholar ] [ CrossRef ]

Click here to enlarge figure

Share and Cite

Rozas, C.; Erazo, O.; Ortiz-Araya, V.; Linfati, R.; Montero, C. Experimental and Statistical Determination of Convective Heat and Mass Transfer Coefficients for Eucalyptus nitens Sawn Wood Drying. Forests 2024 , 15 , 1287. https://doi.org/10.3390/f15081287

Rozas C, Erazo O, Ortiz-Araya V, Linfati R, Montero C. Experimental and Statistical Determination of Convective Heat and Mass Transfer Coefficients for Eucalyptus nitens Sawn Wood Drying. Forests . 2024; 15(8):1287. https://doi.org/10.3390/f15081287

Rozas, Carlos, Oswaldo Erazo, Virna Ortiz-Araya, Rodrigo Linfati, and Claudio Montero. 2024. "Experimental and Statistical Determination of Convective Heat and Mass Transfer Coefficients for Eucalyptus nitens Sawn Wood Drying" Forests 15, no. 8: 1287. https://doi.org/10.3390/f15081287

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals