principles of experimental design ap stats

7   Experimental Design

A sample survey aims to gather information about a population without disturbing the population in the process. Sample surveys are one kind of observational study .

Observational studies only involve observing individuals and measuring variables of interest with no attempt to influence the responses

Unfortunately, this means that many variables are not controlled for so we cannot establish causation between the explanatory and response variables.

Response variables measure an outcome of a study.

Explanatory variables may help explain or predict changes in a response variable.

Because of the lack of control, we cannot be certain that variables other than the explanatory variable confound the response instead.

Confounding occurs when the effects of two variables on a response variable cannot be separated from each other.

Confounding variables are variables other than the explanatory variable that may have an effect on the response variable.

When our goal is to understand cause and effect, an experiment is the only source of fully convincing data since results in observational studies typically have some confounding variable in play. For this reason, the distinction between observational study and experiment is one of the most important in statistics.

An experiment deliberately imposes some treatment (the specific condition applied) on individuals (experimental units or subjects if the experimental units are human) to measure their responses.

7.1 Experiment Principles

In general, the quality of experiments (their internal validity) can be judged based on the degree to which they have four things: comparison, randomization, control, and replication. Stronger internal validity gives us a better cause-effect link in our experiment. Whenever you are describing or evaluating the design of an experiment, you need to be sure to discuss all four of these!

These four determine how much internal validity we have in our experiment.

The logic of a randomized comparative experiment depends on our ability to treat all the subjects the same in every way except for the actual treatments being compared. Good experiments, therefore, require careful attention to details to ensure that all subjects really are treated identically.

7.1.1 Placebos

The response to a dummy treatment is called the placebo effect . Subjects are given a placebo treatment to control for the placebo effect.

For example, If I tell someone that I am giving them an energy drink (when it in fact has doesn’t actually provide “energy”), and they feel like they have energy after, they have fallen for the placebo effect.

It’s well known that someone’s mental state can easily affect their physical state, so it’s important to control for the placebo effect. Typically, this applies to medicine settings, where you might give a pill with the actual medicine and a placebo (a pill with everything but the actual medicine) and conduct it in a blind .

Conducting an experiment in a blind means that you give treatments to patients without allowing them to know which treatment they are taking.

However, Whenever possible, experiments with human subjects take it a bit further and conduct their experiment in a double-blind , where neither the subjects nor those who interact with them and measure the response variable know which treatment a subject received.

7.2 Experiment Designs

7.2.1 completely randomized design.

In a completely randomized design, the experimental units are assigned to the treatments completely by chance. This is similar to (but NOT the same as) a simple random sample (SRS), because in both cases we ignore other variables. Here’s the difference: In an SRS, we’re picking some people (our sample) to study, and ignoring the rest. In a completely randomized experiment, however, we already have our sample (the people in our experiment), and we’re randomly deciding how we’re going to study each person (or, which treatment they’re going to get). So in complete randomization, the randomization is in the assignment , not in the selection , of people in our study.

7.2.2 Randomized Block Design

In a randomized block design, the experimental units are first assigned to blocks according to the different types/status of the experimental units in the experiment. This is similar to stratified random sampling , however, we are not taking a sample. Any reference to stratified random sampling is wrong when describing an experiment design.

After each experimental unit is assigned to their block, the experiment is carried out in each block, where a completely randomized design is carried out within the block.

Afterwards, you compare and analyze results from each block and finally combine all results and analyze the differences between blocks.

7.2.3 Matched Pairs Design

A matched pairs design is a special case of a randomized block design that uses blocks of size 2. In this kind of design, you have to have “matched pairs.” In other words, you need to have two extremely similar individuals that make up each block. In some cases, you have a single person for each block and that person recieves both treatments in randomized order (because who is more similar to a person than themselves?).

7.3 Inference

The main purpose of experiments is to be able to infer something about what we did. Does A actually affect B? Is it true for anyone else other than the people we experimented on?

An observed effect so large that it would rarely occur by chance is said to be statistically significant . If we test something according to a single assumption that we make and find out that the data that we collect doesn’t really match up with that claim (if the chance of seeing data like the one we obtained is too low), then we’d say that it is statistically significant evidence.

7.3.1 Scope of Inference

The scope of inference refers to the type of inferences (conclusions) that can be drawn from a study. The types of inferences we can make (inferences about the population and inferences about cause-and-effect) are determined by two factors in the design of the study.

All Subjects

AP Statistics

Study guides for every class, that actually explain what's on your next test, experimental design, from class:.

Experimental design refers to the process of planning an experiment to ensure that it can adequately address the research question being investigated. This involves selecting how treatments are assigned, ensuring randomization, and controlling for variables that may affect the outcome. A well-structured experimental design allows for valid conclusions about cause-and-effect relationships by isolating the effects of the independent variable on the dependent variable.

congrats on reading the definition of Experimental Design . now let's actually learn it.

5 Must Know Facts For Your Next Test

• A key principle of experimental design is randomization, which helps eliminate selection bias by ensuring that each participant has an equal chance of being assigned to any treatment group.
• In a well-designed experiment, researchers often use control groups to compare outcomes against those who receive the treatment, helping to attribute effects specifically to the treatment.
• Experimental designs can be classified into different types, such as completely randomized designs, block designs, and factorial designs, each with unique methods for treatment assignment.
• Replication within an experiment is important because it allows researchers to confirm findings and improve the precision of their estimates by minimizing the impact of random variation.
• Confounding variables must be controlled for in experimental design, as they can distort results and lead to incorrect conclusions about the relationship between independent and dependent variables.

Review Questions

• Randomization enhances the validity of an experimental design by ensuring that participants are assigned to treatment groups in a way that eliminates bias. This process leads to comparable groups, minimizing the influence of confounding variables that could affect outcomes. Consequently, randomization allows researchers to make more reliable conclusions about cause-and-effect relationships.
• Using a control group in experimental design is crucial because it serves as a baseline for comparison against groups receiving treatments. It allows researchers to isolate the effects of the treatment by showing what happens without it. This comparison helps in interpreting results more accurately by attributing any observed changes directly to the treatment rather than other external factors.
• Different types of experimental designs, such as block or factorial designs, significantly impact data analysis and conclusion validity. Block designs help control for variability within specific subgroups, increasing precision in estimating treatment effects. Factorial designs allow researchers to assess multiple factors simultaneously, providing insights into interactions between variables. Choosing an appropriate design influences both how data is analyzed and how confidently researchers can draw conclusions about causal relationships from their findings.

Related terms

The practice of randomly assigning subjects to different treatment groups to eliminate bias and ensure that the groups are comparable.

A group in an experiment that does not receive the treatment being tested, used as a benchmark to measure how the other groups perform.

The repetition of an experiment or study to confirm findings and ensure that results are consistent and reliable.

" Experimental Design " also found in:

Subjects ( 58 ).

• AP Physics 2
• AP Psychology
• Additive Manufacturing and 3D Printing
• Advanced Communication Research Methods
• Advanced quantitative methods
• Advertising Strategy & Consumer Insights
• Art Conservation and Restoration
• Art Therapy
• Art and Artificial Intelligence
• Business Process Optimization
• Children's Television
• Cognitive Bias in Business Decision Making
• Cognitive Psychology
• College Introductory Statistics
• Combinatorics
• Communication Research Methods
• Communication Technologies
• Data Visualization
• Editorial Design
• Educational Psychology
• Enumerative Combinatorics
• Foundations of Education
• Haptic Interfaces and Telerobotics
• Honors Statistics
• Improvisational Leadership
• Intro to Astronomy
• Intro to Business Statistics
• Introduction to Education
• Introduction to Environmental Science
• Introduction to Industrial Engineering
• Introduction to Political Communications
• Introduction to Political Research
• Introduction to Public Health
• Language and Popular Culture
• Linear Modeling: Theory and Applications
• Market Research: Tools and Techniques for Data Collection and Analysis
• Media Effects
• Media Expression and Communication
• Media and Democracy
• Media and Politics
• Metabolomics and Systems Biology
• Motor Learning and Control
• Music Psychology
• Nuclear Fusion Technology
• Persuasion Theory
• Physiology of Motivated Behaviors
• Professionalism and Research in Nursing
• Psychology of Economic Decision-Making
• Psychology of Language
• Public Policy Analysis
• Reproducible and Collaborative Statistical Data Science
• Science Education
• Social Psychology
• Sports Psychology
• Theoretical Statistics
• Wearable and Flexible Electronics

Practice Questions ( 1 )

• Which experimental design should be used in a study of college students' grades considering major and course load as factors?

Additional resources ( 3 )

• AP Statistics - Unit 3 Overview: Collecting Data
• AP Statistics - Unit 6 Overview: Inference for Categorical Data: Proportions
• AP Statistics - Unit 7 Overview: Means

© 2024 Fiveable Inc. All rights reserved.

Ap® and sat® are trademarks registered by the college board, which is not affiliated with, and does not endorse this website..