How do you know if retrieval practice improves learning in your classroom? Conduct research!
/By Pooja K. Agarwal, Ph.D.
I love receiving emails from educators who are interested in conducting research in their classroom. We know that retrieval practice improves learning. But does it improve learning in your classroom? It’s important to find out!
Here’s how to get involved and start conducting your own research:
Find a cognitive scientist near you and email them to see if they’d like to conduct research in your school. Don’t be shy!
Check out this podcast interview about research with Jennifer Gonzalez (Cult of Pedagogy) and cognitive scientist Dr. Kripa Sundar, including resources and YouTube videos.
Email us at ask@retrievalpractice.org and simply let us know if you’re interested in conducting research in your classroom, school, or university. We’re exploring how we can provide support for both educators and scientists, and it’s really helpful to know who would like to get involved.
Continue reading for tips and a discussion activity on how to decide what data to collect and how to brainstorm measurement tools.
How do you decide what data to collect?
Photo by Katerina Holmes
Question from Matthew, a history teacher in West Virginia: What would you consider to be solid baseline data to collect to look at retrieval practice and its effects? Test scores, anecdotal data, something else?
Recommendations from me: I commend you on your interest in collecting data to determine the benefits of retrieval practice in your own classroom and school. It can be difficult to measure learning – it’s quite a complex human process. You could compare previous grades to current grades and/or a mix of the measurements you listed.
You may also want to think about your ultimate goal: what does learning look like to you at the end of the day/month/school year? Once you have that in mind, you could then work backwards to construct a relevant survey or test to give students at the beginning and end of the school year.
From Matthew: You are correct that it is quite difficult to measure human learning, especially in a small mini-inquiry cycle within a school year. I like that you asked us to zoom out and look at what learning looks like at various points of the year. Our main focus could be on what we want to see improve; what it will look like if retrieval works/doesn't work; and how we can measure learning in a short cycle. I will take this back to my PLC and discuss!
What are some practical tools to collect data?
Question from Stephanie, a school leader in New York: We often struggle to find the right levers for collecting student outcome data. Do you have advice to share regarding getting past implementation fidelity and getting to outcomes in student achievement? I’d love to know if our instruction around the science of learning is impacting teacher practice, and more importantly student outcomes.
Recommendations from me: Measuring the impact of evidence-based teaching strategies on student achievement can be a challenge, for sure. A few options:
A basic start is to provide a pre-survey and post-survey to measure students' thoughts on retrieval practice. Here is survey research we published about students’ study strategies and their thoughts on retrieval practice (1,500 K–12 students total).
You could provide "pop" ungraded pre- and post-exams, to measure student learning without consequences to students' grades. Here is a research article by Dr. Maya Khanna on retrieval practice with ungraded pop quizzes.
As we did in our classroom research with Powerful Teaching co-author and middle school teacher Patrice Bain, you could implement retrieval practice during some lessons–but not all lessons–and compare how students perform on an assessment for retrieved vs. non-retrieved content. Here is an article we wrote together about the research process.
Read about recent research on retrieval practice in a wide range of classrooms and download our spreadsheet to take a closer look at how the research was conducted and how learning was measured.
From Stephanie: This is amazing. Thank you so much for taking the time. Much appreciated!
How do you logistically (and ethically) carry out research?
In my 10+ years of classroom research, we used a specific type of research approach, called a “within-subjects design.” I just looked through my book, Powerful Teaching, and I'm surprised I didn't write about this!
Essentially, all students received retrieval practice. We did not have a “retrieval practice group” and a “control group” in the way people typically think of research.
Each week, students received retrieval practice on some lesson content (for example, half of the textbook chapter’s vocabulary was included on 5-min clicker quizzes each week) and no retrieval practice on the rest of the lesson content. Importantly, all lesson content was still taught by the teacher.
After a few weeks or months, we gave students a low- or no-stakes test to see how students did on questions about content that was included on the mini-quizzes vs. content that was not on the mini-quizzes. There was a lot of work behind the scenes to make this happen, especially to divvy up all the lesson content on different quizzes for different classes.
If you’d like to adapt a within-subjects design for research in your classroom, school, or university, start small: Cover your content and classes exactly like you do now, but simply add in a few non-graded questions on an assessment (ideally at the beginning and end of the course, semester, or school year). It might not be enough data to draw conclusions (yet), but it would be a great way to start a conversation with teachers and students about how to conduct research on retrieval practice.
Get creative with this discussion activity
Here is a 30-minute discussion activity I use with my college students to get their creative research ideas flowing, which you can adapt for your teachers and/or students.
Question posed: Which shoe brand makes people jump higher, Nike or Adidas?
Teachers and/or students need to decide two things:
How could someone figure this out?
How could they measure jumping height?
After brainstorming and discussing ideas from the shoe brand example, you can then pose the same two questions about research on retrieval practice. How could someone figure out if retrieval practice improves learning, and how could you measure learning? Have fun!
