IES Practice Guides - An Unfortunate Secret

TL;DR

• IES Practice Guides should be widely known and used by educators

• The Practice Guides are discipline agnostic

• Use these to focus on objective elements rather than opinion

• Refer to Practice Guides to identify critical elements in products considered for purchase or continued use

There’s nothing like a worldwide pandemic to cool the Reading Wars...a little. However, a quick skim of Twitter reveals there continue to be pedagogical arguments and heated discussions. There are several sources that educators can visit to review the research base behind various products such as interventions or curricula. A few of my most frequented sites are as follows.

• The National Center for Intensive Intervention Tools Charts - screeners, progress monitoring, and interventions are reviewed here.

• Evidence for ESSA - Reviews of K-12 products

• Google Scholar- search a product and review relevant research

• ERIC- search products, review research

When searching for products or narrowing in on interventions or practices for teams to review more in-depth, I would review products in the sites above, then seek research conducted with the product. If research was not conducted to validate the product, that was less convenient, but not a complete deal-breaker. There are interventions or products that lack research validation but are developed with research-validated practices.

Even if an intervention or product was studied and peer-reviewed research was conducted, the study usually was conducted in a setting or with students that were not an exact copy of the target population the product was intended to be applied. In this case, the research is helpful but not completely generalizable. Effect sizes are now required by the What Works Clearinghouse but this recent paper by Wolf et al., (2020), suggests that effect size magnitude can be influenced by numerous factors such as N size, study design, publisher or independent creation. The bottom line, the effect size is valuable but can vary based on many factors.

When charged with evaluating interventions or guiding a process to select new products, reviewing the sites above is recommended as a first step. Gather up as much existing evidence as you can. Backing up may also be necessary. Think back to the skill or skills that are in need of intervention. Then, what does the research indicate as the most effective method of intervention or instruction?

This is where the Institute for Education Sciences (IES) Practice Guides can be an incredibly valuable resource. When hosting or participating in large groups seeking to reach consensus on substantial purchases, opinions can be numerous and emotions can run high. The IES Practice Guides are discipline-agnostic and provide the most impactful recommendations to improve outcomes for students in specific skill areas.

The guides are arranged in a manner where summaries are presented first and more information can be found in the linked PDFs of the entire report. Experts were selected in groups to examine practices and provide the level of evidence to support each recommendation provided. Even when evidence is “low”, the recommendation is listed as the team of experts determined that the recommendation warranted mention and use.

How could the Practice Guides be used when reviewing products or existing interventions?

Let’s say a team is evaluating interventions for reading decoding. The Foundational Skills Practice Guide could be reviewed and the four recommendations could be used as a checklist to review products to ensure quality. If recommendations were not present in an intervention, that intervention could be passed on and the next could be reviewed. The Florida Center for Reading Research produced such a rubric a few years back. Depending on the skills or interventions reviewed, Practice Guides can provide elements to review for each intervention.

Using the Practice Guides and recommendations allows facilitators to have central pillars to refer back to when discussing elements of interventions. If the recommendations are not present, the discussion can be more objective rather than individuals indicating why they do not ‘like’ an intervention or why they prefer another product. The Practice Guides are based on expert reviews and evidence to support the recommendations. Teams working through a selection process can also document the evidence observed in the final choice. In this way, sharing how the process was completed can be shared with stakeholders if requested.

The Practice Guides are widely used by many researchers and administrators. However, too often I mentioned the Practice Guides and the team or individual was unaware. The Practice Guides are high-quality, useful resources available at no cost. Teams and educators need to be aware and increase use of these resources. Don’t even get me started on the Ceedar Center or Iris resources….another time. But, educators need to know about those too.