CHAPTER

3

The extent to which education interventions have been studied and the range of effects typically observed

DOI: https://doi.org/10.56383/HFEY6916

This chapter should be cited as:

Kay, J., Higgins, S.E., Villanueva Aguilera, A.B., Dobson, E.S., Gascoine, L., Katsipataki, M., Rajab, T., Zaman, M., Ellis-Thompson, A., Madgwick, H., Patel, R. and Blausten, H. (2022) ‘The extent to which education interventions have been studied and the range of effects typically observed’ in Mercier, J. (ed.) Data and Evidence in, Duraiappah, A.K., van Atteveldt, N.M., Borst, G., Bugden, S., Ergas, O., Gilead, T., Gupta, L., Mercier J., Pugh, K., Singh, N.C. and Vickers, E.A. (eds.) Reimagining Education: The International Science and Evidence Based Assessment. New Delhi: UNESCO MGIEP.

Coordinating Lead Authors

Jonathan Kay

Steve E. Higgins

Alaidde Berenice Villanueva Aguilera

Emma Sian Dobson

Louise Gascoine

Maria Katsipataki

Taha Rajab

Mohammad Zaman

Amy Ellis-Thompson

Harry Madgwick

Rupal Patel

Hannah Blausten

3.1 Introduction

This chapter has been compiled from the findings of the Education Endowment Foundation’s (EEF) education database project, a joint study conducted with Durham University.

The EEF’s education database is comprised of thousands of education research studies from across the globe, all focused on measuring the impact of education interventions on students’ outcomes.

The EEF’s education database is comprised of thousands of education research studies from across the globe, all focused on measuring the impact of education interventions on students’ outcomes. The studies in the database have been coded to enable analysis and searching across a range of factors, including country, pupil age and type of intervention.

Rather than simply focusing on the impact of interventions, the database also records information about the delivery of interventions (such as the frequency and intensity of the intervention) and detailed quantitative impact data, such as variations in effects based on subject or delivery mechanism (such as whether an intervention is delivered by a qualified teacher or a classroom assistant). Impact is translated from standardized effect sizes to ‘months of learning’ for ease of communication and to aid discussion around the impact of interventions. Months of learning, communicated as a headline figure for each approach, however, can hide important variation caused by duration of intervention, group size and the test measures used. Building the database containing all of this data allows researchers to examine which factors are driving the impact behind the overall average to find the signal amongst the noise. It is this detailed data which makes this education database unique. It will significantly reduce the time and effort needed to review the impact of different types of interventions, and to analyse the factors that increase or reduce effectiveness.

3.2 The database and the EEF Teaching and Learning Toolkit

The database has been designed to underpin updated versions of the EEF’s Teaching and Learning Toolkit and Early Years Toolkit.

The EEF Toolkits are accessible summaries of education research for teachers and decision-makers. With over forty approaches for improving teaching and learning, each is summarized in terms of its average impact on attainment, its cost and the strength of the evidence supporting it.

The database and the Toolkit are living reviews of the evidence. They are updated whenever new studies become accessible and coded. This document – the International Science and Evidence based Education (ISEE) Assessment – cannot replicate the living nature of the reviews, and so readers should consult the live versions which are available at https://educationendowmentfoundation.org.uk/evidence-summaries/teaching-learning-toolkit/, and via any of the EEF’s global partner organizations, listed at https://educationendowmentfoundation.org.uk/about/international-work/eefs-international-partnerships/.

3.3 The future of the database

Much like the Cochrane Library and other living reviews of evidence, the database is a long-term project that will continue to grow and develop over time. Among the goals for the database over the next five to ten years are:

- inclusion of non-English - language studies by EEF partner organizations around the world, allowing the Toolkit to be further tailored to different contexts;

- national and international partnerships and fellowships, enabling external researchers to use the database for research and analysis thus contributing to the global education evidence base;

- use of machine learning and artificial intelligence to automatically search for, identify and extract data from new studies, reducing the time and cost of reviewing education research;

- automatic live updating of the EEF Toolkit from the database, allowing for the ‘living’ systematic review to be updated quickly with the most recently published studies.

3.4 Methods

The first set of studies for the database has been identified from the current version of the Sutton Trust–EEF Teaching and Learning Toolkit. Nearly all of the strands are based on meta-analyses and systematic reviews which have been identified through a systematic updating process (EEF, 2018[1]) since the initial version of the Toolkit was published by the Sutton Trust in 2011 (Higgins, Kokotsaki and Coe, 2011).

These meta-analyses have been systematically ‘unzipped’ so that the included studies which contribute to the overall pooled effect are identified and screened ... for inclusion in the database.

These meta-analyses have been systematically ‘unzipped’ so that the included studies which contribute to the overall pooled effect are identified and screened (a two-stage process of title and abstract and then full text screening) for inclusion in the database (Higgins et al., 2022).

3.4 .1

INCLUSION CRITERIA FOR THE EEF EVIDENCE DATABASE

The inclusion criteria aim to identify relevant educational evidence for schools and policy-makers interested in school-based education, consistent with the mission of the EEF, which is dedicated to breaking the link between family income and educational achievement. Specifically, the EEF aims to:

- raise the attainment of three- to eighteen-year-olds, particularly those facing disadvantage;

- develop their essential life skills; and

- prepare young people for the world of work and further study.

PICOS and SPIDER analyses (Methley et al., 2014) were used to define the scope of the database:

3.4.1 .1

PICOS SPIDER DATABASE SCOPE: EXPLANATION AND EXAMPLES

3.4 .2

INCLUSION CRITERIA EXCLUDED

The majority of the sample (greater than 50 per cent) on which the analysis is based are learners or pupils aged between three and eighteen (further education or junior college students are to be included where their study is for school-level qualifications).

The majority of the sample are: those are post-secondary education; in higher education; adults; infants under three; other students over 18.

The intervention or approach evaluates the impact of an educational intervention or approach, including named or clearly defined programmes and recognizable approaches classifiable according to the Toolkit strand definitions (see the statistical analysis plan here "https://educationendowmentfoundation.org.uk/public/files/Toolkit/EEF_Evidence_Database_Protocol_and_Analysis_Plan_June2019.pdf").

The intervention or approach is not classifiable with regard to the current Toolkit strand definitions (see the statistical analysis plan here "https://educationendowmentfoundation.org.uk/public/files/Toolkit/EEF_Evidence_Database_Protocol_and_Analysis_Plan_June2019.pdf" ).

The intervention or approach is undertaken in a normal educational setting or environment, such as a nursery or school or a typical setting (e.g. an outdoor field centre or museum).

Laboratory studies; specially created environments (both physical and virtual) designed for theoretical research questions, rather than educational benefit[6].

3.4 .3

SEARCH STRATEGY FOR IDENTIFICATION OF RELEVANT SINGLE STUDIES

Where there were no existing meta-analyses or systematic reviews with quantitative data in the existing Toolkit strands, a new systematic search was undertaken for primary studies to update the existing single studies identified for the Toolkit. The following sources (gateways and databases) were used:

- First search

• Article First

• ECO

• Papers First

• World Cat Dissertations

- EBSCO

• BEI

• Education Abstracts

• Education Administration Abstracts

• ERIC

• PsycArticles

• PsycINFO

- Taylor and Francis

• Educational Research Abstracts Online

- ProQuest

• ProQuest Dissertations & Theses (Global)

- Elsevier

• Science Direct

- Thomson Reuters

• Web of Science

In addition, informal searching for ‘grey’ literature (reports and unpublished studies) was undertaken using Google, Google Scholar and Microsoft Academic.

We did not use citation searching, ‘pearl growing’ (Schlosser et al., 2006) or expert nomination, although we did use these techniques to ensure the adequacy of search terms (Papaioannou et al., 2010). Our rationale for this is that the use of such approaches on their own, without subsequently adapting the search criteria, is likely to increase the risk of publication bias (Higgins, 2018). Where we identified relevant studies from non-systematic approaches we aimed to refine our search criteria and to run additional searches to find other similar studies retrieved with the amended search strings.

3.4 .4

RELEVANCE FOR TEACHERS, SCHOOL LEADERS AND POLICY-MAKERS

The database allows users to review the impact of different approaches to improving outcomes for children and young people by understanding not just the average impact of an intervention, but how that impact varies depending on subject, age of pupils and country. It will give teachers, school leaders and policy-makers a much better idea of whether an approach that has worked elsewhere can also work in their particular context.

3.4 .5

LIMITATIONS

The evidence summaries that follow describe the average impact and some of the reasons for variation that have been identified. They cannot predict the impact of an approach in any classroom.

In particular, a search of the global evidence reveals gaps in research on pedagogical strategies in particular contexts. While randomized control trials (RCTs) have proliferated in the context of development, few of these studies look at pedagogical approaches. Rather, they frequently focus on structural approaches or efforts to increase access to education. While these efforts are critical to improving education, they will need to happen alongside efforts to improve pedagogy.

Many of the topics that follow draw heavily on evidence from high-income countries – in particular the United States (USA). It is therefore crucial to carefully consider the contextual barriers to implementation before adopting any strategy to improve learning.

In order to improve accessibility to teachers – the meta-analytic results, average cost and evidence security are communicated using months progress, a five-point cost scale and padlocks. The details of these headline estimates can be found below.

Summary of headline figure data. More information can be found here.

Evidence strength

Evidence strength is communicated in padlocks. The evidence is awarded an initial padlock based on the number of studies that meet the inclusion criteria (e.g. 70 or more for 5 padlocks). Padlocks are then reduced where threats to validity are identified:
- Small percentage of recent studies
- Small percentage of randomised controlled trials
- Threats to ecological validity (e.g. delivered by researchers)
- Large percentage of studies are not independently evaluated
- High unexplained heterogeneity

Cost

The cost estimates are based on the average cost of delivering an intervention. A components based approach is used to measure the per pupil cost, which is then reported on a 5 point scale. For example, one £ is awarded for approaches that cost less than £80 per pupil per year. Full details on cost calculations can be found here.

Months progress

The months progress estimates are based on the effect size calculated using a random-effects meta-analysis. This meta-analytic result is translated into months progress to support accessibility. It uses the assumption that pupils make approximately 1 standard deviation of progress over a year (e.g. an effect size of 0.5 will be 6 months progress.

REGION

INCLUDED
(N)

TOOLKIT
(%)

North America

1924

76%

Europe & Central Asia

349

14%

East Asia & Pacific

83

3%

Middle East & North Africa

82

3%

Sub-Sahara Africa

21

1%

Latin America & Caribbean

9

<1%

South Asia

5

<1%

No Code

62

2%

Table 3.1 Frequency of studies in Toolkit by region overall

3.4 .6

FURTHER INFORMATION

The statistical analysis plan for the database can be found here.

Data extraction for the database is undertaken with three data extraction tools:

- EEF main data extraction, used for all studies;

- EEF Toolkit effect size data extraction, used for all studies;

- strand-specific data extraction (additional codes for each Toolkit strand, such as information about tutors and tutees in peer tutoring, or groups size in small group – used for studies in each strand).

Strand title

Arts participation

Update date

28 June 2021

Number of studies

80

Summary

Moderate impact for very low cost, based on moderate evidence

Cost

Very low

Padlocks

3

Impact

+3 months

What is it?

Global

Arts participation is defined as involvement in artistic and creative activities, such as dance, drama, music, painting or sculpture. It can occur either as part of the curriculum or as an extra-curricular activity. Arts-based approaches may be used in other areas of the curriculum (e.g. the use of drama to develop engagement and oral language before a writing task).

Participation may be via regular weekly or monthly activities, or more intensive programmes such as summer schools or residential courses. Whilst these activities have important educational value in themselves, this Toolkit entry focuses on the benefits of arts participation for core academic attainment in other areas of the curriculum, particularly literacy and mathematics.

Key Findings

Global

Arts participation approaches can have a positive impact on academic outcomes in other areas of the curriculum.

Global

The research here summarizes the impact of arts participation on academic outcomes. It is important to remember that arts engagement is valuable in and of itself and its value should be considered beyond mathematics or English outcomes.

Global

If the aim of the arts approach is to improve academic attainment it is important to identify the link between the chosen arts intervention and the outcomes that need to be improved.

Global

Arts-based approaches may offer a route to re-engage older pupils in learning, though this does not always translate into better attainment. It is important to consider how increased engagement will be used to improve teaching and learning for these pupils.

How effective is the approach?

Global

Overall, the average impact of arts participation on other areas of academic learning appears to be positive but moderate, about an additional three months’ progress.

Improved outcomes have been identified in English, mathematics and science. Benefits have been found in both primary and secondary schools.

Some arts activities have been linked with improvements in specific outcomes. For example, there is some evidence of the impact of drama on writing and a potential link between music and spatial awareness.

Wider benefits such as more positive attitudes to learning and increased well-being have consistently been reported.

Behind the Average

Global

The impact is similar for both primary and secondary school pupils.

Global

The effects tend to be higher for writing and mathematics than reading.

Closing the disadvantage gap

Local

There is intrinsic value in teaching pupils creative and performance skills and ensuring disadvantaged pupils have access to a rich and stimulating arts education. Arts participation may be delivered within the core curriculum, or through extra-curricular or cultural trips but the latter can be subject to financial barriers for pupils from deprived backgrounds.

There is some evidence to suggest a causal link between arts education and the use of arts-based approaches with overall educational attainment. Where the arts are being taught as a means to boost academic achievement for those eligible for the pupil premium, schools should carefully monitor whether this aim is being achieved.

How could you implement it in your setting?

Global

Arts participation relates to a broad range of subjects including traditional fine arts, theatre, dance, poetry and creative writing. It also includes teaching strategies that explicitly include arts elements, such as drama-based pedagogy.

Some components of arts education approaches might include:

- explicit teaching of creative skills and techniques;
- opportunities for pupils to practise, reflect on their strengths and identify areas for improvement;
- access to materials, equipment, extra-curricular activities and cultural experiences.

Global

Arts education may take the form of regular lessons or monthly activities, after school clubs, small group or one-on-one tuition, or whole school programmes. Activities can also be delivered through more intensive programmes such as summer schools or residential courses.

Local

The average cost of arts education is expected to be very low, with costs ranging from very low to high depending on the type of provision. Costs to schools are largely based on teacher professional development and resources. Costs are greater where activities fall outside of the school day or involve small group or one-on-one tuition from specialist teachers.

Implementing arts education will require a small amount of additional staff time compared with other approaches as it is part of the core curriculum. Arts activities may also involve professional artists, and certified drama or music teachers.

In addition to time and cost, school leaders should consider how to maximize the professional development needs of staff to effectively integrate arts activities (such as drama, visual arts or music) in the classroom and evaluate their impact on pupil outcomes.

When introducing new approaches, schools should consider implementation. For more information see Putting Evidence to Work – A School’s Guide to Implementation.

How secure is the evidence?

Global

The security of the evidence around arts participation is rated as moderate. Eighty studies were identified. The topic lost a padlock because a large percentage of the studies were not independently evaluated. Evaluations conducted by organizations connected with the approach, for example, commercial providers, typically have a larger impact, which may influence the overall impact.

As with any evidence review, the Toolkit summarizes the average impact of approaches as researched in academic studies. It is important to consider the context and apply professional judgement when implementing an approach.

Strand title

Behaviour interventions

Update date

18 June 2021

Number of studies

89

Summary

Moderate impact for very low cost, based on moderate evidence

Cost

Very low

Padlocks

2

Impact

+4 months

What is it?

Global

Behaviour interventions seek to improve attainment by reducing challenging behaviour in school. This entry covers interventions aimed at reducing a variety of behaviours, from low-level disruption to aggression, violence, bullying, substance abuse and general anti-social activities. The interventions themselves can be split into three broad categories:
1. Approaches to developing a positive school ethos or improving discipline across the whole school which also aims to support greater engagement in learning.
2. Universal programmes that generally take place in the classroom and seek to improve behaviour.
3. More specialized programmes that are targeted at students with specific behavioural issues.
Other approaches, such as parental engagement and SEL programmes, are often associated with reported improvements in school ethos or discipline, but are not included in this summary, which is limited to interventions that focus directly on behaviour.

Key Findings

Global

Both targeted interventions and universal approaches have positive overall effects, about an additional four months’ progress. Schools should consider the appropriate combination of behaviour approaches to reduce overall disruption and provide tailored support where required.

Global

There is evidence for a range of different interventions with the highest impacts for approaches that focus on self-management or role-play and rehearsal.

Global

Even within programme types there is a range of impacts. In selecting a behaviour intervention, schools should look for programmes that have been evaluated and shown to have a positive impact.

Global

When adopting behaviour interventions – whether targeted or universal – it is important to consider providing professional development to staff to ensure high-quality delivery and consistency across the school.

How effective is the approach?

Global

The average impact of behaviour interventions is four months’ additional progress over the course of a year. Evidence suggests that, on average, behaviour interventions can produce moderate improvements in academic performance along with a decrease in problematic behaviours. However, estimated benefits vary widely across programmes.

Approaches such as improving teachers’ behaviour management and pupils’ cognitive and social skills are both effective, on average.

School-level behaviour approaches are often related to improvements in attainment, but there is a lack of evidence to show that the improvements are actually caused by the behaviour interventions, rather than other school interventions happening at the same time. Parental and community involvement programmes are often associated with reported improvements in school ethos or discipline and so are worth considering as alternatives to direct behaviour interventions.

Behind the Average

Global

Effects are slightly lower for secondary school pupils, about an additional three months’ progress.

Global

Impact seems to apply across the curriculum with slightly greater impact, about an additional five months’ progress, for mathematics than for literacy or science.

Global

Frequent sessions several times a week over an extended period of up to a term appear to be the most successful.

Global

Approaches that focus on self-management and those involving role play or rehearsal are associated with greater impact.

Closing the disadvantage gap

Local

According to figures from the Department for Education in the United Kingdom (UK), pupils who receive free school meals are more likely to receive a permanent or fixed period exclusion compared to those who do not.

The most common reason for exclusion is persistent disruptive behaviour. Pupil behaviour will have multiple influences, some of which teachers can directly control though universal or classroom management approaches. Some pupils will require more specialist support to help manage their self-regulation or social-emotional skills.

How could you implement it in your setting?

Global

Behaviour interventions have an impact by increasing the time that pupils have for learning. This might be achieved by reducing low-level disruption that impacts learning time in the classroom or by preventing exclusions that remove pupils from school for periods of time. If interventions take up more classroom time than the disruption they displace, engaged learning time is unlikely to increase. In most schools, a combination of universal and targeted approaches will be most appropriate.

- Universal approaches to classroom management can help prevent disruption but often require professional development to administer effectively.

- Targeted approaches that are tailored to pupils’ needs such as regular report cards or functional behaviour assessments may be appropriate where pupils are struggling with behaviour.

In all approaches it is crucial to maintain high expectations for pupils and to embed a consistent approach across the school. Successful approaches may also include SEL interventions and parental engagement.

Global

Evidence suggests that programmes delivered over two to six months produce more long-lasting results. Whole school strategies usually take longer to embed than individually tailored or single classroom strategies.

Local

The costs of behaviour interventions vary widely and overall are estimated to range between very low to moderate. The costs to schools to deliver whole school strategies are largely based on staff time and training. More intensive, targeted interventions are likely to incur higher staffing and training costs.

Behavioural interventions can require a large amount of staff time, compared with other approaches. Targeted or one-on-one approaches, delivered by trained school staff or specialists, will require additional staff time compared to universal approaches. Overall, effective approaches can promote better engagement with teaching and learning by reducing challenging behaviour and improving pupil engagement.

In addition to time and cost, school leaders should reflect on the impact of whole school behaviour policies and support staff in maintaining a consistent approach. When adopting new approaches, school leaders should consider programmes with a track record of effectiveness. Improving classroom management may involve intensive training where teachers reflect on their practice, implement new strategies and review progress over time.

When introducing new approaches, schools should consider implementation. For more information see Putting Evidence to Work – A School’s Guide to Implementation.

How secure is the evidence?

Global

The security of the evidence around behaviour interventions is rated as low. Eighty-nine studies that meet the inclusion criteria for the Toolkit were identified. Overall, the topic lost two additional padlocks because:

- only a small percentage of studies were conducted recently, which might mean that the research is not representative of current practice;

- a large percentage of the studies were not independently evaluated. Evaluations conducted by organizations connected with the approach, for example, commercial providers, typically have larger impacts, which may influence the overall impact of the strand.

As with any evidence review, the Toolkit summarizes the average impact of approaches as researched in academic studies. It is important to consider context and apply professional judgement when implementing an approach.

Strand title

Collaborative learning

Update date

23 June 2021

Number of studies

212

Summary

Moderate impact for very low cost, based on moderate evidence

Cost

Very low

Padlocks

3

Impact

+5 months

What is it?

Global

A collaborative (or cooperative) learning approach involves pupils working together on activities or learning tasks in a group small enough to ensure that everyone participates. Pupils in the group may work on separate tasks contributing to a common overall outcome, or work together on a shared task. This is distinct from unstructured group work.

Some collaborative learning approaches put pairs, groups or teams of mixed attainment to work in competition with each other in order to drive more effective collaboration. There is a very wide range of approaches to collaborative and cooperative learning involving many different kinds of organization and tasks. Peer tutoring can also be considered as a type of collaborative learning, but is reviewed as a separate topic in the Toolkit.

The collaborative learning approaches adopted by schools were typically implemented for 8 weeks over the course of a school year. The average impact, however, includes longer and shorter collaborative learning approaches.

Key findings

Global

Collaborative learning approaches have a positive impact, on average, and may be a cost-effective approach for raising attainment.

Global

Pupils need support and practise in working together; it does not happen automatically. Professional development can support the effective management of collaborative learning activities.

Global

Tasks and activities need to be designed carefully so that working together is effective and efficient, otherwise some pupils may struggle to participate or try to work on their own. It is important to ensure that all pupils talk and articulate their thinking in collaborative tasks to ensure they benefit fully.

Global

Competition between groups can be used to support pupils in working together more effectively. However, an over emphasis on competition can cause learners to focus on winning rather than succeeding in their learning.

Global

The most promising collaborative learning approaches tend to have group sizes between three and five pupils who have a shared outcome or goal.

How effective is the approach?

Global

The impact of collaborative approaches on learning is consistently positive, with pupils making about an additional five months’ progress, on average, over the course of an academic year. However, the size of impact varies, so it is important to get the detail right.

Collaborative learning can describe a large variety of approaches, but effective collaborative learning requires much more than just sitting pupils together and asking them to work in pairs or a group; structured approaches with well-designed tasks lead to the greatest learning gains.

There is some evidence that collaboration can be supported with competition between groups, but this is not always necessary, and can lead to learners focusing on the competition rather than the learning it aims to support. Most of the positive approaches include the promotion of conversation and interaction between learners.

The evidence indicates that groups of three to five are most effective for collaborative learning approaches – there are smaller positive impacts for both paired work and collaborative learning activities with more than five pupils in a group. There is also some evidence that collaborative learning approaches are particularly promising when they are used to teach science.

Behind the average

Global

The effects of collaborative learning are slightly higher in secondary schools (about an additional six months’ progress) than in primary schools (about an additional five months’ progress).

Global

The impact of collaborative learning is slightly lower in literacy (about an additional three months’ progress) than mathematics (about an additional five months’ progress) and science (about an additional ten months’ progress).

Global

Small groups of three to five pupils with responsibility for a joint outcome appear to be the most successful structure.

Global

Studies that deliver collaborative learning through digital technology tend to have a lower impact, about an additional three months’ progress overall.

Closing the disadvantage gap

Local

There is limited evidence on differential impact for pupils from disadvantaged backgrounds. There is some evidence that collaborative learning approaches may benefit those with low prior attainment by providing opportunities for pupils to work with peers to articulate their thinking, share knowledge and skills, and address misconceptions through peer support and discussion.

It is crucial that support is provided through well-structured and carefully designed learning activities to ensure that lower-attaining pupils are involved, challenged and learn successfully. If collaborative learning approaches just involve high-attaining pupils solving problems with no input from their peers, this is likely to widen existing gaps in attainment.

Applications and Approaches

Name

Collaborative learning with joint outcomes

Impact

+7 months

Number of studies

111

Summary

When groups conducting collaborative learning activities are given a joint group outcome to work towards, the impact of the approach is typically higher than average.

One hundred and eleven studies in which pupils worked towards a joint outcome were identified.

Applications and Approaches

Name

Collaborative learning with individual outcomes

Impact

+4 months

Number of studies

101

Summary

Some collaborative learning activities give different children within the group different objectives to accomplish. Overall, these approaches have positive outcomes, but the impact is typically slightly lower than those with shared group outcomes.

One hundred and one studies in which individual outcomes were given to pupils within collaborative learning activities were identified.

How could you implement it in your setting?

Global

There are many theories about how collaborative learning might benefit pupil outcomes. Through collaboration, pupils may develop explanation, demonstration, problem-solving and metacognitive skills, or pupils may benefit from sharing the load of challenging tasks. It is important that schools ensure that within collaborative learning:

- all pupils, particularly pupils with low prior attainment, are supported to fully participate;

- the make-up of pairings and groups is carefully considered;

- teachers promote good practice in collaboration, for example, modelling high-quality discussions so that collaborative activities are productive;

- teachers carefully monitor collaborative activities and support pupils who are struggling or not contributing.

Global

There is a broad range of approaches to collaborative or cooperative learning involving different kinds of organization and tasks across the curriculum. Not all of the specific approaches to collaborative learning adopted by schools have been evaluated, so it is important to evaluate any new initiative in this area. Professional development is likely to be required to maximize the effectiveness of approaches and monitor the impact of different approaches in the classroom.

Local

The average cost of collaborative learning is expected to be very low with the cost to schools largely in teacher training and resources. As a classroom-based approach, implementing collaborative learning will also require a small amount of staff time for planning and monitoring, compared with other approaches.

In addition to time and cost, school leaders should consider how to maximize the effectiveness of collaborative learning through teacher professional development to support the use of well-designed tasks. They should carefully monitor the impact of approaches on lower-attaining pupils.

When introducing new approaches, schools should consider implementation. For more information see Putting Evidence to Work – A School’s Guide to Implementation.

How secure is the evidence?

Global

The security of the evidence around collaborative learning interventions is rated as low. Two hundred and twelve studies that meet the inclusion criteria of the Toolkit were identified. The topic lost three padlocks because:

- only a small percentage of studies have taken place recently, which might mean that the research is not representative of current practice;

- a large percentage of the studies were not independently evaluated. Evaluations conducted by organizations connected with the approach, for example, commercial providers, typically have larger impacts, which may influence the overall impact of the strand;

- there is a large amount of unexplained variation between the results included in the topic. All reviews contain some variation in results, which is why it is important to look behind the average. Unexplained variation (or heterogeneity) reduces certainty in the results in ways that we have been unable to test by looking at how context, methodology or approach is influencing impact.

As with any evidence review, the Toolkit summarizes the average impact of approaches as researched in academic studies. It is important to consider context and apply professional judgement when implementing an approach.

Strand title

Extending school time

Update date

9 June 2021

Number of studies

74

Summary

Moderate impact for moderate cost, based on limited evidence

Cost

Moderate

Padlocks

2

Impact

+3 months

What is it?

Global

Extending school time involves increasing learning time in schools during the school day or changing the school calendar. This can include extending core teaching and learning time in schools as well as the use of targeted before and after school programmes (including additional small group or one-on-one tuition). It also includes revisions to the school calendar to extend the total number of days in the school year.

The median intervention length for approaches that extended the school year was forty weeks (an extended school day for one academic year). Other approaches to increasing learning time, such as homework and summer schools, are included in other sections of the Toolkit.

Key Findings

Global

Programmes that extend school time have a positive impact on average but are expensive and may not be cost-effective for schools to implement. Schools will also need to consider the workload and well-being of staff.

Global

Planning to get the most from any extra time is important. It should meet pupils’ needs and build on their capabilities. Where additional time is voluntary, it is important to monitor attendance to ensure pupils who need additional support can benefit.

Global

Before and after school programmes with a clear structure, a strong link to the curriculum, and well-qualified and well-trained staff are more clearly linked to academic benefits than other types of extended hours provision.

Global

Additional school time may be more effective if it is used for one-on-one support, in contrast to small or large group teaching.

Global

Enrichment activities without a specific focus on learning can have an impact on attainment, but the effects tend to be lower and the impact of different interventions can vary a great deal (see entries for physical activity or arts participation). These interventions may, however, be beneficial for their own sake outside of any attainment impacts.

How effective is the approach?

Global

The average impact of approaches involving extending school time is about an additional three months’ progress over the course of a year. The average impact is influenced by the targeted use of before and after school programmes, which have higher impacts, on average. The impact is also slightly lower when school time is extended in secondary school.

In addition to providing academic support, some school programmes aim to provide stimulating environments and activities or develop additional personal and social skills. These programmes are more likely to have an impact on attainment than those that are solely academic in focus. However, it is not clear whether this is due to the additional activities or to improved attendance and greater engagement.

The research also indicates that attracting and retaining pupils in before and after school programmes is harder at secondary level than at primary level. To be successful, any extension of school time should be supported by both parents and staff. It should also be noted that more extreme increases may have diminishing effects if engagement of pupils is reduced.

While the impact on academic attainment is, on average, positive, the cost of extending school times might mean that it is not a cost-effective approach to implement at the school level without additional funding.

Behind the average

Global

More studies have been undertaken in primary schools. Effects are higher for primary (about an additional three months’ progress) than secondary (about an additional two months’ progress) schools.

Global

Most of the evidence relates to literacy and mathematics with similar effects in both subjects.

Global

More intensive approaches in extended time, such as one-on-one, appear to be more effective than either small group or large group teaching.

Local

Most studies have been conducted in the United States of America (USA) – this could pose a risk to the transferability of findings as impacts may be influenced by the average length of regular education in any given context.

Closing the disadvantage gap

Local

There is some evidence to suggest that disadvantaged pupils might benefit more from additional time at school.

To increase the likelihood of additional school time benefiting disadvantaged pupils, school leaders should consider how to secure engagement and attendance among those from disadvantaged backgrounds. It is possible that if targeted tuition or enrichment activities are offered universally, those who could benefit the most would be the least likely to participate or engage. However, adopting a more targeted approach also has its challenges, as selected pupils may feel singled out and stigmatized.

Additional non-academic activities may also provide free or low-cost alternatives to sport, music and other enrichment activities that more advantaged families are more likely to be able to pay for outside of school.

How could you implement it in your setting?

Global

The theory behind extending school time is that extra hours of allocated learning mean that pupils have more exposure to teaching, more time to engage with content and a greater amount of learning overall. When implementing approaches that extend school time it is important to acknowledge that allocated learning time and actual learning time are not the same thing. Schools should:

- carefully monitor attendance to ensure that extensions to the school day or term do not lead to reductions in overall learning time for some pupils;

- carefully consider and monitor pupil engagement – if more time is spent managing pupil behaviour in a longer school day then engaged learning time may not increase;

- monitor staff well-being and workload to ensure that additional teaching time does not reduce quality (e.g. through less time for professional development or planning lessons).

Extending school time is likely to require a significant reconfiguration of working patterns for staff, especially if this involves an altered school calendar. It is important that school leaders are clear regarding the purpose of introducing additional learning time and secure parental support prior to making changes.

Global

Approaches to extending school time are likely to be spread over an academic year. Some schools may also decide to target additional support at specific classes or pupils during particular school terms or times of the year.

Local

If additional teachers are not hired to cover the increase in teaching time that comes from extending school time, any increases to school calendars or timetables may also require a large amount of staff time, compared with other approaches.

In addition to time and cost, school leaders should consider how to ensure the quality of teaching during additional school time and avoid approaches that could increase teacher workload without making significant impacts on pupil learning.

When introducing new approaches, schools should consider implementation. For more information see Putting Evidence to Work – A School’s Guide to Implementation.

What does it cost?

Local

Overall, costs are estimated as moderate. The basic cost of teaching a pupil is about £3,120 a year (£16 per day) in primary school and about £4,680 a year (£25 per day) in secondary school. Extending the school year by two weeks would therefore require about £160 per pupil per year for primary schools and about £250 per pupil per year for secondary schools. Estimates suggest that after school clubs cost, on average, £7 per session per pupil. A weekly session would therefore cost £273 per pupil over the course of a thirty-nine-week school year.

How secure is the evidence?

Global

The security of the evidence around extending school time is rated as moderate. Seventy-four studies that meet the inclusion criteria of the Toolkit were identified. Overall, the topic lost an additional padlock because a large percentage of the studies are not randomized controlled trials. While other study designs still give important information about the effectiveness of the approaches, there is a risk that results are influenced by unknown factors that are not part of the intervention.

As with any evidence review, the Toolkit summarizes the average impact of approaches when researched in academic studies. It is important to consider context and apply professional judgement when implementing an approach.

Next steps

Local

Putting Evidence to Work – A School’s Guide to Implementation.

Strand title

Feedback

Update date

4 June 2021

Number of studies

155

Summary

High impact for very low cost, based on extensive evidence

Cost

Very low

Padlocks

4

Impact

+6 months

What is it?

Global

Feedback is information given to the learner about their performance relative to learning goals or outcomes. It should aim to produce (and be capable of producing) improvement in students’ learning.

Feedback redirects or refocuses the learner’s actions to achieve a goal by aligning effort and activity with an outcome. It can be about the output or outcome of the task, the process of the task, the student’s management of their learning or self-regulation, or about them as individuals (which tends to be the least effective).

Feedback can be verbal or written or can be given via tests or digital technology. It can come from a teacher or someone taking a teaching role, or from peers (see Peer tutoring).

Key findings

Global

Providing feedback is well-evidenced and has a high impact on learning outcomes. Effective feedback tends to focus on the task, subject and self-regulation strategies: it provides specific information to the student on how to improve.

Global

Feedback can be effective during, immediately after and some time after learning. Feedback policies should not over specify the frequency of feedback.

Global

Feedback can come from a variety of sources – studies have shown positive effects of feedback from teachers and peers. Feedback delivered by digital technology also has positive effects (albeit slightly lower than the overall average).

Global

Different methods of feedback delivery can be effective and feedback should not be limited exclusively to written marking. Studies of verbal feedback show slightly higher impacts overall an additional seven months’ progress). Written marking may form one part of an effective feedback strategy but it is crucial to monitor impacts on staff workload.

Global

It is important to give feedback when work is correct – not just when it is incorrect. High-quality feedback may focus on a task, subject or self-regulation strategies.

How effective is the approach?

Global

Feedback studies tend to show high effects on learning. However, there are a wide range of effects and some studies show that feedback can have negative effects and even make things worse.

There are positive impacts from a wide range of feedback approaches – including when feedback is delivered by technology or peers. Impacts are greatest when feedback is delivered by teachers. It is particularly important to provide feedback when work is correct, rather than just using it to identify errors.

Many studies of feedback also include other practices. For example, mastery learning approaches combine feedback with additional support for pupils who are falling behind, while approaches like formative assessment also include work to understand specific gaps in learning that need to be addressed and how the teacher wants the pupil to progress.

Feedback has effects across all age groups. Research in schools has focused particularly on its impact on English, mathematics and, to a lesser extent, science.

Local

Embedding formative assessment explicitly can be a key component in laying the foundations for effective feedback. The EEF has trialled ‘Embedding Formative Assessment’ in English schools and found a positive impact, on average.

Behind the average

Global

Feedback appears to have slightly greater effects for primary school pupils (about an additional seven months’ progress) than for secondary school pupils (about an additional five months’ progress).

Global

Effects are high across all curriculum subjects, with slightly higher effects in mathematics and science.

Global

Low-attaining pupils tend to benefit more from explicit feedback than high attainers.

Global

Although some studies have successfully demonstrated the benefits of digital feedback, effects are typically slightly smaller (about an additional four months’ progress).

Closing the disadvantage gap

Local

There is evidence to suggest that feedback involving metacognitive and self-regulatory approaches may have a greater impact on disadvantaged pupils and lower prior attainers than other pupils. Pupils require clear and actionable feedback to employ metacognitive strategies as they learn, as this information informs their understanding of their specific strengths and areas for improvement, thereby indicating which learning strategies have been effective for them in previously completed work.

Applications and Approaches

Name

Written feedback

Impact

5 months

Number of studies

104

Summary

Written feedback typically involves both marks or grades and comments. It is generally given to pupils after they have completed a task and is usually intended for them to read on their own.

The impact of written feedback is typically a little lower than the overall impact. Average progress is five months.

This impact includes all forms of written feedback. The evidence for specific approaches such as ‘triple marking’ is much more limited.

It is especially important that schools monitor teachers’ workload in the use of written feedback. Given it is not clear when feedback can be most effective, feedback policies should not over specify the timing of feedback.

Applications and Approaches

Name

Oral feedback

Impact

7 months

Number of studies

67

Summary

Oral feedback typically involves spoken comments from the teacher, either to an individual, group or class. It tends to be more immediate than written feedback and is usually given either during or at the conclusion of a task or activity.

The impact of oral feedback is higher, on average, than the impact of feedback overall. Average progress is seven months. Whilst recognizing the potential benefits of oral feedback, this finding should not supplant the necessity to consider the principles that underpin the teacher feedback to improve pupil learning guidance report.

While oral feedback has a slightly higher positive effect on average, most schools will want to use a range of methods for providing feedback and it is important to focus on quality within each medium.

How could you implement it in your setting?

Global

Feedback may have a positive impact by: supporting pupils to focus future learning on areas of weakness; identifying and explaining misconceptions; supporting them in taking greater responsibility for their own improvement; or increasing pupils’ motivation to improve.

Implementing feedback successfully will require:
- communication with pupils, teachers and parents/caregivers about practices and expectations that relate to feedback policies;
- assessment of pupil understanding to ascertain what needs to be improved;
- consideration of the 'opportunity cost' associated with different feedback practices;
- ensuring that feedback can be acted upon, for example, by including specific information regarding what a pupil has done successfully or not, and an explanation as to why;
- careful consideration of how feedback will be received, including impacts on self-confidence and motivation;
- providing opportunities for pupils to act upon the feedback after it has been given;
- evaluation of how effective the feedback has been.

Global

Feedback interventions vary in length. Some are short, targeted approaches that address pupil misconceptions within weeks or even days. Others are used as more extended methods of tracking and supporting pupil progress over many months.

Local

The average cost of feedback and feedback interventions is very low. The cost to schools is largely in training.

Implementing feedback and feedback interventions will also require a moderate and sustained amount of staff time, compared with other approaches.

In addition to time and cost, school leaders should consider how to maximize teacher professional development in supporting them to deliver effective feedback and avoid approaches that increase teacher workload without providing pupils with the necessary information to improve performance.

When introducing new approaches, schools should consider implementation. For more information see Putting Evidence to Work – A School’s Guide to Implementation.

Relevant EEF studies

Embedding Formative Assessment

Anglican Schools Partnership

How secure is the evidence?

Global

The security of the evidence around feedback is rated as high. One hundred and fifty-five studies that meet the inclusion criteria of the Toolkit were identified. The topic lost a padlock because a large percentage of the studies are not randomized controlled trials. While other study designs still give important information about the effectiveness of approaches, there is a risk that the results are influenced by unknown factors that are not part of the intervention.

As with any evidence review, the Toolkit summarizes the average impact of approaches as researched in academic studies. It is important to consider context and apply professional judgement when implementing an approach.

FEEDBACK & THEORY OF CHANGE

Evidence suggests that disadvantaged pupils and low prior attainers can benefit more from meta-cognitive interventions than the average pupil.

In order for pupils to employ metacognitive processes and thinking to their learning, they need clear and accurate feedback on their strengths and areas of improvement to know what actions and practices to repeat, and what to do differently.

Therefore, feedback that is clear and accurately reflects pupils’ strengths and weaknesses may therefore bring about larger improvements for disadvantaged pupils and/or low prior attainers.

Strand title

Mastery learning

Update date

24 June 2021

Number of studies

80

Summary

Moderate impact for very low cost, based on very limited evidence

Cost

Very low cost

Padlocks

2

Impact

+5 months

What is it?

Global

Mastery learning was originally developed in the 1960s. According to an early definition of mastery learning, learning outcomes are kept constant but the time needed for pupils to become proficient or competent in these objectives varies.

Subject matter is broken into blocks or units with predetermined objectives and specified outcomes. Learners must demonstrate mastery on unit tests, typically 80 per cent, before moving on to new material. Pupils who do not achieve mastery are provided with extra support through a range of teaching strategies such as more intensive teaching, tutoring, peer-assisted learning, small group discussions or additional homework. Learners continue the cycle of studying and testing until the mastery criteria are met.

More recent mastery approaches do not always have all the characteristics of mastery learning. Some approaches without a threshold typically involve the class moving on to new material when the teacher decides that the majority of pupils have mastered the unit. Curriculum time varies according to the progress of the class. In other approaches, pupils are required to demonstrate mastery on a test to progress to new material, but there is not a specified threshold of at least 80 per cent.

Local

Mastery learning should be distinguished from a related approach which is sometimes known as ‘teaching for mastery’. This term is often used to describe the approach to mathematics teaching found in high-performing places in East Asia, such as Shanghai and Singapore. Like mastery learning, teaching for mastery aims to support all pupils to achieve deep understanding and competence in the relevant topic. However, teaching for mastery is characterized by teacher-led, whole class teaching; common lesson content for all pupils; and use of manipulatives and representations. Although some aspects of teaching for mastery are informed by research, relatively few interventions of this nature have been evaluated for impact. Most of the studies in this strand should be distinguished from this related approach.

Key findings

Global

Mastery learning is a cost-effective approach, on average, but is challenging to implement effectively. Schools should plan for changes and assess whether the approach is successful within their context.

Global

A high level of success should be required before pupils move on to new content – it is crucial to monitor and communicate pupil progress and to provide additional support for pupils who take longer to reach the required level of knowledge.

Global

Mastery learning approaches are often associated with direct instruction, but many of the high-impact studies identified included elements of collaborative learning.

Global

There is a large variation in the average impact – mastery learning approaches have consistently positive impacts, but effects are higher for primary school pupils and in mathematics.

How effective is the approach?

Global

The impact of mastery learning approaches is an additional five months’ progress, on average, over the course of a year.

There is a lot of variation in this average. It seems to be important that a high bar is set for achievement of ‘mastery’ (usually 80 per cent to 90 per cent on the relevant test). By contrast, the approach appears to be much less effective when pupils work at their own pace (see also Individualized instruction).

Mastery learning also appears to be particularly effective when pupils are given the opportunity to work in groups or teams and take responsibility for supporting each other’s progress (see also Collaborative learning and Peer tutoring).

Local

The EEF evaluation of ‘Maths Mastery’ – an example of the ‘Teaching for Mastery’ approach, found positive impacts overall – but with a slightly lower effect than the average impact for more traditional mastery approaches.

Behind the average

Global

Studies involving primary school pupils have tended to be more effective (about an additional eight months’ progress) than for secondary school pupils (about an additional three months’ progress).

Global

Mastery learning has been used successfully across the curriculum but particularly for reading, mathematics and science. Effects are higher in mathematics and science (about an additional six months’ progress) than reading (about an additional three months’ progress).

Global

A high level of mastery of about 80 per cent is associated with more successful approaches.

Global

Mastery learning approaches that include collaborative learning can be particularly effective.

Closing the disadvantage gap

Local

Mastery learning approaches aim to ensure that all pupils have mastered key concepts before moving on to the next topic – in contrast with traditional teaching methods whereby pupils may be left behind, with gaps of misunderstanding widening. Mastery learning approaches could address these challenges by giving additional time and support to pupils who may have missed learning, or who take longer to master new knowledge and skills.

In order for mastery approaches to be effective for pupils with gaps in understanding, it is crucial that additional support is provided. Approaches that simply build upon foundational knowledge without targeting support for pupils who fall behind are unlikely to narrow disadvantage gaps.

How could you implement it in your setting?

Global

Mastery learning works through designing units of work so that each task has a clear learning outcome, which pupils must master prior to moving on to the next task. Core components of the mastery approach that schools should be careful to implement include:

- effective diagnostic assessment to identify areas of strength and weakness;

- careful sequencing of topics so that they gradually build on foundational knowledge;

- flexibility for teachers on how long they need to spend on any particular topic;

- monitoring of pupil learning and regular feedback so that pupils can master topics prior to moving to the next;

- additional support for pupils who struggle to master topic areas.

Global

Mastery learning interventions are typically delivered over the course of an academic year, as choosing to take longer on a topic or scheme of work requires flexibility in the planning and teaching of curriculum content.

Some schools may decide that certain topics are more suited to a mastery approach than others, and therefore the delivery time could be as short as half a term.

Local

Overall, the median costs of implementing mastery learning approaches are estimated as very low. The costs associated with mastery learning approaches mostly arise from professional development training for teaching staff, which is most commonly a start-up cost for introducing the new approach.

Whilst the average cost estimate for mastery learning is very low, the range in costs of professional development training, and the option to pay for ongoing training and additional staff to provide greater timetable flexibility, mean that costs can range from very low to moderate.

Implementing mastery learning also requires a moderate amount of staff time, compared with other approaches. School leaders should be aware of the extra staff time required and think carefully about other activities they might need to cut back on to provide this additional support.

In addition to time and cost, school leaders should consider how to maximize support for struggling learners and avoid some pupils getting bored or frustrated whilst they wait for others to master content.

When introducing new approaches, schools should consider implementation. For more information see Putting Evidence to Work – A School’s Guide to Implementation.

How secure is the evidence?

Global

The security of the evidence around mastery learning is rated as low. Eighty studies that meet the inclusion criteria of the Toolkit were identified. Overall, the topic lost two additional padlocks because:

- only a small percentage of studies have taken place recently, which might mean that the research is not representative of current practice;

- a large percentage of the studies are not randomized controlled trials. While other study designs still give important information about the effectiveness of approaches, there is a risk that results are influenced by unknown factors that are not part of the intervention.

As with any evidence review, the Toolkit summarizes the average impact of approaches when researched in academic studies. It is important to consider context and apply professional judgement when implementing an approach.

MASTERY LEARNING – THEORY OF CHANGE

On average, disadvantaged pupils are more likely have higher school absence rates

Mastery learning approaches moderate against lost learning by giving pupils with less proficiency in a topic further support and opportunity to address gaps in their understanding.

Mastery learning mitigates against gaps in understanding widening as pupils progress through school, and as a result can help support pupils who may be left behind through other approaches.

Disadvantaged pupils are more likely to be previously low-attaining pupils – meaning they gain less knowledge or proficiency in skills than their more privileged counterparts.

Strand title

Mentoring

Update date

4 June 2021

Number of studies

44

Summary

Low impact for moderate cost, based on moderate evidence

Cost

Moderate

Padlocks

3

Impact

+2 months

What is it?

Global

Mentoring in education involves pairing young people with an older peer or adult volunteer who acts as a positive role model. In general, mentoring aims to build confidence and relationships, develop resilience and character, or raise aspirations, rather than develop specific academic skills or knowledge.

Mentors typically build relationships with young people by meeting with them one-on-one for about an hour a week over a sustained period, either during school, at the end of the school day or at weekends. In some approaches mentors may meet with their mentees in small groups.

Activities vary between different mentoring programmes. While some include academic support with homework or other school tasks, approaches focused primarily on direct academic support (sometimes referred to as ‘academic mentoring’) are not covered in this strand. See One-on-one tuition and Peer tutoring.

Mentoring has increasingly been offered to young people who are deemed to be hard to reach or who are at risk of educational failure or exclusion.

Key findings

Global

The impact of mentoring varies but, on average, it is likely to have a small positive impact on attainment.

Global

Positive effects on attainment tend not to be sustained once the mentoring stops, so care must be taken to ensure that benefits are not lost. It is important to consider how pupils who have benefited from mentoring can be supported to retain positive changes in their confidence and behaviour.

Global

Both community-based and school-based approaches can be successful.

Global

Mentor drop-out can have detrimental effects on mentees. It is important to consider how to support mentors.

How effective is the approach?

Global

On average, mentoring appears to have a small but positive impact on academic outcomes. The impacts of individual programmes vary. Some studies have found more positive impacts for pupils from disadvantaged backgrounds, and for non-academic outcomes such as attitudes to school, attendance and behaviour.

There are risks associated with unsuccessful mentor pairings, which may have a detrimental effect on the mentee, and some studies report negative overall impacts.

Programmes that have a clear structure and expectations, provide training and support for mentors, and recruit mentors who are volunteers, are associated with more successful outcomes.

There is no evidence that approaches with a single focus on improving academic attainment or performance are more effective; programmes with multiple objectives can be equally or more effective.

Behind the average

Global

Studies have been undertaken in both primary and secondary school settings with similar impacts.

Global

Overall impact on mathematics and general school subjects tends to be higher than on reading or science outcomes.

Global

Regular meetings of once a week or more frequently appear to be most effective.

Closing the disadvantage gap

Local

While mentoring is not generally as effective in raising attainment outcomes as small group or one-on-one tuition, it is possible to target the approach to pupils from disadvantaged backgrounds and those with particular needs. Some evidence suggests that some pupils from disadvantaged backgrounds show low engagement with or have low expectations of schooling. Mentoring interventions may be more beneficial for these pupils, as the development of trusting relationships with an adult or older peer can provide a different source of support.

Mentors dropping out of programmes can result in detrimental effects for pupils, particularly for those who may have already experienced disillusionment at their perceived lack of support from teachers and school. Therefore, additional care should be taken in the recruitment of reliable mentors when interventions are being used to support disadvantaged pupils.

How could you implement it in your setting?

Global

Mentoring requires close interaction between an adult or older peer and one pupil or a small group of pupils. Conversations between mentors and mentees may address but would not be limited to: attitudes to school; specific academic skills or knowledge; self-perception and belief, particularly in relation to schoolwork; and aspirations for future studies and career options. It is important to consider what support mentors might require to effectively deliver mentoring.

Mentoring interactions normally occur one-on-one between mentor and one mentee – although mentors can mentor multiple pupils. Some mentoring approaches also include small group interactions.

Global

Mentoring interventions are typically delivered over an extended period of time (often at least the length of a school year) in order to allow mentors and mentees to develop more lasting and trusting relationships. Frequent regular meetings of once a week or more tend to be more beneficial.

Local

The average cost of a mentoring intervention is moderate. The cost to schools largely involves mentor training, salary costs (for non-volunteer mentors) and resources. Some programmes also include continuous training and support for mentors which may increase costs.

Compared with other approaches, implementing mentoring interventions requires a moderate and sustained amount of staff time.

In addition to time and cost, school leaders should consider how to maximize the recruitment of effective and reliable mentors who are well matched to mentees. Consideration should also be given to how any gains made in pupil confidence, resilience or aspiration are to be maintained after the intended period of mentoring, as studies show these changes can be difficult to sustain.

When introducing new approaches, schools should consider implementation. For more information see Putting Evidence to Work – A School’s Guide to Implementation.

NOTES – MENTORING THEORY OF CHANGE

Strand title

Metacognition and self-regulation

Update date

29 June 2021

Number of studies

246

Summary

+7 months’ impact for very low cost, based on extensive evidence

Cost

Very low

Padlocks

4

Impact

+7 months

What is it?

Global

Metacognition and self-regulation approaches to teaching support pupils to think about their own learning more explicitly, often by teaching them specific strategies for planning, monitoring and evaluating their learning.

Interventions are usually designed to give pupils a repertoire of strategies to choose from and the skills to select the most suitable strategy for a given learning task.

Self-regulated learning can be broken into three essential components:

- cognition – the mental process involved in knowing, understanding and learning;

- metacognition – often defined as ‘learning to learn’; and

- motivation – willingness to engage our metacognitive and cognitive skills.

Key findings

Global

The potential impact of metacognition and self-regulation approaches is high (more than seven months of additional progress), although it can be difficult to realize this impact in practice as such methods require pupils to take greater responsibility for their learning and develop their understanding of what is required to succeed.

Global

The evidence indicates that explicitly teaching strategies to help plan, monitor and evaluate specific aspects of students’ learning can be effective.

Global

These approaches are more effective when they are applied to challenging tasks rooted in standard curriculum content.

Global

Teachers can demonstrate effective use of metacognitive and self-regulatory strategies by modelling their own thought processes. For example, teachers might explain their thinking when interpreting a text or solving a mathematical task, alongside promoting and developing metacognitive talk related to lesson objectives.

Local

Professional development can be used to develop a mental model of metacognition and self-regulation, alongside an understanding of teaching metacognitive strategies.

How effective is the approach?

Global

The average impact of metacognition and self-regulation strategies is an additional seven months’ progress over the course of a year.

Metacognition and self-regulation strategies can be effective when taught in collaborative groups so that learners can support each other and make their thinking explicit through discussion.

Behind the average

Global

Studies involving primary school pupils have typically been more effective (about an additional eight months’ progress) than those involving secondary school pupils (about an additional seven months’ progress).

Global

Metacognitive and self-regulation strategies have been used across curricula, with approaches in mathematics and science particularly successful.

Global

Studies that use digital technology, for example, intelligent tutoring systems that scaffold learning, have particularly high impacts on pupil outcomes.

Closing the disadvantage gap

Local

There is some evidence to suggest that disadvantaged pupils are less likely to use metacognitive and self-regulatory strategies without being explicitly taught them. Explicit teaching of metacognitive and self-regulatory strategies could therefore encourage such pupils to practise and use these skills more frequently in the future. With explicit teaching and feedback, pupils are more likely to use these strategies independently and habitually, enabling them to manage their own learning and overcome challenges on their own in the future.

How could you implement it in your setting?

Global

Self-regulation and metacognition strategies work through learners monitoring and evaluating their own learning strategies. Some necessary components for successful metacognitive strategies might include:

- explicit teaching of metacognitive strategies;

- teachers modelling their own thinking to demonstrate metacognitive strategies;

- opportunities for pupils to reflect on and monitor their strengths and areas for improvement, and plan how to overcome current difficulties;

- providing enough challenges for learners to develop effective strategies, but not so difficult that they struggle to apply them.

Global

Metacognition and self-regulation strategies are most effective when they are embedded in a school’s curriculum and a specific subject lesson. For example, teaching metacognitive strategies to self-evaluate an essay in history will be different for a pupil evaluating their methods for mathematical problem-solving.

When introducing new approaches, schools should consider implementation. For more information see Putting Evidence to Work – A School’s Guide to Implementation.

Local

Overall, the median costs of implementing metacognition and self-regulation strategies are very low. The costs associated with metacognition and self-regulation are mostly in professional development training for staff, which is most commonly a start-up cost to embed the approach into the school’s curriculum.

Whilst the median cost estimate for metacognition and self-regulation strategies is very low, the variation in the cost of professional development training, and the option to purchase additional materials and provide ongoing training and support, means that costs can range from very low to low. Evidence suggests that the effectiveness of metacognition and self-regulation strategies is influenced by teachers’ understanding of how to develop pupils’ metacognitive knowledge.

These cost estimates assume that schools are already paying for staff salaries, materials and equipment for teaching, and facilities to host lessons. These are all prerequisite costs of implementing metacognition and self-regulation strategies, without which the cost is likely to be higher.

Implementing metacognition and self-regulation strategies also requires a small amount of staff time, compared with other approaches, as staff need to develop their own understanding of metacognitive and self-regulatory processes to model effective use of these strategies and skills to pupils.

In addition to time and cost, school leaders should consider how to maximize explicit teaching of metacognitive strategies by supporting teachers to use these approaches in their practice. At the same time, school leaders should be careful to avoid alienating teachers who do not feel confident in their knowledge or implementation of these strategies.

How secure is the evidence?

Global

The security of the evidence around metacognition and self-regulation strategies is rated as high. Two hundred and forty-six studies were identified. The topic lost a padlock because a large percentage of the studies were not independently evaluated. Evaluations conducted by organizations connected with the approach, for example, commercial providers, typically have larger impacts, which may influence the overall impact of the strand.

As with any evidence review, the Toolkit summarizes the average impact of approaches when researched in academic studies. It is important to consider context and apply professional judgement when implementing an approach.

METACOGNITION AND SELF-REGULATION – THEORY OF CHANGE

There is some evidence to suggest that disadvantaged pupils are less likely to use metacognitive and self-regulatory strategies.

Explicit teaching of metacognitive strategies should help learners who are less likely to practise/use these skills to use them more frequently in the future.

Disadvantaged pupils may develop these skills and start to use them independently and out of habit. This will help them overcome challenges in the future.

Strand title

One-on-one tuition

Update date

27 May 2021

Number of studies

123

Summary

Moderate impact for moderate cost, based on moderate evidence

Cost

Moderate

Padlocks

3

Impact

+5 months

What is it?

Global

One-on-one tuition involves a teacher, teaching assistant (TA) or other adult providing intensive individual support to a pupil. It may happen outside of normal lessons as additional teaching, for example, as part of extending school time or a summer school, or as a replacement for other lessons.

Key findings

Global

On average, one-on-one tuition is very effective at improving pupil outcomes. It might be an effective strategy for providing targeted support for pupils who have low prior attainment or are struggling in particular areas.

Global

Tuition is more likely to make an impact if it is additional to and explicitly linked with normal lessons.

Global

One-on-one tuition can be expensive to deliver, particularly when it is delivered by teachers. Approaches that deliver either instruction through TAs or in small groups rather than one-on-one have smaller positive effects, on average, but may be a cost-effective solution to providing targeted support.

Global

For one-on-one tuition led by TAs, interventions are likely to be particularly beneficial when the TAs are experienced, well-trained and supported, for example, delivering a structured intervention.

How effective is the approach?

Global

Evidence indicates that one-on-one tuition can be effective, providing approximately five additional months’ progress on average

Short, regular sessions (about thirty minutes, three to five times a week) over a set period of time (up to ten weeks) appear to have optimum impact. Evidence also suggests tuition should be additional to, but explicitly linked with, normal teaching, and that teachers should monitor progress to ensure the tutoring is beneficial. Studies comparing one-on-one with small group tuition show mixed results. In some cases one-on-one tuition has led to greater improvement, while in others tuition in groups of two or three has been equally or even more effective. The variability in findings may suggest the importance of the particular type or quality of teaching enabled by very small groups, rather than the precise size of the group.

Programmes involving TAs or volunteers can have a valuable impact, but may be less effective than those using experienced and specifically trained teachers. Where tuition is delivered by volunteers or TAs, training and the use of a structured programme is advisable.

Behind the average

Global

Studies undertaken in primary schools tend to show greater impact (about an additional six months’ progress) than those undertaken in secondary schools (about an additional four months’ progress).

Global

Effects in mathematics appear to be substantially lower (about an additional two months’ progress) than in literacy (about an additional six months’ progress).

Global

Short, regular sessions (about thirty minutes, three to five times a week) over a set period of time (up to ten weeks) appear to result in optimum impact.

Global

Studies involving digital technology show broadly similar effects.

Local

Studies have been undertaken in seven countries around the world with broadly similar effects.

Closing the disadvantage gap

Local

Studies in England have shown that pupils eligible for free school meals typically receive additional benefits from one-on-one tuition. Low-attaining pupils are particularly likely to benefit.

One-on-one tuition approaches can enable pupils to make effective progress by providing intensive, targeted academic support to those identified as having low prior attainment or who are at risk of falling behind. The approach allows the teacher or tutor to focus exclusively on the needs of the learner and provide teaching that is closely matched to each pupil’s understanding. One-on-one tuition offers greater levels of interaction and feedback compared to whole class teaching and can support pupils in spending more time on new or unfamiliar material, overcome barriers to learning, and increase their progress through the curriculum.

How could you implement in your setting?

Global

One-on-one tuition provides additional support that is targeted at a pupil’s specific needs. Reducing the ratio of pupils to teacher allows for closer interaction between educators and pupils. When adopting one-on-one tuition, schools should consider how to ensure that these active ingredients have a positive impact by:

- accurately identifying the pupils who require additional support;

- understanding the learning gaps of the pupils who receive tuition and using this knowledge to select curriculum content appropriately;

- ensuring that teachers are well prepared for high-quality interactions with pupils, such as providing well-planned feedback;

- ensuring that tuition is well linked to classroom content and allowing time for the teacher and tutor to discuss the tuition;

- monitoring the impact of tuition on pupil progress and adjusting provision accordingly.

Global

One-on-one tuition may be delivered by teachers, trained TAs, academic mentors or tutors. Interventions are typically delivered over an extended period, often over the course of several weeks or a term.

Local

The average cost of one-on-one tuition is moderate. The costs to schools are largely in additional salary costs and learning resources, the majority of which are recurring costs. Through the UK’s National Tutoring Programme Year 1 (2020–21), schools could purchase subsidized in-person or online one-on-one sessions in fifteen-hour blocks for an average cost of £167 to £180 per pupil. Costs are lower for online delivery compared to in-person tuition and are higher when provided by qualified or specialist teachers.

When delivering teacher or TA-led small group tuition, implementation is likely to require a large amount of staff time compared with whole class approaches. Given the lower costs, small group tuition may be a sensible approach to trial before considering one-on-one tuition. See Small group tuition.

In addition to time and cost, school leaders should consider using providers with a track record of effectiveness. To increase the impact of school-led one-on-one tuition, school leaders might consider professional development for teachers, TAs and tutors to support high-quality teaching in areas such as formative assessment, curriculum knowledge, instruction and feedback, which will build capacity in schools.

When introducing new approaches, schools should consider implementation. For more information see Putting Evidence to Work – A School’s Guide to Implementation.

How secure is the evidence?

Global

The security of the evidence around one-on-one tuition is rated as moderate. One hundred and twenty-three studies that meet the inclusion criteria for the Toolkit were identified. The topic lost padlocks because:

- a large percentage of the studies were not independently evaluated. Evaluations conducted by organizations connected with the approach, for example, commercial providers, typically have larger impacts, which may influence the overall impact of the strand;

- there is a large amount of unexplained variation between the results included in the topic. All reviews contain some variation in results, which is why it is important to look behind the average. Unexplained variation (or heterogeneity) reduces certainty in the results in ways that we have been unable to test by looking at how context, methodology or approach is influencing impact.

As with any evidence review, the Toolkit summarizes the average impact of approaches as researched in academic studies. It is important to consider context and apply professional judgement when implementing an approach.

Strand title

Peer tutoring

Update date

9 June 2021

Number of studies

127

Summary

Moderate impact for very low cost, based on extensive evidence

Cost

Very low

Padlocks

4

Impact

+5 months

What is it?

Global

Peer tutoring includes a range of approaches in which learners work in pairs or small groups to provide each other with explicit teaching support, such as:

- fixed-role, cross-ability tutoring in which one learner, who is often older, takes the tutoring role and is paired with a tutee or tutees, who are often younger;

- reciprocal-role tutoring, in which learners alternate between the roles of tutor and tutee.

The common characteristic is that learners take on responsibility for aspects of teaching and for evaluating their success.

Key findings

Global

Peer tutoring, on average, has a positive impact on both tutors and tutees and may be a cost-effective approach for delivering one-on-one or small group tuition in a school.

Global

Peer tutoring seems to be most effective when it is used to review or consolidate learning, rather than introduce new material.

Global

Training for staff and tutors is essential for success. It is crucial to allocate sufficient time to train both staff and tutors, to ensure training provides structure to the tutoring, and to identify and implement improvements as the programme progresses.

Global

Four- to ten-week intensive blocks with regular sessions (four to five times a week) appear to provide maximum impact for both tutors and tutees.

How effective is the approach?

Global

Peer tutoring approaches have been shown to have a positive impact on learning, with an average positive effect equivalent to approximately five additional months’ progress within one academic year. Studies have identified benefits for both tutors and tutees, and for a wide range of age groups. Though all types of pupils appear to benefit from peer tutoring, there is some evidence that pupils who are low-attaining and those with special educational needs make the biggest gains.

Peer tutoring appears to be particularly effective when pupils are provided with support to ensure that the quality of peer interaction is high, for example, questioning frames to use in tutoring sessions, and training and feedback for tutors. In cross-age peer tutoring some studies have found that a gap of less than three years is optimal, although ensuring that the gap is wide enough so that the work is challenging to the tutee whilst easy enough for the tutor to support them is key. Regular tutoring sessions (four to five times a week) of up to ten weeks appear to be more effective than less intensive or longer programmes.

Successful approaches may also have other benefits, such as supporting the social and personal development of pupils and boosting their self-confidence and motivation for learning.

Behind the average

Global

Effects are similar (about an additional five months’ progress) for both primary and secondary pupils.

Global

Impact is similar (about an additional five months’ progress) for both literacy and mathematics.

Global

Lower-attaining pupils tend to benefit more (about an additional six months’ progress) than higher-attaining pupils.

Global

A number of studies involving digital technology have been undertaken, with similar overall impact.

Closing the disadvantage gap

Local

While there is limited evidence that specifically examines pupils from disadvantaged backgrounds, studies have shown that pupils who are low-attaining typically receive additional benefits from peer tutoring. Peer-led tutoring approaches may help pupils to close gaps in their learning by offering targeted, peer-led support to consolidate within-class learning, practise skills, and identify and overcome misconceptions. There is also some evidence to suggest that peer-led tutoring can offer tutors the chance to revisit and revise skills and prior knowledge, and develop metacognitive understanding of topics.

Name

Peer tutoring: tutors

Impact

+6 months

Number of studies

12

Summary

In peer tutoring pupils are taught by other pupils, of the same age or sometimes older. This section focuses on the academic impact of delivering tuition on the tutors themselves.

Some schools are concerned that the tutors may not benefit and may be losing learning time. However, the impact of peer tutoring on tutors is typically slightly higher than the overall impact of six months’ additional progress, on average.

The evidence base for this is weaker than the overall evidence for peer tutoring, as only twelve of the 127 studies examined the impact of the approach on tutors.

Applications and Approaches

Name

Reciprocal tutoring

Impact

+5 months

Number of studies

43

Summary

In reciprocal tutoring pupils take turns to be the tutor and the tutee, usually in the same session. Each pupil experiences being taught by a peer and being the tutor.

The impact of reciprocal peer tutoring is typically about the same as the overall effect. The average months’ progress is five.

How could you implement it in your setting?

Global

Peer tutoring relies on close interaction between two or more students with learners taking responsibility for aspects of teaching and for evaluating their success. When implementing peer tutoring approaches, schools should consider how to ensure high-quality interactions between pupils. This might include:

- carefully structuring tasks so sessions focus on existing knowledge;

- training peer tutors on teaching approaches, such as modelling knowledge, overcoming common misconceptions, feedback and evaluating progress;

- carefully considering appropriate pairing of tutors and tutees;

- providing teaching aids and learning frames to guide tutors on how to structure learning, or the types of questions to ask tutees.

Global

Peer tutoring interventions are typically delivered over four- to ten-week intensive blocks. Approaches may involve cross-age or same-age tutoring, usually in pairs. Approaches may be based on a fixed tutee–tutor relationship, while others may be reciprocal.

Local

The average cost of peer tutoring is expected to be very low. The cost to schools is largely in teacher training and learning resources. Implementing peer tutoring also requires a moderate amount of staff time, compared with other approaches.

In addition to time and cost, school leaders should consider how to maximize the quality of peer tutoring interactions and ensure sufficient time is allocated to identify and implement improvements to approaches. When utilizing programmes, school leaders should assess the quality and strength of evidence behind them.

When introducing new approaches, schools should consider implementation. For more information see Putting Evidence to Work – A School’s Guide to Implementation.

How secure is the evidence?

Global

The security of the evidence around peer tutoring is rated as high. One hundred and twenty-seven studies that meet the inclusion criteria of the Toolkit were identified. The topic lost a padlock because a large percentage of the studies were not independently evaluated. Evaluations conducted by organizations connected with the approach, for example, commercial providers, typically have larger impacts, which may influence the overall impact of the strand.

As with any evidence review, the Toolkit summarizes the average impact of approaches as researched in academic studies. It is important to consider context and apply professional judgement when implementing an approach.

Strand title

Phonics

Update date

10 June 2021

Number of studies

121

Summary

Moderate impact for very low cost, based on extensive evidence

Cost

Very low

Padlocks

5

Impact

+5 months

What is it?

Global

Phonics is an approach to teaching some aspects of literacy by developing pupils’ knowledge and understanding of the relationship between written symbols and sounds. It involves hearing, and identifying and using sound patterns or phonemes to read written language. The aim is to systematically teach pupils the relationship between these sounds and the written spelling patterns, or graphemes, which represent them. Phonics emphasizes the skills of decoding new words by sounding them out and combining or ‘blending’ the sound-spelling patterns.

Key findings

Global

Phonics is an important component in the development of early reading skills, particularly for children from disadvantaged backgrounds.

Global

The teaching of phonics should be explicit and systematic to support children in making connections between the sound patterns they hear in words and the way these words are written.

Global

The teaching of phonics should be matched to children’s current level of skill in terms of their phonemic awareness and their knowledge of letter sounds and patterns (graphemes).

Global

Phonics improves the accuracy of a child's reading but not necessarily their comprehension. It is important that children are successful in making progress in all aspects of reading, including comprehension, and the development of vocabulary and spelling, which should also be taught explicitly.

How effective is the approach?

Global

The average impact of the adoption of phonics approaches is about an additional five months’ progress over the course of a year.

Phonics approaches have been consistently found to be effective in supporting younger pupils to master the basics of reading, with an average impact of an additional five months’ progress. Research suggests that phonics is particularly beneficial for younger learners (four- to seven-year-olds) as they begin to read. Teaching phonics is more effective on average than other approaches to early reading (such as whole language or alphabetic approaches), though it should be emphasized that effective phonics techniques are usually embedded in a rich literacy environment for early readers and are only one part of a successful literacy strategy.

While there have been fewer studies examining phonics with older readers, there is evidence that it can be a positive approach. With any reading intervention, careful diagnosis of the difficulties the reader is experiencing, regardless of age, is required. If an older reader is struggling with decoding, phonics approaches will still be appropriate. Where readers are struggling with vocabulary or comprehension, other interventions may be more appropriate.

There is some variation in impact between different phonological approaches. Synthetic phonics approaches have higher impacts, on average, than analytic approaches. Analytic phonics approaches have also been studied less overall (only nine studies). The small number of analogic phonics approaches identified in this review (six studies) have a negative impact on average.

Behind the average

Global

The majority of studies have been conducted in primary schools, though there are a number of successful studies with secondary school pupils with a similar overall impact (about an additional five months’ progress).

Global

Most studies of phonics are of intensive support in small groups and one-on-one with the aim of supporting pupils to catch up with their peers. The effects of one-on-one tend to be a little higher (about an additional five months’ progress) compared with small group interventions (about an additional four months’ progress), but this needs to be offset by the number of pupils who can receive support.

Global

Approaches using digital technology tend to be less successful than those led by a teacher or TA. Studies of intensive support involving TAs show slightly lower overall impact (about an additional four months’ progress) compared to those involving teachers. This indicates the importance of training and support in phonics for interventions led by TAs.

Global

Synthetic phonics approaches have higher impacts, on average, than analytic phonics approaches.

Local

Studies have been conducted internationally (seven countries), mainly in English-speaking countries. Those conducted outside of the USA have typically shown greater impact.

Closing the disadvantage gap

Local

Studies in England have shown that pupils eligible for free school meals typically receive similar or slightly greater benefit from phonics interventions and approaches. This is likely due to the explicit nature of the instruction and the intensive support provided.

It is possible that some disadvantaged pupils may not develop phonological awareness at the same rate as other pupils, having been exposed to fewer words spoken and books read in the home. Targeted phonics interventions may therefore improve decoding skills more quickly for pupils who have experienced these barriers to learning.

How could you implement it in your setting?

Global

Phonics approaches aim to quickly develop pupils’ word recognition and spelling by developing pupils’ ability to hear, identify and manipulate phonemes (the smallest unit of spoken language), and to teach them the relationship between phonemes and the graphemes (written letters or combinations of letters) that represent them. Successfully implementing a phonics might involve:

- using a systematic approach that explicitly teaches pupils a comprehensive set of letter–sound relationships through an organized sequence;

- training staff to ensure they have the necessary linguistic knowledge and understanding;

- carefully monitoring progress to ensure that phonics programmes are responsive and provide extra support where necessary;

- carefully considering any adaptions to systematic programmes that might reduce impact.

Good implementation of phonics programmes will also consider pupils’ wider reading skills and will identify where pupils are struggling with aspects of reading other than decoding that might be targeted through other approaches such as the explicit teaching of reading comprehension strategies.

Global

Where phonics is delivered as an intervention targeted at specific pupils, regular sessions (up to four times a week) of thirty minutes or so over a period of up to twelve weeks appear to be the most successful structure.

Local

Overall, the median costs of implementing a phonics intervention are estimated as very low. The costs associated with teaching phonics arise from the need for specific resources and professional training, the majority of which are initial start-up costs paid during the first year of delivery.

Whilst the median cost estimate for phonics programmes is very low, the range of prices between available programmes and the option to purchase additional ongoing training and support for teaching staff means that costs can range from very low to low. Evidence suggests that the effectiveness of phonics is related to the pupil's stage of reading development, so it is important that teachers have professional development in effective assessment as well as in the use of particular phonics techniques and materials.

These cost estimates assume that schools are already paying for staff salaries to deliver interventions, facilities to host lessons, and basic stationery materials for staff and pupils. These are all pre-requisite costs of implementing a phonics intervention, without which the cost is likely to be higher.

When introducing new approaches, schools should consider implementation. For more information see https://educationendowmentfoundation.org.uk/tools/guidance-reports/a-schools-guide-to-implementation/" Putting Evidence to Work – A School’s Guide to Implementation.

How secure is the evidence?

Global

The security of the evidence around phonics is rated as very high. One hundred and twenty-one studies that meet the inclusion criteria of the Toolkit were identified.

As with any evidence review, the Toolkit summarizes the average impact of approaches as researched in academic studies. It is important to consider context and apply professional judgement when implementing an approach.

NOTES – PHONICS THEORY OF CHANGE

Some disadvantaged pupils may have less advanced decoding skills due to lack of exposure to books and print as young children.

Phonics interventions aim to improve decoding skills, so that pupils read words more fluently and therefore comprehend sentences and whole texts.

Strand title

Reading comprehension strategies

Update date

8 June 2021

Number of studies

141

Summary

High impact for very low cost, based on extensive evidence.

Cost

Very low

Padlocks

4

Impact

+6 months

What is it?

Global

Reading comprehension strategies focus on learners’ understanding of written text. Pupils learn a range of techniques which enable them to comprehend the meaning of what they read. These can include: inferring meaning from context; summarizing or identifying key points; using graphic or semantic organizers; developing questioning strategies; and monitoring their own comprehension and then identifying and resolving difficulties for themselves (see also Metacognition and self-regulation).

Strategies are often taught to a class and then practised in pairs or small groups (see also Collaborative learning).

Key Findings

Global

Reading comprehension strategies have a high impact on average (about an additional six months’ progress). In addition to phonics it is a crucial component of early reading instruction.

Global

It is important to identify the appropriate level of text difficulty, to provide appropriate context to practise the skills, desire to engage with the text and enough of a challenge to improve reading comprehension.

Global

Effective diagnosis of reading difficulties is important in identifying possible solutions, particularly for older struggling readers. Pupils can struggle with decoding words, understanding the structure of the language used, or understanding particular vocabulary, which may be subject-specific.

Global

A wide range of strategies and approaches can be successful, but many pupils need to be taught explicitly and consistently.

Global

It is crucial to support pupils to apply the comprehension strategies independently to other reading tasks, contexts and subjects.

How effective is the approach?

Global

The average impact of reading comprehension strategies is an additional six months’ progress over the course of a year. Successful reading comprehension approaches allow activities to be carefully tailored to pupils’ reading capabilities, and involve activities and texts that provide an effective, but not overwhelming, challenge.

Many of the approaches can be usefully combined with collaborative learning techniques and phonics activities to develop reading skills. Techniques such as graphic organizers and drawing pupils’ attention to text features are likely to be particularly useful when reading expository or information texts.

There are some indications that approaches involving digital technology can be successful in improving reading comprehension (although there are relatively few studies in this area), particularly when they focus on the application and practice of specific strategies and the use of self-questioning skills.

Supporting struggling readers is likely to require a coordinated effort across the curriculum and a combination of approaches that include phonics, reading comprehension and oral language. No particular strategy should be seen as a panacea, and careful diagnosis of the reasons why an individual pupil is struggling should guide the choice of intervention strategies.

Behind the average

Global

More studies have been conducted with primary school pupils, but the teaching of reading comprehension strategies appears effective across both primary (about an additional six months’ progress) and secondary (about an additional seven months’ progress) schools.

Global

Although the main focus is on reading, comprehension strategies have been successfully used in a number of curriculum subjects where it is important to be able to read and understand text.

Global

Lower-attaining pupils in particular appear to benefit from the explicit teaching of strategies to comprehend text.

Global

There are some indications that approaches involving digital technology can be successful in improving reading comprehension, particularly when they focus on the application and practice of specific strategies and the use of self-questioning skills.

Global

Shorter interventions of up to ten weeks tend to be more successful. However, there are some examples of successful longer interventions.

Closing the disadvantage gap

Local

Studies in England have shown that pupils eligible for free school meals may receive additional benefits from being taught how to use reading comprehension strategies. However, the UK evidence base is less extensive than the global average, and UK studies show lower impact for all pupils.

Reading comprehension strategies involve the teaching of explicit approaches and techniques a pupil can use to improve their comprehension of written text. Many learners will develop these approaches without teacher guidance, adopting the strategies through trial and error as they look to better understand texts that challenge them. However, we know that, on average, disadvantaged children are less likely to own a book of their own and read at home with family members, and for these reasons may not acquire the necessary skills to read and understand challenging texts.

How could you implement it in your setting?

Global

Reading comprehension strategies work through a number of different mechanisms – all focused on improving the understanding of meaning of text effectively. Common elements include:

- explicit teaching of strategies;
- teachers questioning pupils to apply key steps;
- summarizing or identifying key points;
- metacognitive talk to model strategies;
- using graphic or semantic organizers;
- using peer and self-questioning strategies to practise the strategies (such as reciprocal questioning);
- pupils monitoring their own comprehension and identifying difficulties themselves.

Global

Reading comprehension strategy interventions are typically delivered between one to three terms of a school year, either by teachers within class settings, or by TAs with smaller groups.

Evidence suggests that reading comprehension approaches need to be tailored to pupils’ current reading capabilities, so it is important that teachers receive professional development in effective diagnosis as well as training in the use of particular techniques and materials.

Local

The average cost of reading comprehension strategies is estimated as very low. The cost to schools is largely in training and professional development, books and learning resources, the majority of which are initial start-up costs paid during the first year of delivery. Whilst the median cost estimate for reading comprehension programmes is very low, the range of prices between available programmes and the option to purchase additional ongoing training and support for teaching staff means that costs can range from very low to low.

Effective teaching of reading comprehension strategies also requires a moderate amount of staff time, compared with other approaches. In addition to time and cost, school leaders should consider how to develop teachers’ ability to use specific techniques for particular pupils’ needs and ensure they use texts that provide an effective challenge to readers.

When introducing new approaches, schools should consider implementation. For more information see Putting Evidence to Work – A School’s Guide to Implementation.

How secure is the evidence?

Global

The security of the evidence around reading comprehension strategies is rated as high. One hundred and forty-one studies that met the inclusion criteria for the Toolkit were identified. The topic lost a padlock because a large percentage of the studies were not independently evaluated. Evaluations conducted by organizations connected with the approach, for example, commercial providers, typically have larger impacts, which may influence the overall impact of the strand.

As with any evidence review, the Toolkit summarizes the average impact of approaches as researched in academic studies. It is important to consider context and apply professional judgement when implementing an approach.

[1] https://educationendowmentfoundation.org.uk/public/files/Toolkit/Toolkit_Manual_2018.pdf

[2] A study of Spanish-speaking students learning mathematics in English would be excluded. A study of Spanish-speaking students learning French in a Spanish medium school would be included.

[3]Specific design features are identified through coding so that these can be investigated as moderators.

[4] This includes other measures of impact such as correlational and categorical effect sizes where these result from a counterfactual comparison and where they can meaningfully be converted to a standardized mean difference (Borenstein et al., 2009).

[5] Sample size is not included in these criteria. This is because we intend to undertake an analysis of the relationship between sample size and effect size based on the existing evidence of an inverse relationship in education (e.g. Slavin and Smith, 2009) and other fields (e.g. Button et al., 2013; Kühberger, Fritz and Scherndl, 2014). This has implications for meta-analysis as methods for publication bias and the use of a random effects model assume sample size and effect size are independent.

[6] For example, by using the conversions available in programs like Comprehensive Meta-Analysis or David B. Wilson’s online conversion tool: https://campbellcollaboration.org/escalc/html/EffectSizeCalculator-Home.php.

REFERENCES

Borenstein M., Hedges L.V., Higgins, J.P.T. and Rothstein, H.R. (2009) ‘Subgroup analyses’, in Introduction to meta-analysis. London: John Wiley & Sons, pp. 59-86.

Button, K.S., Ioannidis, J. P., Mokrysz, C., Nosek, B.A., Flint, J., ... and Munafò, M. R. (2013) ‘Power failure: why small sample size undermines the reliability of neuroscience’, Nature Reviews Neuroscience, 14(5), pp. 365-376.

EEF (2018) Sutton Trust-EEF Teaching and Learning Toolkit & EEF Early Years Toolkit: technical appendix and process manual (working document v.01). London: Education Endowment Foundation.

Higgins, S. (2018) Improving learning: meta-analysis of intervention research in education. Cambridge: Cambridge University Press.

Higgins, S., Katsipataki, M., Villanueva Aguilera, A.B., Dobson, E., ... and Uwimpuhwe, G. (2022) ‘The Teaching and Learning Toolkit: communicating research evidence to inform decision-making for policy and practice in education’, Review of Education. https://doi.org/10.1002/rev3.3327.

Kühberger, A., Fritz, A. and Scherndl, T. (2014) ‘Publication bias in psychology: a diagnosis based on the correlation between effect size and sample size’, PLOS ONE, 9(9). https://doi.org/10.1371/journal.pone.0105825.

Methley, A.M., Campbell, S., Chew-Graham, C., McNally, R. and Cheraghi-Sohi, S. (2014) ‘PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews’, BMC Health Services Research, 14(1). https://doi.org/10.1186/s12913-014-0579-0.

Papaioannou, D., Sutton, A., Carroll, C., Booth, A. and Wong, R. (2010) ‘Literature searching for social science systematic reviews: consideration of a range of search techniques’, Health Information & Libraries Journal, 27(2), pp. 114–122.

Schlosser, R.W., Wendt, O., Bhavnani, S. and Nail-Chiwetalu, B. (2006) ‘Use of information-seeking strategies for developing systematic reviews and engaging in evidence-based practice: the application of traditional and comprehensive pearl growing: a review’, International Journal of Language & Communication Disorders, 41(5), pp. 567–582.

Slavin, R. and Smith, D. (2009) ‘The relationship between sample sizes and effect sizes in systematic reviews in education’, Educational Evaluation and Policy Analysis, 31(4), pp. 500–506.