Cool New Study on Text Difficulty and Adolescent Literacy

  • 02 November, 2019
  • 18 Comments

I don’t do this often, but occasionally a study that catches my eye is particularly pertinent to questions that teachers are asking me.

National surveys suggest that middle and high school teachers are increasingly likely to place kids in texts that are relatively easy to read (Rand, 2017; Thomas Fordham Foundation, 2018); texts that are supposedly at the students’ “instructional levels.”

Teachers ask me all the time how they can be expected to use high school level texts when so few kids in their classes are reading at grade level.

And, yet, high school students often tell me that they hate being placed in what they refer to as the “stupid books.”

That’s where this new study comes in.

This study examined the reading comprehension of 293 ninth graders (Lupo, et al., 2019), who were randomly assigned within classes to easy or challenging versions of the instructional texts.

The students’ reading comprehension on the instructional text was evaluated at the end of each lesson, and general reading comprehension level was tested at the end of the 12-week intervention.

One interesting finding: “Only a small subset of students who read significantly below average, many of whom were identified as English learners, benefited from reading the easier versions.”

In other words, as long as there was instructional support, most of the students were able to make sense of the texts.

The two approaches to instruction support were either Listen-Read-Discuss or KWL; the latter being the most successful of the two in supporting student reading. In other words, neither of these instructional approaches were aimed at providing any kind of targeted support for dealing with the actual variables that were making the challenging texts so difficult (e.g., vocabulary, cohesion, tone).

Nevertheless, except for a small number of particularly struggling second language students, shifting to easier text was not beneficial in terms of increasing student understanding of the instructional texts. Which means there is no good reason, for most students or situations, to shift older students to easier texts to facilitate their reading—as long as you are ready to provide instructional support.

That means not using texts that poorly support content standards.

That means not trying to manage multiple text levels.

That means not stigmatizing or isolating the lower readers.

Why do it if there isn’t an instructional benefit.

Interesting finding 2: These students made some learning gains in general comprehension over the 12-weeks of instruction. They made the same amount of gain whether they worked with the easier or harder texts.

In other words, working with texts that were likely closer to the students’ instructional levels provided no learning advantage. This finding matches with the results found in several elementary grade studies (such comparisons either find no learning benefits due to the use of the easier texts, or that the easier texts actually are a detriment to student learning).  

Gosh. I wish the researchers had asked the kids how they felt about their text placements. Experience tells me the ones with the more challenging text will feel more respected.

Another interesting finding: There was no difference in reading comprehension due to text difficult between even most of the low readers.

But what about the small number of particularly low students (mainly second-language learners) who actually did do better with the easier texts?

The study doesn’t do much with this finding, so my thoughts are just speculation.

For example, I wish they would have identified that small group of students to see what happened to their general reading comprehension over the 12 weeks. It seems likely that such an analysis would be spoiled by small sample size, but it might be interesting just to see what happened with these students.

Also, remember, there was no specific instructional support aimed at the linguistic or conceptual factors that may have been consequential in making sense of these texts. KWL focuses on prior knowledge and Listen-Read-Discuss focuses on decoding. I wouldn’t necessarily expect either of those interventions to be particularly helpful for second-language learners.

Again, man, I wish they would have had a vocabulary intervention, or one aimed at “juicy sentences” (thank you, Lily Wong-Fillmore), or cohesion, or text structure. Those kinds of interventions may have been more successful, but even without that, classes clearly were not hindered by teaching students with complex text.

We have so many opinions on the importance of instructional texts for student learning (e.g., Betts, Fountas & Pinnell, Calkins, Richardson), and attempts to reason from irrelevant studies by analogy (Allington)… but there just aren’t that many direct tests of those claims.

Lupo and company have made a valuable contribution, and one that is entirely consistent with past direct tests of the proposition that easier texts facilitate comprehension and learning. I know it’s easier and I know its popular, but putting kids in text below grade level is a bad idea in most cases.

Reference

Lupo, S. M., Tortorelli, L., Invernizzi, M., Ryoo, J. H. (2019). An exploration of text difficulty and knowledge support on adolescents’ comprehension. Reading Research Quarterly, 44(4), 457-479.

Comments

See what others have to say about this topic.

Ann
Nov 02, 2019 04:04 PM

We know that research on high school readers is not directly applicable to primary grades. When you scaffold learning, whether it is learning to read or to ski, the conditions and equipment may be adjusted. The similarity I see is that in both early reading with leveled text and in struggling high school readers, the teacher’s have to teach hard so students can read the text. Books do not teach students to read.
Teachers that hand children ‘grade level’ text because the research said to, will see most students who expect failure, withdraw. “They don’t even try,” “They are unmotivated,” are then teacher responses.
Good teaching creates successful experiences, undermines helplessness, and gives students access.
There is no easy path
-Use leveled texts
-Give all kids OG
-Use grade level texts
-Take away Science and Civics
The teacher has to work hard everyday for every child.
I did just that for 40 years:)

Harriett
Nov 02, 2019 05:07 PM

Tim, you're the first person I thought of when I read this study two weeks ago. Yep--you've been right all along and have certainly influenced my practice. When I started teaching a third grade class once a week last year, I made sure to teach all the students grade-level text. I differentiated the instruction, not the text--and I certainly needed to pull my lowest readers into a small group in order do provide that differentiation.

Jeffrey White
Nov 02, 2019 05:42 PM

Our German program uses authentic and difficult texts with a genre approach to teach German starting in the 4th semester. Many students have reported that the texts feel more real and relevant than texts in the German textbook that's used in earlier semesters. They like the connections to more complex themes. (See Heidi Byrnes' research of the Georgetown German program. Her work inspired what we are doing)

Gail Brown
Nov 02, 2019 10:18 PM

I agree with what you've posted and I think this works for experienced teachers who have effective implementation of KWL within their knowledge and experiences... So, yes, given appropriate instructional support, the text difficulty effects can be reduced with effective teaching support and scaffolding. The problem that I have seen is twofold. Firstly, there are many new, inexperienced teachers out there, and they are not skilled in KWL - they might not even have heard of this before - depends on their teacher training program? Also, from my classroom visits and experiences, broad strategies like KWL are quite variable across teachers. So, again, I'm supporting your conclusion, with a caveat that it's the skills and knowldge of the classroom teacher that will ensure students understand and learn from texts they read, and I think this is independent of text difficulty.

Narelle
Nov 03, 2019 05:04 AM

Volume 54

Katie
Nov 03, 2019 11:52 AM

Teachers definitely need to differentiate their instruction when using only grade level texts, but sometimes it becomes the teacher reading it to them or other students reading it to them. The student then gains the background/content knowledge but they are not gaining the skills to be able to read that text on their own sometime soon and gain that knowledge on their own. It will always take them tons of teacher support to get them to be independent with grade level texts. I have never had a below level student who was only taught with grade level materials be able to access grade level material on their own by the end of the year. But I have certainly had students that were giving instructional level materials at first and built up their reading skills and strategies be able to access grade level materials by the end of the year.

Audrey
Nov 03, 2019 12:40 PM

How much work was the teacher doing? What are we indirectly teaching these students by doing most of the work? What happens when the teacher isn’t there? Is their increased comprehension transferable to other texts or other content?

It seems like this is another example of a push for one way of teaching versus another. Instead, why not embrace the benefits of both instructional practices? Teachers could provide students with on grade level materials during whole group lessons and differentiated support. Then they could provide students with opportunities to read and be instructed on reading strategies to grow them as readers with texts at their instructional level.

When we embrace multiple approaches to teaching reading, we allow for all students to be met exactly where they are and give them access to on-grade level texts.

Tim Shanahan
Nov 03, 2019 01:28 PM

The teachers in this study—or in several others—are not doing all the work. Limiting instructional materials to texts that students can already read reasonably well severely limits what they can learn and, at least for adolescents, cuts them off from text they find intellectually interesting, age-level appropriate, and socially acceptable. Telling kids what the text says or reading the text to the kids, however, is not sound scaffolding and it is not what is done in such studies.

Tim Shanahan

Ann
Nov 03, 2019 05:15 PM

Making Thinking Visible- the work of Project Zero scaffolds thinking in many ways that teachers find useful.

Norka Padilla
Nov 04, 2019 02:09 PM

Excellent article which supports the work that proficient teachers are implementing as social justice for ELs/MLs with the learning they've gained at Standards Institute with UnboundEd.org under the leadership of Amy Rudat. Hannah Turner and Michelle Hunsberger in MCPS, Maryland are equity educators implementing standards aligned curriculum, including juicy sentences that is resulting in undoing systemic and systematic racism of chronic long term ELs- who were only long term because we didn't know how to do this yet. Now we do. "Know better, do better," M. A.
Ensuring we are putting forward accurate research like this is so important. Previous comments also confirm that it's about our proficiency as educators. When we know how, so do our students. Thank you.

Lynn
Nov 04, 2019 05:46 PM

Just a quick note that I think you have a typo on the second line of paragraph 2. "texts that are supposedly as the students’ “instructional levels.” - should be "at" the students' instructional levels.

Tim Shanahan
Nov 04, 2019 11:01 PM

Thanks Lynn

Catherine Whittal-Williams
Nov 06, 2019 11:12 PM


I have STRONG feelings about this. I concur that "dumbing down" literature is a dangerous and slippery slope. Increasingly we are thinning our thinking by offering so much "textual stimulus" that we forget to finish the job with "literature". To me, there's a profound difference. No challenge = no growth. - Cathy Whittal-Williams

Alicia
Nov 07, 2019 07:33 PM

I am all about giving students experience with texts of grade-level complexity with the instructional support they need to be successful with those texts. I did it for twelve years in my own classroom with struggling readers and their Lexile levels grew and many of them were proficient on their state tests after 5 or 6 previous years of not reaching proficiency.

But when you speak of Fountas & Pinnell and those who followed them (Calkins, Richardson) are you sure that you have a complete understanding of their comprehensive design for literacy? You write as though the ONLY texts they suggest a teacher use are ones on students' instructional levels, and that is not true at all. According to Fountas & Pinnell, the ONLY part of a student's day that should involve instructional level text is guided reading, and guided reading should only comprise about twenty minutes of a student's day. In later grades, teachers do not even always meet with every group every day. (See Guided Reading: Responsive Teaching Across the Grades by Fountas & Pinnell for more information.) And I'm not sure we can safely sweep the idea of instructional levels (or learning zones, or zones of proximal development, or whatever your preference) aside as though it has no place in literacy instruction at all. If that is the case, why not cut to the chase and support kindergarteners with reading Moby Dick? It is not the concept of instructional levels that is the problem, but the degree. And Fountas & Pinnell would not advocate for instructional level texts that are "easy books" or "dumb books" or "baby books", either. An instructional level text should be such a stretch that a student could not read it on her own without instructional support. By the secondary level, that would frequently be a grade level text.

I agree with so much of what you write, but whenever it comes to this issue, I feel that you have seen teachers implement the work of Fountas & Pinnell (and others) poorly and incompletely and that you base your assessments of their approach on the poor implementation you have observed.

Tim Shanahan
Nov 08, 2019 02:45 AM

Alicia— I do understand that kids are free to read harder texts anytime when they are NOT being taught to read. But teachers will guide their relatively easy reading, just not the harder reading. That makes no sense.

Do you know that studies show that guided reading when compared with teaching kids with grade level text showed greater learning gains for guided reading?

Tim

Alicia
Nov 08, 2019 03:31 PM

Yes, I'm right there with you. I did use grade-level texts for guided reading with great success, and by far the vast majority of the guided reading groups in my district are working at grade level or above. Those working below grade level are the exception. That's how it should be, and those who find they use instructional levels to stymie student growth rather than promote it are misapplying the work of the researchers you mentioned.

When I last saw you in a conference, you talked about how students needed exposure to many varied texts of varied complexity day after day, and that is exactly what we are doing using Fountas & Pinnell's design for responsive literacy with success.

I assure you, I greatly respect your message for utilizing challenging texts. I just hate to see the work of those other researchers reduced to the concept of teaching with "easy books" because of some educators who have misunderstood the purpose and work of guided reading.

Rhonda Dion
Nov 23, 2019 09:24 PM

Tim, Can you confirm when to use instructional texts?
Thanks!
Rhonda

Sarah Lupo
Nov 23, 2019 10:14 PM

Thanks for sharing our work, Tim. I'm intrigued by your interpretation and others ideas and enjoyed reading this and the comments. I have garnered many ideas for my next study too- I'm looking at science a-z leveled texts and the impact of science knowledge- and hoping to implement a text structure intervention.
A few thoughts:
- We compared two different ways of approaching knowledge: activating (KWLs) and directly building knowledge (Listen-Read-Discuss by Manzo & Casale, 1986). Activating was the winner in this race. Many of the teachers speculated that LRD would help ELs and students who "struggle" because it focused on building knowledge. But it didn't. ELs actually had the strongest benefits for activating knowledge. The important take away was that knowledge was gained through reading- which is the point of reading!
- Most of the teachers in this study were inexperienced and had 0-3 years of teaching experience.
- I did interview students! It didn't make it into this article. In general kids didn't realize that one version was easier or harder- but they did pick up that some had read longer vs. shorter texts as the easier Newsela texts are on average 200 words shorter.
- The differences between EASIER texts (where language is not manipulated) and easier VERSIONS of texts merits a mention. In our study we found that texts manipulated to be easier were disjointed and lacked cohesion (thus not actually easier). So I agree with some points above about offering a range of texts, both easier and more challenging, but I'd question the use of leveled texts as the lower levels can be disjointed and hard to read.
- In terms of a push for an instructional method, this is an experimental study and we had to stick to the methods in order to compare the two methods. I agree that there is room for both in instruction but the finding that activating knowledge over direct building was really interesting- and a surprising finding. One that reminds us to value all students knowledge that they bring to the table.

Thank you all for your thoughts on this work!

What Are your thoughts?

Leave me a comment and I would like to have a discussion with you!

Comment *
Name*
Email*
Website
Comments

Cool New Study on Text Difficulty and Adolescent Literacy

18 comments

One of the world’s premier literacy educators.

He studies reading and writing across all ages and abilities. Feel free to contact him.