Cool New Study on Text Difficulty and Adolescent Literacy

  • adolescent literacy
  • 02 November, 2019
  • 27 Comments

I don’t do this often, but occasionally a study that catches my eye is particularly pertinent to questions that teachers are asking me.

National surveys suggest that middle and high school teachers are increasingly likely to place kids in texts that are relatively easy to read (Rand, 2017; Thomas Fordham Foundation, 2018); texts that are supposedly at the students’ “instructional levels.”

Teachers ask me all the time how they can be expected to use high school level texts when so few kids in their classes are reading at grade level.

And, yet, high school students often tell me that they hate being placed in what they refer to as the “stupid books.”

That’s where this new study comes in.

This study examined the reading comprehension of 293 ninth graders (Lupo, et al., 2019), who were randomly assigned within classes to easy or challenging versions of the instructional texts.

The students’ reading comprehension on the instructional text was evaluated at the end of each lesson, and general reading comprehension level was tested at the end of the 12-week intervention.

One interesting finding: “Only a small subset of students who read significantly below average, many of whom were identified as English learners, benefited from reading the easier versions.”

In other words, as long as there was instructional support, most of the students were able to make sense of the texts.

The two approaches to instruction support were either Listen-Read-Discuss or KWL; the latter being the most successful of the two in supporting student reading. In other words, neither of these instructional approaches were aimed at providing any kind of targeted support for dealing with the actual variables that were making the challenging texts so difficult (e.g., vocabulary, cohesion, tone).

Nevertheless, except for a small number of particularly struggling second language students, shifting to easier text was not beneficial in terms of increasing student understanding of the instructional texts. Which means there is no good reason, for most students or situations, to shift older students to easier texts to facilitate their reading—as long as you are ready to provide instructional support.

That means not using texts that poorly support content standards.

That means not trying to manage multiple text levels.

That means not stigmatizing or isolating the lower readers.

Why do it if there isn’t an instructional benefit.

Interesting finding 2: These students made some learning gains in general comprehension over the 12-weeks of instruction. They made the same amount of gain whether they worked with the easier or harder texts.

In other words, working with texts that were likely closer to the students’ instructional levels provided no learning advantage. This finding matches with the results found in several elementary grade studies (such comparisons either find no learning benefits due to the use of the easier texts, or that the easier texts actually are a detriment to student learning).  

Gosh. I wish the researchers had asked the kids how they felt about their text placements. Experience tells me the ones with the more challenging text will feel more respected.

Another interesting finding: There was no difference in reading comprehension due to text difficult between even most of the low readers.

But what about the small number of particularly low students (mainly second-language learners) who actually did do better with the easier texts?

The study doesn’t do much with this finding, so my thoughts are just speculation.

For example, I wish they would have identified that small group of students to see what happened to their general reading comprehension over the 12 weeks. It seems likely that such an analysis would be spoiled by small sample size, but it might be interesting just to see what happened with these students.

Also, remember, there was no specific instructional support aimed at the linguistic or conceptual factors that may have been consequential in making sense of these texts. KWL focuses on prior knowledge and Listen-Read-Discuss focuses on decoding. I wouldn’t necessarily expect either of those interventions to be particularly helpful for second-language learners.

Again, man, I wish they would have had a vocabulary intervention, or one aimed at “juicy sentences” (thank you, Lily Wong-Fillmore), or cohesion, or text structure. Those kinds of interventions may have been more successful, but even without that, classes clearly were not hindered by teaching students with complex text.

We have so many opinions on the importance of instructional texts for student learning (e.g., Betts, Fountas & Pinnell, Calkins, Richardson), and attempts to reason from irrelevant studies by analogy (Allington)… but there just aren’t that many direct tests of those claims.

Lupo and company have made a valuable contribution, and one that is entirely consistent with past direct tests of the proposition that easier texts facilitate comprehension and learning. I know it’s easier and I know its popular, but putting kids in text below grade level is a bad idea in most cases.

Reference

Lupo, S. M., Tortorelli, L., Invernizzi, M., Ryoo, J. H. (2019). An exploration of text difficulty and knowledge support on adolescents’ comprehension. Reading Research Quarterly, 44(4), 457-479.

Comments

See what others have to say about this topic.

Ann Nov 02, 2019 04:04 PM

We know that research on high school readers is not directly applicable to primary grades. When you scaffold learning, whether it is learning to read or to ski, the conditions and equipment may be adjusted. The similarity I see is that in both early reading with leveled text and in struggling high school readers, the teacher’s have to teach hard so students can read the text. Books do not teach students to read.
Teachers that hand children ‘grade level’ text because the research said to, will see most students who expect failure, withdraw. “They don’t even try,” “They are unmotivated,” are then teacher responses.
Good teaching creates successful experiences, undermines helplessness, and gives students access.
There is no easy path
-Use leveled texts
-Give all kids OG
-Use grade level texts
-Take away Science and Civics
The teacher has to work hard everyday for every child.
I did just that for 40 years:)

Harriett Nov 02, 2019 05:07 PM

Tim, you're the first person I thought of when I read this study two weeks ago. Yep--you've been right all along and have certainly influenced my practice. When I started teaching a third grade class once a week last year, I made sure to teach all the students grade-level text. I differentiated the instruction, not the text--and I certainly needed to pull my lowest readers into a small group in order do provide that differentiation.

Jeffrey White Nov 02, 2019 05:42 PM

Our German program uses authentic and difficult texts with a genre approach to teach German starting in the 4th semester. Many students have reported that the texts feel more real and relevant than texts in the German textbook that's used in earlier semesters. They like the connections to more complex themes. (See Heidi Byrnes' research of the Georgetown German program. Her work inspired what we are doing)

Gail Brown Nov 02, 2019 10:18 PM

I agree with what you've posted and I think this works for experienced teachers who have effective implementation of KWL within their knowledge and experiences... So, yes, given appropriate instructional support, the text difficulty effects can be reduced with effective teaching support and scaffolding. The problem that I have seen is twofold. Firstly, there are many new, inexperienced teachers out there, and they are not skilled in KWL - they might not even have heard of this before - depends on their teacher training program? Also, from my classroom visits and experiences, broad strategies like KWL are quite variable across teachers. So, again, I'm supporting your conclusion, with a caveat that it's the skills and knowldge of the classroom teacher that will ensure students understand and learn from texts they read, and I think this is independent of text difficulty.

Narelle Nov 03, 2019 05:04 AM

Volume 54

Katie Nov 03, 2019 11:52 AM

Teachers definitely need to differentiate their instruction when using only grade level texts, but sometimes it becomes the teacher reading it to them or other students reading it to them. The student then gains the background/content knowledge but they are not gaining the skills to be able to read that text on their own sometime soon and gain that knowledge on their own. It will always take them tons of teacher support to get them to be independent with grade level texts. I have never had a below level student who was only taught with grade level materials be able to access grade level material on their own by the end of the year. But I have certainly had students that were giving instructional level materials at first and built up their reading skills and strategies be able to access grade level materials by the end of the year.

Audrey Nov 03, 2019 12:40 PM

How much work was the teacher doing? What are we indirectly teaching these students by doing most of the work? What happens when the teacher isn’t there? Is their increased comprehension transferable to other texts or other content?

It seems like this is another example of a push for one way of teaching versus another. Instead, why not embrace the benefits of both instructional practices? Teachers could provide students with on grade level materials during whole group lessons and differentiated support. Then they could provide students with opportunities to read and be instructed on reading strategies to grow them as readers with texts at their instructional level.

When we embrace multiple approaches to teaching reading, we allow for all students to be met exactly where they are and give them access to on-grade level texts.

Tim Shanahan Nov 03, 2019 01:28 PM

The teachers in this study—or in several others—are not doing all the work. Limiting instructional materials to texts that students can already read reasonably well severely limits what they can learn and, at least for adolescents, cuts them off from text they find intellectually interesting, age-level appropriate, and socially acceptable. Telling kids what the text says or reading the text to the kids, however, is not sound scaffolding and it is not what is done in such studies.

Tim Shanahan

Ann Nov 03, 2019 05:15 PM

Making Thinking Visible- the work of Project Zero scaffolds thinking in many ways that teachers find useful.

Norka Padilla Nov 04, 2019 02:09 PM

Excellent article which supports the work that proficient teachers are implementing as social justice for ELs/MLs with the learning they've gained at Standards Institute with UnboundEd.org under the leadership of Amy Rudat. Hannah Turner and Michelle Hunsberger in MCPS, Maryland are equity educators implementing standards aligned curriculum, including juicy sentences that is resulting in undoing systemic and systematic racism of chronic long term ELs- who were only long term because we didn't know how to do this yet. Now we do. "Know better, do better," M. A.
Ensuring we are putting forward accurate research like this is so important. Previous comments also confirm that it's about our proficiency as educators. When we know how, so do our students. Thank you.

Lynn Nov 04, 2019 05:46 PM

Just a quick note that I think you have a typo on the second line of paragraph 2. "texts that are supposedly as the students’ “instructional levels.” - should be "at" the students' instructional levels.

Tim Shanahan Nov 04, 2019 11:01 PM

Thanks Lynn

Catherine Whittal-Williams Nov 06, 2019 11:12 PM


I have STRONG feelings about this. I concur that "dumbing down" literature is a dangerous and slippery slope. Increasingly we are thinning our thinking by offering so much "textual stimulus" that we forget to finish the job with "literature". To me, there's a profound difference. No challenge = no growth. - Cathy Whittal-Williams

Alicia Nov 07, 2019 07:33 PM

I am all about giving students experience with texts of grade-level complexity with the instructional support they need to be successful with those texts. I did it for twelve years in my own classroom with struggling readers and their Lexile levels grew and many of them were proficient on their state tests after 5 or 6 previous years of not reaching proficiency.

But when you speak of Fountas & Pinnell and those who followed them (Calkins, Richardson) are you sure that you have a complete understanding of their comprehensive design for literacy? You write as though the ONLY texts they suggest a teacher use are ones on students' instructional levels, and that is not true at all. According to Fountas & Pinnell, the ONLY part of a student's day that should involve instructional level text is guided reading, and guided reading should only comprise about twenty minutes of a student's day. In later grades, teachers do not even always meet with every group every day. (See Guided Reading: Responsive Teaching Across the Grades by Fountas & Pinnell for more information.) And I'm not sure we can safely sweep the idea of instructional levels (or learning zones, or zones of proximal development, or whatever your preference) aside as though it has no place in literacy instruction at all. If that is the case, why not cut to the chase and support kindergarteners with reading Moby Dick? It is not the concept of instructional levels that is the problem, but the degree. And Fountas & Pinnell would not advocate for instructional level texts that are "easy books" or "dumb books" or "baby books", either. An instructional level text should be such a stretch that a student could not read it on her own without instructional support. By the secondary level, that would frequently be a grade level text.

I agree with so much of what you write, but whenever it comes to this issue, I feel that you have seen teachers implement the work of Fountas & Pinnell (and others) poorly and incompletely and that you base your assessments of their approach on the poor implementation you have observed.

Tim Shanahan Nov 08, 2019 02:45 AM

Alicia— I do understand that kids are free to read harder texts anytime when they are NOT being taught to read. But teachers will guide their relatively easy reading, just not the harder reading. That makes no sense.

Do you know that studies show that guided reading when compared with teaching kids with grade level text showed greater learning gains for guided reading?

Tim

Alicia Nov 08, 2019 03:31 PM

Yes, I'm right there with you. I did use grade-level texts for guided reading with great success, and by far the vast majority of the guided reading groups in my district are working at grade level or above. Those working below grade level are the exception. That's how it should be, and those who find they use instructional levels to stymie student growth rather than promote it are misapplying the work of the researchers you mentioned.

When I last saw you in a conference, you talked about how students needed exposure to many varied texts of varied complexity day after day, and that is exactly what we are doing using Fountas & Pinnell's design for responsive literacy with success.

I assure you, I greatly respect your message for utilizing challenging texts. I just hate to see the work of those other researchers reduced to the concept of teaching with "easy books" because of some educators who have misunderstood the purpose and work of guided reading.

Rhonda Dion Nov 23, 2019 09:24 PM

Tim, Can you confirm when to use instructional texts?
Thanks!
Rhonda

Sarah Lupo Nov 23, 2019 10:14 PM

Thanks for sharing our work, Tim. I'm intrigued by your interpretation and others ideas and enjoyed reading this and the comments. I have garnered many ideas for my next study too- I'm looking at science a-z leveled texts and the impact of science knowledge- and hoping to implement a text structure intervention.
A few thoughts:
- We compared two different ways of approaching knowledge: activating (KWLs) and directly building knowledge (Listen-Read-Discuss by Manzo & Casale, 1986). Activating was the winner in this race. Many of the teachers speculated that LRD would help ELs and students who "struggle" because it focused on building knowledge. But it didn't. ELs actually had the strongest benefits for activating knowledge. The important take away was that knowledge was gained through reading- which is the point of reading!
- Most of the teachers in this study were inexperienced and had 0-3 years of teaching experience.
- I did interview students! It didn't make it into this article. In general kids didn't realize that one version was easier or harder- but they did pick up that some had read longer vs. shorter texts as the easier Newsela texts are on average 200 words shorter.
- The differences between EASIER texts (where language is not manipulated) and easier VERSIONS of texts merits a mention. In our study we found that texts manipulated to be easier were disjointed and lacked cohesion (thus not actually easier). So I agree with some points above about offering a range of texts, both easier and more challenging, but I'd question the use of leveled texts as the lower levels can be disjointed and hard to read.
- In terms of a push for an instructional method, this is an experimental study and we had to stick to the methods in order to compare the two methods. I agree that there is room for both in instruction but the finding that activating knowledge over direct building was really interesting- and a surprising finding. One that reminds us to value all students knowledge that they bring to the table.

Thank you all for your thoughts on this work!

Teacher Jan 27, 2020 03:11 AM

Why then do new reading programs like Wonders 2020 equip their program with leveled texts of the same story and topic?

Lydia Feb 06, 2020 03:40 AM

Although I teach upper elementary school, I absolutely agree with your point about students who are given the "easier" books or versions of text tend to feel discouraged, isolated, and disrespected. I teach many students who have been given the label of "struggling reader" for much of their school experience, and many have learned to dislike reading for this reason. It is difficult to blame them for feeling discouraged! I have been approached about changing my instruction from novel studies to using remedial basal readers in the past; I truly feel that authentic experiences with novel studies (AND exposing my general education students to the very same texts that students in the gifted classes in our grade level read) helps my students to feel more confident and engaged with reading. With appropriate scaffolding, my students are able to comprehend the texts that we are reading, and they also can tackle more difficult skills, such as making complex inferences and determining themes.

When children are made to feel that they are operating at a lower level than other children, they will eventually throw in the towel and feel as though there is no need to put in the effort if they will always be knocked down (much like adults, honestly). Students need the opportunity to experience texts that are on grade level, even if that means that the teacher must provide additional support. Reading great books and taking part in intellectual discussions improves students' academics and confidence levels.

Kathy Cobb Feb 06, 2020 11:21 PM

The key words in your comments are “as long as there was instructional support”. Students can be successful in grade level text with, in some cases, significant support. We know this comes from the best practices of explicit teaching in all five areas of reading, usually with small group and individual support. With this support, students can grasp the concepts being taught in the content areas. However, providing that support can be a tricky accomplishment within the time parameters of a segmented school day and balancing the needs of more advanced readers.
In writing “they made the same amount of gain whether they worked with the easier or harder texts”, was the research addressing gain in content comprehension or reading ability? With so much mental energy being spent on those concepts, when is it that students increase their reading skills?
It seems if students only read texts for which they need support, they won’t value reading on their own (or attempt it). In your opinion, do struggling readers need to read text on their independent levels to increase their reading ability? I’m not familiar with all of Richard Allington’s work, but I have read his article in which he asserts that all children should read something they can read accurately and with understanding every day to practice the skills they have been explicitly taught so they can progress through the reading process.
My thoughts are that both points are valid. In core instruction, students do need support reading grade level text to address the content and reading standards we teach. But, in addition, they need to read their choice of texts successfully to become better readers.
Thank you for your time. I would appreciate your clarification!

Elizabeth Deal Feb 10, 2020 01:04 AM

I find this very interesting in that most schools, especially elementary, tend to use guided reading texts that are below their reading level. The idea and research may differ in the fact of studying elementary versus middle or high school students; however, I would think the overall teaching concept behind reading below their levels may be similar. My thoughts is that giving a student a text that is below their reading level would allow for the opportunity to use this type of text as a learning tool that the students can then apply when reading a text on a higher level. The research you indicated did not show this though. Could it possibly be due to the age of the students studied? If the only students that didn't show progress or growth during the study were the English Language Learners, could it be that those students had not be taught the skills needed to read a more challenging text? I am like you, I wish there would be more research to indicate these findings with this study. I work with students who have special needs, and use texts below their reading level on a daily basis. Each student learns differently and we as teachers have to understand that meeting the needs of these students comes in all forms of instructional strategies, meaning reading on level or sometimes a little below until they can catch up.

Tiffany Wissmach Feb 10, 2020 02:34 AM

As a first grade teacher of six years, I find this declaration of not using differentiated texts surprising. I have been differentiating text for students as directed by my district and do see need for it. I am not disagreeing with the positive outcomes that were seen in the study of the ninth graders. It is especially helpful to learn of the negative impact “easy levelled” texts for older students affects self efficacy. It is very vague and arbitrary for a study to state that students gained the “same amount of learning and comprehension” and teachers be expected to assume that that means the students are on grade level. Did they master the text? Did they read with clarity and understanding? Can they connect their learning with previous knowledge? Maybe they are all just learning on a low level. “Same” doesn’t tell us the level of mastery. I still see the importance of differentiating and holding students to a certain level of mastery. Teachers must know their students and their skills. Rather than give multiple levels of a text, why not use a variety of texts to teach the same topic. Adjust the tasks and text to the student’s level as taught with text complexity. Teach reading strategies and help students own their learning and become equipped with lifelong skills.

Ladee Feb 09, 2021 02:26 AM

There are many opinions about how to teach children how to read. My district uses the Fountas and Pinnell basal assessments where students read a text and teachers determine their instructional level for guided reading. According to their benchmark assessment, students’ instructional levels should be one level above their independent reading level. I subscribe to this opinion because it has been my experience that students who struggle through texts, that are more complex than they can handle, tend to get frustrated and give up easily. However, according to this blog, text complexity does not deter students from reading. Instead, it is their unpreparedness that causes their anguish. That is where I must develop better habits and strategies. It is my responsibility to make sure my students have the tools necessary to absorb the text. According to the studies mentioned in the blog, less complex texts do not equal more comprehension for students. Instead, things like vocabulary building can lead them to successfully comprehended the content that is above their own assessed reading level. The Common Core Standards have made their mark on the educational world by suggesting that many of the readings used in education have traded complex texts for books that are easier for students to digest. However, some studies have suggested that texts have not gotten less complex. In fact, commonly used readings in the 2000’s have had a higher LEX score than all other decades. Meaning, the fears proponents of the CCSS were trying to convey were not as research based as initially assumed. Yet they used those studies to implement a national set of standards that were not fully vetted. While I don’t agree with the CCSS stance that text have gotten less complex, I do agree that complexity in text is absolutely necessary to “grow” your students. My students used to check books out of the library on Fridays. They are allowed to choose two books. One way I made sure they were reading difficult texts for leisure is to have them choose one book on level and one above their level. I love how this blog addresses the issue of how students, who may be low readers, feel about being given the less difficult, or stupid, books. While I am an avid user of leveled readers, I plan on trying to afford my students the opportunity to read the same books as their peers that are on or above level. While it would be more work for me (building vocabulary, scaffolding, preparing for how to deal with student frustration if all other strategies fail) I would like to see how well my low readers do with more challenging texts (above their instructional level) during guided reading.

Suzanne Swindell Feb 21, 2021 12:19 AM

In the article Letting the Text Take Center Stage, the role of Common Core State Standards in denouncing the traditional reading method of matching students to texts based on their reading ability is discussed. Traditional methods for teaching reading would have students that read at a higher reading level given more complex texts to read, while students with lower reading levels were given easier texts to read. This study proves that this method is ineffective and does not benefit the students at all. On the contrary, it appears that students given more complex texts, along with teacher support, made more gains than students that read easier texts. Dr. Heidi Anne Mesmer spoke about text complexity and how to choose books in her podcast. She stated that there are text analysis tools that can be used to estimate text complexity, but there is more to the equation. It is more about observing the learner interacting with the text and seeing how they work together. What we as teachers have to realize is that in order for students to grow and learn using the more complex texts, we have to provide more instructional support to promote understanding and comprehension of the text. The article Letting the Text Take Center Stage stated that “students will be frustrated by challenging texts and this means more instructional supports will be needed to help and encourage them.” As a Kindergarten teacher, many of our English Language Arts Standards begin with “with prompting and support.” Being that I teach very young students, this is not a very hard requirement to fulfill as most everything we teach is a new concept and a lot of prompting and support is automatically required. If I were asked to teach older students, I would be unsure of how to provide that support and would need guidance in how to do so. The article also stated that “teacher preparation typically includes few tools for helping students to learn from challenging texts.” I agree whole heartedly with that statement. After reading both of these articles and listening to the podcast, I have come to the conclusion that utilizing more complex texts is in the best interest of my students. I plan to begin doing so and seek out the best methods for providing the support necessary to help them succeed.

Ash Davis May 31, 2022 10:40 PM

A lot of varied comments across a wide range of age groups. What is adolescence? It appears that conversations that begin with children in kindergarten are looking at this topic from an unintended viewpoint. The whole concept of 'adolescence' seems to be overtaken with the reference to text difficulty. If this is the case, why do we not just disregard age/grade/level and focus on students more holistically. It appears to easy to pigeon-hole sets of students for research purposes when we know that literacy is an issue across all ages (including into adulthood) and that there is no easy fix. Referencing studies and methods of certain individuals seems to diminish the plethora of work that is being done by the 'boots on the ground' educator to influence a new generation of readers. Does the difficulty level of the text affect the level of comprehension required depending on the skill-level of the student?

Timothy Shanahan May 31, 2022 11:00 PM

Ash--
Indeed, text difficulty affects comprehension. There is no question of that. If you are a science or other content teacher that is a real concern because if you assign a text to impart information to the kids, they may not get it. However, if you are responsible for teaching reading, it isn't the right question. The issue for a reading teacher is, what is the right kind of text to teach reading with. By the adolescent years (or really grade 2), the issue is what kind of text leads to the greatest growth in reading ability -- and that is text that students cannot already read well. The instruction needs to be aimed at enabling success with texts that students cannot already do well with.

tim

What Are your thoughts?

Leave me a comment and I would like to have a discussion with you!

Comment *
Name*
Email*
Website
Comments

Cool New Study on Text Difficulty and Adolescent Literacy

27 comments

One of the world’s premier literacy educators.

He studies reading and writing across all ages and abilities. Feel free to contact him.