Prior Knowledge, Or He Isn't Going to Pick on the Baseball Study

Nancy--

Indeed, we can overdo (and often overdo) prior knowledge emphasis. In many of the studies it took no more than providing a topic to readers to marshal their prior knowledge. Too many teachers think prior knowledge refers to the information in the text itself (so their prior knowledge preparation seems aimed at telling them what the text said). Yesterday I spoke with an educator who had a fine understanding of this. She suggested that if she were teaching the science of tsunami's she might show the students a brief video of a tsunami so the kids would have some idea of what it was, and this would be their preparation for taking on a text that was to explain how tsunami's form in terms of weather processes and their biological effects (but she would not use such a video if the point of the article/chapter was simply to explain what a tsunami was).

thanks.

Tim, I can't thank you enough for writing on this topic! E.D. Hirsch started the Core Knowledge Foundation in 1986 based on this claim that background knowledge is essential in supporting reading comprehension. He attempted to identify and promote a "core" set of knowledge that students need to learn at each grade level. It's a nice concept to think that someone, or even a group of content experts, can determine exactly what knowledge across many subjects students need to learn at each grade. However, it's just not practical. And as you note in your post, in real life we often have to read text for which we have minimal or no knowledge. Hirsh's 2006 book The Knowledge Deficit promoted the notion that background knowledge is THE reason why students can't comprehend. Daniel Willingham later made a similar claim in several articles, including a 2014 Teacher College article "Can Reading Comprehension be Taught?" which begins with this line: "In this commentary we suggest that reading comprehension strategy instruction does not actually improve general?purpose comprehension skills. Rather, this strategy represents a bag of tricks that are useful and worth teaching, but that that are quickly learned and require minimal practice." I use the word "claim" because as you noted above, interpretations of one or two studies is not the same as findings that might be concluded from a meta-analysis of multiple, replicable studies that are valid and reliable. Willingham himself acknowledges this when he says "in this commentary we suggest..." It always bothered me that he felt comfortable to use such rhetorical language about comprehension strategy instruction "...represents a bag of tricks." He also claims that comprehension strategies are "quickly learned and require minimal practice." I have been teaching comprehension strategies to students, especially those who struggle with reading, for over 35 years. This includes teaching them some of the close reading strategies that they must be taught in order to unpack complex and challenging text. While a few "high flyer" students can quickly pick up on a strategy and apply it independently, the vast majority of the students I have worked with need a lot of explicit instruction and guided practice before they are able to independently apply them to text across all subjects. I have also trained thousands of teachers, including subject area teachers, how to provide this instruction so it is embedded in their content teaching. And the feedback I hear from them is the same.... it's hard work for both the students and the teacher.

N. Wexler's new book is just an extension of Hirsch's and Willingham's claim. However, she goes a step further and claims that teaching comprehension strategies is not necessary. She cites the "baseball study" and refers to Hirsch and Willingham's claims, but rather than admitting that the premise of her book is an interpretation, in interviews and articles promoting the book she often claims that research backs this interpretation. This is why teachers such as the one who posed the question at the start of your post are confused. Since the publication of the book, I have had numerous educators contact me and ask if it's true that research says comprehension strategy instruction is not useful. So again thank you for addressing this issue in your post.

In my work, I try to explain to teachers that assuming students are already fluent readers, there are several components that support reading comprehension: vocabulary knowledge, background knowledge, knowledge of text structures, and meta-cognition/comprehension strategies. No one of these components is THE answer to improving comprehension.... teachers need to provide instruction for all of them using real text from different subject areas.

Luke--

Most of the research on this problem has focused on science reading, mainly physics, where readers' conception of the world contradict the information to be presented in the texts (though I know of at least one study in which primary grade kids allow their background information to overwhelm a text's meaning even with more typical texts). Studies on that kind of thing discourage attempts to focus readers on their prior knowledge (making hypotheses and such), since these tend to reinforce the misconceptions. Better to launch right in on the texts. There are decades of research now on what are called refutation texts in which the misconceptions are addressed directly in the text (and that is great for teaching science but doesn't encourage an independent skeptical reader.

One thing that I have used (but I have no research on this) is Five Questions--something my dear friend, Dorsey Hammond, taught me: In preparing kids to read, instead of emphasizing what the students know about a topic, focus their attention on what they want to know and what questions they have about the topic. Then they read the topic with the purpose of answering 5 questions:

1. Which of my questions did the text answer?
2. Which of my questions didn't the text answer?
3. What did I learn from the text I wasn't looking for?
4. What surprised me?
5. What else would I like to know?

Question number 4 is the key here, because information that violates our expectations is what we find surprising (and that means it is not consistent with our prior knowledge). That's where the learning can take place. A reader who gets used to reading with a sense of surprise will be less likely to allow their prior knowledge to overwhelm a text.

thanks.

Joan-

Good hearing from you. I've come to think of productive reading comprehension instruction as including three foci:

1. Comprehension strategy teaching (including teaching students both to use what they know, but not to allow that to overrun their understanding of the what the author wrote)

2. Knowledge of the world (using the texts used in reading instruction to increase student knowledge of their social and natural worlds -- which obviously includes literature)

3. Written language (including vocabulary, morphology, syntax, cohesion, discourse structure -- along with things like learning to make sense of the relationships between prose and graphic elements)

Those are the key things that I would try to teach now if I had my own classroom again. Of course, teaching these includes both explicit explanation by the teacher along with opportunities for the kids to practice them and extend them through supervised reading. Reading folks have hit #1 too hard at times, now it looks to me like the prior knowledge contingent is overdoing it on #2. With exception of vocabulary, i think Item #3 is particularly neglected and has been for a long time (despite strong research supporting such teaching--and except with the youngest kids and second language learners the idea that you can teach oral language in ways that improve reading comprehension is an unproven claim).

thanks.

tim

Hello! Can you clarify the difference between these strategies and the skills that you refer to in the ACT study that do not lead to stronger readers? Thank you!

Thanks Tim for reacting to my post.... I think we are aligned in our thinking about this. And, you are so right in pointing out that teaching text structure is often neglected. It has several sub-components, including those you just mentioned. Text structure starts at the simples level of developing syntactic awareness which helps kiddos unpack long, complex sentence when reading but it also helps them write longer sentences. Activities such as sentence combining have long been found to be helpful for this across all grade levels. I also think teaching them to be aware of what I call "patterns of organization" (e.g., description, sequence, cause/effect, etc.) that can be found within a paragraph and combined across multiple paragraphs, along with the transition words and phrases that signal these patterns. And finally, it is helpful to teach how to recognize the similarities and differences of larger text structures, such as how the body of information text, vs argument, vs narrative is developed --- this improves both their comprehension and writing.

Thanks
A great blog. I have asked a few times on Twitter if the Baseball study had been replicated and didn’t get a response.
Great advice here.

The tsunami example is a good one. My district refers to this approach as "Creating Curiosity for New Content." And with our access to YouTube, it's very easy. And effective.

As an early childhood literacy instructor, I have to thank you for pointing out that what is true for beginning readers is often different from what is true for more experienced readers. You said, “ With exception of vocabulary, I think item #3 is particularly neglected and has been for a long time (despite strong research supporting such teaching--and except with the youngest kids and second language learners the idea that you can teach oral language in ways that improve reading comprehension is an unproven claim).”
The development of language, the vessel for background knowledge, is difficult to quantify. DIBELS, Istation, etc. do little to help primary teachers understand the impact of low language on children’s problem solving, metacognition, social negotiation, or literacy development. Background knowledge also impacts motivation. Many six year old boys would work hard to read about T-Rex and understand all the information. But a book about becoming a ballerina, not so much.

Shyla—
The comprehension skills that are often taught (but make no difference) tend to be question types... main idea, drawing conclusions, key details, etc.

I think the strategies like self questioning or visualizing are actions that help readers to pay attention to the information, to think about it, to spend more time with the text... the supposed skills don’t appear to do anything for the reader.

Thanks

Tim

Harriet and Ann—

I agree that prior knowledge has some motivational value but know of no research supporting that view. The best explanations of prior knowledge link it to memory but there are others as well that make some sense. I hope to write about that sometime in the future.

A couple of people emailed me, saying "Tim's doubting the baseball study!" or similar. I've been very occupied w/ putting courses online so I haven't read Tim's blog until just now....but actually I don't think he and I are far apart.

First, Tim notes that background knowledge influences how students put together texts, and that that is settled science. His comment on Recht & Leslie is that it hasn't been replicated...and it's true that it hasn't been directly replicated, although a couple of other labs have done something similar--pitted more general skill vs. knowledge, e.g.,

Hall, V. C., & Edmondson, B. (1992). Relative importance of aptitude and prior domain knowledge on immediate and delayed posttests. Journal of Educational Psychology, 84(2), 219.

Schneider, W., Körkel, J., & Weinert, F. E. (1989). Domain-specific knowledge and memory performance: A comparison of high-and low-aptitude children. Journal of educational psychology, 81(3), 306.

What I take to be Tim's point in this blog is what this all means for practitioners. If you have background knowledge, are you good to go, with nothing else. Here again, I actually think Tim and I are quite close.

1) One set of skills to teach readers are reading comprehension strategies, of the sort evaluated by the National REading Panel. There's no question that such instruction is beneficial--the literature is enormous and convincing. What I've emphasized is that there's not evidence that extended practice with such strategies helps. (I've suggested the literature indicates as little as two weeks might be enough, but I've also so said you'd probably want to do this two-week stint in three successive grades. If memory serves, Tim thought two weeks was too lean, and suggested six was more appropriate.)

2) ALTOGETHER DIFFERENT in my view is instruction in how to *work* with texts, to do the sort of tasks, mostly academic, that differentiate the sort of tasks we ask students to do in academic contexts. This is in addition to (not instead of) the strategies aimed to help with comprehension of the text, which #1 above is about. There is a good literature here showing gains, though inevitably not as rich as for #1, in part because it becomes fragmented as you get subject-specific goals. I don't know if the question of practice has been evaluated for these sorts of texts....I suspect there would be practice effects, but again, I don't know.

We might also ask "is there experimental evidence, not just correlational evidence of knowledge on reading comprehension?" In other words has someone tried to teach kids some new knowledge and then show that they read on that topic with better comprehension? I don't have time to refresh my memory on this (maybe someone can help me) but my memory is that a few people have tried and failed. If I'm right, that matches similar efforts in memory and problem-solving, tasks for which you also see sizable knowledge effects. The probable reason is that for knowledge to be helped it needs to be really integrated with other long-term memory representations....the strength and inter-connectedness of representations you get in the course of a one-hour experiment isn't going to do it.

I thoroughly enjoyed reading your article! I would love to know your perspective on how curriculum companies are building topical units of study and then testing students’ comprehension/writing on that same topic. Are we really just assessing background knowledge because of the knowledge-base students now have from what they read during the unit of study?

Tim,
Part of why we are pushing hard on the importance of knowledge in Hawaii is because Wonders pushes pretty relentlessly on comprehension strategy instruction, every single story , all year long - all seven years - and often the strategy is a poor match for the story. The themes and topics are pretty loosely connected making the job of building knowledge that much harder.
Sometimes teachers and practitioners don’t have time to sift through the nuance of such an argument, so when you say the baseball study hasn’t been replicated - you communicate - perhaps unwittingly - let us go back to an exclusive and relentless focus on strategy.

Petra
I definitely do not think that strategies are the only thing to teach with regard to reading comprehension and there definitely should be a lot of attention to both how to make sense of written language and building world knowledge.

Thanks

Tim

I plan to write at greater length about this topic in the near future, but since I've seen my name mentioned here a couple of times--including in the question that is used to kick off this blog post--I'd like to say a few words. I'm not going to wade into whether the baseball study has been exactly replicated or not, but I've seen the results from one of the studies Dan Willingham mentions, Schneider, Körkel, and Wiener (1989). That study was done in Germany, with kids in 3rd, 5th, and 7th grade, and the topic was soccer: after being tested on both general reading comprehension and knowledge of soccer, the kids were divided into groups as in the baseball study. Each student then heard and read a story on a young player's experiences in a soccer game. They were then tested 15 minutes later with a cloze version of the test with 20 blanks. (I'm basing this description on information from Dylan Wiliam, a well-respected education scholar.) Admittedly, I haven't seen the study itself--and admittedly, it's not BASEBALL--but it seems pretty similar! And the results were similar too: the kids with high knowledge of soccer and "low" reading ability did much better on the cloze test than the "high" readers with low knowledge of soccer (16.4 vs 11.1 points). There is of course other evidence to support the idea that building knowledge is crucial to creating good all-around readers, which I'll address in another venue as soon as possible.

There are a number of other points that have been made here that I intend to respond to later, but for now I would just say this: Everyone seems to agree that background knowledge is helpful to comprehension, and that there is evidence to support SOME kinds of strategy instruction. The disagreement seems to center on whether schools should continue to teach "reading comprehension strategies" in isolation from the content areas.

First, as Tim acknowledges, a lot of the reading comprehension instruction that schools currently spend time on -- finding the main idea and details, drawing conclusions, etc. -- doesn't actually have evidence behind it. Basal readers are largely organized around those supposed skills, not the handful of strategies for which the National Reading Panel found evidence. Generally, publishers and educators don't seem to draw much of a distinction between comprehension "skills" and "strategies" and the former predominate. So if we say schools should "teach reading comprehension strategies," the result will be many hours spent on "skills" for which there is no evidence. And as for the strategies for which the NRP did find evidence (visualizing, self-questioning): they will only work if readers have enough background knowledge to be able to make sense of the text at least at a superficial level. You won't be able to answer the questions you're asking yourself or to "visualize" if you can't make head or tail of the text.

Second, I'm not arguing that kids shouldn't be taught how to unpack complex sentences, to think about the structure of a text, etc. Background knowledge is only part of the comprehension equation; kids also need to become familiar with the conventions of complex written language, which differs significantly from those of spoken language. But I don't think long-term instruction in comprehension strategies, in isolation from teaching content, is the best way to get kids familiar with the conventions of written text. Rather, I would say that teachers of history, science, literature, etc., should incorporate these techniques--having kids ask questions about their understanding, etc.--as part of their teaching of CONTENT. And what's really powerful is to teach kids to use the conventions of written language in their own writing, using a method like The Writing Revolution. Once they learn to do that, they're much better able to understand the conventions of written language when they encounter them in their independent reading.

When I was trained in my single subject credential program for English and social studies, I took a class on "reading in the content areas". When I became a high school English teacher, there was never a shortage of complaints from us English teachers that reading wasn't being taught in the content areas. Now, if we call it "comprehension in the content areas" rather than reading, that might have more sway. Yes, the students can decode "From each according to their abilities; to each according to their needs", but what does that actually mean?

Great thoughts that are much needed in our polarizing world.

One aspect of reading comprehension that scholars don't seem willing to discuss is motivation. Strategies seem meaningless without the internal desire to know the material. Too many of our students lack this type of motivation, and empirical studies run in laboratory settings with 'students' who are very motivated to be part of the study don't really help me in the classroom with a majority of students who are not motivated to understand the material they are being taught. I listened to your seminar ("How to Plan Effective Literacy Instruction") today, and the comment I have about independent work in a classroom being less effective than teacher-led instruction is "Of course!" There is a human being vying for the child's attention when the instruction is led by a teacher. If cameras were in my classroom and researchers with clipboards observing my students during their activities, whether independent, or teacher-led, you can bet the buy-in for ALL activities would be much higher than without all the hype, attention, and cameras. I have very little time or patience for research in best practices; the best thing we can do for our kids (other than hire competent teachers) is reduce class size. That said, I found your remarks in the seminar more truthful and fact-based (regarding what actually happens with students) than most researchers I have ever studied.

I'm quite confused whose version of the science of reading and its claims regarding knowledge and comprehension we should go with.

First in this blog post, if my understanding of your point is correct, you were saying that the baseball study is a one off and we shouldn't generalize from it. Also, you said that we should focus on brief strategy instruction and encourage students to tackle texts even if they have limited background knowledge as they can build it by reading the text.

But in your Science of Reading (https://shanahanonliteracy.com/upload/publications/228/pdf/Science-of-Reading-2020.pdf) article you quote Cervetti & Hiebert who said "...In these studies that have examined these different contributions to comprehension, knowledge is the most important contributor." Then on p.13 you cite the very same baseball study that you warned us about in this blog. You wrote "The study showed that prior knowledge of a topic has a greater impact on reading comprehension than general reading ability." Then on p.14 you say that a teacher's time is best spent teaching vocabulary and knowledge. So knowledge seems like it is of great importance.

Then, if we look at another article about the Science of Reading, we find the very same reference to the baseball study. This time, it is the 2020 updated version of Teaching Reading is Rocket Science by Louisa Moates (https://www.aft.org/sites/default/files/moats.pdf). On page 14 of Moat's article, she says " Ironically, while background knowledge can be gained from reading, it is also true that those who already know more about a topic make better inferences and retain meanings better than those who know little about it." Moates reference Hirsch's (2016) book Why Knowledge Matters and an oft-cited article by D.T. Willingham 'The Usefulnesss of Brief Instruction in Comprehension Strategies' (https://www.aft.org/sites/default/files/periodicals/CogSci.pdf).

Almasi & Hart in the book 'Best Practices in Literacy Instruction' (p.225) say that many who struggle to comprehend struggle to do so regardless of whether they have content knowledge. They go on to say that those who argue in favor of the content of knowledge position are often long on rhetoric and short on empirical evidence.

So is there enough evidence or is the issue of knowledge and the science of reading more about rhetoric at this stage?

You have reminded me of a moment of insight I had while observing at a preschool I was considering for my daughter. It was a farm-based Montessori in Yorkshire England that encouraged children to spend as much time as possible outside. They had a large cloakroom for the children to put on their waterproof outerwear and the obligatory wellies. I witnessed as a teacher provided nonintrusive supportive companionship as she assisted other children while a little boy struggled to figure out how to put his wellies on by himself. It took around 30 agonizing minutes for that child to get his wellies on, and even then I think he had them on the wrong feet. But I was in awe, not of him, but of the teacher, because she didn't hurry him or fuss at him or take the wellies from him and put them on his feet. Because she had her eye on the real goal, not the apparent goal. She wanted to teach all of the children how to master the tasks in their life and he was clearly at work. Getting ready to go outside wasn't wasted time while he 'missed out' on whatever he could have been learning outside, it was an opportunity for productive struggle.
It is easy for us to get confused and think the goal of a reading class is for the student to comprehend the text of the day, but that isn't our goal. Our goal is for the student to increase their ability to decode and comprehend text in the rest of their lives. Today's text, for all its charm, is mostly an exercise program for their reading skill. We need to optimize text selection for increase in skill in comprehension, not opportunity to enact comprehension without improvement. The goal isn't to understand the baseball text, it is to struggle productively with the baseball text.
I wish there had been a follow up to see if those who clearly struggled to comprehend showed any improvement in their skillset or if they, when compared to a similar group without the difficulty comprehending, upon meeting a similar challenge did better, for having had that struggle. Which struggles are productive and which are demotivating? Which are both? Can a teacher use a challenging or frustrating text as an opportunity to model ways to achieve partial success or informative failure. Too many children and adults think getting the right answer is more valuable than learning from having gotten the wrong answer. I wish confidence in pursuit of productive struggle for my students, not pursuit of ease.

I also have a question about the baseball study. I saw in another article that this was part of the paragraph read:
“Churniak swings and hits a slow bouncing ball toward the shortstop. Haley comes in, fields it, and throws to first, but too late. Churniak is on first with a single, Johnson stayed on third. The next batter is Whitcomb, the Cougars’ left-fielder.”

For 7th and 8th graders, this doesn’t seem like a challenging passage. The names would be challenging for my dyslexic soon-to-be 5th grader but she’d be otherwise able to read it. Obviously, I haven’t seen the whole passage and I don’t know if it was leveled in anyway to match what would be found on reading comprehension assessments for 7th and 8th graders. But this little blip makes me wonder if part of the result of reading ability having no impact simply came from the fact that the passage was simple enough that no one struggled to read it, which left background knowledge as the only thing to distinguish the kids.

Also, while the passage is presented as a narrative piece, one of the comprehension tasks was to re-enact the half inning of baseball with physical pieces, which seems to heavily reinforce the need for background knowledge to be successful at the comprehension task. It doesn’t seem like they asked many questions related to the narration that didn’t seem directly related to a baseball game. I have a feeling that a passage like, Casey at Bat, which focuses lengthily on one at bat, wouldn’t be as affected by overall baseball knowledge.

Eric--

Your analysis of the study is a wise one. Indeed, it appears likely that one of the problems here is that that passage was relatively easy for these students, which only allowed background knowledge differences to affect the outcomes. Part of the reason for this is that they excluded all students whose reading was below the 30th %ile. If that hadn't been done, I expect reading would have mattered.

Mark Seidenberg has also shown that given the small sample size, etc., the study did not actually result in a statistically significant difference between reading and knowledge effects.

Knowledge matters in reading, but being able to read is likely the most important issue in reading comprehension.

tim

It allows us to break the bonds of experience, superceding what we know.
I love this statement and it describes what I feel is the ultimate purpose of reading. Whether the text is Fiction or Non-Fiction prior knowledge and what we learn afterwards helps to make the text interesting and more connections can be made with our young readers.

It is massive to understand the difference between making claims based on research vs what you initially interpret.

This is a great article. I think it's part of human nature to overreact to one case and immediately think it must be universal. This is a great example of why its important to look at the whole body of research surrounding a topic and not merely accept the first thing we're told. So many people are misled by poorly conducted or even intentionally misleading research because they treat one study as ultimate proof and confuse interpretation with results.

This is a great article. I think it's part of human nature to overreact to one case and immediately think it must be universal. This is a great example of why its important to look at the whole body of research surrounding a topic and not merely accept the first thing we're told. So many people are misled by poorly conducted or even intentionally misleading research because they treat one study as ultimate proof and confuse interpretation with results.

I greatly appreciate Shanahan's take on the baseball experiment. I must say, I am guilty of reading it and taking it as the truth. Although it may not be wrong in its findings, assuming that it applies to everyone and believing it to be the "end all be all," is whats more dangerous. There needs to be more research done on this topic, I would be very interested in the findings

Hope--
I don't want to mislead. Both before and after the publication of the baseball study, there have been a number of studies showing the importance of knowledge in reading comprehension. It does matter if you know stuff and that you be able to make connections between what you know and what you are reading. The idea that reading comprehension depends more upon what you know than your ability to read (the apparent finding of the baseball study) is silly, but that doesn't mean that knowledge doesn't matter. We, in schools, need to focus both on making certain that students can read and that they know a lot about their social and natural worlds.

tim