Blogging Disrupting Class #10: Chapter 7 -- Improving Education Research (Part 1)

The previous post summarizes how Chapter 6 ("The Impact of the Earliest Years on Student's Success") detours a bit from the book's main argument with a rather weak recommendation on how best to spend money on early childhood education. Chapter 7 discusses some ideas for improving education research. I didn't like this chapter much the first time around, and I suspect I won't like it much better this time either...

The opening vignette for Chapter 7 (pp. 159-160) is designed to make the point that educational research isn't very helpful because most of it is "preliminary and incomplete" (p.161). The authors then launch into their main argument, which is that it is possible for education, as in other fields, to have a body of research which allows "people to predict with great certainty the results of actions," despite the claims of many education practitioners that such certainty is impossible. This happens because the existing paradigm "causes researchers to stop their work when it is half done," resulting in "statements of correlation but not casuality," which results in contention rather than consensus (p.161).

Right away I am skeptical, but on 2nd reading, let's see what the authors have to say. The idea of predicting "with great certainty the results of actions" is at a closer glance too vague to discern its meaning quickly. What actions are we talking about? Are they actions which are worth knowing? In what sense is certainty the outcome we are looking for? I suspect that the answers to these questions will also reveal the limitations of the authors' approach, but let's see.

Their next important point: Research on best practices or what works best on average across education is inadequate (p.162). As someone who's researched effective practices (which is distinct from 'best practices'), my skepticism deepens, but it's not clear yet what they mean by this. The latter point re the inadequacy of finding out what works best on average is much more agreeable -- after all, the argument of this book is all about customizing learning rather than standardizing it, right? So I'm on board with this one.

The authors then launch into an explanation of how descriptive bodies are built by claiming that researchers build bodies of understanding in two major stages, descriptive and prescriptive. Description is preliminary, while prescriptive is more advanced (p.163). This is the next yellow flag for me -- I'm not a big fan of prescriptions in social science. Medicine? That's different. Then they describe a process -- Step 1: Observation (pp.163-164) -- no argument here. Step 2: Classification -- no argument here either (yet), although the authors point out that "many people are still arguing over and researching what the proper categories should be in this still nascent field of understanding how people think and learn (pp.164-165). This is the first hint of what may be bothering me about their approach, but more on this later. Step 3: Defining Relationships, which leads among other things to correlative findings and measures, which "causes paralysis" because formulations of the average cannot describe specific situations (pp.165-166). I agree with this point as well.

Next, the authors describe how to improve descriptive bodies of understanding: instead of deriving correlations inductively from data sets, educational researchers should do what researchers in other fields do, which is to seek exceptions ("anomalies") to average tendencies (pp. 166-167). Two points to make here: 1) this is reminiscent of the arguments made in The Nassim Nicholas Taleb in his book The Black Swan, which I have some sympathy with; and 2) this is starting to sound rather condescending -- for instance, Taleb spends much of his time pointing out how most economists build models which fail to take anomalies into account, which is why they're so bad -- but I thought it was only educational researchers who fail to seek exceptions? Still, looking for anomalies in and of itself is a good strategy, and the topic is how to improve descriptive bodies of understanding. So I'm still on board here.

Unfortunately, this next section is where I jump off. I thought this was going to be a relatively short post on a relatively short and tangential chapter. However, it turns out that the latter part of this chapter reveals an important truth about why Disrupting Class's argument is off the mark and how it goes astray. So, more on that in the next post...

Chapter 6
Chapter 5
Chapter 4 (Part 3)
Chapter 4 (Part 2)
Chapter 4 (Part 1)
Chapter 3
Chapter 2
Chapter 1
Introduction