Your right about the lack of information about instructional delivery, IWB’s mediate instruction (ie. the teacher instructs the students through the technology) unlike a lot of other instructional technology. Therefore it is imperative that any research that looks into IWB’s, explores exactly what the teachers are doing with them and why (what training and support did they receive) b/c the technology is simply a mediator for the instruction between the teacher and student, unlike one-to-one laptops in which the relationship of technology, teacher, and student is different.

From what little I know about research in instructional technology and instructional design, there’s already a huge body of good research that has repeatedly demonstrated that a specific media has very little to do with increases in learning. This is not a new question. It is asked every time a new technology is introduced into learning environments. The repeated and, what I think empirically valid result from media comparison studies over 50 years shows that if instructional methods stay the same, learning stays the same. (Clark, 1994; Dillon and Gabbard, 1998)

This is not an insignificant body of evidence. If the same instructional methods were used in non-IWB classrooms and in IWB classrooms, and everything else was controlled(not even close for Marzano), the result that one would expect from this history of research would be one of no causality. A causal effect from using a different media would be like finding a human fossil in the cretaceous period.

From this study, we have no clue about instructional delivery. That’s key to learning and the development of instructional strategies that take full advantage of the capabilities of interactive technologies should be the focus of any school that purchases these devices. That’s the really, really hard part.

Your insights regarding the design of his research are excellent and thought provoking. I wonder if the problem with Marzano’s research isn’t more with all of his research, not just with this particular example of his research?

However, I think both of following two criticisms of his research while valid, are really criticisms of a lot of research into educational technology.

“Suffice it to say that none of the measures of student achievement used by any of the teachers who participated in the study are either valid or reliable.”
A valid measure of achievement may be a test score, but do test scores really represent a valid measure of all the learning that is going on (I believe that they don’t) and if they don’t, then what would a valid measure be? That isn’t to say, that the teachers did use a valid or reliable measurement of student achievement.

“just the fact that the teacher had to take an existing unit and figure out how to integrate the IWB technology means that the teacher was biased toward that group (i.e. she/he was more planful about that teaching).”
Don’t most teachers who integrate technology do so willingly? Isn’t the biggest problem with integrating technology convincing people to do it? If you included people who didn’t want to use the technology or were ambivalent, wouldn’t that bias the study the other way.
Also, are teachers who are more planful, more likely to integrate technology, since they are planning (ie. thinking what they could do vs. doing what they always have done) to teach content?

In conclusion, I think your review was excellent, however I tend to see Marzano’s research as more a very basic starting off point than anything at all conclusive.