If you’ve ever taken a college or graduate level course, surely you’ve completed some kind of summative evaluation form at the end of the semester.  At Hofstra University, where I worked for 5 years before this past academic year, we called them CTRs (Course and Teacher Ratings).  They consisted of a bunch of Likert scale items (strongly disagree to strongly agree) and a few open-ended questions.  For the most part, students hated doing them and faculty members hated having to use them.  I didn’t love the wording of many of the items, but I always asked my students to please take them seriously as an opportunity to let me know how I was doing.  I told them that I would receive an analysis of the data and their actual responses to the open-ended items.

As part of applying for tenure at VCU, I have to demonstrate growth as an instructor.  So, I plugged the CTR data from my 5 years at Hofstra into EXCEL and discovered some very interesting things.  The graph below represents the data from a scale (composed of 5 items) that purports to be an overall measure of the course and the instructor.  The x-axis represents the time points from Fall 2002 to Spring 2007.  The y-axis represents the range of scores (which can range from 1 to 5).  For this particular scale, the lower the number the better.  But, I flipped the y-axis so that it looks like “better is higher;” a more standard look for such a line graph.  The blue line represents my ratings; the red line represents the average score of the other faculty members (including adjuncts) within the program area.

[NOTE: click on image for larger view]

I entered the professoriate with NO teaching experience.  I guest lectured once while I was getting a masters degree, but that was it.  Hofstra took a bit of chance on me in that respect and I am eternally grateful to them for that.  But, the graph clearly shows that my ratings were not as good early in my teaching career as they were last year.

I should also add that in my first couple of years as a professor, i was asked to teach a few sections of an undergraduate foundations of education course.  I thought I would really enjoy working with undergraduates considering a future as an educator.  But, after teaching a few semesters, I began to really dislike it.  I had a hard time dealing with the students’ limited understanding of and experiences with education.  Seemingly simple concepts such as “charter schools” were completely foreign to them.  My ratings were not terrible for those course sections, but my department chair and my colleagues and I decided that my time and energy was better spent working with graduate students.

Overall though, I think the graph tells an accurate and interesting story.  Quite simply, I’ve improved significantly as an instructor.  The more comfortable I’ve become in my own skin and the more I’ve been able to find my own voice, the more I’ve been able to engage my students.  That’s my interpretation of the data.

Academics bemoan the use of “quantitative” ratings of their work as instructors.  But, I think it’s critically important that we ask our students to reflect on their experiences in our classes and to provide us with data about our work.  I wonder how many of my P-12 colleagues/readers have similar systems in place to collect and analyze summative or formative data about their performance directly from their students.  Do you?

Tags: data, evaluation, teaching

Thanks to Will Richardson for alerting us to an op-ed piece in the Washington Post that’s disturbing to the core (IMHO).  Clearly, there are details of the “technology initiative” (odd language, seems more like they just built a new school with cool stuff in it) that we’re not getting.  But, I wonder, did, as the author suggests, the administration just buy a whole bunch of stuff without considering the curriculum and/or the teaching needs?  I doubt that’s entirely the case, but it did remind me…

There have been many great sports coaches who were successful based on a “system” they installed.  Bill Walsh and the so-called “West Coast Offense” is one example.  Pete Carrill’s “Princeton Offense” is another.   Numerous proteges of those coaches have left the nest to coach their own teams using the system they learned.  These coaches struggle at first because certain types of players thrive within their system and those players are not necessarily the ones they inherit on the new team.  In other words, they inherit a team of players not necessarily suited to thrive in the system to be implemented.  Thus, it takes the coaches a couple of years to get appropriate players in place for their system to succeed.  Urban Meyer, the head football coach at the University of Florida brought his “Spread Offense” from the University of Utah to Florida.  He inherited a quarterback who was pretty good (although U of  F fans tend to disagree on that), but who did not possess the skill set to thrive in Meyers’ offense.  Once Meyers got a quarterback who could operate his system well, he was more successful.  Now, we see Rich Rodriguez bringing his version of the Spread Offense from WVU to the University of Michigan; the rising sophomore quarterback who would’ve been the starter had a coaching change not happened has transferred because he’s not at all the type of QB who can run the new “system.”

Some professional coaches have a harder time implementing systems because they can’t  as easily recruit the right kinds of players.  Long-term contracts and salary caps often force professional coaches to think differently.  Therefore, a successful pro coach is often one who is able to adjust his system to the personnel.  Bill Parcells is a great example.  He won the Super Bowl as the coach of the New York Giants with his system (ball control/rushing attack on offense; tough D).  When he took over as coach of the Patriots, he had a team more suited to the passing game.  So, he adjusted and was quite successful.  He made similar adjustments when he took over the Dallas Cowboys.

So, what does this all mean for education?  Well, I think educational leaders/policymakers are guilty of installing systems without regard to the personnel.  Much more so than in professional sports, school leaders inherit a team; absent retirements or mass exoduses, teaching staffs often remain fairly stable.  Yet, the ed. tech. policy agenda has been dominated by a focus on infrastructure development supported by a bit of professional development.  In other words, the “systems” have been installed and the leaders are then forced to try to fit the personnel into the system.  “Hey, we’ve got all these great technologies, now figure out how to use them!” (and that may or may not have been the case in the high school referenced in the op-ed piece; we don’t know)

Technology planning needs to be done with curriculum and teaching at the forefront.  In other words, the technology should be mapped to the curriculum (NOT the other way around) and the infrastructure should be shaped around the strengths/weaknesses of the team members (the teachers).  From there, as new teachers are added, they can be purposefully selected (i.e. they should be hired because they “fit” within the system that’s been implemented).  That means including curriculum specialists in the technology planning process and, more importantly, involving teachers. 

Make the system fit the team, not vice versa.

Tags: coaching, curriculum, education, leadership, teaching, technology, technology planning