Continuing my sniffing through the NAEP Data Explorer, today I “explored” differences in digitally-infused pedagogy by race.  One of the items on the background questionnaire of the 8th grade NAEP in 2007 was as follows: “When you are doing math for school or homework, how often do you use these different types of computer programs?”  One of the listed programs was “A program to practice or drill on math facts (addition, subtraction, multiplication, division).”   Looking at the results for that item disaggregated by race, we get the following (click on image to enlarge):

Overall, African-American students are much more likely to use computers to practice or drill on math facts than White students.  Given the significant achievement gap that exists, these differences partly explain why, overall, the there is a negative correlation between using computers to practice or drill on math facts and math achievement.  I can’t be entirely sure about the degree to which race confounds that overall relationship without access to the raw (restricted-use) NAEP data.

But, more importantly, is the figure above problematic?

Tags: computers, drill, math, NAEP

One of my posts from almost four months ago has been resurrected by comments from Tina K. and Amir.  In that post, I suggested I’d do some more digging.  So, I dug.

Some background…these are NAEP data with tables and statistics generated by the NAEP Data Explorer.  The NDE is an awesome (free!) tool for analyzing NAEP data.  It’s kinda amazing to me that more folks, including the media, haven’t picked up on this tool to do some really quick and easy data analysis.  Anyway, to satisfy the inquiries of Amir and particularly Tina, I analyzed 8th grade data from the 2007 NAEP administration.  The 8th grade assessment includes the best “type of computer” use data; i.e. we can break down computer use by some specific applications.  That’s what I did.  The math achievement results disaggregated by response category follow:

[NOTE: click on images to enlarge them]

So, quite clearly, the same results appear as with the 4th grade data in my earlier post.  The group of students who never or hardly ever use computers score significantly higher than the other groups, across all applications.  Again, I don’t know anything about those students demographically.  But, still…

Going one step further, I ran a regression analysis with four of the independent variables (i.e. the “types” of uses).  The NDE would only allow me to use four; it’s a statistical/psychometric thing…don’t ask.  So, I took out word processing and drawing as those seemed likely the most remotely associated with math achievement.  The results are as follows (again, click on the image to enlarge):

Make sense? Yeah, I know, unlikely. Unless you are well versed and regularly practiced in regression analysis, there’s no reason that would make any sense to you. So, let me try to summarize some key results:

• Of all the variance in math achievement, differences in these four types of computer use for math account for 16%. That’s not that high; not terrible, but it’s safe to say that, overall, computer use for math does not explain much of why kids differ on their math scores.

The independent variables are “contrast coded” which is the right way to do this analysis. But, it limits what we can say. That being said,…

• The average score for a student who never or hardly ever uses computers in any of those ways is 291.
• Students who use the Internet for math once every few weeks score a bit higher than the previously mentioned student (i.e. never or ever uses in any of the ways).  That is, by simply adding Internet use for math once every few weeks adds a little bit to the average score of the non-computer using student.
• Same story for using graphing programs for charts.
• Adding Internet use once every few weeks AND graphing programs once ever few weeks has a cumulative positive effect on the non-computer using student (again, though, VERY small positive effect).
• The more frequently kids use math programs to drill on math facts, the lower they score.

So, there you have it.  I’ll likely play around a bit more with the NDE to see what else I find with respect to other subjects and other uses of computers.  Fun times!

Tags: achievement, math, NAEP, technology

If you’ve ever taken a college or graduate level course, surely you’ve completed some kind of summative evaluation form at the end of the semester.  At Hofstra University, where I worked for 5 years before this past academic year, we called them CTRs (Course and Teacher Ratings).  They consisted of a bunch of Likert scale items (strongly disagree to strongly agree) and a few open-ended questions.  For the most part, students hated doing them and faculty members hated having to use them.  I didn’t love the wording of many of the items, but I always asked my students to please take them seriously as an opportunity to let me know how I was doing.  I told them that I would receive an analysis of the data and their actual responses to the open-ended items.

As part of applying for tenure at VCU, I have to demonstrate growth as an instructor.  So, I plugged the CTR data from my 5 years at Hofstra into EXCEL and discovered some very interesting things.  The graph below represents the data from a scale (composed of 5 items) that purports to be an overall measure of the course and the instructor.  The x-axis represents the time points from Fall 2002 to Spring 2007.  The y-axis represents the range of scores (which can range from 1 to 5).  For this particular scale, the lower the number the better.  But, I flipped the y-axis so that it looks like “better is higher;” a more standard look for such a line graph.  The blue line represents my ratings; the red line represents the average score of the other faculty members (including adjuncts) within the program area.

[NOTE: click on image for larger view]

I entered the professoriate with NO teaching experience.  I guest lectured once while I was getting a masters degree, but that was it.  Hofstra took a bit of chance on me in that respect and I am eternally grateful to them for that.  But, the graph clearly shows that my ratings were not as good early in my teaching career as they were last year.

I should also add that in my first couple of years as a professor, i was asked to teach a few sections of an undergraduate foundations of education course.  I thought I would really enjoy working with undergraduates considering a future as an educator.  But, after teaching a few semesters, I began to really dislike it.  I had a hard time dealing with the students’ limited understanding of and experiences with education.  Seemingly simple concepts such as “charter schools” were completely foreign to them.  My ratings were not terrible for those course sections, but my department chair and my colleagues and I decided that my time and energy was better spent working with graduate students.

Overall though, I think the graph tells an accurate and interesting story.  Quite simply, I’ve improved significantly as an instructor.  The more comfortable I’ve become in my own skin and the more I’ve been able to find my own voice, the more I’ve been able to engage my students.  That’s my interpretation of the data.

Academics bemoan the use of “quantitative” ratings of their work as instructors.  But, I think it’s critically important that we ask our students to reflect on their experiences in our classes and to provide us with data about our work.  I wonder how many of my P-12 colleagues/readers have similar systems in place to collect and analyze summative or formative data about their performance directly from their students.  Do you?

Tags: data, evaluation, teaching

In response to Scott McLeod’s 140-character book review contest, I submitted an entry that summarized/synthesized two books in less than 140 characters.  I wrote:

ORGANIZING (EVERYBODY AND/OR EVERYTHING) IS DIFFERENT NOW.

That’s it.  Put together “Here Comes Everybody” and “Everything is Miscellaneous” and that’s what you get.

For those who read the books, those 57 characters should make perfect sense.

For those who’ve read either or neither, it should be fairly self-explanatory.  But, if not, basically, HCE is about how Web 2.0 lowers (and in many cases, eliminates) the transaction costs of organizing a group of like-minded individuals to achieve a common goal.  EIM is about how the digitizing and tagging of information means that old organizing schemes developed for physical information (e.g. the Dewey Decimal system) have given way to digitzing+tagging+aggregating.

That’s it.  Class over.  Thanks for coming and you can now go read something that actually requires book-length treatment.

Tags: digital, organization, Web 2.0

I’ve long had an idea of what kind of school I’d like my child(ren?) to attend, but I’ve had a hard time articulating it.  Fortunately, there are plenty of smart and creative bloggers and academicians out there that help me learn and think.  In fact, I still can’t articulate everything fully, so you’ll have to settle for a few links.

So, for me, the future of schooling is approximately:

THIS +THIS + THIS + THIS + THIS

Ubiquitous computing and, therefore, ubiquitous learning.  Knowledge as rhizomatic and negotiated.  IEPs for every child.

I can dream, right?

Tags: education, future, knowledge, learning, school