Member Login

Lost your password?

Registration is closed

Sorry, you are not allowed to register by yourself on this site!

To what effect?

This is the question I asked this morning on Twitter.

Before I unpack that, though, let me first offer kudos to the many, many educators with whom I interact on Twitter and other forms of social media who have spent many, many hours this summer engaged in both formal and informal professional learning activities. Many educators I know spend far too little time over the summer thinking and learning systematically about how to improve their practice in the coming year. So, good on you for taking the time and energy to learn and think and share…

But, back to the question…

That’s just from this summer. And, I’m sure I’ve missed stuff…

Many of the educators who participate(d) in those events report that they are/were perfectly wonderful; “amazing” even. Apparently, this social media-aided PD is more powerful than any PD they’ve ever done; better than any grad school course they’ve taken. And, it may very well be.

But, it seems to me that many of the folks who take part in these events have been at it for a couple/few years now. And, they’ve become pretty good at sharing what they’re learning and even doing. There’s value in talking about and sharing ideas and actions, but that only gets us so far. Furthermore, I hear/read many knowledge claims about how awesome these ideas are. “Students are learning more!” “Students are so much more engaged!” etc.

So, I ask, what is your warrant for those knowledge claims? What evidence is there that all of these new forms of professional learning are making a difference for kids?

I’m not asking for the sorts of formal, systematic inquiry that most people associate with “research” (though, that would be nice). Among those I interact with on Twitter and other forms of social media, there are some wonderfully creative storytellers. So, here’s an idea: what if you spent this coming school year documenting how the professional learning you did this past summer translated into improved outcomes for students? What if you systematically collected artifacts that would help you tell a story (or stories) about how what you did differently this year had a positive impact on kids? Then, tell that story. Don’t tell a story of what you did; tell a story of what happened for kids.

If you really think that these forms of professional learning are making a difference for kids and that more educators need to be doing it, it behooves you to make the case.

“In god we trust; all others must bring data.” -attributed often to W. Edward Deming, though the origins of the phrase are contested.

Tags: , , , ,

21 Responses to “To what effect?”

  1. While I see value in this process, it’s complicated. I’m certainly in the camp of “here’s something that we’ve done” without necessarily sharing all the results. My hope is that I’ve shared enough of my failures that I never come across as someone with all the answers.

    However, in terms of sharing the results, I run into a few problems. For example, I shared at #rscon3 about customized learning. I’ve shared stories in my blog posts about what it has looked like (conferences, standards grids, paperless math assessments, inquiry days).

    I’ve been more reticent to share the “results” that are quantitative (my students scored 85% on the district tests compared to the district average of 36% and 32%) The problem with sharing this data is that it confirms the validity of standardized tests (which I don’t agree with).

  2. Jon Becker says:

    Hi John,
    I don’t necessarily need “quantitative” results. I even specifically suggested crafting stories.

    Think of it this way… imagine I’m a parent of a student in your classroom and I want to know if the new stuff you’ve tried with my kid this year “worked.” How would you respond? What evidence would you offer?

  3. Erin says:

    I think perhaps the line ‘students are learning more’ is a ubiquitous kind of ‘it’s all good’ comment for teachers on twitter trying to spread enthusiasm or convince skeptics. It’s misleading because only real constant in a classroom from year to year is the teacher- today’s students are learning what they’re learning and last year’s group no longer exists. (I mean, to correlate with data- the students themselves still exist).
    Maybe we on twitter want to stay away from lines like “I’m learning so much” or “I’m so much better than I used to be” to avoid appearing narcissistic (although, to the uninitiated, that’s probably the trait most associated with Twitter and other social media). I think that the stories you’re after do exist, and they’re made up of 140 character glimpses of insight and discovery.
    I’m very lucky because I teach English. My story of improving myself is developing into one where I get to help other students learn to tell their own stories. My reply to the hypothetical parent in your comment, as maddeningly passive as it is, would be to turn to the student and ask him or her: ‘What did you learn?’


    Erin (@erinneo)

  4. I appreciate the challenge, but I think that some of us are already sharing the kinds of stories you’re looking for, although we could probably do it more often. Independent School magazine is publishing an article I wrote with students about our blended learning history class. Here’s a post in which I write about a professional conference at which they presented. Neither of those would have been possible/likely without the connections I’ve made on Twitter (you, included :) ) and conferences such as EduCon. I also have student evaluations of all the units in the courses I teach. I generally try to pull from those when I present. I’ve talked about the results of reading aloud to my students, which I only started doing because of conversations on the English Companion Ning. Part of this article came out of the way an Ignite presentation I gave and the feedback I received pushed my thinking on reflection. Maybe not perfect examples, but are those the kinds of things you’re getting at?

    I guess where things sometimes break down for me is narrating effect on students after the fact. Most of that is a result of the sheer workload of being a full-time classroom teacher with three preps, but you’re right to push us to do that reflection and offer that evidence. I’ll just say that on a very gut level, it’s hard to read that post and not think about the extra freedoms the academy affords in terms of structuring your time.

    What’s more frustrating to me is people who present and don’t present evidence because they don’t have it to present because they’re not in the classroom. I don’t want to discount their voices entirely because of their previous experiences and the time that they have to think and read by virtue of their non-classroom jobs. However, I’d love to see them partnering with classroom teachers in their presentations and publications, and I’d like to see more teachers giving the opportunity to address conference participants.

  5. Jon Becker says:

    You lost me a little at the end, there, Meredith. I’m not sure what you’re getting at with the statement about “the extra freedoms the academy affords…” I’m not excluding myself from this expectation. I should (and do/will) hold myself to the same standards of documenting “success” in my own teaching.

    Also, I offer for your (and anyone else’s) perusal, this article on the Reggio Emilia approach and especially the section on “Documentation as communication” ( There, documentation happens as part of the learning environment; it doesn’t take much “extra” time. I wish I could share how my son’s teachers document his growth, both with the usual portfolio one would expect to see in a Reggio Emilia classroom but also via multimedia stories. We get regular (few times a week) emails with wonderfully written narrative accounts of the students’ experiences accompanied by web-based albums of still and moving images of the learning that’s happening. I know that’s a lot to ask of teachers who have larger “caseloads” of students and multiple classes, but there are possibilities that are really worth considering.

  6. Thanks for the response, Jon. Your post made me feel defensive, and I was trying to work through why that might be. I think my perception (rightly or wrongly) that you were talking down to (a certain sub-section of) educators rather than with them probably drove some of that line. (Lots of you language in the post.) Perhaps I should ask- Are you thinking that folks promoting conferences, chats, etc. have a greater burden of proof than those who aren’t particularly keen on them? In what ways are you narrating effects on students of your learning?

    I love the idea of documentation as communication and think that in some way our students’ blogs might be some of the same kind of thing.

    I also wrote a longer response here (which seriously cut into my nap time ;) )

  7. I tend to read posts like this from a too critical eye. It was the same with the posts about ISTE being cliquish (Holy crap, am I in a clique?) or the comments about teachers being angry whiners (I was critical of testing, does that include me?).

    I want to push innovation. I want to share what’s working. I want to offer solutions.

    However, I don’t want to come across as narcissistic. This becomes tricky. If I saw, “I’m learning” and “this worked for me,” it can become self-centered. If I saw, “this works” and “try this out,” it becomes narcissistic.

    In terms of artifacts, most teachers I know have a class blog where students are already posting artifacts. Most teachers are also sharing stories. Yet, it’s a fine line between sharing stories about one’s crafts and using students as a personal PR campaign. I’ve felt that tension at different moments when someone says, “Why don’t you have your students post something on this?”

    Bottom line for me is that it’s complicated and being complicated, it’s easy for me to move into a place of overly analytical self-criticism.

  8. Russ Goerend says:

    Why are only the “informal” events highlighted? What about ISTE? Edubloggercon? Same expectation of evidence for those events?

  9. Jon Becker says:

    Yes, Russ, same expectation. But, I highlighted those events because I’m empirically and professionally interested in the lines among/between informal and formal learning. And, I’m interested in open education. So, I want to know if people really are learning and if that learning transfers… or, are those events just “amazing!”

  10. [...] more about the benefits of connected learning. Thank you to everyone who continues to contribute. A post recently written by Jon Becker really made me think. Yes, many of us are good at sharing, collaborating, creating. But what do we [...]

  11. [...] more about the benefits of connected learning. Thank you to everyone who continues to contribute. A post recently written by Jon Becker really made me think. Yes, many of us are good at sharing, collaborating, creating. But what do we [...]

  12. I am going to give you a straight answer to here from my own experience:

    - I started using social media for my own professional development in December of 2009. After seeing my own growth of learning go, many of my colleagues both in my school and in my division have joined from my rave reviews of it.

    Through the use of social media, I came upon Daniel Pink’s video, read his book, and our staff had some conversations that removed awards ceremonies from our schools and focusing on motivating students intrinsically first.

    By September of 2010, we began the implementation of a school wide digital portfolio initiative for our 500 students that gave students more ownership of their learning, improved literacy, and helped to create a personalized learning environment for our students while creating a strong digital portfolio. We also implemented Google Apps in our school that has helped students and teachers to increase collaboration between one another. Although this is a long term goal and we see it as something that unifies our school and students in our vision, our division was so impressed with the project that I am now working to implement this for our entire division for 10,000 students. Although I cannot give you test scores, I have seen student engagement increase dramatically while restructuring the way we look at and create learning environments. It has made an impact.

    On a personal note, through seeing many admin, I was happy to create Connected Principals which has shared best practices with other colleagues around the world to share best practices. This has been a great resource for admin and I have seen many of the practices shared in this site implemented in other schools, while also relooking at traditions that have been forever embedded in schools that are not necessarily beneficial to learning.

    I could honestly say that my learning through social networking started a snowball that led to some great conversations with my staff, students, and community that helped these and other initiatives forward. Is it the only reason? Absolutely not as I am only one part of my staff team, but I know that getting immersed in this environment and helping others locally see the same thing, we are making huge strides in our schools in the last two years.

    As tough as it was to “toot” my own horn, you wanted this evidence how this has impacted learning. Our kids are learning in a more relevant way, because honestly, myself and others that I work with are focused on our own consistent and continuous learning.

    Hope that helps.

  13. Mikkel Storaasli says:

    Great conversation. Especially since I’m planning my EdD dissertation on this very topic. The type of “data”I would like to gather is tricky: it seems to me that there is not a 1:1 connection between “Connected Professional Learning” and student achievement, particularly because the quantitative data we might gather is itself flawed. (standardized test scores). This type of learning plants seeds that may bear fruit over a long period. George spoke beautifully about this That’s complex stuff to get at- learning environment, professional culture, expectations of students, engagement.

  14. Jon Becker says:

    Good stuff, George. Here’s where I’m looking for more, though… You wrote, “By September of 2010, we began the implementation of a school wide digital portfolio initiative for our 500 students that gave students more ownership of their learning, improved literacy, and helped to create a personalized learning environment for our students while creating a strong digital portfolio.” How do you know literacy improved? How do you know students “own” their learning more now?

    Also, you write, “I have seen student engagement increase dramatically while restructuring the way we look at and create learning environments…” What do you mean by “student engagement?” And, “I see it happening” is not going to convince most decision makers. Is there actual evidence of this? (and I’m not necessarily looking for test scores here).

    And, as you know, the story of Connected Principals will be documented and told after a thorough, systematic inquiry by my dissertation advisee. I’m eager to tell that story.

    Finally, digital portfolios + Google apps for edu… so, there’s a significant addition of digital platforms, but I guess I’m still asking the “so what” question. Folks will be doing some things differently in your system, but, so what?

  15. @Jon Do you believe that through connected learning we get better? That sharing innovation and idea, we will not all improve our learning. If you read things like Stephen Johnson’s work or the Power of Pull, you will see that we are trying to bring classes together where we are using social networks as a way to leverage learning. Personally, my own work has improved dramatically through this which I outlined.

    Literacy has improved as it has expanded. Yes our students writing has improved but they also are using digital literacies to create works that they could never do before. The way they represent their work is more diverse than ever.

    How we see the classroom has changed significantly. Students have more power of their learning as they explore their passions and interests, while still ensuring that they meet curriculum standards as a classroom. This is coming through my experiences and conversations with staff and students. Hard data is not there yet as it is still relatively new, but through my narrative, I have seen a dramatic improvement. Are we there yet? No. But I believe that our students are truly becoming learners. The thing with innovation is that it takes a long time to get hard data.

    Why do our stories not matter?

  16. Jon Becker says:

    Yes, George, I believe “we” learn lots through social media and other modern forms of connection. I guess I’m looking for evidence of transfer of that learning. And, yes, stories matter. But, stories about how things are different is not the same as stories about how things are better. Are things dramatically “better” for our students now?

    So, again, you say: “Literacy has improved as it has expanded. Yes our students writing has improved.” You KNOW this, but how do you know what you know?

    “Personally, my own work has improved dramatically through this which I outlined.” How do you know your work has improved? I ask this question of myself all the time. I’m trying all kinds of new things, trying to be innovative in my own teaching, but I don’t have enough evidence yet that what I’m doing now is any better than before. Again, I am doing things DIFFERENTLY, but am I BETTER now than I was before engaging with social media for professional learning?

    Here’s what I wrote to John T. Spencer in an earlier comment: “… imagine I’m a parent of a student in your classroom and I want to know if the new stuff you’ve tried with my kid this year ‘worked.’ How would you respond? What evidence would you offer?”

  17. Does it matter at all that our students are more prepared for the world we live in? Whether you agree with the shift in our society, the work that is happening in our schools is relevant to the world our students are in. I know this through the work that they have archived through their digital portfolios and the conversations that I have seen.

    As for literacy, when students are writing, it is now better. Grammatically and sentence structure has improved. Is it with 100% of students, I could not tell you, but I have seen it in many. The truth is that if it does not work for students, we do what is needed to help them whether digital or not. This is something that I have learned through my own networking.

    My knowledge and implementation has increased of what I am doing. I have learned to tap into the power of networks. Maybe I am not smarter, but I am more resourceful. Do you not think that if we get more people involved and have them share their ideas that our organizations are better?

    I guess my focus is more on my organization getting better. Tapping into them will help. As I sit here away from my school I have no data I can throw into the comment section of this blog.

    Questions we need to ask: Is learning more meaningful? Is learning more relevant? Is it really?

    I can say that cursive handwriting has improved in our school and show you a student work from one month to the next which will PROVE that they have improved, but does that mean it will help them in our world?

  18. I know my students are becoming more successful because I spend so much time with them. I see how they come to me and I see how they leave me.

    I know we have had conversations about anecdotal evidence before, but that is what how we do it. We compare our students from where they started to where they finished.

    I understand your desire for better “evidence” whether that be test scores, the results of action research, or before/after types of evidence. I don’t think you should be surprised that we furnish little of that. We aren’t (necessarily) taught that this type of sharing is important. Many of us still work in buildings with teachers that won’t even share their lessons with others.

    Funny though, you are asking for the exact same thing we expect our students to give us, proof that they are learning…. Maybe our professional learning plans should include student work….

  19. Paul Oh says:

    I have to say, I love the idea of “telling the story of what you did.” Of course, I’m no longer in the classroom, so it’s easy for me to get behind that notion given how time-consuming it is as a practice.

    I would suggest, though, that the frame for this question be not just “tell the story to prove your exciting new ideas worked, to parents/guardians, administrators, etc.” but also “tell the story so that you can reflect and learn from your practice.” Better yet, if there’s a shared culture in the school environment that this kind of reflective practice is the norm and has value, then perhaps teachers would feel comfortable going public with their stories. Not just about the successes, but also stories that focus on the why. “Here’s why I did what I did, and this is what happened.” And also allow for critical conversations to build around that story.

    Please excuse the plug, but stories of teacher practice – written by teachers – that focus on the “why” is one of the central features of a website that my organization, the National Writing Project, has helped develop (with funding from the MacArthur Foundation). Check it out if it sounds interesting: (Anyone from this comment thread can develop resources for Digital Is, fyi.)

    Really appreciate this post and resulting conversation, Jon. I’m going to have this in my head today, gratefully, as I attend edcamp sfbay.

  20. [...] At Educon 2.4 a couple of months ago, I co-facilitated a session called #chats and #camps: Examining the Impact of Social Media-Fueled PD on Classroom Practice and Student Learning.  That conversation was driven, partly, by a blog post I had written called “To What Effect?” [...]

  21. [...] To What Effect? You may remember this blog post which turned into this Educon discussion which then spawned an idea for a massive, distributed [...]

Leave a Reply