August 12, 2008

Nobody Is Representative

Posted by Kevin | Print This Article

An interesting conversation in the comments at Acronym on their “blog backlash” case study, and then continued in Lisa Junker’s post on reaction to criticism. I said my piece in both comment threads, but there are some very interesting concepts here that merit further exploration.

First, going back to my original comment on the case study, in which I said it was patently ridiculous to suggest that bloggers are representative of majority viewpoints. That’s because I think it’s absurd to suggest that any one individual member is “representative” — be they a blogger, a listserve participant, a committee member, a past chairman, an officer, or whatever. She might be insightful, she might be wise, she might have good ideas, she might have found an issue that no one else had considered, she might be cranky and contrary — but none of that makes her representative.

As I said in my comment on the second post, we cling to comments from active or vocal members all too often because we don’t have anything else on which to rely. This is because many associations fall into a structure along the following lines:

• We answer to a Board, comprised of volunteers who by the very virtue of their self-selection as board members have proven that they are unlike most of the members
• We work with committees, comprised of volunteers who by the very virtue of their self-selection as committee members have proven that they are unlike most of the members
• We do a survey of the full membership every year or two that by its very nature is outdated by the time the results are compiled

So when we hear from real, actual members, we enthusiastically embrace what they say as “representative” because we don’t really know anything else about what our members think. (Lisa’s story about an association taking a year to revise a program because one member wrote a letter — this is NOT unusual, I’ve seen very similar things happen.)

BUT, now we have lots of tools and data available to determine what our members really “think” (I’ve written about this before), and so we should be a lot more clued in and able to respond to criticism in an appropriate way (when it’s merited, fix the problem; when it’s not, explain why it’s not). HOWEVER — when I refer to what members “think,” I’m really referring to what large percentages of them “do” (actual behavior) coupled with some ongoing qualitative queries.

I will use ASAE as an example. I did a search of old Blogoclump posts to find someone saying something about ASAE that I didn’t agree with. It wasn’t hard; Ben Martin apparently wrote a post saying that he’d decided ASAE’s listservers were no longer valuable and he dropped out of them.

I, however, find the listserves to be very valuable, and in fact the only tangible benefit to my ASAE membership (except maybe the magazine), and if I were to drop out of the listserves, dropping out of membership altogether wouldn’t be far behind (though I agree the software could use an upgrade).

So, the question is –

Is Ben representative of ASAE members?

Or am I representative of ASAE members?

I would posit that NEITHER OF US is representative, we’re just two individuals with blogs, and ASAE should no more base its decisions on what Ben thinks, or what I think, than on what the man in the moon thinks. What matters is the actual behavior of the actual members on a large scale.

How many subscribe, and to how many listserves, and is it growing or shrinking? What’s the percentage of lurkers versus active participants, and is it growing or shrinking? How many people are “dropping out” and how does that compare to prior years? And very important from the quality side of it, are the number of “frustrated” emails sent to the listserves (me-toos, complaints about repeats, people asking how to unsubscribe) growing or shrinking? This sort of data (both the numbers and considered quality reviews of the listserve conversations), reviewed on an ONGOING basis, could easily tell you if the software needs revamping (and, I’m willing to bet, would have revealed that the listserves needed some tweaking a long time ago) or if more drastic changes should be considered.

This is just one example — but it’s an important example of the type of analysis that associations should be doing about all their programs, all the time. Weighing programs and analyzing member behavior is not something to do once in a while, or when you get some complaints, but a regular part of what you do. Kill the programs that aren’t working (or change them). Create new ones. But if something is working, you should know it’s working, and not go all frantic when a small number of people don’t like it. You should be doing enough that there are other things you offer that they DO like … and if they don’t like anything, well, then why are they members at all?

(Oh, and I know the case study actually detailed a complaint about a board meeting, not a program, which I think may have warranted all of a thirty-second response rather than a day of hand-wringing … but I enjoyed where the comments took us.)

Comments

Leave a comment

Name (required)

E-mail (required)

URI

Your Comments

Notify me of followup comments via e-mail