Monday, January 28, 2008

Update and Status of Charter Survey

Thanks to all who have taken the time to fill out the charter survey. We are more than half way through with our data collection, and my initial reviews of the data have already proved fascinating. I am hopeful that we can start posting some very basic data as early as late March or early April. However, I expect to continue analyzing data and posting results for many months following the initial post.

Some of you have written to ask questions about how many survey responses we plan to collect, and how many we truly need to get a good read on the community. Please understand that sampling is a combination of science and art, and I am taking great pains to get a reliable sample for the charter study. Let me just take a minute to explain some of the thinking behind my sampling methodology and goals.

If you have ever taken an introductory statistics class, you may recall that a truly random sample can yield very reliable results with as few as thirty respondents. Unfortunately, random samples are almost impossible to achieve in the social sciences. Add in the fact that our subject matter is of a deeply personal and highly confidential nature, and sampling becomes even more challenging. Furthermore, I do not just intend to post the aggregate results of the study. In other words, I plan on running crosstabs and segmenting respondents into groups based on their answers so I can then report the statistics from just these groups. To do this I need a quality sample not only in terms of numbers, but also in terms of inclusiveness of key groups.

With my sampling methodology I am purposely seeking out groups of respondents that represent a mix of, initially, geographic and demographic groups. I then compare these different groups to the broader sample to determine if these groups have a tendancy to respond differently to the survey than do other groups. If they do, I go back and assure that I have an adequate number of respondents for each group, and balance or "stratify" the sample so that they are adequately represented relative to appropriate mix in the population. Also, if I find that key geographic or demographic categories are under-represented, I can go back and recruit respondents from these categories. In the end, I expect that the sample will include submissive men from every state and every major ethnic/demographic category, with enough respondents in each to at least assure a confidential and directionally accurate sample

I also do this on other key variables that come from the data itself. For example, because I am posting for survey respondents on Around Her Finger, I want to make sure that those who heard about my survey on that site, if they are significantly different than the total sample, are not over-represented in the results. I am doing the same thing with other sites where either I have mentioned the study, or others have mentioned the study on my behalf.

All this takes a great deal of work, and I believe this effort will make my results unique and far more valuable than the "poll and post" approach to survey research that dominates the web today.

Thank you again for your participation and patience while I work through the results.

No comments: