Wiki Surveys: Open and Quantifiable Social Data Collection

In the social sciences, there is a longstanding tension between data collection methods that facilitate quantification and those that are open to unanticipated information. Advances in technology now enable new, hybrid methods that can combine some of the benefits of both approaches. Drawing inspiration both from online information aggregation systems like Wikipedia and from traditional survey research, we propose a new class of research instruments called wiki surveys. Just as Wikipedia evolves over time based on contributions from participants, we envision an evolving survey driven by contributions from respondents. We develop three general principles that underlie wiki surveys: they should be greedy, collaborative, and adaptive. Building on these principles, we develop methods for data collection and data analysis for one type of wiki survey, a pairwise wiki survey. We then present results from www.allourideas.org, a free and open-source website we created that enables groups all over the world to deploy wiki surveys. To date, more than 7,000 wiki surveys have been created, and they have collected over 400,000 ideas and 10 million votes. We describe the methodological challenges involved in collecting and analyzing this type of data and present a case study of a wiki survey created by the New York City Mayor’s Office. The talk will end with some more general claims about social research in the digital age. [Joint work with Karen E.C. Levy]