J. Nathan Matias, Sayamindu Dasgupta, Benjamin Mako Hill

Abstract

This paper contributes to a growing body of work that attempts to measure informal learning online by revisiting two of the most surprising findings from a 2012 study on skill progression in Scratch by Scaffidi and Chambers: users tend to share decreasingly code-heavy projects over time; and users' projects trend toward using a less diverse range of code concepts. We revisit Scaffidi and Chambers's work in three ways: with a replication of their study using the full population of projects from which they sampled, a simulation study that replicates both their analytic and sampling methodology, and an alternative analysis that addresses several important threats. Our results suggest that the population estimates are opposite in sign to those presented in the original work.