The kids are going to be alright

Catchy headlines like “Heavy Social Media Use Linked With Mental Health Issues In Teens” and “Have Smartphones Destroyed a Generation?” advance a common trope of generational decline. But a new paper in Nature uses a new and rigorous analytical method to understand the relationship between adolescent well-being and digital technology, finding a “negative but small [link], explaining at most 0.4% of the variation in well-being.”

What really sets apart the new paper from Amy Orben and Andy Przybylski is that it aims to capture a more complete picture of how variables interact. The problem that Orden and Przybylski tackle is endemic one in social science. Sussing out the causal relationship between two variables will always be confounded by other related variables in the dataset. So how do you choose the right combination of variables to test?

An analytical approach first developed by Simonsohn, Simmons and Nelson outlines a method for solving this problem. As Orben and Przybylski wrote, “Instead of reporting a handful of analyses in their paper, [researchers] report all results of all theoretically defensible analyses.” The result is a range of possible coefficients, which can then be plotted along a curve, a specification curve. Below is the specification curve from one of the datasets that Orben and Przybylski analyzed.

Amy Orben and Andrew Przybylski explain why this method is important to policy makers who are interested in the tech use question:

Although statistical significance is often used as an indicator that findings are practically significant, the paper moves beyond this surrogate to put its findings in a real-world context. In one dataset, for example, the negative effect of wearing glasses on adolescent well-being is significantly higher than that of social media use. Yet policymakers are currently not contemplating pumping billions into interventions that aim to decrease the use of glasses.

Truthfully this is the first time I have encountered specification curve analysis and am quite impressed with its power. For more details, check out the OSF page,the writeup in Nature, and the full paper. Also, Michael Scharkow details how to apply SCA to variance and includes some R code.