Could a computer be better at identifying depression than a primary care physician?
That’s the suggestion of a new study that focused on using machine learning to analyze Instagram photos. The study, conducted by a researcher from the department of psychology at Harvard University and another from the University of Vermont, analyzed nearly 44,000 photographs posted to Instagram, exploring factors like what filter was used and how makes “likes” a photo received.
The study included photographs from 166 people, some of whom were depressed, and some of whom were not.
Instagram offers a variety of filters to change how a photo appears, and the researchers discovered that healthy participants were more likely to use a filter than depressed people. But if a depressed person did use a filter, the most common choice was Inkwell, which is one of the filters that turn a photo black and white.
Using earlier studies, the group taught the computer system to identify markers for depression. For example, since earlier research showed that depressed individuals preferred darker, grayer colors, the program analyzed color as one factor, including the role that Instagram’s filters play on those colors.
The program also considered factors like how often users post, since depression often correlates with reduced social activity. In addition, the system also counted the number of people in the image using face detection and also factored in the number of likes and comments each photo received.
Photos posted from depressed Instagramers were more likely to get comments, but less likely to receive likes. The study participants with a depression diagnosis were more likely to use no filters at all, and when they did, often favored the Inkwell filter. And while depressed individuals were more likely to show faces in their posts, they often had fewer faces in each image compared to other users.
To see how well the average person is able to pick up on signs of depression using only Instagram photos, the researchers also had a group of people rate each photograph. The participants didn’t know they were looking at Instagram photos, and didn’t know that the study was being conducted on depression. The researchers asked each person to rate the image from 1-5 in four categories: happy, sad, likability and interestingness. The ratings for happy and sad tended to correspond with the images shot by individuals diagnosed with depression, while the other ratings, including “likability” and “interestingness,” did not.
The researchers also took the same machine learning program and applied it to only the posts that were dated before the individual was diagnosed. Using earlier photos, the programs were still able to accurately predict which users were depressed more than half the time.
The most common filter choice for healthy users was Valencia, which makes photos brighter. Other popular filters included X-Pro II and Crema, for healthy and depressed photographers, respectively.
In the comparison below, the image on the right contains the type of photo features exhibited by someone with depression—darker, grayer, and bluer, researchers explained.
“Photos posted by depressed individuals tended to be bluer, darker, and grayer” the study stated. What’s more, photos from depressed users garnered more comments, but fewer likes.
Ultimately, the study concluded that not only was depression discernible in Instagram photos, but their method was actually more accurate than some professionals’ diagnosis success rate. “Our model showed considerable improvement over the ability of unassisted general practitioners to correctly diagnose depression,” the researchers reported.
By "general practitioners," the study was essentially referring to a primary care physician, and not a psychotherapist or psychiatrist, and by "unassisted," the researchers meant a doctor who was evaluating a patient simply by talking with him or her, and not using a test, like a questionnaire.
“Using only photographic details, such as color and brightness,” the study says, “our statistical model was able to predict which study participants suffered from depression, and performed better than the rate at which unassisted general practitioners typically perform during in-person patient assessment.” Based on 118 earlier studies, the researchers found that general practitioners only correctly diagnosed depression 42 percent of the time.
But the researchers caution that the comparison between people and machines in their study was "strictly informal" and included to contextualize their findings, and wasn't the main point of their research.
"We'd like the emphasis to be more on the fact that this [is] a new way of thinking about improving early detection of mental illness, than on the possibility that this early groundwork is already outperforming trained medical professionals," said Andrew Reece, a doctoral candidate at Harvard University and the study's first author.
Sources and Additional Information: