Could a computer be better at identifying depression than
a primary care physician?
That’s the suggestion of a new study that focused on
using machine learning to analyze Instagram photos. The study, conducted by a
researcher from the department of psychology at Harvard University and another
from the University of Vermont, analyzed nearly 44,000 photographs posted to
Instagram, exploring factors like what filter was used and how makes “likes” a
photo received.
The study included photographs from 166 people, some of
whom were depressed, and some of whom were not.
Instagram offers a variety of filters to change how a
photo appears, and the researchers discovered that healthy participants were more
likely to use a filter than depressed people. But if a depressed person did use
a filter, the most common choice was Inkwell, which is one of the filters that
turn a photo black and white.
Using earlier studies, the group taught the computer
system to identify markers for depression. For example, since earlier research
showed that depressed individuals preferred darker, grayer colors, the program
analyzed color as one factor, including the role that Instagram’s filters play
on those colors.
The program also considered factors like how often users
post, since depression often correlates with reduced social activity. In
addition, the system also counted the number of people in the image using face
detection and also factored in the number of likes and comments each photo
received.
Photos posted from depressed Instagramers were more
likely to get comments, but less likely to receive likes. The study
participants with a depression diagnosis were more likely to use no filters at
all, and when they did, often favored the Inkwell filter. And while depressed
individuals were more likely to show faces in their posts, they often had fewer
faces in each image compared to other users.
To see how well the average person is able to pick up on
signs of depression using only Instagram photos, the researchers also had a
group of people rate each photograph. The participants didn’t know they were
looking at Instagram photos, and didn’t know that the study was being conducted
on depression. The researchers asked each person to rate the image from 1-5 in
four categories: happy, sad, likability and interestingness. The ratings for happy and sad tended to
correspond with the images shot by individuals diagnosed with depression, while
the other ratings, including “likability” and “interestingness,” did not.
The researchers also took the same machine learning
program and applied it to only the posts that were dated before the individual
was diagnosed. Using earlier photos, the programs were still able to accurately
predict which users were depressed more than half the time.
The most common filter choice for healthy users was
Valencia, which makes photos brighter. Other popular filters included X-Pro II
and Crema, for healthy and depressed photographers, respectively.
In the comparison below, the image on the right contains
the type of photo features exhibited by someone with depression—darker, grayer,
and bluer, researchers explained.
“Photos posted by depressed individuals tended to be
bluer, darker, and grayer” the study stated. What’s more, photos from depressed
users garnered more comments, but fewer likes.
Ultimately, the study concluded that not only was
depression discernible in Instagram photos, but their method was actually more
accurate than some professionals’ diagnosis success rate. “Our model showed
considerable improvement over the ability of unassisted general practitioners
to correctly diagnose depression,” the researchers reported.
By "general practitioners," the study was
essentially referring to a primary care physician, and not a psychotherapist or
psychiatrist, and by "unassisted," the researchers meant a doctor who
was evaluating a patient simply by talking with him or her, and not using a
test, like a questionnaire.
“Using only photographic details, such as color and
brightness,” the study says, “our statistical model was able to predict which
study participants suffered from depression, and performed better than the rate
at which unassisted general practitioners typically perform during in-person
patient assessment.” Based on 118 earlier studies, the researchers found that
general practitioners only correctly diagnosed depression 42 percent of the
time.
But the researchers caution that the comparison between
people and machines in their study was "strictly informal" and
included to contextualize their findings, and wasn't the main point of their
research.
"We'd like the emphasis to be more on the fact that
this [is] a new way of thinking about improving early detection of mental
illness, than on the possibility that this early groundwork is already
outperforming trained medical professionals," said Andrew Reece, a
doctoral candidate at Harvard University and the study's first author.
Sources and
Additional Information:
