Documentary ﬁlms are powerful storytelling vehicles. The best social justice ﬁlms take us to places we’ve not been before, engaging our minds and our hearts as they explore some of the most signiﬁcant issues of society: poverty, education, the environment, health, social equality. In ninety minutes or less, we are moved by these stories. But are we moved to action?
That is the question that Jana Diesner, assistant professor, set out to answer. “A great number of documentary ﬁlms are made with the idea that by bringing an issue to the public in an artistically appealing, compelling, and dramatic way, they can have an impact by effecting change,” she said. “Often that means an incremental contribution towards some bigger goal.”
However, it has been difficult to tell whether watching a movie on climate change, for example, actually moves people to alter their perception or habits. In fact, the success of a social justice documentary is often measured either by quantitative metrics such as the number of screenings, or in a more qualitative fashion by conducting interviews with small focus groups before and after a screening.
Diesner took a different approach that brings these solutions closer together. She wanted to ﬁnd out how people thought and felt about an issue before and after a movie was released, and to identify how much the ﬁlm’s content and its coverage in both traditional and social media contribute towards any change in the public discourse. She knew that by combining methods from natural language processing, social network analysis, and data visualization, she could get a more comprehensive, rigorous, and efficient grasp on the impact of a documentary.
With support from the JustFilms division of the Ford Foundation—a major funder of documentary ﬁlms—Diesner put together a team of graduate students from a diverse set of backgrounds to design and build ConText, a publicly available tool that facilitates impact assessment. In addition to evaluating impact after a ﬁlm has been released, they also bring this computational solution to ﬁlmmakers in earlier stages of work. Using ConText, Diesner and her team work in collaboration with various directors, producers, and funders to identify strategic opportunities that make the most of outreach and campaigning, ultimately increasing the impact of ﬁlms that are about to be released.
This collaboration doesn’t only make the ﬁlm’s impact more powerful, it also makes ConText work even better. “We gain a tremendous amount of valuable insights from working directly with ﬁlmmakers and those in the media. Their subject matter expertise is fundamental to our ability to improve our technology such that it becomes more useful and gets used to solve a speciﬁc real-world problem here and now,“ said Diesner.
Photo caption: Team members include, from left Andrew Higgins, doctoral student in philosophy; Shubhanshu Mishra, doctoral student in GSLIS; Sean Wilner, doctoral student in Informatics; Diesner; Kiumars Soltani, doctoral student in Informatics; Jinseok Kim, doctoral student in GSLIS; Liang Tao, master's student in agricultural engineering; and Amirhossein Aleyasen, master's student in computer science.
Putting ConText to work
After realizing she could apply a computational method to a seemingly abstract question, Diesner and her team had to decide what indicators they would use to measure impact. They then got to work identifying and developing methods and software that enable the measuring and analysis of these indicators.
It isn’t only important, though, to ﬁnd the right opportunities for connection. Diesner found that ﬁlmmakers must also be strategic in the way in which they communicate with journalists if they want to increase a ﬁlm’s impact. “In our analyses we sometimes see a strong focus in the press on portraying a ﬁlm as a high-quality art product, which is certainly a major goal and success for a ﬁlmmaker. However, if they more strongly present their ﬁlm as a vehicle for raising public awareness around an issue when talking to reporters, the focus turns to the issue, not the art. We have observed this trend for classic media as well as social media. On social media, interestingly, it is often the user community who ties the art product to a lively debate on a broad range of aspects of the issue,” said Diesner.
The House I Live In
One documentary that aimed to increase awareness and education concerning an issue is the award-winning ﬁlm, The House I Live In. Written and directed by renowned ﬁlmmaker Eugene Jarecki, the documentary investigates drug use in America, acknowledging the devastating effects drug use has on families while also challenging the notion that the war on drugs, as it has played out in the political arena as well as the criminal justice system for the last forty years, is having any positive effect. In fact, the movie asserts that it is the war on drugs and mandatory minimum sentencing that has made “America the world’s largest jailer, and destroyed impoverished communities at home and abroad, all while drugs are cheaper, purer, and more available today than ever.”
In their baseline model, Diesner’s team discovered that the issues of mandatory minimum sentencing and prisons were jointly discussed with other topics, mainly drugs and legislation. Diesner asserts that facilitating discussion of these topics when talking about the documentary could increase the ﬁlm’s chances of making an impact. Following the movie’s release, Diesner found that The House I Live In was covered by the press both as an art product as well as a vehicle for discussion around the war on drugs and the social issues that surround the topic. Early social media posts mainly related to screening announcements and introducing the movie's director, main characters, and issues. Eventually, the commenters on these posts not only tied the ﬁlm to the issues at hand, but also included themes not addressed in the original posts, such as the workings of the prison system, the experiences of people of color, and how drugs have affected people’s lives. The ﬁlm’s social media accounts continued the dialogue, and ultimately it is this engagement that has kept the issue—and the movie—in active discussion.
The movie was released in 2012 and, among its many honors, received the Grand Jury Prize at the Sundance Film Festival. Two years later, the front page of the ﬁlm’s website is devoted to educating readers and providing ways to act on the issue. Similarly, the documentary’s Facebook page remains active, notifying its nearly 20,000 fans about legislative developments as well as coverage of the drug war in the media.
Beyond the screen
ConText was developed to further investigate social justice documentaries, but the tool functions across a wide set of uses and users. From the very beginning, Diesner built ConText with a bigger goal in mind: to create a tool that conducted text analysis and network analysis in an integrated and efficient fashion, and to make this tool available across disciplines.
Diesner and her team are currently expanding the reach of ConText by branching out into the assessment of other types of media, such as feature stories and campaigns. “We have begun to think about impact assessment in a more generalized way as it also applies to other domains, including academia, politics, and business. We also continue to develop, apply, and evaluate methods and algorithms for constructing different types of network data based on natural language text data and jointly analyzing text data and network data. ConText is our sandbox for providing these novel solutions.“
In order to bring ConText to the end user, Diesner and her team are continuously running hands-on training workshops to participants from academia, the private sector, and governmental units. Diesner has conducted training workshops for this open-source tool with scholars in South Korea, Denmark, South Africa, Peru, Russia, and the U.S. “Seeing how people interact with the tool helps us to continuously improve the usability and routines offered in ConText. Users come to us with projects in mind and suggestions for new features that we might never think of, and we greatly appreciate that feedback.”
Note regarding image: Topics present in the dataset containing press releases about the documentary The House I Live In after the documentary was released. Each cloud represents a topic comprised of several words. The words in the cloud have been tagged according to the sentiment related to them. The topic positioning represents how closely the text represents the topic. The center cloud represents a more central topics in the text which in this case is gun, assault, county and judge. Other topics around the center in clockwise direction are in decreasing order of their fit to the overall text. The font size of each word represents the frequency of that word relative to other words.