In advance of the Leaders Debate, the EDV team turned to a polling company to recruit more than 300 people to use the Democratic Reflection app. Participants were given a pre-debate survey about their views on the election and a post-debate survey about the event and their experience using the app. At the end of it all, the team gathered data from 242 participants; some didn’t watch all of the debate, or didn’t complete both surveys.
Plüss said most users were active for the full two hours, with activity peaking near the end, during closing statements. The EDV team was concerned 20 buttons would making viewing more complicated, but that didn’t seem to be an issue.
“It leaves the question open to see how many [statements] people would be able to take,” said Plüss, adding that some users indicated that they would have liked more options, particularly more emotionally charged responses—the impolite things people say when they’re watching a debate and something happens that makes them yell at the TV.
On April 16, during a BBC election debate featuring the leaders of the five main opposition parties, the EDV team made the app available to everyone.
It wasn’t planned as part of the study, and they did very little promotion, only a couple of tweets and a mention on the Open University’s Facebook page. The server was optimized to support up to 700 users, said Plüss, and that night, as he watched the number of logins inching upward, he grew anxious. Close to 2,000 people joined in. Sure enough, the server crashed.
“Obviously, if we had collected that amount of data without any interruptions, it would have been amazing, but the fact that we got that amount of interest, I think it’s wonderful,” said Plüss. “It’s not that people were just coming in, taking a look, and leaving—they were actually wanting to interact.”
In fact, that interaction is probably what caused the crash. The team had modified the platform, allowing users to view a live feed of other viewers’ reflections as well as their own responses.
“The idea was to create a bit of a more of a social community kind of experience,” said Plüss. “Technically, that’s what got us in trouble, because that stream of information, when you have such a high number of users, is huge.”
The server was down for about five minutes. But Plüss considers the extra run a success of sorts: To his surprise, some users returned, and the EDV team ended up with about 400 streams of data—not useful for longitudinal data, due to the gap in the middle, but researchers could still study responses to specific moments of the debate.
Besides harnessing more data, said Plüss, “It was amazing to see that people had the appetite for this concept.”
The EDV team is open to discussions with media partners in and outside Britain interested in using the Democratic Reflection app in future debates. The entitlements on which the questions are based might be different, and the technology might be applied in different ways, but the ultimate goal of providing debate viewers with the means to express their emotions as well as their needs could be applied in any country, said Coleman.
“When we were doing this in April and May of 2015, we were saying to each other, ‘Wouldn’t it be great if we could do something for the presidential election in the States in 2016?’ My guess is it is almost certainly too late for that,” said Coleman, adding that Germany’s election in 2017 might be a more realistic possibility.
“What we’re looking at is who we can work with for the greatest public good,” he added.
Launching Democratic Replay
Democratic Replay will be made public in May as a tutorial website. It will include an analysis of data gathered from the Democratic Reflection app—51,964 pieces of data to be exact, one for each time a viewer clicked on a reflection during the ITV Leaders Debate on April 2.
The EDV team is working on multiple layouts with programmer Alberto Ardito, a visiting research student from the Polytechnic University of Bari in Italy. One shows a timeline of audience responses; click anywhere on the timeline to bring up that point in the debate. Another option shows a histogram as the video plays. There’s also a “feedback flower,” its size corresponding to how many people chose a particular reflection during each 10-second segment.
“What our analysis will show is what was going on during the debate when people felt that particular entitlements were either being satisfied or being particularly not satisfied,” said Coleman.
Users will be able to filter the demographic profiles of viewers who provided feedback, making it possible to compare, say, how men and women responded to a candidate’s statement. Argument maps, fact-checking, and other components will also be available.
The EDV team finished the first iteration of Democratic Replay in October, five months after the election—too late to test if the platform could affect civic engagement or influence voting behavior. But between May and the end of the EDV project in September 2016, the team will continue to assess its use.
In the future, it might be possible to produce a Democratic Replay within a week of a debate, said Plüss. That’s well outside the 24-hours news cycle, but the team thinks the value is in turning the debate into an educational resource and a hub of data for journalists and other researchers.
As for the general public, the platform is going to be most useful for those who watched the debate, said Coleman. Both technologies, Democratic Replay and Democratic Reflection, are aimed at people who are “taking some notice of what’s going on,” but who may not have followed everything closely or who have not yet decided who to vote for.
“What I don’t think Democratic Replay as a technology does is open up a lot of space for people who are completely disengaged from the process,” said Coleman. “I think that will involve us in a different piece of work.
“It can be done, but I think that the problem is to throw everything in and try to take the disengaged, the engaged-but-confused, and then the engaged-but-highly opinionated all together and assume that you can create technologies for all of them. You can’t, I think.”
After all, this isn’t the worm.