Sep 23, 2016
Scott and Randy tear into the history and problems of comments on "news" sites, and identify the most overlooked problem. They then talk about current and future solutions (well, other than just giving up an shutting down.)
Popular Science -"Why We're Shutting Off Our Comments" -Sept 23,
Shadow of the future: "The shadow of the future promotes cooperation in a repeated prisoner's dilemma for children"
Original paper: Bargaining, Enforcement, and International Cooperation by James D. Fearon
Shutting down onsite comments: a comprehensive list of all news
How the Huffington Post handles 70+ million comments a year
We discussed the history of HuffPo comments with Justin Isaf in Jan 2015
Reforming the Trollosphere: Creating Conversation in the Comments
The Coral Project:
New York Times: Quora: How does the NYT determine which articles have comments?
New York Times: A Community Manager Walks Into A Bar:My AMA with Bassey Etim, Community Desk Editor at The New York Times The Engaging News Project:
Hey reporters: An alternative to #DontReadtheComments: Jump
Case Study: Yahoo! Answers Community Content Moderation from Building Web Repuation Systems
The Washington Post is using Slack to create a reader community focused on the gender pay gap
Scott: Hi listeners, in this episode we ask why do comments on sites suck so much, and what can we do about it?
Randy: They're sucking because they lack context, and we'll tell you what that means.
Scott: Now, this isn't a new problem, and many are trying to address it. We'll share their approaches ...
Randy: ... And give our recommendations based on our personal experiences.
Welcome to the Social Media Clarity podcast, 15 minutes of concentrated analysis and advice about social media and platform and product design.
Scott: I'm Scott Moore.
Randy: I'm Randy Farmer.
Scott: We're discussing the problem with comment sections. You may have heard that a number of news sites have been shutting down comment sections in the last couple of years, or generally complaining about the poor quality of comments they receive on their articles, and we think that there's a real simple problem here, and it's the model in that people are presented with just a blank text box with no context about what to say or how to behave.
Randy: Part of that is because we don't know who the audience is. It's not clear from a plain text box who you have in mind when you're writing a comment, and what you're actually writing about. Are you writing to the publisher of the article? The reader? The commenter? The author? It's not at all clear, and I don't think the publishers were even sure. I think they assumed that the post, the content it self, would be a sufficient context for commenting, if they thought about context at all. One way I like to put it is, there's no "to:", expressed or implied, when a visitor creates their own context. Is it to the author? The publisher? The topic? Or a reply to another commenter? There is one context that I like to refer to all the time, which is when you post a public content, it's actually to God, Google and everyone.
Scott: This creates an attractive nuisance. The vicious circle goes like this. Publishers are not creating a clear context to their commenters, and without that clear context, people don't have enough context to care about each other, so they don't really focus on developing relationships. They tend to focus on being an audience to the rest of the world, and they have their own axes to grind or they ignore the content of the article, and post anyway, and these low-quality comments tend to wind up drawing more bad comments than good comments, and the circle starts all over again.
Randy: In contrast, there are communities with blank text boxes that have strong context and therefore have less difficulty with comment quality, because they're constructed around either topical content, or group goals, and they tend to be smaller and more intimate.
Scott: These tighter contexts provide what's known as "the shadow of the future," and that is, that's the probability of future interaction. If you expect to interact with other people in the future, you treat them differently. If you don't expect to interact with somebody in the future, then your cooperation is going to be lesser than. It's like comparing a small town diner where you expect to see the same people over and over, you're going to be nice to them, verses a bus station where everyone's passing through, and bus stations aren't really known for their friendliness.
Randy: Now it's time to discuss how others are trying to address poor quality comments on their sites.
Scott: For too long, folks have been treating the symptoms. There's a long list of sites that have closed their comments absolutely, completely, but there's a cost to that. You lose your SEO from comments. You lose potential ad revenue from people participating on the page where you're selling ads. Some sites exert editorial control over which content can have comments enabled, and for how long. For example, the Philadelphia Enquirer, the Guardian, Fox News, all pick and choose which pieces of content are going to allow comments at all. This can increase your editorial costs and you can also suffer from a dip in your SEO and ad revenue from people commenting.
Randy: Some, like Ars Technica and Boing Boing have put comments behind a click. This is an editorial speed bump. It's complex and it's all about context, and bad comments can cost you significant revenue. When I worked at Yahoo, Yahoo Health had comments related to articles about drugs and treatments, and when the drug companies were advertising, they were paying the highest ad rates on the internet, and they didn't like the detracting and often medically dangerous comments that were showing up on the same page as the article about their treatment or drug. They moved the comments off of that page in order to recover that revenue. The critical context turns out to be the advertisers for many of these applications, not the users.
Scott: Then we can't forget the ever popular increasing your moderation. Whether moderation happens before or after publication, these systems wind up being expensive, mostly because they don't scale well, and definitely don't respond quickly.
Randy: Some are pushing moderation tasks to Facebook comments, and for me, this is completely baffling, because now you have confused the context one more time, because now, instead of just the other people reading and the other people commenting, you now have brought in the entire user's social graph. Anyway, who gets notified when you post on a site using remote Facebook comments? Who are you talking to? Are you writing for your Facebook friends or are you writing for the author of an article?
Scott: We've come across a really novel approach. Make your commenters pay before they can view or comment. Tablet magazine is a magazine for a non-profit organization, and they actually charge for commenting. They have a daily rate of 2 dollars, or a monthly rate of 18 dollars, or a yearly rate of 180 dollars, and you might think, "Who would pay to comment?"
Scott: Well the answer is no one. They killed the comments on their site. All of their commenting happens on their Facebook page where they repost the articles anyway. According to them, this is exactly what they wanted. They are very happy with it exactly happening this way. If somebody wants to comment from the wild web, then pay for it, and they'll be happy to moderate your comment. That works out well for them, but it might not work out well for you if you're relying on things like advertising revenue and SEO.
Randy: This has been a problem for quite a
while. It's well known. The grousing about it is everywhere, and we
now join that group, but there have been several efforts to
standardize and platformize. One of them is Civil Comments. Civil
comments is a platform launched by Aja Bogdanoff who is involved
with Ted Talks Communities, and Krista Morgan. Aja told Tech Crunch
in October, "We need to change how people are submitting their
content. We need to make sure that we're giving them good reasons
to behave well," so when you write your comment, in order to post
it, you actually have to review 2 other comments on the site for
quality and civility. This gives you a chance to edit your comment
before submitting. Then comments may go live or be held for review
based on whoever is using the platform.
This approach is a definite improvement, and it starts to set the context, but it's only after the user has invested in writing a potentially context-less comment. We think this approach might be able to be improved by changing the order, getting people to read comments from others before composing a new one.
Scott: Another tool for publishers to facilitate curation in moderation comes out of the Coral Project. They have one tool called Trust. It's actually part of 3 planned modules, Trust, Ask and Talk. The Trust module is largely so that journalists can find new sources, reveal potential troublemakers and identify useful contributions within all the contributions that are going on. It's not out yet. Some people are experimenting with it, and it still doesn't address what we're talking about, which is the whole idea of context. There's nothing to provide context. It's really looking at things after all of these context-less comments come out.
Randy: So far it looks like it's very early and there might be a little bit of reinventing the wheel, but we'll see how it turns out, and I'm certain they'll uncover some useful lessons I hope they share with us all.
Scott: The New York Times has really changed how they're operating with comments. They started by following the popular but misguided, generic goal of "building community", but that lacks context, and that's the whole problem we're talking about. Since then, they have transformed into a better "letters to the editor." As Bassey Etim, the New York Times community editor, says, "Our goal is to have every New York Times comment thread offer tangible, added value to each article for our readership."
Randy: That's a goal.
Scott: That's a goal, and it limits the scope. With that goal clearly stated internally and externally, that allows them to select which stories have comments and how long the comments are going to be open on a particular story, and when those comments come in, they're human reviewed. They prioritize what comments are reviewed based on whether it's on the home page, whether it's getting a lot of attention, and when they review their comments, they hold off on publishing every comment just because it passes reviews. They actually hold on to a couple of comments until there is a spectrum of positions that support or add to the article.
Randy: This does a great job of modeling.
Scott: Exactly. It's done by your paid staff and it teaches regulars what's going to wind up getting published, and then also they curate and they highlight. They do this either by having picks of particular comments that really add to the context of the rest of the article, or they do modeling by highlighting the New York Times Picks community by doing profiles of folks.
Randy: I really like that they preserve the context, as I was talking about earlier. They put the comments behind a click, which is the same kind of speed bump as I described before, but it's displayed as a pullout sidebar that allows the article and comments to scroll independently, so it keeps them connected, but it doesn't detract from either thing. It's great.
Scott: You're right, but ultimately with all of this, there's still just an empty text box at the top of the comments, which makes it way too easy to skip a lot of this really great context modeling that they're doing, so good effort, but we're still calling them out on the one piece that we're saying is, we're missing context.
Randy: My advice to them would be instead of just putting the FAQ link there, if you've never posted before, actually making you read the shorthand version of their FAQ, which is pretty short. After the break, our recommendations.
Randy: There are plenty of problems to go
around and lots of people are trying different things, but we're
going to tell you what we think. We think the most important thing
is to decide the context. Ask yourself the questions that are
important. Why do you want comments at all? What do you want from
the commenters? This is a behavioral question. What benefits do the
commenters get? How do we want them to behave? Gather those
questions clearly in your minds, and then talk about how you might
be gathering this information. Once you know these things, you want
to communicate them clearly. You want to be transparent about your
answers to those questions. If it's about ads, is that why you want
comments? What are you going to do to trade off to get that
revenue? You want to communicate everything about that.
Don't just present a white box. It has no context. Make it really, really obvious by putting context everywhere. Make knowing the context a speed bump. Some examples include making context a click through, as we talked about a couple different ways already. Put the text box after or to the side of other comments, and we added that to the side of after we saw the New York Times. That model is pretty cool. Reward and require reading other comments. I worked on a project called Discourse.org, which is a message board system, and before you could contribute, you actually had to read through threads.
Scott: Part of communicating your context
clearly is making sure that you've got the right technical bits for
communicating the context. One of the challenges I found with
working with traditional UX and UI approaches is that they tend to
focus on what happens with one user behind the keyboard and not
what's happening with hundreds of thousands of users in a social
context, all taking that same action, and this plays out in terms
of whether you want to have a community talking to each other or
whether you want to have an audience talking to the publisher. If
you want a community, set the context. Put the reply box on the
bottom. Put the replies in chronological order. If you're looking
at having the audience speak to the publisher, having the text box
on top or having comments in reverse chronological order lends
itself to talking to the article as opposed to other people.
Labels can help. Consider changing the word "comment" to something else. Discuss, reply, contribute, and consider changing the label of your "like" buttons to "respect." There's something that backs this up. The Engaging News Project did a little research on simply changing "like" buttons to the word "respect," and they found that respondents who saw the respect button clicked on more comments in the comment section, and from an angle of participation, respondents seeing the "respect" button clicked on more comments from other political perspectives in comparison to either using "like" or even "recommend."
Randy: That sounds great.
Scott: Once you've set and communicated the context, it's important to shape it and reinforce it. Be willing to decide if comments are even needed on individual stories, much like the New York Times does, and this is a decision that can be based on your staffing capability, if the author is willing to or even able to respond in the comment section, if the topic is polarizing or if the top has recently had comments enabled. Also, set expectations for how you would like people to behave. Bassey Etim at the New York Times recommends you consistently tell your readers that they're part of a quality club, and this is the kind of quality that we're looking for, and this goes right back to our questions, "what benefit do the commenters get?" And, "how do we want them to behave?"
Randy: You also want to model commenting behavior. Calling back to the New York Times policy of holding comment publication until a balance is available is a way to teach people what gets published. Make sure your content authors and staff are participating to your best standards.
Scott: Don't forget, we're talking about people here, so directly engage with your commenters. Work with the folks who are writing the content, creating the content, to engage in the comments on their posts. This increases the quality of user contributions, and again, there's a little bit of experimentation from the Engaging News Project that found that when a reporter interacted in a comment section, the chances of an uncivil comment declined by 15 percent, and the commenters were 15 percent more likely to provide evidence over opinion when the reporters participated. Make sure your moderation staff is there supporting authors, so they can focus on meaningful engagement with the people commenting.
Randy: It's really popular to talk about
recognition, reward and celebration, and in appropriate measure,
these are good practices. Just don't substitute recognition and
reward for diligent moderation. Here are some recommendations that
don't require special technology, just people.
Dedicate a portion of your moderation strategy to finding good content. Reward the kind of comment quality you want. Reply to the person directly, and thank them or congratulate them for a particular quality. Of course, highlight and promote the best contributions, and do a retrospective of model commenters.
Sometimes the technology you have just won't cut it, and you need to consider an alternative approach. One approach I've used with several clients, and has been implemented in the Telegent platform is the ability for users to be the frontline in identifying the worst content, and allowing them to flag it and then automatically hiding it, perhaps for later review, but I must warn you, we're calling this out as an alternative approach because if you do it wrong you can actually make things worse. If you do it right, the response time between a bad post it disappearing can drop to seconds, fundamentally improving the apparent quality of your content, and discouraging bad contributions from vandals.
Scott: Sometimes you have to go back to your
very first question, and ask yourself why you want comments, and
realize that free text comments are not the right choice for your
community or for feedback, and if you really want a community,
consider a purpose-built community platform such as Discourse.org
or even Slack. Now, the Washington Post is trying out an experiment
where they have a small Slack community of Washington Post readers,
mainly women, who are focusing initially on issues of pay gap, but
it's turning into other discussions, which are becoming rich
sources for the Washington Post to generate more stories or greater
insight on this particular topic, or consider personal stories from
your audience or your community that can give insight into a topic
without a lot of back and forth that can be the source of conflict,
and also consider that text is not your only tool.
Submissions of images or videos, particularly if they are coming from members who are close to a particular topic or close to a particular location, where they can supplement or add flavor to the original topic are also ripe sources for content. That can give a sense of community without resorting to a blank text box.
Randy: It was a challenge putting together this episode because it's difficult that there are so many different variants of text boxes for user feedback out there. There are so many techniques that people are using in an attempt to fashion quality out of that content. For example, we didn't talk much about moderation techniques here today. That's what people talk about a lot and we hope to have an episode about that in the very near future. If you have ideas for specific things we should discuss in this area, please give us feedback. You can reach us on Facebook at Social Media Clarity and on Twitter as @SMClarity.
Scott: Thank you so much for listening.
Randy: Yeah, see you next time.
For links, transcripts and more episodes, go to SocialMediaClarity.net. Thanks for listening.