The ethics of experimentation
The learning curve for figuring out how people will behave on your website is a steep one. It’s hard to predict their emotions. It’s hard to see how they would interact with each other. It’s even harder to figure out why user emotion is so contagious on the Internet.
Companies are probably all too familiar with the frustration this situation breeds. Most have felt the sting of their very best ideas being ignored, their ads not being clicked, and their product launches flopping.
While some companies might try to comfort themselves with the notion that everyone is facing the same problems online, major companies are conducting tests to see if they could affect users’ behavior, emotions, and the way they interact. They are doing it in a multitude of ways, from simple A/B testing where they change colors on a web page to see if it changes the rate at which people buy to showing different information to different audiences to gauge their reactions.
Recent User Testing Sparks Debate
Facebook confessed to a test that experimented with news feeds of 700,000 users in 2012 to see how people responded after reading negative or positive stories, and then published a scientific paper on it.
Then OkCupid, a dating site, made a similar confession of conducting an experiment that tried showing two users as a good match when they were actually a mismatch to see how users reacted. OkCupid founder Christian Rudder stated in a blog post: “When we tell people they are a good match, they act as if they are, even when they should be wrong for each other.”
The impact of these two experiments was small, but they sparked debate about the ethics of testing and experimentation on users. Many users were angered by the lack of permission or awareness about these experiments. And they were especially angered by the fact that their emotions (in the case of Facebook) and dating lives (in the case of OkCupid) were toyed with.
Consent or Pre-Approval Wanted by Public
Facebook didn’t submit any proposal to the Institutional Review Board (an independent ethics committee that requires scientific experiments to abide by consent and safety standards) for pre-approval of the study, according to Forbes. Instead, the social giant considered it another A/B test, which most tech companies and startups engage in on a regular basis.
Facebook technically does ask people for consent in its Data Use Policy agreement when people sign up, specifically saying that they may use the information they receive about you for data analysis, testing, research and service improvement. Many consider it to be a weak form of consent, as they don’t know the intent or the risks of the experiment, and believe that Facebook should provide an option to opt out from these experiments.
James Grimmelmann, Professor of Technology and the Law at the University of Maryland, believes Facebook may have crossed the line. He told Slate that exposing users to something that causes physiological status changes is experimentation is the kind of thing that requires user consent.
On the other hand, Dr. Adam Irish, Mellon Postdoctoral Teaching Fellow in Political Science and International Relations at Wheaton College, said that anyone posting on social networks should be aware that online behavior is traceable. If people will use free services online, they should be aware that their data is likely being collected, used and distributed, till it is determined what is ethical. Facebook also made an apology stating the test was poorly communicated to the users.
Unlike Facebook, OkCupid didn’t issue an apology. They simply went on falsifying their results and mismatching users. Though the company’s founder said that they simply did what other companies were already doing, conducting experiments without users consent can quickly erode trust, according to an article on BBC, especially when it involves presenting users with wrong information, like in OkCupid’s case. They simply put the users trust at risk.
The Impact on Conversion Rate Optimization
So how does this affect businesses that implement A/B and multivariate testing to optimize conversion rates on their websites? Washington Post’s Brian Fung asked: “If you’re lying to your users in an attempt to improve your service, what’s the line between A/B testing and fraud?”
When you’re conducting testing to create a better product or improve your service, there should be a clear ethical stance on testing for your company and how the user data will be handled. A New York Times article says these experiments are important, but they often have ethical issues associated with them.
If you are changing button colors or modifying text in a way that has the same meaning, but is presented in a different way, then your experiments are different from those in the examples above. You are not harming your potential customers with A/B and multivariate testing, so long as you are not conducting experiments that lead visitors to believe something false about any aspect of your products and services or themselves.
Nonetheless, as a business, you should offer a disclosure on your website that lets visitors know that their data is being tracked anonymously. You may even want to follow the example of sites like Pinterest that offer the strict “legalese” terms and conditions with a snippet simplifying each section. This way, your visitors can feel confident about how their data is being used, and how their actions are being measured.
Transparency in this manner should have little affect on your ability to optimize conversions, but it will have a major affect on the level of trust your visitors have when on your website. That kind of trust can be the difference between people buying from your versus buying from your competitors.
What are your thoughts?
Join in the discussion below by leaving a comment about your thoughts on the issue.