By Peter Svarre

This article is published in connection with The Digital Ethics Compass project.

Ethics is often an abstract and distant concept for individual designers. What does it matter if I behave slightly unethically? Everyone else does it too! And in the big picture, my actions really don’t matter, do they?

Sometimes we can better understand design ethics if we go in the opposite direction and design a product that is 100 per cent unethical. You do not have to actually design the product, you can just make a ‘provocatype’, i.e. a provocative model that illustrates how bad things can turn out if we forget ethics in our design.

In this small post, we will make a ‘provocatype’ of a fictional future world where no designers think about ethics. How would such a world look? And what consequences would it have for the people living in this world?

Let us start by abolishing all rules and all ethics related to the collection and use of data. Companies and public authorities can freely collect data, they can store them however they like and they can use them for any conceivable purpose. First of all, this means that you friends and colleagues all have full access to the results of your latest examination at the venereal disease clinic. Your insurance company knows everything there is to know about your health and they have scanned all of your social media updates and just increased your insurance premium because you have been moved up into a higher risk group. Your commute time to work (on a bicycle) has suddenly doubled from 20 to 40 minutes because the police have installed surveillance cameras on all street corners and are issuing tickets for the slightest traffic violations. It turns out that the traffic on the bicycle paths was a little smoother when people did not slavishly have to follow the traffic rules and were able to improvise a bit on their journey.

Let us also remove all ethics and legislation for decisions made by algorithms. The Danish Government has realized that they cannot meet the 70 per cent reduction targets for greenhouse gases, and therefore they have passed legislation saying that all residents of Denmark need to install smart meters in smart light bulbs in their homes and let an algorithm developed by Netcompany manage heating and lights in all homes in Denmark. The basic sentiment is perhaps fine, but Netcompany has trained its algorithm to base its decisions on ‘the average Dane’, which is actually just 3.4 per cent of the population. The remaining 96.6 per cent of the population now live in homes where things are constantly either too hot or too cold or where lights turn on far too early in the morning or turn off right as someone is brushing their teeth. However, on the upside, Denmark has now reduced its CO2 emissions by a few percentage points!

Amazon has also finally arrived on the Danish market, and it has realized an old dream of ‘anticipatory shipping’, which means that Amazon will send orders that have not yet been made to its customers. Amazon’s algorithms have simply become so good at predicting our needs that we no longer need to order products ourselves. Amazon can use the data it already has about us to predict what we will buy before we know it ourselves (fun fact: Amazon has already patented this technology). The result is, of course, that customers are happy but they also spend a lot more than they did before this technology was introduced. And Amazon’s monopoly has now been strengthened in the e-commerce space and this means that they can now practically price their products however they like.

Now let us also remove all ethical considerations from the use of nudging in digital design. Here, of course, it is to a great extent children that are in the target group. Netflix has reintroduced the feature where a new episode of Peppa Pig automatically starts after the last one has finished. They have also optimized the design and buttons so that it becomes far easier for children to start new episodes and thereby sit around in front of a screen for longer. Netflix has also just launched a special children’s channel that can be operated easily be children as young as two.

Pension companies have also finally figured out how to get people to understand the importance of saving up for retirement. Using hordes of nudging experts, they have developed new digital designs that make it almost irresistible to save up for retirement. The result is that people move what money they can to their pension savings, the overall consumer spending in society drops and more and more elderly people die with large amounts of assets that are inherited to their children, causing them to have less incentive to take a job. The pension companies have more assets under management than ever before and become a new political power factor in Denmark, as they can decide on the direction of almost any investment in Denmark.

Finally, we remove all ethical scruples concerning the A/B testing of solutions on ordinary people. Tesla was the first to step up. They have had trouble getting their autonomous driving function to work in snowy weather, so they make an A/B test with two different versions of their algorithm. They do not know in advance which algorithm performs best in snowy weather, but after running the test for a month, they can see that one algorithm results in seven more deaths per million kilometres driven than the other. However, Tesla wants to be absolutely sure, so they let the test go on for another few months!

The stories could continue. But it should already be clear how important ethical digital design is to our common future as humankind. A world without either legislation or ethical considerations would simply be impossible to live in for both companies and private citizens.