Anglerfish are famous for the glowing lure they dangle from their heads that they use to catch small fish. But this is not the most interesting thing about them. Their most fascinating trait is how they mate. Males of the species, who are much smaller, lack a functioning digestive system. To survive they must combine, or fuse-mate, with the females. This means they bite into the female’s skin, releasing an enzyme that bonds them together for life. In exchange for the nutrients they need to live, they provide her with sperm. If this sounds desperate that’s because it is: only 1% of males live long enough to find a mate.

If the equivalent of the shiny lure atop the anglerfish were part of a tech product, deceiving people into making bad choices, we’d call it a dark pattern: a deliberate deception that causes harm to one party but benefits the other. But we don’t use this label to apply to nature because… well I’m not sure why. Evolution is about good design for survival and we can learn from its history, even if we don’t agree with its morality. But the history of design most UX designers know starts with the Macintosh. We have a narrow view of how design fits into the world, even into the world of business, and that works against us.

Corporations, like anglerfish and other living things, are designed to survive. They evolve and use tactics that manipulate perception. Consider how the tail of a male peacock is a kind of advertising. Or how that ultraviolet light in orchid petals, crafted to attract insects for fertilization, is part of its “plant marketing strategy.” There’s perhaps nothing wrong with trying to get others interested in what you have to offer. It seems fair and natural, doesn’t it?

Yet there is also the Cuckoo bird. It lays an egg in other bird species’ nests without the mother knowing. When the young cuckoo hatches, it pushes its unhatched step-siblings out of the nest to fall to their death. It’s a killer baby bird! We’d call this immoral, or homicidal, if humans did it, but we struggle to rationalize our morality with the natural world we’ve evolved from and are still a part of.

Founders, as a kind of creature to stay with the metaphor, don’t start companies to “make great user experiences.” Instead their goals, as explained to investors, are to generate profits, growth and value for shareholders. How society, the environment, customers and employees fare is often secondary. They may make things that improve our quality of life, but they are not generally altruistic. They hire people to help with their stated goals, and not for any other reason.

Remember that we had to shame automobile companies for years to put in seat belts. We had to fight for decades to get the makers of cigarettes to admit they were unhealthy. These were among the most profitable industries of their eras, and Big Tech has parallels to Big Auto and Big Tobacco. We might want a better designed world for everyone, but the literal design of corporations, the nature of their species so to speak, makes them creatures with more selfish ambitions.

In Mark Hurst’s recent essay, Why I’m losing faith In UX, he explains how he sees a decline in the last 30 years of User Experience design. He calls out how Amazon Prime’s cancellation process is now 6 pages long. And how leaders of Facebook, Google, Amazon, and Apple have lied under oath about the harm of their products. He calls out how ” design thinking” as popularized by Stanford’s d.school has empowered Silicon Valley to exploit people. He offers that in Big Tech the meaning of UX has changed from user experience to user exploitation. There’s merit to what Hurst is saying, but he’s missing the larger picture of the animal he is grappling with.

Hurst references Erika Hall’s popular tweet that offers a diagram (shown below) about the relationship: “This is all too often how UX design is considered and practiced.” I think she’s saying that while some designers do good work on the (small) things they control, they contribute to a negative net impact on the people they are designing for, given the dubious ethics of the underlying business that pays them (actually the meme implies customers will be eaten alive! But smarter predatory businesses want you to be a zombie, who buys things in ads and has good cash flow).

Angler fish labeled "business model". Lure labeled "UX".

This should not be a surprise. We know that often the user is the product, but many of us have absorbed that cynically into the equation of UX work. Sometimes it’s confusing though. Hurst doesn’t mention that you can have a great user experience in one sense and be exploited, or exploit others, at the same time (e.g. Uber, Facebook or even cocaine, at least at first). Maybe we shrug now and then, saying “I’m just the designer” and it’s not our jobs to define the business or defend its ethics. Or that working for a questionable company, given our personal situation, is the only option that we have (like the lonely male anglerfish). For many reasons it’s easy to feel like we’re attached to a larger creature, hanging on, along for the ride.

One mistake we make is thinking this is specifically about UX. It’s not. Advertising, marketing, branding and even engineering are more likely to be the lure, as entire apps and services can be the free temptation hovering above a darker business model. It’s far more a diagram about predatory businesses than anything else.

Somewhere in Hurst’s story, and in the design community, was the latent hope that UX was going to reform capitalism. Or be immune from it. That sounds ridiculous, but there is no other explanation for our surprised outrage that these businesses do what they have always done, but now use UX as part of their tactics. As if we are so special and our knowledge beyond appropriation.

I don’t think Hurst’s lost faith is actually in UX. User Experience design is primarily a set of skills. You can’t lose faith in UX design any more than you can lose faith in carpentry. Instead he has lost his faith in the willingness of predatory (big tech) corporations to do the right thing. Placing faith there was the mistake, given what we know of the species. To believe most corporations, given their history, would, without regulations and against the wishes of their stockholders, invest in good design and ethical practices above other considerations is about as foolish as hoping anglerfish will go vegetarian. Even the billionaires who choose to do good do their philanthropy through foundations, not corporations.

This is depressing and I’m sorry. It’s not entirely your fault that we’re in this situation. Design culture has always been self-aggrandizing. Something rarely mentioned in design schools or books is the possibility that good design may have little impact on why some businesses succeed or not. There is a reason why Norman doors, for all of their usability flaws, are still popular in the world. We laugh at them, and countless other obvious “unusable” designs of confusing microwave ovens, tech gadgets or the latest trending social media app, but the joke is on us. We think we are seeing incompetence, but instead it’s evidence there can be successful businesses where our services aren’t important and we’d prefer to stay in denial.

Hurst overlooks that Amazon customers are happy even with bad UX. It is one of the top five most loved and respected brands in the United States, despite the design and ethical issues he rallies against. Google, a common target of distain from UX experts, is among them too. The truth about Amazon customers is they care more about cheap prices, two-day delivery and a great return policy than anything UX designers are likely to improve. In the places where this isn’t true, perhaps Alexa or Kindle or their next new service, they likely will invest more in design. Currently Amazon’s strongest revenue growth is in AWS and it’s possible their consumer businesses won’t be as important in the future. There’s a logic here, we just don’t like it.

Should customers have different priorities? Including concerns about how poorly Amazon employees, and suppliers, are treated to achieve those low prices? Should citizens demand corporations do less damage to society? I say yes, but my opinion is irrelevant to Amazon. And Walmart. And the stock market. And it’s irrelevant in a way to all of us who want the growth stocks in our mutual funds and 401ks to rise, while we rarely ask what the real cost of that growth might be. I want this to change but that can only start by being honest about how things are.

Returning to the anglerfish, for no particular reason at all, certainly not for any metaphoric relevance to where I’m headed, there’s another fascinating fact about its biology. Females accept multiple male sexual partners. And when each male latches on, it slowly loses organs it no longer needs. It loses its eyes and its fins, its kidneys, even its precious heart gets reabsorbed into its new host. It is no longer a creature itself. It is just a sexual appendage. It hangs on, sensing its fellow men but unable to see them, surviving day after day with no purpose and no function, except to provide parasitic underwater sex on demand.

For a typical business, UX design is seen as a specialization. Businesses have many specialized roles they hire for. They fit into a larger strategy for profit and growth decided by people who are in generalized roles. Even if you work on a large team, under a Director of UX, many of the important design decisions you have to work with are likely not made by designers. The budgets, schedules, and requirements come from people who know far less about making great/ethical products than you or your team does, in part because making great products isn’t their goal. Unless someone influences their thinking, that is the way it will stay.

Your faith in UX, as promised in the title of this essay, comes in an unexpected form and it starts here. You must trust that you are good at UX design and that your focus, for a time, must go elsewhere. If most of the big design decisions are made by people who you think are bad at design, or are unethical, you need to broaden what you think design is. The VP, the PM, the engineer, if they’re making decisions that “should be yours” or are “terrible”, then they’re designers too and powerful ones. This means there are only three reasonable choices:

  1. Move into a role where you make the important decisions.
  2. Become better at influencing decision makers.
  3. Find a place to work that has higher standards (or start your own).

Unfortunately the most common choice might be #4: complain and/or do nothing. It’s human nature to prefer the familiar, even if disappointing, to the unknown. It’s a safe choice because many make it. It’s less scary than the alternatives and it’s often immediately rewarded as others will complain with you. They may also tell you #1 is a betrayal. That you won’t be a “designer” anymore, despite how if you do it you’ll be making the decisions you felt you should have been yours all along (as well as paving the way for the next “designer” to have more power). For #2, some will say “Why should I have to?” and the answer is you don’t. You don’t have to do anything. However if you want something to change, participation is required.

The day you were hired you likely knew more about good design than most of your organization, including the executives. Put your faith in your UX knowledge and use it as a foundation to stand on, instead of a weight holding you down. Ignore the standard design books, design conferences and design media for a time. Don’t keep sharpening the sharpest tool you have if it’s clearly not the one you need.

Instead learn about how consultants persuade. Studying business for designers is good, but even better is to immerse yourself in a founder or product management community. Study their arguments and their theories on their turf. Ask questions. Learn to speak their language, so you can be a better translator. Maybe you can do it locally, if there’s someone on “the other side” in your org you get along with well. Ask them to mentor you. Make them an ally. Treat the powerful people you want to influence as if they were your users: you want to study them, charm them, so they can be users of your ideas.

It can seem insurmountable that one individual could ever gain much influence. If the host creature, or organization, is large and powerful, what hope could there ever be to redesign how things work? This is how we must imagine the sad male anglerfish feels as he rides along, playing the same small role for all time. But what he can’t know, that you can, are other stories. What he can’t do, that you can, is make new choices. You are free to change your relationship to your organization, to your work and to yourself.

Consider, for example, the abilities of the emerald wasp. It’s a fraction of the size of the creature it needs to influence (a roach). It’s so small it’s not seen as a threat. It’s underestimated, but it has studied well and perfected its skills. Once it finds the right place and time, in an instant it strikes and lets the design of its persuasive chemistry do the work. Soon it has control of the mind of what once was its superior, pulling it gently along, without even a struggle, to do its bidding. Now imagine if our emerald coated friend wasn’t working alone, but had allies with a shared mission? What might be achievable then? In nature, and design, anything is possible.

(Note: American corporations should be better regulated to act more as members of society, and penalized for profiting from behavior that damages the greater good, but this is a systemic issue beyond the scope of design which is why I didn’t address it directly in this essay. I’ve received constructive feedback on this omission and wanted to address it in some fashion. Added 2-6-2021).

(Minor edit: “being honest about where things are”, 20/2021).


Scott Berkun is the author of How Design Makes The World. You can read a related thread on #designethics on twitter.


Thanks to Bryan Zug and Phillip Hunter for draft feedback.

On Tuesdays I often write about the top voted question on Ask Berkun (see the archive). This week’s question is from C. with 101 votes] is Why Do Idiots Get Ahead?

I’m a diligent individual, but find it frustrating to continually clean up after the mess “idiots” create – but yet the “idiots” cannot be stopped. I have clout in my organization, yet individuals below me are supported by peers, while I’m ignored.

Your question is an interesting one, but not just for workplaces. As an experiment, lets turn your question around. Why Do Smart People Get Ahead?

I’m not sure they always do. The greatest single factor for how far ahead we get in life is simple: where and when we are born. If you were born in ancient Rome it was 50/50 you’d live past 10 years old no matter how smart you were. Then again, if your Dad was Louis XIV, King Of France in 1644, and you were the first son, you’d be far ahead before you said a single word. There were thousands of other smarter kids born that same day in France, but none were given the same advantages. Monarchy seems pretty limiting to us now, but even today who our parents were defined hundreds of advantages or disadvantages we didn’t pick, but often take credit for. Of course there are no guarantees: many children of the rich and famous often have a terrible time living up to the burdens of those legacies. But my point is there are many factors that define who succeeds in the universe, some we control but many we don’t. Some seems fair to us and some unfair.

Specific to the question of idiots, smarter people get ahead only when they are able to successfully apply their abilities to the situations and challenges they face. Some challenges in life depend more on social skills, passion, empathy, dedication and ambition than smarts. More so, words like smart, dumb, intelligent and idiot are used very loosely. Howard Gardner defined at least nine types of intelligence, including spatial and inter-personal smarts. Depending on what we’re talking about (life? work? sports?) different kinds of intelligence yield different advantages. Some titans of industry have terrible social lives. Many of our most prolific artists struggle with depression. Life is more complex than the simple scorecard we often use to judge others, and ourselves, with. “Getting ahead” seems a lousy measurement, since it demands the question: ahead of whom?

There are five different ideas hidden inside your question, as it relates to the working world:

Also see:

idiots

(I gave the opening comments at a keynote panel on The Next Industrial Design Revolution for IDSA’s Future of the Future event. Here’s an edited version of my brief talk).

The first industrial revolution may have been the most dramatic we will ever have. This is an unpopular notion as we suffer from what Tom Standage called chronocentrism – which is the belief that the present is the most amazing time ever in history and our most recent inventions will transform the world like nothing before. I don’t believe that. I don’t think you will either if you thought about it for a minute.

Consider life 100 years ago, and the the shift from hauling water on your back, walking up from the river every morning to having indoor plumbing or “instant water” as a modern marketer might have called it. Or the shift from horse power to electricity, and lighting dozens of candles with your hands to indoor light at the push of a button. Electricity had far more profound impacts on society than many of our hyped inventions of today.

As a simple test: if you could only have one of A) your mobile phone with internet, or B) running water, electricity for your home and modern medicine, which would you choose? We’d all eventually choose B. We take for granted the most profound technological advancements central to our lives.

We also forget that the  first industrial revolution centered on steam power and the mass manufacturing of textiles, the central industry of the industrial revolution. It wasn’t consumer technology, it was factory machines. And it’s overlooked that this revolution was predicated on slavery. Central to the revolution was a cheap mass labor force. It created the economic advantages these new inventions accelerated. And the lesson for us today is that in every revolution, at least in every industrial revolution, ethics and morality of some kind are likely overlooked. Here are three questions to help us.

Question #1: How Is Your Work Moral For The Future?

If we believe that “design is an extension of our identity”, as the conference program defines it, how do we explain consumerism? How do we explain advertising? The enormous consumer debt in the U.S. is predicated on the desire to upgrade to the latest versions of products we make. We are paid to manipulate people into buying and upgrading. How then do we reconcile our salaries with the moral challenges of American capitalism? How do we explain the environmental crisis and it’s connection to product and technological manufacturing? To the invasion of privacy that many of the most popular technologies today inflict on their own customers? Just as slavery was the unspoken crime of the first industrial revolution, what is the silent immorality of the one we are in now?

The next generation is more aware of moral issues than perhaps any generation before. They were born into a world with major economic, environmental and social problems, a troubling legacy that we are leaving for them. Is what you are working on today  designed for 5 years? 10? 50? If not, you are designing more for our generation than the next. This is not generational design, so much as indulgent and selfish creation. Our chronocentricsm blinds us from what we claim design does: improve the world.

Question #2: Will you respect “unprofessional” creativity?

When a new technology lowers barriers to entry, progress and regress happen simultaneously. For example, HTML was a huge step backwards for design, in that it took away the layout and typography control the technology of print had developed for centuries. But it was a huge step forward in inviting an entire new generation of young people without preconceptions to create and publish.

This is a fine line we have to balance: we have to be capable of respecting creative but untrained outsiders, and finding constructive ways to engage and elevate what their work. Rather than taking the natural stance that “people without our background are not designers”, we should be generous and curious. If we want to influence the future we have to make our knowledge accessible to the next generation. If we don’t they will simply pass us by.  They are not waiting for a torch to be handed to them, as that’s a metaphor so old it  predates all of us in this room.

Question #3: Is the value of your expertise more than pretense?

We are here at a glamorous professional event that presumes design degrees and professional events are valuable. But we are biased: all the people who question the value of these things are not in the room to disagree.

We must admit that as tools continue to improve, and the affordability of creation increases (kickstarter, 3D printing, etc.), the assumption that our profession and our professional society is necessary will be continually challenged. Great designs are being made by people without our pedigree and we are likely to dismiss them for this reason alone, presuming we have the power to dismiss.

But generational change is unforgiving. They are not waiting for our approval. The tools this generation has allows them to go directly to making, and to finding an audience, and for many of us this is terrifying. We can assume they will fail, or find their way to the path we’ve been on, but the history of revolutions suggests otherwise. Only if we are lucky will we even be asked, by younger and faster creators, how our past experience is relevant. It’s up to us to reach out to them, with open minds, to apply our wisdom to their work on their terms, not ours. Our terms are dying while theirs are just being born.

Recently Mike Monteiro wrote about whether the AK-47 is worthy of study for a design student. I agree and disagree with him at the same time, which led me to write this response. He wrote:

If a thing is designed to kill you, it is, by definition, bad design.

This sounds powerful but it makes little sense because it pretends design and ethics are the same and they’re not. I know he wants them to be the same (and in a way I wish they were too), but he’s mixing design, which is a practice, with ethics, which is a system of beliefs. They overlap but they are different lenses.

For example, a house cat’s front claws have evolved into a wonderful design: sharp, compact, strong, lightweight and retractable. But by Monteiro’s definition if you’re a mouse or a bird, the claws are a bad design, since they are made to kill you. It might be unfortunate, or even evil (from the bird’s perspective), that such a design exists, but for the purpose it was designed for it’s an excellent design. If you’re a starving cat, those claws are designed well enough to save your life, even if through killing. (Also consider assisted suicide devices, things designed to kill you, but by your own hand. Is that bad design?).

This leads to the very idea of violence: when, if ever, is it ok to be violent and to kill a person? An animal? These are good ethical questions, but not design questions as you don’t need to question the ethics of a supermarket or a slaughterhouse to design one well (as defined by the client), even if you should (and I agree with Monteiro that you should). Most people most of the time don’t ask ethical questions about their daily work, or anything at all.

Regarding the AK-47, I don’t like guns. I don’t like most violence on TV or in movies. I wish the AK-47 did not need to exist, but I can’t deny the staggering amount of violence in human history. Much of that violence thousands of years ago was necessary to survive in Darwin’s world, but just 70 years ago the entire world was at war for a second time (because the first world war just wasn’t bloody enough).

I hope we grow out of our violence but moral progress is far slower than technological progress. In which case, when guns are pointed at you by evil people, and there is no alternative, a good designed defense may just include having one of your own. Albert Einstein, a pacifist, encouraged the United States develop the Atomic bomb, because the prospect of the Nazi’s having one before the rest of the world was far worse than the alternative.

We forget that civilization itself is an experiment (Freud thought it’s one that makes us crazy) and in a short time we’ve threatened to end our experiment ourselves. Studying something like an AK-47, and the history of conflicts that surround its use, explains a great deal about human nature, which I’d hope any designer would want to understand. It leads to asking about Einstein (a pacifist) and the atomic bomb, and dozens of more complex collisions of ethics, violence and technology.

Design is an ethical trade.

No it isn’t. I wish it were, but it’s not. Who designed all the junk in our landfills? Who designs pop-up ads? Who designed TMZ? Who designed our culture of conspicuous consumption and the advertising that promises salvation if we just buy one more thing (that we don’t need)? Who designed newspapers that lie to us? Government technology that spies on us? It’s designers. Designers were paid to do all of those things. Some designers are ethical, but some are not. Some designers refuse projects because of their ethics, some do not. But both design things and both are designers.

Modern design is dominated by consumerism and while consumerism has been great for the U.S. economy it has also been bad for the planet and for the human psyche.  If design were primarily a noble profession centered on the progress of humanity designers would worship Victor Papanek and Buckminster Fuller instead of Steve Jobs and Mark Zuckerberg. But we don’t. Most design students today don’t know either of that first pair, which makes me very sad. Most designers today, especially in the tech world, aren’t making the world any better at all. They are paid very well to make shiny things that attempt to solve largely superficial first world problems of extreme convenience.

And to design is to take purpose into account — as my friend Jared Spool says: design is the rendering of intent. You can’t separate an object’s function from its intent.

I’m friends with Jared too, and Monteiro and I have hung out together with him, but I disagree with both of them. I can use a hammer to build a chair, or a torture device, or to knock you unconscious. What the designer intended is mostly irrelevant once it’s in my hands. And even if the designer showed up and chose to tell me “Hey! You’re not using my object with the designed intent!” I could use the very object they’re talking about and bash them over the head with it.

All tools can be weaponized or used for evil, even a spreadsheet (“track the monthly wilding budget”), even an email application (“fire the missiles now!”) , even a calendar (“reminder: blow up building today”). Of course most tools are not designed to be weapons, and some designs are clever in minimizing their uses for evil, but so what. User intent trumps designer intent (See MacGyver, and then imagine him not as a hero but as a terrorist). Designers are arrogant and often forget they have the most influence only over the most trivial of their user’s decisions.

Monteiro wrote: Your role as a designer is to leave the world in a better state than you found it. You have a responsibility to design work that helps move humanity forward and helps us, as a species, to not only enjoy our time on Earth, but to evolve.

I do love this idea. The problem is almost no one who hires a designer sees this as what they are paying for, and as a result, most designers don’t see it either. It’s likely this ambition requires designers to make sacrifices, to do pro-bono work or to start their own companies that uphold a higher moral standard than their past clients. They have to redesign design which is far scarier than simply designing more things consumer companies hire them to do. If anything studying an AK-47 and its impressive and horrible history connects young designers with a world far larger, bigger and more inspiring towards truly noble works than the latest gadgets can. For designers to change the world for the better they first need to understand how the world works at all.

Related:

Each Tuesday I take the top voted question from readers and answer it.  With 146 votes, this week’s winner was from James:

When a system is broken at work, should you work to fix the system, game the system, or avoid the system?

It depends! Do you like your job? Do you like your community? If yes, you should naturally work to fix. If you don’t, you should quit your job or move before you start quitting your ethics (I’ve written previously on Why We Accept Bad Systems and you might want to peak at that post before reading on). Here are three answers to your three options:

1383370_40646018

[Note 12-5-17: this question came up this week for a 2nd time, with 368 votes, by Richard Ferrers, and thoughtful reuse is also one my favorite rules]

Each week I take the top voted question from readers and answer it (submit yours here).  With 225 votes, this week’s winner was “What Are The Rules You Live By?” submitted by Max. He wrote:

Over the course of life you make experiences, learn from them and have certain key “nuggets” of wisdom that you hold on to. These are rules that you found that make you more successful and happy, make you avoid stupid things and make better choices. What are those?

I have rules about rules:

Instead there are pairs of contradicting rules I think about often. Each side of each pair are true, but never to the exclusion of the other. The uncertainty generated by these “rules” keeps them alive and pushes me to revise and review the rules themselves.

What are your rules to live by? How did you decide on them? Leave a comment.

There are 4 points and 5 questions at the heart of the latest drama around Facebook and ethics, and the report of a recent experiment they did regarding emotions.

1. All media manipulates emotions by design

Newspapers editors have always chosen which news stories we see, choices made with emotional impact in mind. Television and print news have a long history of over-reporting stories of violent crime, far out of proportion to the rate they happen. One argument is the belief it’s easier to draw attention for negative stories. Another is that making people feel bad eases the challenge of selling them things in ads (See Do Emotions In Ads Drive Sales?). The news feed from any news source has always been editorialized, or in more cynical terms, manipulated. There has never been a news report that is purely objective, although some are more balanced than others.

Advertising by definition is “a form of marketing communication used to encourage, persuade, or manipulate an audience” (#). Television, print and web news generally depend on advertising for income. Any service you use that depends on advertising feels pressure on on all of their choices to help advertisers succeed in their goals.

2. Clickbait and headline crafting are emotion manipulations

News services choose which stories to cover and how to title them. There is a natural incentive to want stories to draw attention, and the writing of titles for articles is an important skill, a skill based on understanding reader’s emotions. Buzzfeed and Upworthy are notorious for their careful crafting of the title of the articles (The Atlantic reported on Upworthy’s successful headline style). The common use of puns (humor), tension (fear), and teases (curiosity) are all based on attempts to use the reader’s emotions to make them more or less likely to want to read the story. Headlines predate the web, and the history of newspapers documents yellow journalism, a practice still in use today.

3. Designers of most major websites experiment on their users

For more than a decade it has been standard practice for people who design websites to use A/B testing on unknowing users of a site. These experiments are done regularly and without any explicit user consent.  A designer’s job is to learn about user behavior, and apply their knowledge of humor behavior to help their employer with their goals through redesigning things. Any website that depends on advertising for revenue is designed with the goal of selling ads. This includes most of the major news services that have been critical of Facebook’s behavior, including The Economist, The Atlantic, The Wall Street Journal, The Guardian and The New York Times. All of them have likely done research on unsuspecting users regarding how design changes impact user behavior.

Most major software companies employ user researchers and user experience designers, people who study human behavior for the corporation. Many of these people are hired for their expertise in experimental psychology. Most formal experiments are done in labs with explicit participant permission, but as software world has shifted to the web, the formality of online user experiments has changed.

4. The Facebook news feed has always been controlled by an algorithm

At any moment there are 1500 possible stories Facebook can show the average user. Much like Google or any search engine, Facebook has an algorithm for deciding which of many competing links (or stories) to share with any user at any time. That algorithm is not public, just as the algorithm in the minds of any news editor anywhere isn’t public either. That algorithm is perhaps the greatest piece of intellectual property Facebook has, and it is something they are likely developing, changing, and experimenting with all the time in the hopes of “improving” it. It’s no accident Facebook calls it the news feed.

Just like any media source, improvements from Facebook’s perspective and a particular user’s perspective may be very different. However all sources of media have some kind of algorithm. When you come home from work and tell your spouse about your day, you’re using a kind of algorithm to filter what to mention and what to ignore and have your own biases for why you share some news and not others.

Facebook is a social network that by its central design mediates how “friends” interact.

5. The 5 questions that decide if Facebook crossed the line

Outrage is emotional itself. When we get upset, based on a headline, we already have confirmation bias about the story. None of us heard about what Facebook did from a purely objective source. We all entered the story the same way we enter all stories, with preconceptions of our own and influences born from the editor and writer.

Ethics gets grey fast once you get past initial emotions and dig in to how one practice compares to others. There is never just one bright yellow line, instead it’s a series of many gray lines, where the details make a difference. There are 5 questions that probe at the entire issue and yield perspective on what’s right, what’s wrong and what’s somewhere in the middle.

  1. Is it ethical to manipulate people’s behavior? Perhaps, but with media it’s a matter of degree. All communication has the implicit goal of effecting people in some way. All media organizations, which includes Facebook, have a combination of motivations, some shared by their users and some not, for the changes they make to their products. Advertising has strong motivations to explicitly manipulate people’s emotions and any organization that use ads are influenced by these motivations.
  2. Is it ethical to use design and experimental psychology to serve corporate goals? Probably. This doesn’t mean all experiments are ethical, but using experiments of some kind has long been the standard for software and technology design, and similar methods have long been employed by advertising agencies. There may be nuances that should be adjusted, but in principle research on human responses to media is standard practice and has been for a long time. In the pursuit of making “better” products, research is an essential practice. I don’t think Facebook’s Terms of Service needs to state anything specific about experimentation if the terms of service for news and media outlets that do similar experiments don’t have to either.
  3. Was there something wrong with explicit, rather than implicit, emotion research? The outrage may stem from this difference. Rather than they study being about increasing clicks, which has implied emotional factors, the stated goal was purely about influencing how people feel. This makes it feel categorically different, even if the methods and motivations are largely the same as much of the research done in most software and media companies. Plenty of magazines and websites have the primary goal of changing how you feel about a topic or issue (e.g. Adbusters). The results from the study were moderate at best – a tiny amount, about .1%, of influence was found. There are other possible issues with the study design, described here.
  4. Was there something wrong in doing an experiment at this scale? Maybe. If the study had sampled 20 people I doubt there would have been much outrage. Something about the scale of the study upsets people. The rub is that when your service has 1 billion users, a small experiment involves thousands of people (the study involved 689k users, less than 1%).
  5. Was it unethical to publish the results in a research journal? Possibly. On one hand research journals have specific protocols for participants in studies. But on the other hand, if it’s common for an an organization to do the experiment in private, why is it unethical to report on it? (I’m not saying it’s necessarily ethical, I’m just raising the question)

This month I’m posting every day, picking the top voted reader question and answering it.  With 50 votes, today’s winner, submitted by Peter Colligan, was:

 Software Ethics – When is it acceptable to ship a low quality product?

I develop enterprise software. Sometimes the decision is made to ship when quality is low. I have as a professional software engineer no ethical recourse to such actions. Sometimes the impact is not death but extreme inefficiencies that cause overspending and unstable conditions for customers… Other professions such as medicine, drug research, etc. have professional guilds that extend beyond employment boundaries. Is this really a problem?

The short answer is there are no short answers on ethical questions.

If you sell something as “low quality” and the person buying it wants a cheap low quality product, what’s the problem? If both parties feel they got what they asked for, there is no ethical challenge regardless of quality (for addictive drugs or products producing toxic waste there may be communal ethics, but that’s another discussion). The rub then is what a product promises to do vs. what it actually does which leads to marketing ethics (e.g. are  infomercials ethical? What about alpha or beta-software?) 

Deciding when something is finished is highly subjective. The Brooklyn bridge was designed by Roebling to have cables 6x stronger than necessary, a very high engineering standard. This level of “quality” is rarely used in modern engineering work. Is this lower standard ethical? Or was Roebling unethical in wasting so much in city resources to build a “wasteful” bridge? Subjective indeed.

Quality, as the book Zen and the Art of Motorcycle Maintenance painfully explores, is hard to define. Creators and consumers often have widely varying standards for what good and bad mean (see Why Software Sucks). McDonald’s, the fast food restaurant, has nearly 2 million employees world wide. Is it unethical to work there because the quality of food is so low? Even if the quality is low, is it ok to sell low quality things if people want them anyway? When is it ethical to publish low quality writing? Could you win a law suit against a musician because you thought the song you bought was poorly sung? Is it wrong to post grammatically incorrect status updates on Facebook every day about your favorite socks? Objectivity in ethics is hard to find.

The simplest place to start is to spend as much time as possible with people who share your ethics. Does anyone else meet your standards? Can you find people who want to pay a premium for your higher quality work? If the answers are yes then your problem is solved. if the answers are no, then your standards might be too high.

The standards for many kinds of products are often set by the market. Progress often happens by companies making superior products rather than a committee decreeing a new higher standard. Sometimes there is a role for government to solve certain market limitations, e.g. automobile seat belts or fuel economy regulation, but that’s more of an exception than the rule.

The birth of medical malpractice and professional negligence as legal concepts are also worthy of study. The ISO had its first standard in 1951 and that standard was for… getting engineers to agree on how to measure things. Certainly an important development, but not a triumph of product quality. Professional associations are slow and even when they create good standards they mostly impact the reduction of the worst malpractices. The ACM does have a software engineering code of ethics but until doing research for this post I’d never seen or heard of it before and I bet most makers and consumers of software haven’t either.

It’s most germaine element for our interests is:

3.01. Strive for high quality, acceptable cost and a reasonable schedule, ensuring significant tradeoffs are clear to and accepted by the employer and the client, and are available for consideration by the user and the public.

The statement ensuring significant tradeoffs are clear is great advice for anyone making anything. In your case the decision to ship a lower quality product might be the best choice to balance out all of the clients tradeoffs. Software, unlike bridges, is easy to upgrade allowing for quality to be improved over time.

And in this notion of tradeoffs is perhaps the real answer you’re looking for. Improve your skills at selling the positive trade-offs of high quality work. Express how much clients will gain in the long term if they’re willing to invest more in quality work upfront. That’s an important skill that has nothing to do with design or engineering.

Each week I take the top voted question from readers and answer it.  With 45 votes, this week’s winner was “Why do we accept and perpetuate bad systems?” by Peter Saumur:

Examples – 2 lanes of checkout on a busy day. Automated parking systems that break down with no backup. Sitting in traffic for over an hour each way to get to work. It’s madness yet we accept it as the way things are.

The easy answer is we’re lazy. It takes energy to reject a system, especially if it’s a complex system you do not control. Sitting in traffic is not an active choice, it’s a passive decision that comes with the choices for where to live and work. Like waiting in a long line, it’s far simpler psychologically to plod along with your head down, than to challenge the dozens of people in front of you who may have no objections to lines at all, or worse, diverging ideas for what the alternatives might be. It takes courage to challenge a system and wisdom to improve it, a rare combination.

The depressing answer is some systems are difficult, or impossible, to change. Even if you were sufficiently motivated, problems like traffic operate on a large enough scale, with multiple conflicting objectives, that it might not be solvable. Certainly not without the cooperation of many other people, some of whom benefit from the inefficiencies you find frustrating (e.g. the land developers who profited by overpopulating an area).

For all frustrating systems ask: Who benefits (Cui Bono)? That is the first step to understanding how a system truly works. There are always people who benefit more from a system staying the way it is, and as long as they are in power nothing will change. Any hope of improving a system starts with understanding the reasons forces are actively protecting the status quo.

The stoic answer is that all systems are designed, and all designs are tradeoffs. They are good for some things and bad for others. There is no perfect system, and even if there were, as soon as the world changed in a significant way the same elements that made the system seem perfect would suddenly become a frustration. Gas powered cars seemed a wonderful idea in 1925, before anyone could imagine global warming, rush hour traffic, or fatalities from drivers distracted by text messages on their phones.

The relative answer is: bad compared to what? We naturally take good things for granted since there are evolutionary advantages to constantly seeking to improve one’s situation. But for any bad system, go back in time. What was that system like 10, 20, 50, 100, 500 years ago? In some ways things will seem better, but in many ways that older system will seem worse. It’s better to wait a few minutes in poorly managed checkout lines at the grocery store, than to hunt and gather all day long with little guarantee of having enough to eat.

The positive answer is that many people do reject bad systems. They make sacrifices to give themselves the power to avoid systems they find distasteful. With sufficient motivation and the willingness to make tradeoffs, many modern frustrations can be avoided or minimized. For deeper systems, they protest in public to draw attention to how a system is failing. They complain to representatives in the system and try to influence them to prioritize change.

What “bad” system bothers you the most? Leave a comment.

[minor edits: 3-6-2021]

sociopaths bookThe other day I saw some high school students in a coffeeshop, and on the table was a book titled Why We Love Sociopaths. I laughed when I saw it, thinking it was an excellent book title. Then I looked it up on kindle and read the whole thing that night. At 100 pages it covers a wide range of ideas, questions and criticisms of modern television, a second cousin to Postman’s classic Amusing Ourselves to Death.

While I’m a fan of the high quality of television we’ve seen in the last decade, perhaps the highest of all time, I’m often bothered by how horrible the lead characters are as people. Certainly Shakespeare had some lead characters who were cruel or rotten and it’s not a new structure for drama. But if you run through The Sopranos, The Simpsons, The Wire, Mad Men and even South Park, you find consistent reprehensible behavior among central characters, sometimes played for comedy but often seriously.

Kotsko, a professor of humanities at Shimer college, improved my thinking( and I interviewed him about his book here). He takes many popular shows and breaks them down, exposing, through a conversational and refreshingly non-academic style, deeper questions and theories as to why these characters are popular to modern America.

I didn’t agree with all of his analysis, but that’s part of the fun. What do I believe? His take was consistently interesting enough to provoke me into deeper thought, a compliment for any book.

If you’ve never seen The Wire, House, The Simpsons, Dexter, Breaking Bad or the shows I’ve mentioned above enough to know the main characters and themes, I’d pass on this book. His analysis lands well if you’ve seen the shows, but for shows I didn’t know it was harder to follow.

Here are some good quotes from the book.

On evil sociopathic characters being a form of wish fulfillment of escape from morals:

 The sociopath is an individual who transcends the social, who is not bound by it in any gut-level way and who can therefore use it purely as a tool… If only I didn’t give a fuck about anyone or anything, we think—then I would be powerful and free.

On reality TV shows rewarding sociopathic behavior (manipulations and betrayals):

Many of the adult games that Augustine was so skeptical of remain with us today: the quest for wealth or fame, or public debates and trials won by rhetoric and technicalities rather than truth and justice. Since his time, however, at least one new adult game has emerged—the reality TV show.

The centrality of exclusion to the model can be seen in the fact that even apparently skill-based shows, such as America’s Next Top Model or Hell’s Kitchen, with outside judges determining the winner rather than a vote of one’s peers, typically rely on the slow work of “voting people off the island” rather than any kind of positive score-keeping.

On Dan Draper’s inherent sociopathy:

What kind of man’s first thought, in the split second after witnessing a horrific death, is not to panic, not to head for cover, but to consider how he can take advantage of this situation to raise his social standing?

This mercenary approach to human identity is a necessary correlate to his ability to buy his own pitch, showing the thorough inner consistency of his worldview: everyone can be manipulated by the same emotional triggers, including me, and everyone is interchangeable, even me.

I read the whole book in one sitting. If you’re a curious fan of these shows, you’ll enjoy the book.

Good movies about software are hard to make. That’s why few people try and those who do mostly end up with mediocre films. David Fincher (Fight Club, Curious Case of Benjamin Button) gets at least one thing right in his new film: pace. This movie, for a drama about young adults making software, is smart, quick and unrelenting in its progression. The Social Network is a good, well acted, well scripted drama, which provokes questions about ideas, ownership, ethics and relationships (End of short review).

I didn’t read the book the film is based on, Accidental Billionaires, as the author’s style of dramatization and invention (He also wrote Bringing down the house) has earned him a reputation for stretching the limits of what can be called reporting. Its foolish to expect Hollywood films to have much interest in upholding literal truths.

However I have researched Facebook’s history and it’s clear Zuckerberg was not a good guy in his early years (The New Yorker profile suggests Zuckerberg conceeds this). He managed to upset many people he worked with, was sued by some of his first employers and his best friend / co-founder of Facebook. The details of the movie are exaggerated as films, by their nature, tend to be, but the spirit seems not far from the Mark (pun!).  And it’s this spirit that’s the most interesting aspect of  the film.

Nearly everyone is portrayed as shallow, arrogant, selfish and superficial. Some are fools, others are brilliant, but the tone is youthful confusion over what matters most. And this reflects our dilemma over what to make of the worst elements of social media: a playing out of high school cliques, displays of ‘status’ to impress others, and a confusion over what a friend or authenticity actually are. The movie itself shows “a social network” with Mark at it’s center – but its a sad, broken and treacherous one.

The film has been criticized for poorly portraying women, which is true, but this misses how the film poorly portrays everyone. Nearly every character is an embarrassment in some significant way, and the movie is largely criticizing the shallowness of elites (Harvard, Silicon valley, lawyers, VCs, the upper class, etc.).  The movie is a critique of the kinds of people who would choose to profit from changing the world based on the model of “facebooks” (e.g. yearbooks), relationship status, feeds and friending people. The point is:  it’s a 19 or 20 year old’s view of the universe, for better and, as the movie emphasizes, for worse. It’s notable Zuckerberg’s fiancé, with whom he was dating the entire duration of the time shown in the film, isn’t mentioned much less seen. But otherwise it’s hard to find particular bias: I doubt anyone feels great about how they are portrayed in this movie.

I’ve never met Zuckerberg, but his portrayal is reminiscent of people I knew in my Computer Science classes at CMU and in the tech sector today: young men who are arrogant, shy, brilliant, awkward, angry, passive-aggressive, misunderstood and possibly vicious.You’ll find some people like this in any CS classroom or in any tech start-up or IT department. One appeal of computers is they do exactly what you tell them, much unlike people, which tends to attract people with particular abilities and disabilities. Bill Gates near cameo in the film is telling – go watch his deposition video and it’s clear he and Zuckerberg, as portrayed, have much in common (Of course anyone deposed is bound to be cranky, but still).

And more interesting, Bill Gates, Steve Jobs, Larry Ellision, Andrew Carnegie, Andrew Mellon, Henry Ford and dozens of others of captains of industry weren’t lovable, likable or ethical either early in their careers (if ever), despite how we lionize them later. Films of their early lives would have similarities to The Social Network. Simply put, no one is forced to be a CEO or start a company. Those who do are often fueled by greed, arrogance, pride, insecurity or a need to prove something to someone who probably isn’t even paying attention, a point it’s clear David Fincher intended to make.

But much like the film Wall Street, which showed the tragedy of the power brokers in finance but instead created a hero (Gordon Gecko) for a certain group of people, The Social Network, which was clearly designed as a tragedy, will have the same fate. Many young entrepreneurs will see having a business card with “I’m CEO, bitch” as a goal worthy of spending their lives chasing, missing how much personal carnage this psychology created for this particular CEO and everyone around him.

When I was a kid I watched my NY Giants, hold on to a short lead late in Superbowl XXV in 1991. They were up by 1 point, and with 8 seconds left the Bills had a chance to win the game by kicking a field goal. It’s awful to have your fate, or your team’s, clearly in the hands of your competitor. Those long waiting moments were stressful, but something made it much worse.

In the seconds before the Bills kicked the ball, the TV showed the Giants sidelines. A circle of players huddled together in prayer. Praying for what? I wondered. For the kicker to miss? Yes, indeed. At the time, I found this troubling. What kind of god would honor a prayer not only as selfish as this, but clearly at the expense of someone else?

To be hated forever for losing a game (e.g. Bill Buckner) which the kicker would be if he missed, is much worse than anything the Giant’s would feel, watching from the sidelines, as he kicked in the winning points and sealed their fate. And I wondered while waiting for the kick, what would happen if a similar number of equally faithful Bill’s players were praying just as piously on the other side of the field. Can you out pray someone? Is that really how prayer works? Or how an intelligent, attentive, loving god would make decisions about our fates? By counting prayers? And wouldn’t you have to consider, if this is prayer warfare, about what the other team’s prayer strategy was before kneeling down to pray for yours? A drop of logic makes all of this fade away into foolishness, as the machinery by which these specific acts effects life defies any reasonable person’s imagination.

As it turned out, Norwood, the Bill’s kicker, missed the kick. And as predicted, despite a great career he is best known for one kick that he missed. My Giants won the Superbowl. I was happy, for sure, but as much as I’d wanted this outcome all season, I felt there was something wrong. A win is not quite the same as the other team losing. Sometimes I’d rather have a solid loss than a rotten tasting win. But I’m weird – and Giant’s fans everywhere may disown me (watch this video of the missed kick to see what I mean). This game makes highlight reels as an amazing game, but it’s not for me. I put myself in the kicker’s shoes ever time. I don’t have a major problem with the idea of God, or some kinds of faith in God. I have an open mind and am open to many different kinds of ideas. But I do have a problem where the name of God is used to justify behavior that runs against ordinary natural human integrity.

Take for instance, the Golden Rule. I like the Golden Rule. It’s a core idea in nearly every religion, nation, culture or tribe, and I see it as a kind of integrity and basic ethics. I will treat others in the same way I wish to be treated (or as they wish to be treated). Many of the ten commandments and similiar moral codes in other cultures are specific implementations of the core theme of the Golden Rule. But to pray for victory, without considering that the people on the other side might also be fans of your flavor of god, or even if not fans of your flavor they are still people worthy of your respect, can not be a high integrity act. No one would want a competitor with God’s ear to ask for their failure. The whole idea makes God a possession – MY GOD. A god who is listening to help me and my needs. Rather than shared, OUR GOD. A god that has the collective interest of all life, or human life, in mind.

I know from history when anyone starts claiming sole dominion over spiritual territory and believes in VIP access to the deities the only place it’s sure to send us all is straight to hell (metaphoric or literal depending on what you believe). The only high integrity prayer, or act, I can imagine is to hope that the team that plays best, wins. To wish that everyone plays well. And that no one gets hurt. And like the Klingons and their wish for a noble death that in the end everyone can walk off the field proud that they played well, and hard, and gave it their best. That even if they lost they feel there’s nothing else they could have done – there is nobility in that. That to me, as a competitor, is the most noble outcome of all: everyone played well and was at their best.

A part of me would rather play well and lose, than embarrass myself with incompetence, and win anyway. To put the game winning shot in the hands of your opponent is, strictly speaking, a failure on it’s own. I have similiar questions of integrity when I see an athlete or award winning star point up the sky when they win, or score a touchdown. What exactly is this intended to mean? I’m a big fan of humility, and giving thanks to people, life, the universe at large, or anything really, but it’s not clear at all to me this is what’s happening. Would they point to the sky if they lost? Isn’t god, or whatever they’re pointing to, up there in all cases, regardless of the outcome? If they catch a winning touchdown pass, shouldn’t they point at least a little bit to the guy who threw them the ball? Or the coach who put them on the field? It seems a better demonstration of devotion, or faith, or humility, is what you do when there is no spotlight on you.

Or, as is often in life, you are not the center of attention for a big reward, and instead are in the muck with the rest of us, with plenty around you worth complaining and feeling disappointed about. What do you choose to do then? Who do you point to and what does that pointing mean? Or more precisely, how generous and humble are you in your treatment of others in and below your station then? It’s out of the spotlights and the glory where the real essence of people, whether you call it their spirit, soul or integrity, is found, and that’s the only place worth looking for it. When I’m able to remember this I can find heroes and saints worthy of emulation without needing someone else to point them out for me, or draw attention to themselves by pointing up to the sky.

Related:

Flipping through old essays I found this one from 2005 – it’s one my favorite but most overlooked essays.

Good, evil and technology: A fun philosophical inquiry

Much of my recent ranting on social media is tied to this sort of philosophical questioning, as the making of good tools shouldn’t be confused with doing good in the world.

If you like my stuff, and are interested in tech,  I think you’ll like this: