Corporations seem evil because we anthropomorphize them

you can't model companies as people. to change their behavior effectively, you have to change their incentives.

Lots of people think corporations are the source of all problems. Why is this? Although “corporations are people” is almost always the punchline of a joke these days rather than a serious opinion about the 14th Amendment, in a strange turn that would make Gettier proud, I think over-anthropomorphization is an accurate description of what's going on when people lament about companies.

Corporations exist to maximize profit. This is usually a good thing, because profits measure how much value they are producing for the world using the capital entrusted to them by their shareholders, as long as all externalities are correctly priced.¹ Corporations nonetheless often try to be person-shaped because that makes them more relatable or something; I’m not sure if this works, but at least we get some funny memes.

denny’s is a stack of pancakes in a trenchcoat

People exist to, uh, live life. People have lots of goals. Usually a big one is money, but the vast majority of people also care about various other things, like happiness, love, friendship, belonging, status, and such. Most of us have virtues, like feeling empathy towards others and following some moral code, but also various flaws—greed, wrath, envy, etc. Throughout our lives, we meet many people, and get a pretty good sense for the different types of guy that are out there.

So, on the one hand, corporations are one of the greatest tools for coordination ever invented; our entire civilization is vastly wealthier because of the existence of corporations. They enable people to do more ambitious things by pooling resources, limiting personal liability, and decoupling managing the company from providing capital for the company, allowing further specialization of labor.

On the other hand, if the Coca Cola company were a dude living on my street, I would think of him as a crazy amoral sociopath.

My Neighbor Coca-cola

Mr. Cola doesn't experience any of the normal human drives, except the one for money. He doesn't care about finding joy in life, or making friends. His relationships are all transactional; he doesn’t really care about Amalgamated Sugar, even though they’ve been doing business for decades. If Amalgamated were to have a rough day and beg Mr. Cola for help, and the cold hard math for helping him just doesn’t pencil out, he would simply stand there and watch him die.² Mr. Cola also doesn’t really care about whether he’s causing his consumers to develop coronary heart disease. Unless this perception is hurting his sales, of course, in which case he will feel no shame lying or misleading people. He portrays an image of ethicalness only insofar as is necessary to not lose customers. Of course, he’s fallible—he’s only inhuman after all. He makes mistakes and miscalculations all the time, though they’re often different from the ones you or I would tend to make.

vitaminwater, owned by the coca cola company, is basically as unhealthy as soda.

To be fair, most companies don’t start out this way. A small company mostly reflects the vision of a small number of founders, who are often reasonable people. Some companies manage to remain heavily founder controlled even as they grow; others never grow past this point and stay in small niches where the competitive pressures are not so fierce.

But the inexorable pressure of the free market pushes any company that wants to expand towards becoming an optimizer, because any company that has a shred of humanity in its heart will get ruthlessly outcompeted by those that don’t.

How do entities composed of mostly non-evil people end up doing evil things? While smaller companies might allow employees to act on compassion, larger companies are more bureaucratic, making it easier for individual bureaucrats to justify overriding their innate compassion because they’re just “doing it by the book”. As labor specialization increases, individual employees might even be completely abstracted away from the people they’re harming, making it easy to just do what they were hired to do and not think too hard about it. And of course, the tiny fraction of people who happen to actually be amoral optimizers tend to gain lots of influence.

Increasing market efficiency accelerates this process, and squeezes companies which previously were not subject to strong competitive pressures. Even the small companies are feeling the pressure now; the whole idea of private equity is to find the small local mom and pop shops and make them much efficient. People are quite reasonably upset at private equity for taking person-shaped small companies and replacing them with company-shaped companies, destroying personal charm in the name of economic efficiency.³

Optimizers have a place in the world

To be clear, I’m not arguing that there shouldn’t exist powerful non-human entities like companies. There are a lot of other good things in the world that would sound equally bad if described as people. My lawnmower⁴ would also be an amoral sociopath if it were a human. The lawnmower does not love you, nor does it hate you, but your hand is in the way of the grass it is cutting, and it is not capable of experiencing remorse about injuring you. And companies are incredibly useful: so many of our modern affordances would not exist without them.

Rather, the problem is that we are treating companies the way we would treat humans. People are our only reference point for intelligent behavior, so it’s natural to anthropomorphize all intelligent processes. Perhaps we might even assume that because companies are made of people, they will also behave like people; though this notion is easily disabused by being an employee in a large company, or visiting the DMV.

Another way this shows up is people often try to metonymize companies by their CEOs. This too is a fundamental misunderstanding of how companies work. The CEO certainly plays a huge role in setting the vibe of a company, but for any largish company, they do not hold absolute power over their realms.

The solution is to simply think of companies as optimizers rather than people. To change a person’s actions, you can appeal to their conscience, or their non-monetary interests, or your relationship with the person. To change a company’s actions, you have to change the incentives of its environment. The simplest case is regulation, but there are so many more knobs that can be tuned to create better incentives, especially in the modern age where the legislative machinery of our democracy has gotten a bit rusty. For example, consumer choice can also be very powerful, especially if switching costs are low and there are competitors abound.

You didn’t think this post could actually avoid talking about AI, did you?

Unfortunately, everything is a metaphor about AI, whether you want it to be or not.⁵

First, AI labs are companies too, and are subject to all the same incentives. People talk a lot about which CEOs are more trustworthy, and certainly this matters, because the market for AI companies is not perfectly efficient, but it’s less important than shaping the incentives so that the right outcomes are incentivized.

Another angle is that because of rapid AI advancement, people are now suddenly getting access to a new kind of vaguely human shaped but also fundamentally non-human intelligence. Companies can provide some useful intuitions for thinking about amoral optimizers, but there are limitations to this analogy. It’s likely we’ll see the same thing all over again, with anthropomorphized thinking about AI causing us to make bad decisions about how to interface correctly with them.

1. This is a big “if”! Pricing externalities correctly is very difficult.

2. Also, uh, he sometimes unhinges his jaw like a rattlesnake and devours other companies whole. I don’t know what the kabbalistic implications of this are.

3. The economics solution here if people really prefer an inefficient but human institution is to either increase prices or institute a local tax to pay for the gap, so the business owner is fairly compensated for the value they produce by doing things the old fashioned way. It remains to be seen if people’s revealed preferences would actually be to pay more for the vibes though.

4. This is just illustrative, I don’t actually own a lawnmower. In fact, I’m not sure if I’ve ever owned a lawnmower at any point in my entire life. Unlike the coca-cola company, I would feel just a tiny bit weird if I didn’t make this clarification.

5. Funnily enough, I wasn’t planning on this being an AI post, but after I wrote everything else in this post it became kind of obvious that it’s also applicable to AI. Is anything not related to AI in some way these days? I swear, some day I will actually have an entire full length blog post that is not about AI at all.