The Peak Size of Agentic AI Mega Firms
Also: Magic, Book Sculptures, and Spacetime Maps
Dwarkesh Patel theorises on what will happen to firms as AIs keep proliferating (ht Rafa). Particularly, when you can effectively run far more agentic AI at far lower costs than having that many human workers.
AI firms will have population sizes that are orders of magnitude larger than today's biggest companies - and each AI will be able to perfectly mind meld with every other, from the bottom to the top of the org chart.
…and
Future AI firms won’t be constrained by what's scarce or abundant in human skill distributions – they can optimize for whatever abilities are most valuable. Want Jeff Dean-level engineering talent? Cool: once you’ve got one, the marginal copy costs pennies. Need a thousand world-class researchers? Just spin them up. The limiting factor isn't finding or training rare talent – it's just compute.
One conclusion is that we might actually see the rise of far bigger firms as a result.
Ronald Coase’s theory of the firm tells us that companies exist to reduce transaction costs (so that you don’t have to go rehire all your employees and rent a new office every morning on the free market). His theory states that the lower the intra-firm transaction costs, the larger the firms will grow. Five hundred years ago, it was practically impossible to coordinate knowledge work across thousands of people and dozens of offices. So you didn’t get very big firms. Now you can spin up an arbitrarily large Slack channel or HR database, so firms can get much bigger.
AI firms will lower transaction costs so much relative to human firms. It’s hard to beat shooting lossless latent representations to an exact copy of you for communication efficiency! So firms probably will become much larger than they are now.
I think it’s a reasonable conclusion, and he stops short of saying that the end result would be the world seeing one giga-firm.
But it’s not inevitable that this ends with one gigafirm which consumes the entire economy. As Gwern explains in his essay, any internal planning system needs to be grounded in some kind of outer "loss function" - a ground truth measure of success. In a market economy, this comes from profits and losses.
In a sense, I agree. You *are* going to see much more efficient and “larger” firms in terms of economic mass. Many firms fail out of inability to simply be effective: like, doing proper accounting or following recommended processes. Firms full of agentic AI could go a long ways at lessening execution risk: the day to day struggles of operations in a firm.
But, Dwarkesh ends the article that points to me as the far more intractable problem:
How does the firm convert trillions of tokens of data from customers, markets, news, etc every day into future plans, new products, and the like? Does the board make all the decisions politburo-style and use $10 billion dollars of inference to run Monte Carlo tree search on different one-year plans? Or do you run some kind of evolutionary process on different departments, giving them more capital, and compute/labor based on their performance?
These assumptions about mega firms still run under the assumption that a complex system can be broadly understood as a whole, when markets and society act far closer to a murmuration than say trying to calculate the entropy of a gas in a container.
It’s because markets themselves are socially constructed by its own participants. I’m paraphrasing work from Donald Mackenzie and Michel Callon, but a part of this understanding is to simply give an example: you are strolling on a beach and you suddenly see an ice cream truck. Now you want ice cream.
People don’t arrive at markets with fully formed beliefs, ideas, and wants and then use the market to get what they want. More often than not, the market itself is a conversation that in turn influences each person in it. There’s no objective prediction of a market, some outside “view” that if you knew all the participants and all their wants and needs that you’d be able to divine what they would buy. The want, in part, is generated from interaction with the market.
This makes predictions about what people want intractably complex. To quote Carl Sagan: to bake an apple pie from scratch, you first have to invent the universe.
Firms packed with agentic AI still have to accept the nature of the market itself: that the buyer will be informed by the market itself. Downwind of that is the fact consumers at the end of the supply chain, people buying products and services, buy products for many different reasons that aren’t just about making the most efficient thing. For example, one consumer might simply not want to buy a brand because their ex liked it.
One can then argue: well, why not then create agentic AI that *sets* these trends and interact with the market to produce demand? That *might* work, but then you’re then merely creating another influencer (among many). There won’t ever be one ‘brand’, much in the same way there won’t be one language. People will always have a desire to connect deep and niche alongside shallow and broad (mainstream). At the end of the day, a firm itself also communicates inside a market. No matter how much a mega Amazon might create impeccable products, people might still resist buying from mega Amazon because it’s a mega firm. The firm itself is a part of the conversation even if the firm might create countless brands and products.
So, yes, I think it makes sense to conclude the mega agentic AI firms will be a thing, but it won’t consume the entire market due to simply chipping away at inefficiencies. There’s a limit to the ability to simulate futures. Firms can only grow to the scale of 1) what/who they are and what they represent, and 2) the inherent peak of moving at the speed at which humans interact with the market. In the most extreme case, imagining a world where a company produces custom tailor-made products you didn’t think you might want is just not something that will happen. Markets and people are just far too social for that.
A conversation is like music. It’s enjoyed through time. You can’t ‘complete’ a market in the same way you can’t shortcut a song.
Bonus Content!
📺 Watching - Severance, Conclave
I *really* enjoyed Conclave. It’s not just the stacked cast, directing, and acting, but I found the themes of the story to resonate. I really have to scratch my head to find a film or story whose main theme centers around certainty (and in this context, by nature, faith). Engaging watch.
🏃♂️ Running/Walking
I used to walk a lot more. But slowly replaced it with running. This week, I decided to do a 20km walk again through nearby nature, mostly to test out whether my Altra Lone Peak 7 trail shoes would be a fitting walk shoe at that range. I’ve always wanted to do a longer walking trip and aiming to do so this year. It’s great to walk this distance again (20km+ in a day). As much as it is an exploration of nature and the world, it’s also always a great exploration of the mind: thinking about life, stories (or whatever I’m working on), listening to music and audiobooks. It’s great.
🕹️ Gaming - A Train, Inscryption
Had a long-haul flight back to the US and downloaded A Train and Inscryption for the Switch. I got pilled on trains when I played Factorio, so that idea of building train networks sounded interesting. I’ll keep trying A Train, but it’s almost a bit too management heavy for me. That’s one thing that Factorio actually does well from a management game perspective: it’s not a spreadsheet game. As for Inscryption, it’s a roguelike deck builder with a fun puzzlebox + horror element to it. Easy to enjoy and have fun with. Quirky too alongside the “spooky” elements.
I’m excited for Civ 7, though. I must admit: I’ve always loved Civ games, but the last one I played was Civ 3. Way too long ago. So, it’s high time to play it!
📚 Writing - Novel #2
Progress continues. Currently, I’d say about 60% through draft 2.
💾 Links
Magic as Permission
I enjoyed this perspective from
on magic and enchantment.Calls for “reenchantment” more often desire to grant permission to feel this enchantment, observing the deep surprise that life, mind, and matter exist at all, in whatever religious, spiritual, or materialist means may be available. In this sense, magic doesn’t operate as the borderland between the rational and the irrational, as it’s mostly understood. Magic isn’t always irrational; it seems that way because it has less certain epistemic allegiances. Instead, it operates through the endless, contested edges of permitted experience.
It resonated because I find that there’s a general lack of positive enchantment in much of the Western world (not just in one part of the political spectrum). It’s that feeling of being unable to be sincerely hopeful or positive, because well, *look at the world*. It’s the feeling of: “How can you be happy when tragedy x, or y, or z is happening?!” Being hopeful or optimistic can come across as almost cringe and naive. But, in some sense, I think it’s because cringe optimism is often voiced in opposition to tragedy. It’s usually in the form: “Let’s not be depressed or sad about a visible and clear tragedy, let’s instead hold hands, vibe out, and see the good things in life!” It’s giving very dogdrinkingcupofcoffeewhilefireisburningaroundthem.jpg.
So, I think if there’s any way to re-give us permission to magic, to re-enchantment is to be a bit more like Sam Gamgee.
Sam: I know. It’s all wrong. By rights we shouldn’t even be here. But we are. It’s like in the great stories, Mr. Frodo. The ones that really mattered. Full of darkness and danger, they were. And sometimes you didn’t want to know the end. Because how could the end be happy? How could the world go back to the way it was when so much bad had happened? But in the end, it’s only a passing thing, this shadow. Even darkness must pass. A new day will come. And when the sun shines it will shine out the clearer. Those were the stories that stayed with you. That meant something, even if you were too small to understand why. But I think, Mr. Frodo, I do understand. I know now. Folk in those stories had lots of chances of turning back, only they didn’t. They kept going. Because they were holding on to something.
Frodo: What are we holding onto, Sam?
Sam: That there’s some good in this world, Mr. Frodo… and it’s worth fighting for.
And a splash of Nietzsche’s Amor Fati.
I want to learn more and more to see as beautiful what is necessary in things; then I shall be one of those who makes things beautiful. Amor fati: let that be my love henceforth! I do not want to wage war against what is ugly. I do not want to accuse; I do not even want to accuse those who accuse. Looking away shall be my only negation. And all in all and on the whole: some day I wish to be only a Yes-sayer.
It’s to not look away from that which dampens our magic. To instead, see the tragedies and use our magic against it.
Book Sculptures
Doing work at the intersection of literature and art, I find Ana Maria Caballero’s recent “Book Sculptures” to be amazing.
Caballero’s Book Sculptures are an eight-part series that questions how society values poetry. This series proposes the book as a sculptural object, and will see Caballero print eight different books, each as an edition of one, with a unique ISBN and the anatomy and structure of a traditional book.
The books in this series contain only one poem, printed 197 times in their pages. When taken as integers, the digits in 197 add up eventually to 8, a number that represents abundance.
Collectors of Book Sculptures also receive an MP4 of a single page of the book endlessly turning while Caballero reads the poem, accentuating the artist’s expansion of the ways poetry can be experienced, exhibited and transacted in the digital age.
I’ve had similar ideations along this line: particularly, a project that also borrows ideas from Beck’s Song Reader album (the album is released only as notation). You write a novella, but what gets released to buy in a traditional publishing format is the journal of the novella’s creation (the writer’s daily thoughts on writing). But the novella itself is an artwork where there would only ever be one copy of. One day, I’ll revisit this idea. :)
Literature-as-art objects changes the frame of reference, especially in terms of how it is valued. Doing this for poetry is even better as it is a far harder medium to publish traditionally for. Really cool.
Spacetime Maps
This was a really interesting exploration of how to adjust maps to show time between points on a map.
It’s janky, but conceptually interesting. A different kind of example in everyday life we encounter is how subway maps are sometimes not overlayed over its spatial distances.
It’s for legibility, but also condenses it to show expected time between stops (on top of spatial dimensions).
🎶 Music
Brothertiger - High Tide
Dreamy song (ht Rian)! Enjoy!
Enjoy a lovely sunset this week.
Simon