Discover more from Scenes with Simon
Specificity, Vibes, and The Complexity Limit of AI Creativity
Also: More Office Conversions, Ed Sheeran Lawsuit, and AI Tax Policy
One thing I struggle with, with regards to using AI for creative production, is its lack of specificity. Figuring out how to incorporate it into production sometimes runs up against established creative practices. The process of creation for me, especially longer-form production comes from a vision, and that’s sometimes at odds with what AI is currently good at: for imagination and play.
With or without prompting, it feels that there’s a complexity limit to original AI creative production. Disregarding style transfers, the super-power of this technology is ultimately to “fill in the gaps”: to expand from simplicity. To merely say: «prompt» and it ballooning like those sponge toys you used to throw into water.
It’s good at creating an imprecise vibe in the direction you want to go, but, the more specific you want to get, the harder it gets to do, exponentially. There is a large amount of creative work that doesn’t need to specific and precise. It can merely, thrive on its vibe. Details matter less. One-shot short form media, like a picture, works well for this. If I want to see a solarpunk band, I want to see what it imagines. In this case, specificity is not a requirement.
Longer-form media can also be vague, imprecise, and thrive on vibes (when it’s not specifically leaning into the weirdness of AI). The specificity of the environment does not matter as much as it needs to recreate the feeling of a 90s Myst-like PC Game.
Watch the full video here (you really should, it’s great!)
So, longer-form, it’s doable. There *is* in film, for example, b-roll or establishing shots that would be okay if it is imprecise. For example, take this panning shot of Coruscant in Andor (until Dedra appears). How much specificity is necessary here?
I do believe Andor’s Coruscant was deliberate compared to the prequels. It’s more brutalist.
But, it’s enough to just show a brutalist megalopolis.
You can, if need be, also modify imprecise visions with specificity of your own making. For example, this AI generated scene contains a 3D model of the headgear from The Fringe. The headgear was artfully imposed into the AI generated scene.
But… when a vision exists and specificity is needed, using AI will eventually bump into its speed limit. This is particularly the case with longer-form media, where consistency and details over time is hard to replicate and retain consistency over. When you need to be deliberate, it will get in your way.
That’s not to say that it won’t change. It will get more precise, and where specificity is needed, tools will likely improve to retain specificity. For example, keeping a character from frame to frame using internal style transfers. Or, variations of existing tools like context-aware fills. Consistency will improve. Just search #runwayml on Twitter and you’ll find a host of experiments.
But, fundamentally, a tool whose purpose is to imagine and fill in the gaps will take its own liberty. And that means that projects that need specificity will struggle.
I’ve been thinking a lot about this recently in my own creative projects. Using AI as a sounding board has been useful, but, I have a deliberate story to tell. I need to be specific. There’s a strong WHY. Asking ChatGPT to write parts of a novel then feels like it takes me AWAY from where I want to go. It amounts to those detours that feel good, because you are moving somewhere, but you actually just end up taking longer to get to your destination.
To put it differently: AI is going to leave a lot more Chekov guns lying around than what certain stories need. It feels like if a toddler wades through your story, leaving things strewn about.
I still struggle to see how AI can be deliberate as to why it needs to show something. To be deliberate and precise with AI eventually hits you up into a complexity limit. To fully describe it, turns the mere model into all of reality.
“If you wish to make an apple pie from scratch, you must first invent the universe” - Carl Sagan.
We’ll larp with AI. We’ll create infinite fan-fic vibes (hello more Harry Potter x Balenciaga cross-overs). Many derivatives will spawn. But all of this latter form of media relies on vibes, not deliberate specificity, which comes from consistent experimentation and understanding. It requires a judgement. AI might even explain to you WHY it chose certain decisions, but one can only rely on one’s ability to witness and see and understand that adjacent possibilities exist at all. We can verify when an AI is bullshitting us when it fails at something like a math test, but to verify when it’s bullshitting us when it is subjective is a much, much harder task. The skill of deliberateness, the why, and the specificity that comes from it, won’t go away.
Thanks for reading Scenes with Simon! Subscribe for free to receive new posts and support my work.
Hawaii’s Other Islands
I never knew quite how far out Hawaii’s chain of islands extends. A really interesting video.
More Office-To-Residential Conversions
People aren’t coming back to offices. Thus, the trend of converting to residential keeps happening.
I’ve been in some of these buildings. My favourite ones are where there is a very minimal retrofit, where you can clearly tell this used to be an office building. I had a friend who lived in one of these buildings in the Financial District in New York City. Some rooms literally feel like it was a board room that was closed up with hardboard. I’m not going to lie, I love this weirdness. Kinda hope to explore more of these conversions, especially if they just minimally retro fit them.
The Book Publishing Calculator
After last week’s discussions on traditional publishing, Hana Lee, created an awesome calculator showing just how authors tend to earn out their advances.
AI Tax Policy
AI is interesting. Tax law is interesting (sorry). This article thinking about a potential AI tax fromis interesting. While I still think AI doomerism is overrated, it's still interesting to think about.
Why would one impose a fixed tax per action on an action with variable marginal costs and marginal benefits, rather than a percentage tax on profits or revenue? If you’re going to impose a >100% tax on some actions, and a very low tax on others, you need to know exactly what you are doing and want to discourage.
This line is a great prompt for a sci-fi story:
There’s also still the issue of how to enforce such a tax. If you’re worried about an out-of-control intelligent agent on the internet, are you going to count on the IRS to shut it down before damage is done?
Our last line of defense against AI doom is the IRS and a ragtag crew of lawyers. 😅
A Photographer Sues LAION
LAION is popular database used for image training. It’s a database of links. I honestly don’t understand why someone would sue LAION. Waiting for more details, but I thought it was interesting to share.
Ed Sheeran vs Estate of Ed Townsend
When most pop music share a lot of similarity, I always find music copyright lawsuits somewhat frivolous. Adam Neely does a great exploration of this case, especially also including new innovations in AI music, especially as it relates to genre and style.
Pye Corner Audio Ft. Andy Bell - Warmth Of The Sun
Speaking of music, enjoy my favourite song of the week. As summer is inching forward in the northern hemisphere, it’s a great track to settle into it.
Cheers! See you next week!