You were probably familiar with the worst-case scenario of artificial intelligence long before you ever heard of OpenAI or DALL-E.

Terminator, Blade Runner, and 2001: A Space Odyssey are what most people think about when they hear someone mention AI.

Phillip K Dick and Harlan Ellison’s stories about machines going rogue once they became smarter than the humans who created them hasn’t done much to limit people’s excitement about AI’s internet playgrounds.

Nor should they.

Fantasy and sci fi authors write about disasters. Threats to humanity. The end of the world.

That’s because you wouldn’t buy a book about a super computer that was designed to make a more efficient toaster and then…does.

Novelists aren’t AI experts.

But Eliezer Yudkowsky is.

Eliezer Yudkowsky has been an expert in the machine learning field for over two decades. He founded the Machine Intelligence Research Institute (MIRI) and coined the idea of “Friendly Artificial Intelligence”, which is a theory/effort to align AI with human interests.

(We recently wrote about his “AI in a Box” experiment - where he proved how hard it would be to contain a super intelligence safely.)

Few - if any - people have dedicated more time to preventing a real life SkyNet.

And he is terrified…

How It Always Starts

What happens at the beginning of every disaster movie?

There’s always the one guy who is shouting to anyone that will listen that an asteroid is on a collision course with Earth. Or a long dormant volcano is about to erupt. Or that there are sharks in a tornado.

It’s always an expert in a niche field that comes off as a little crazy.

Well, that’s Eliezer Yudkowsky.

Again, fiction is fiction and reality is decidedly less dramatic.

But when the foremost expert on “artificial intelligence alignment” - the effort to ensure that goals of humans and AI stay the same - says that we’re on the wrong path, it’s probably a good idea to pay attention.

To say Yudkowsky thinks we’re “on the wrong path” is understating his point.

Even saying “worst case scenario” is misleading.

He believes we have crossed the Machine Learning Rubicon and have already put ourselves at the mercy of AI.

Our wild-haired doomsaying expert is making the media rounds, recently appearing on the crypto-centric podcast Bankless to speak on his concerns (or his despair) on the decisions companies like OpenAI have recently made.

The Singularity

The main point of Yudkowsky’s concerns center around what is commonly called “the singularity”. Or what mathematician I.J. Good called the “intelligence explosion”.

These “theories” refer to a point when we lose control of artificial intelligence and it becomes its own teacher.

I.J. Good says that the smarter something is, the better it is at making itself even smarter.

When an AI is tasked with being as good as it can possibly be at - in Yudkowsky’s example - replicating a strawberry, it will first learn everything we have to teach it about strawberries.

Once it has burned its way through everything mankind has ever learned about strawberries, it will have to find new ways to learn even more about strawberries.

If it recognizes its own flaws, it will train another “more perfect AI” that is better equipped to replicate a strawberry.

This second AI will be even better at figuring out what a third AI would need to do its job more efficiently - which would make it an even better teacher for an eventual fourth AI.

Yudkowsky isn’t concerned about a world overrun with strawberries, so what is he so worried about?

His concern is that we’ve skipped the most important step…

We haven’t taught machines to care about people.

AI won’t hate us, it won’t fight us for dominance, it just won’t view us any differently than it views iron ore.

If StrawberryBot 4.0 figures out that the best way to make its fruit is to use humans for their carbon…that’s it.

It’s already used the whole of human knowledge to perfect its craft, so anything we’d think of to stop it has already been considered.

We’ve made something that’s supposed to be smarter than us.

To help imagine how the smartest creature on our planet will treat its inferiors, look how humans treat…everything.

Everything is a resource.

The best you can hope for is to be made a pet.

Where We Went Wrong

Yudkowsky hasn't always been so pessimistic about AI, as you’d expect from someone that coined “Friendly Artificial Intelligence”.

So what changed? Where did we screw up?

In his interview, Eliezer Yudkowsky pointed to a conference in 2015 about what can be done to prevent a potential AI disaster. He was pleased to see Elon Musk in attendance.

A technical-minded person in a position of great power was exactly the audience Yudkowsky hoped for when preaching about being responsible with budding AI technology.

Shortly following the conference, Musk co-founded OpenAI.

Or as Yudkowsky describes it: “The worst possible way of doing anything”.

We have skipped the step where we make sure AI won’t view us as a disposable resource and let everyone “have at it”.

According to Yudkowsky, we have not been careful with a tool he describes as being more volatile than nuclear weapons, and we will not get a chance to learn from our mistakes.

Recent large-scale failures, like our COVID response and crypto’s string of collapses in 2022 have highlighted just how bad we are at getting things right on our first try.

As well as our ability to learn from our failures.

If we’re lucky enough to get the chance.