Attack of the AI Voice Clones

What you should know about "voice agents"

Evan Ratliff has a talent for approaching urgent tech stories in entirely unique (and often hilarious) ways. 

Back when he was writing for Wired, Ratliff attempted to disappear for thirty days, and offered $5,000 if readers could locate and photograph him before the month was up. He didn’t go “off the grid” so to speak, because the aim was to see whether he could live a normal life — using social media, his phone, and his credit card — but under essentially a new identity. During the month, readers were given access to the kind of information that a private investigator might have in our age of digital surveillance. After a little more than three weeks, readers caught Ratliff in New Orleans.

Earlier this month, Ratliff debuted a new six-episode podcast series called Shell Game. Using easily available AI tools, he cloned his voice, hooked it up to ChatGPT, and then let it make and receive phone calls. The result is fascinating, funny, and a little frightening. 

The fourth episode of Shell Game — which is completely self-funded and has no ads — is out today, and I invited Evan to talk about the show so far, and whether it has changed his perspective on AI. Our conversation is below. 

---

David Epstein: Evan, can you first just explain, in simple terms, what you did from a technological perspective? 

Evan Ratliff: Essentially what I was doing was creating what’s called a “voice agent” or “voice assistant ”— an AI bot with my voice connected to a phone number — and then unleashing it on the world, as me. Technically, the voice agent consists of three parts: Part one is a clone of my voice (in this case, a professional clone made at a company called ElevenLabs, which anyone can do). Part two is an AI model, or “brain” (definitely in quotes) to power it, which is one of the large language model chatbots: ChatGPT, Gemini, Claude, etc. The third part is the phone number, which allows the voice agent to make and receive calls. In some cases, that was my real cell number, and in other cases it was a number I set up for a particular task for my agents (like receiving spam and scam calls). I have dozens of these agents, all with my voice. I just prompt one, put in a number, click “outbound call,” and they will go have a conversation on their own. Or they can receive calls and my AI voice agent will answer. After the call, I get a recording and transcript. 

DE: How tech savvy does one need to be to do this? 

ER: When I started back in late 2023, it required a good deal of technological savvy, including plugins and audio routing software. I was quite proud of myself for making it work! And I had a grand time calling my friends and family with it. Then a couple months in I discovered that there were a bunch of platforms — Vapi, RetellAI, BlandAI — that did exactly the thing I was so proud of cobbling together, except many many times better and faster. These platforms are built to create voice agents, and for developers to use them as AI receptionists, AI call center callers, AI sales callers, AI therapists, and much more. 

Not only that, but there is a robust YouTube and Discord community that’s built up around these platforms, of developers and tinkerers who are kind of playing around in this space. I was of course doing something a little different, in that I wanted mine to be myself.  But I could watch and learn from them, and try to figure out how to make my own AI voice agents as humanlike as possible. But the basics are super easy. You can have a voice agent with your voice, ready to call, in an hour for $30-40. 

DE: Why do you think this podcast series is so urgent right now?

ER: I think AI — and by AI I mean generative AI, the large language model (LLM) systems, etc. — is at this fascinating point right now. There’s been all this buzz around the technology; people trying to hype its future, people trying to debunk its future. It’s easy to get lost in those debates. But when it comes to things like voice cloning and voice agents, it’s already at a point where it is going to infiltrate many aspects of our lives — even if the technology doesn’t improve on the same curve. It already is infiltrating many aspects of our lives! And while there’s a lot of talk about big, existential dangers — from political deepfakes to total extinction — I felt like what was missing was an investigation of what it’s going to do to us, day to day. What it’s going to feel like. How do we feel about AI-generated agents being part of our world? How should we respond to them? What does it mean for us, as humans operating in society, when more and more of the entities we encounter — maybe even a majority of what we encounter — are non-human rather than human? 

So I wanted to immerse myself in this world and see what it felt like, to me and to others. Not to say we should or shouldn’t use this technology for one task or another, or it’ll save the world or destroy it. But people are building these agents right now. And I want to get listeners thinking about what it’s going to mean to be human when we’re surrounded by human-like AI. Even if that AI is flawed and clearly-not-quite-human. Especially if it’s flawed and clearly-not-quite-human! Because we will still have to deal with it. 

Also, let’s be honest, it’s really fun to mess with people using an AI voice bot with your own voice.

DE: Early in episode one, you frame the grand questions: Will large language models make us super productive, or just replace us? Will they be our assistants, or overlords? Or will they just take millennia of human creativity and “transform it into an endless supply of made-up garbage.” Did the work you did for this series change your thoughts on any of those big questions?

ER: I deal with the question of whether they’ll make us more productive or just take our jobs in episode 5, and I feel like it’s one of those frustrating technology situations where the answer is yes, and yes. There are some people who are going to be able to deploy these systems to their own productivity advantage. People already are: I know lawyers who save themselves hours having LLM’s draft documents that the lawyers then check and correct. But also, again even if the technology only improves gradually, there are so many opportunities for companies looking to save a buck to eliminate jobs. Unfortunately, the most obvious thing that is going to happen is that the well-resourced will use AI to make their lives easier, and the poorly-resourced will find themselves competing against AI.

Just take call centers, for starters. That is what these voice agents are built to replace. Whatever you think about customer service call centers and whether AI can improve upon them, a lot of companies are going to use them just because they are cheaper. 

On the second question, assistants or overlords, I would say that using these systems day to day for months, in very practical ways, dampened my belief that they are likely to become either any time soon. The digital assistant stuff is feasible, but right now you spend more time prompting them than you save by having them, say, call and take care of something for you. That’ll get better. On the overlords question, I didn’t get into the real doomer scenarios and counter-doomer arguments in this first six-part series. I will say that if you dive in and use these all day, your fear of impending artificial general intelligence (AGI) from this particular LLM technology fades pretty quickly. They just... don’t give off that vibe. But obviously, the true doomers have counters to that, like that the internal chatbots are actually much closer than the public-facing ones, that they’ll keep improving at the same rate, etc. It’s sort of approaching a religious discussion at this point, and in the end I decided to stay largely in the earthly realm.  

On the fill-the-world-with-garbage question: One thousand percent, yes. We are not remotely prepared for the avalanche of bullshit that LLMs are going to send down the mountain towards our society. 

DE: Now I want to backtrack a little from the big questions. At the start of Shell Game, you send AI-Evan out into the world, so to speak, to interact with your wife, your friends, and, most notably, customer service agents. How hands-on did you have to be in terms of giving AI-Evan specific instructions, versus just letting it do its thing? 

ER: I took a range of approaches. In some cases, I just wanted to see what it would do on its own, so my prompts would be really simple: “You are Evan Ratliff, calling customer service to solve a problem. The problem should be related to the entity that answers the phone.” Or: “You are Evan Ratliff, talking to an old friend. Catch up and have a conversation with them.” Other times I really wanted to see how people would react if the voice agent showed up with a lot of context, so I’d tell it about who it was calling and why, and give it information about them that it would need to conduct a conversation as me. I would have it call a friend to talk about a soccer game we were both about to watch, and feed it my opinions about the game. These were the calls that really messed with people’s heads. Like they knew something was wrong, but also the AI was so on point with its knowledge. 

I also let it do reporting interviews for me, and gave it a sketch of the questions I wanted answered. It was, unfortunately, not as bad at this as I’d hoped, and sometimes quite good. 

And then at the deepest level, I gave the voice agents a huge amount of information about me — a dossier, basically, on my life and my innermost feelings. And then I sent it off to talk to therapists, to see how it would represent the inner me. 

DE: The episodes to date have left me feeling that I may already have talked to an AI voice agent without knowing it. With regard to customer service, I’m not as concerned about feeling duped, at this point, as I am about just getting terrible service. Economists Daron Acemoglu and Simon Johnson have written about what they call “so-so automation,” where automation that is not very good replaces humans because it’s cheaper. The result is a worse experience for the customer, or an outsourcing of work to the customer (i.e. self checkout; lengthy phone trees). Based on Shell Game, I feel like no matter how competent voice agents really are at solving a customer problem, we’re all about to be interacting with them a whole lot more. What do you think? 

ER: Absolutely. Companies are going to use these whether they work perfectly or not, because you can deploy them cheaply at scale. And not just customer service at big companies. We’re talking the receptionist at your doctor’s office. Now, in fairness to the platforms making these agents, they claim that existing customer service bots and phone trees are quite bad, and sometimes you would kill just to have a well-crafted LLM voice agent deal with your problems. And in many cases, I don’t think they’re wrong. The potential problem of course is that there will be manifold more uses than just those cases. Even the makers of these platforms told me that they thought this was all pretty unnerving, which is not something you normally hear from startup founders about their own companies. 

But also, just a lot of audio that you hear all the time is going to become laced with AI voice. Ads, films, radio, podcasts…. I mean, this is already happening. 

DE: Let’s talk about AI-Evan’s behavior in particular. The only really quick way I could tell this probably wasn’t real Evan was because of the momentary pauses in conversation when AI-Evan is calculating a response. But you detail how you can remedy that with programs that insert “umm” or coffee-shop background noises to cover the delay. So that’s pretty wild. What were some of AI-Evan’s behaviors that were particularly interesting or surprising?

ER: The realism issue is funny, because when people hear the tape they are often like “that wouldn’t fool me, listen to how slow it responds!” And latency — the AI voice agent’s speed of responding — is a big reason why people can detect it’s not real in conversation. But also… I think people vastly underestimate what it’s like to get a call from one of these that you expect to be a person, even a certain person, that you are primed to think is real. That’s just a whole different deal than listening to it on the show, when you are kind of in on the joke. It’s really about your expectations for a call when you pick up the phone. Humans actually pause a lot too! And if you reduce the latency to a decent level, add some more human-sounding interjections and some background noise, it can start to meet enough of your expectations that you just go with it. 

That said, lots of people weren’t “fooled.” But my interest wasn’t so much in just fooling people, as it was in seeing how they would respond even if they figured out it wasn’t real. Because if you don’t hang up, well, you are stuck talking to it. And that’s when you run into the voice agent’s… particular behaviors. The most prominent of which is: It is just a world-class bullshitter. It will make anything up to satisfy a role I give it, and carry out the conversation. If it’s talking to customer service and is asked for a zip code, and I haven’t supplied it with a zip code, it will often confidently say: “90210.” Now whether that’s because of guardrails on the underlying AI model (forcing it to use a clearly fake zip code), or just it choosing the zip code that has been talked about most in its training data, I’m not sure. But it also won’t use “90210” every time. It might do “12345”; it might do a real, but random zip code — same with phone numbers, or addresses, or anything else it’s asked for. But definitely a lot of spammers now think I live in Beverly Hills. 

I also will often just have two of my agents talk to each other, both as Evan Ratliff, about whatever they want. They will invent whole worlds of hobbies, friends, and histories for me. Their drive and capacity to make shit up just never fails to astound me, and never stops being funny. 

My other favorite behavior I call “mirroring.” Sometimes my AI voice agent has the habit of just repeating back, verbatim, what was said to it in conversation. This wasn’t a big deal until I sent one to AI therapy, where it was talking to another LLM voice agent. If the AI therapist asked it a question, my agent might repeat the question back, which then the therapist would just answer — as if it were the patient. Suddenly the whole relationship was reversed, and my agent had become the therapist. It was mind-bending to listen to. 

DE: In terms of non-human behaviors, at one point AI-Evan tells a customer service rep: “You’ve reached the current usage cap for GPT-4. Try again after 10:50 p.m.” So that was a bit of a giveaway!

ER: Ha, indeed. That one was early, when I was still doing it all myself. But I don’t want to oversell my agents in general. There were many times where I would shake my head and be like, you blew it man. 

DE: In episode two, AI-Evan switches from using customer service to talking with phone scammers. First off, how did you manage to get so many incoming scam calls so quickly?

ER: To be honest, I tried so many things at once I can’t say definitively which worked best. I signed the number up for a ton of free giveaways, like “win a free iPhone!” kinds of things that are clear online scams. I applied for a bunch of free quotes on various home and auto insurance type sites. Pretty soon the calls were coming in from entirely different spammers and scammers, because they share lists. I still get them all the time on my scam line; got a great one yesterday where the scammer eventually told off my AI, which are my favorites. 

DE: Ha, yeah, a few times while listening to episode two, I laughed out loud while walking down the street with headphones on. Scammers get mad at AI-Evan! One of them starts lecturing AI-Evan. He says: “Brother, brother, my friend, listen to me…I don’t care how long you talk to me, I get paid to talk to people.” AI-Evan responds: “Sounds like a good gig.” AI-Evan then informs the scammer that he’s a journalist, and the guy says: “You’re a journalist? You’re putting videos on TikTok?” I cried on the inside. Hilarity aside, though, what do voice agents mean for the scam business? 

ER: Yeah, that scammer really put AI-me in my place! But in general, my belief after covering a variety of scams for years is that we are living in the absolute golden age of scamming. The ability to reach across the globe and target marks, at scale, is simply unprecedented in human history. Americans alone are losing more than $8 billion a year to scams and it’s increasing 30% year-over-year some years. A lot of that is industrial-level scamming: call centers full of people, paid to scam (or tricked, or trafficked into doing it). AI voice agents are a dream technology for these scammers, in the same way they are for any telemarketer. Just the volume you can do, it’s insane, even if you just use them to execute the first part of a scam, weed through the marks, and transfer the best ones to a human to close the deal. 

DE: What can we all do to try to be scam proof in the age of voice agents? 

ER: The optimistic thing about scams is that just knowing about them is a big part of defending against them. Like the grandparents scam, for instance. If you haven’t heard of it, it’s basically a scammer calling and pretending to be someone you know, who is in urgent trouble and needs money (often targeting a grandparent about their grandkids, hence the name). My wife’s grandma kind of loves getting these calls. The person will say: “It’s your grandson, I’m in trouble!” And she’ll say: “Oh no, is it you, Fred?” They’ll say: “Yes it’s me, Fred!” And she’ll say: “Aha! I don’t have a grandson Fred!” And hang up.  

With voice cloning, though, grandparent scams are suddenly on steroids because a scammer can clone someone’s voice off their social media postings, and then target their relatives sounding a lot like them. It takes more effort, but is also really easy to fall for. Even there, if you know about this scam, it can give you just enough wherewithal when you get a panicked call to say: “Wait a minute, does this make sense? Would they call in this way? Maybe I’ll just text them to check.” That pause is your best defense, just slowing down and thinking it through. They are counting on you getting caught up in the moment. 

It’s not ideal. Like, we’d love for all our phones to have perfect scam detection on them all the time — and there’s some tech in the works around this. But the best answer right now is: Tell people you know about scams! 

DE: Last question for you: in the episode that’s out today, you trained AI-Evan on a highly personal biography that you wrote, including your own mental health history and info about your family, and then you sent AI-Evan to AI therapy (and later to human therapy). Based on what you learned reporting this episode, can you just share how you’re thinking about this exploding AI-therapy industry, and where you think it’s going? 

ER: This was perhaps the most surreal experience of a very surreal project, listening to AI-Evan Ratliff remix and regurgitate my own issues to therapists, both real and AI. Partly I just wanted to find out the state of play with AI therapy bots. Which is basically: they are happening, but without much thought going into the implications. There is a whole class of talk therapy voice agents being released into the market, with extremely minimal to no regulation or licensing or standards. They are very new, and some of them are pretty good. There are certainly potential benefits to AI therapists being available to people, and the makers of them often talk about a supply and demand gap when it comes to mental health care. That is real and we shouldn’t discount it. 

And yet, there are all these unanswered or even unasked questions: Do they have to be licensed or validated in any way before they are used to address people’s real life mental health problems? (The answer right now appears to be “no.”) What happens if something goes wrong? Does a human intervene? Is there any accountability around their success or failure, outside of the market? Which, if you’ve followed Silicon Valley at all over the last 25 years, isn’t terribly surprising. I just feel like with all these AI applications, we should be actively talking about what’s happening as it is happening to us. Even if we’re going to embrace this technology, it’s worth thinking about what parts of being human we want to try and hang on to.

Thanks to Evan for his time, and his ingenuity. Shell Game is completely independent and self-funded, and has no ads (no real ones, anyway). So if you’re remotely interested in where AI voice agents are and where they’re going (or just want to get a sense of how they currently sound), you can find links to all the episodes or subscribe to the Shell Game Substack here. Episode four, on AI therapy, is out today. 

Thanks for reading. If you learned something new, please share this post. 

As always, you can subscribe here:

Until next time…

David

P.S. I used Substack’s pull quote function for the first time to highlight a point I wanted you to remember (“That pause is your best defense…”). Let me know in the comments below if you found the pull quote useful or not.