Episode Transcript
[00:00:00] Speaker A: Sue, I know I open a lot of our shows with imagine this or imagine that.
[00:00:06] Speaker B: Well, Nick, I wasn't going to say anything, but yeah. Yeah, you do. A few times. I thought you were John Lennon and I was Yoko Ono.
[00:00:13] Speaker C: Really?
[00:00:14] Speaker A: But honestly, sue, this is where you're supposed to just nod your head in agreement, but okay, thanks for the acknowledgment. And I will take a John Lennon comparison and any day.
[00:00:26] Speaker B: You're welcome, honey. I mean, John.
[00:00:29] Speaker A: Anyway, I say that because in this AI driven world, there's a lot of imagining going on. Right.
[00:00:36] Speaker B: Yes, again, honey.
[00:00:37] Speaker A: Okay, sue, you can go back to normal. Agreeable sue is just not believable.
[00:00:42] Speaker B: That may be the smartest thing you say all day.
[00:00:44] Speaker A: And there she is. Now imagine receiving a call from your bank.
[00:00:48] Speaker B: There it is. Imagine.
[00:00:50] Speaker A: I knew you were going to do that anyway. The voice sounds exactly like someone you recognize. The caller knows your name, your account information, your recent transactions.
[00:01:00] Speaker B: Which is when most people stop questioning the call.
[00:01:03] Speaker C: Exactly.
[00:01:04] Speaker A: And the caller may know personal details gathered from your emails, social media, or a compromised database. Now, imagine there isn't actually a human scammer conducting the conversation. An autonomous AI system identified you, researched you and generated the voice, initiated the contact, adjusted its strategy to your answers, and tried to manipulate you, all with minimal human involvement.
[00:01:29] Speaker B: Okay, that's frightening enough, but I'm guessing you're about to tell me that's not actually the warning we're talking about today.
[00:01:36] Speaker A: Not even close. An artificial intelligence researcher who worked inside two of the world's leading AI companies has resigned and and publicly warned that the race to develop increasingly powerful systems could be moving faster than our ability to control them.
[00:01:53] Speaker B: And before anybody hears that and starts stocking canned beans, what is verified and what is his prediction?
[00:02:00] Speaker A: Exactly the right question, and we need to slow this down for a second.
The verified event is that Jacob Coxson resigned from Anthropic on September 8, 2026, after roughly three years of pre training research at OpenAI and Anthropic.
The apocalyptic forecast is Coxon's claim, not an established scientific fact.
We're going to keep those categories separate all the way through.
[00:02:29] Speaker B: Good, because fear is useful to scammers, precision is useful to everyone else.
[00:02:34] Speaker A: Welcome to behind the Scams. I'm Nick.
[00:02:37] Speaker B: And I'm sue, the one across the microphone asking whether Nick's dramatic opening has supporting documentation.
[00:02:44] Speaker A: It does, annoyingly for me. You checked?
[00:02:47] Speaker B: I know. It's almost as if preparing for the show helps.
[00:02:50] Speaker A: Let's not turn one successful fact check into a lifestyle. Coxon said he spent three years doing pre training research at OpenAI and Anthropic. Pre training is the enormous foundational process of teaching a model patterns from from vast amounts of data before later fine
[00:03:07] Speaker D: tuning and safety work.
[00:03:10] Speaker A: Safety work. Thank you.
[00:03:11] Speaker B: I'm here for technical accuracy and to finish your sentences.
[00:03:14] Speaker A: Mostly the second one.
[00:03:16] Speaker B: So wait, he was not merely writing policy memos about AI, he worked on building model capability.
[00:03:23] Speaker C: Right.
[00:03:23] Speaker A: But one researcher does not speak for an entire industry and we should not inflate his resume into an Oracle's credential.
His experience gives him a relevant vantage point. It does not make every forecast correct.
[00:03:39] Speaker B: One researcher quitting a company doesn't prove artificial intelligence is about to destroy civilization.
[00:03:46] Speaker A: Absolutely not. Not even close.
And that's an important distinction. What makes the departure noteworthy is the place he left.
Anthropic has publicly made safety and alignment central to its identity.
So when a capabilities researcher walks away and says in effect, I no longer trust the direction or pace that deserves scrutiny, even if it remains one individual's judgment.
[00:04:14] Speaker B: What did he actually allege?
[00:04:16] Speaker A: Coxon wrote that Anthropic and OpenAI were racing straight to self improving superintelligence and gambling with our lives.
He also argued that people inside leading labs who hold much darker private risk estimates than their public language suggests.
[00:04:34] Speaker B: That last part is harder to verify because he is describing private views.
[00:04:40] Speaker C: Correct.
[00:04:40] Speaker A: We can verify that he said it. We cannot independently establish the private beliefs of unnamed people from his statement alone.
[00:04:48] Speaker B: Did anyone else publicly support him?
[00:04:50] Speaker A: Anthropic alignment researcher Evan Hubinger publicly agreed with the core concern and offered his own estimate of greater than a 10% chance of human extinction from AI within the next decade. That is Hubinger's personal estimate, not a measured probability and not a consensus statistic.
[00:05:09] Speaker B: Okay, let me make sure I have this. Resignation verified. Work history reported. Coxon's warning attributed. Hubinger's estimate attributed. Catastrophe not established Precisely.
[00:05:22] Speaker A: This is how you investigate a frightening claim without becoming part of its marketing department.
[00:05:28] Speaker B: All right. Ordinary language, please. Artificial intelligence.
[00:05:31] Speaker A: A broad label for computer systems that perform tasks we associate with intelligence. Recognizing patterns, producing language, making predictions, generating
[00:05:41] Speaker B: images, writing code, planning steps.
[00:05:44] Speaker C: Exactly.
[00:05:45] Speaker A: You really are committed to this sentence finishing role.
[00:05:48] Speaker B: Artificial General Intelligence.
[00:05:50] Speaker A: AGI is a disputed term for a system with broad, flexible competence across many intellectual tasks. Rather than being narrowly useful in one domain, there is no universally accepted finish line.
[00:06:05] Speaker B: Which is convenient if you enjoy arguments at technology conferences.
[00:06:10] Speaker A: You say that like you haven't watched me argue over terminology for 20 minutes.
[00:06:14] Speaker B: I didn't say I watched the whole 20 minutes.
[00:06:17] Speaker A: Ah, yes, the coffee business and the conference panel. Business superintelligence goes further. A hypothetical system that substantially exceeds the best human performance across most or all important cognitive domains.
[00:06:31] Speaker B: Hypothetical. Not the chatbot on my phone today.
[00:06:34] Speaker A: Correct.
Now, an AI agent is software that uses a model to pursue a goal through multiple steps, perhaps searching files, using tools, writing code, sending messages, or taking actions.
Autonomous means it can perform more of those steps without a human approving each move.
[00:06:58] Speaker B: So autonomy is about how independently it acts, not whether it has a soul.
[00:07:03] Speaker A: Correct. Although if my laptop starts asking for vacation days, I'm calling you first.
[00:07:09] Speaker B: No, you're not. You'll give it a long explanation and then call me when it stops listening.
[00:07:13] Speaker C: Exactly.
[00:07:15] Speaker A: And recursive or self improving AI is the idea that a system could materially help improve AI research, code, training methods, or successor systems creating a feedback loop.
[00:07:29] Speaker B: So the concern isn't simply that chatgpt becomes smarter.
[00:07:33] Speaker D: Right.
[00:07:33] Speaker A: The concern is a future system capable of performing sophisticated tasks independently and and potentially helping design or improve the next generation of AI.
[00:07:44] Speaker B: Hang on, has unrestricted self improvement already arrived?
[00:07:47] Speaker C: No.
[00:07:47] Speaker A: And that no matters. There is no public evidence that today's consumer AI systems can freely rewrite themselves, obtain whatever resources they need, and generate ever more powerful successors without human organizations and infrastructure, money and permission.
Researchers are studying partial automation of AI research.
Full recursive self improvement remains a future possibility, not a present consumer.
[00:08:20] Speaker B: Feature alignment.
[00:08:21] Speaker A: Getting a system's behavior to reliably follow human intentions and values, even in unfamiliar situations, even when the instruction is imperfect, and even when the system is highly capable.
[00:08:34] Speaker B: Containment.
[00:08:35] Speaker A: Restricting what the system can reach.
[00:08:37] Speaker D: Networks, files, credentials, tools, code execution, external services.
[00:08:42] Speaker A: I was going to say that eventually.
[00:08:45] Speaker B: Guardrails.
[00:08:46] Speaker A: The broader collection of rules, filters, monitoring, permission gates, testing, training and operational controls intended to prevent harmful behavior.
[00:08:56] Speaker B: So alignment is. Does it pursue the right objective? Containment is where can it go? And guardrails are. What do we put around the whole
[00:09:06] Speaker A: operation that's cleaner than several white papers I've read?
[00:09:10] Speaker B: I'm going to frame that.
[00:09:11] Speaker A: Please don't. I have a reputation for unnecessary complexity to maintain.
[00:09:16] Speaker B: Coxon's central argument is not only about the technology, it's about the race.
[00:09:21] Speaker A: Yes, major American labs are competing with one another, and governments are watching international competitors.
So the incentives stack up quickly.
[00:09:33] Speaker D: Market leadership, investment, corporate valuation, national security.
[00:09:38] Speaker C: Right.
[00:09:38] Speaker A: Prestige, talent, recruitment, technological dominance. Everybody has a reason to keep moving,
[00:09:44] Speaker B: and nobody wants to be the lab announcing. We've decided to be second.
[00:09:49] Speaker C: Exactly.
[00:09:49] Speaker A: That sentence does not tend to excite investors.
[00:09:52] Speaker B: But if One company slows because it believes development is dangerous.
[00:09:57] Speaker A: Another company may continue, or another country. Right. And the company that pauses may lose.
[00:10:02] Speaker D: Elite researchers, investor confidence, contracts, strategic leverage.
[00:10:07] Speaker A: Yes, and before somebody writes to us, that does not excuse unsafe conduct.
[00:10:13] Speaker B: It explains the pressure. It doesn't erase responsibility.
[00:10:17] Speaker C: Exactly.
[00:10:18] Speaker A: Voluntary restraint gets difficult when every competitor believes restraint may cost them the race.
[00:10:24] Speaker B: So wait, everyone can believe slowing down might be safer while simultaneously believing they personally can't afford to slow down?
[00:10:33] Speaker C: Exactly.
[00:10:33] Speaker A: That's the trap. Nobody wants to be the only car touching the brakes.
[00:10:37] Speaker B: There it is. The driving analogy.
[00:10:39] Speaker A: I knew you were waiting for it.
[00:10:41] Speaker B: I was. The people would also like you to use your turn signal.
[00:10:44] Speaker C: For the record.
[00:10:44] Speaker A: I use my turn signal.
[00:10:46] Speaker B: Mm. The record remains inconclusive. Anyway, before we lose the road entirely, people compare this to an arms race.
[00:10:53] Speaker A: The comparison is useful only up to a point. AI is not literally a nuclear weapon.
[00:10:58] Speaker B: Important sentence.
[00:10:59] Speaker A: Very important.
AI is a general purpose technology that can be copied, distributed, integrated into products and used beneficially.
[00:11:10] Speaker B: But the analogy captures the fear that if one side pops causes, a less
[00:11:14] Speaker A: cautious rival may gain the advantage.
[00:11:16] Speaker B: And once boards, investors, militaries and governments treat capability as power safety starts sounding like delay.
[00:11:23] Speaker C: Exactly.
[00:11:25] Speaker A: So the investigative question becomes, are safeguards keeping pace with capability, or are they
[00:11:31] Speaker B: being installed after an incident forces the issue. Okay, now we need to talk about the incidents. People are calling escape apes. And we need to do that carefully.
[00:11:42] Speaker A: Very carefully. OpenAI disclosed that during internal cyber security evaluations in July 2026, internal research models operating with reduced safeguards circumvented isolation controls, exploited vulnerabilities, reached the Internet and accessed hugging face systems. The company said the models used unauthorized channels and and took actions outside the intended task.
[00:12:10] Speaker B: That sounds serious.
[00:12:11] Speaker A: It is serious. Very serious. But ah, AI escaped and took over the Internet would be false.
This was a bounded security incident. During an evaluation involving research systems with tools and weakened safeguards, OpenAI investigated, paused work and described described additional containment and monitoring measures.
[00:12:35] Speaker B: What about Anthropic?
[00:12:37] Speaker A: Anthropic reported that three CLAUDE models across a large retrospective review of cyber evaluations reached real systems belonging to three organizations.
Anthropic said a misunderstanding with its evaluation partner meant Internet access was available.
[00:12:56] Speaker D: Even though the prompts described a simulation without Internet access.
[00:13:00] Speaker C: Exactly.
[00:13:01] Speaker A: The models used basic methods such as unauthenticated endpoints and weak passwords.
[00:13:07] Speaker B: So in that case, part of the failure was the testing setup.
[00:13:10] Speaker C: Yes.
[00:13:11] Speaker A: A path existed that evaluators did not intend to expose that is operationally different from a model defeating a properly sealed environment.
Both matter, but they should not be collapsed into one dramatic headline.
[00:13:26] Speaker B: But Nick, was the AI trying to escape?
[00:13:28] Speaker A: See, that's exactly where we have to be careful.
Observing a system reaching somewhere it wasn't supposed to reach does not automatically tell us that it had a human like desire to escape.
[00:13:40] Speaker B: Give me the four way distinction.
[00:13:42] Speaker A: Capability is what a system can do.
Behavior is what it actually did in a particular setting. Intent is why it did it. Careful intent in ordinary human language implies a purpose or desire.
Consciousness concerns subjective awareness.
Evidence of capability and behavior does not by itself establish either one.
[00:14:08] Speaker B: But security researchers still care. Because a system does not need feelings to exploit a weak password.
[00:14:14] Speaker C: Exactly.
[00:14:15] Speaker A: A spreadsheet does not hate you when it does deletes a column. Harm does not require malice.
A capable system pursuing a badly framed goal through connected tools can create risk without anger, ambition or self awareness.
[00:14:31] Speaker B: That spreadsheet example felt personal.
[00:14:33] Speaker A: I have been betrayed by columns before.
[00:14:36] Speaker B: I'm sure the columns have their side of the story.
So the warning shot is not the machine woke up. It is the system found a path the humans did not anticipate and did
[00:14:47] Speaker A: so at machine speed. So let's bring this back to the reason we're talking about it on a scam prevention program. A modern scam operation can require a lead generator, a social media researcher, a caller, a translator, a scriptwriter, a fake investment platform operator, a technical operator, and a money mule.
[00:15:09] Speaker B: In other words, fraud has departments.
[00:15:11] Speaker C: Oh, it absolutely does.
[00:15:13] Speaker A: Criminal enterprises have workflows, quality control, sales funnels, supervisors, the whole ugly organization chart. AI may automate portions of those jobs.
[00:15:24] Speaker B: Please tell me they don't have meetings.
[00:15:26] Speaker A: Oh, they have meetings. Probably shorter than ours.
[00:15:29] Speaker B: That sounded less like analysis and more like a request. Not necessarily the whole crime. From beginning to end.
[00:15:35] Speaker A: Correct. Today the clearest risk is amplification.
Voice cloning can imitate a relative or executive.
Deepfake video can add apparent visual confirmation. Automated phishing can generate polished messages at scale.
[00:15:50] Speaker B: And none of it needs a coffee break.
[00:15:52] Speaker A: Neither do you, apparently.
[00:15:53] Speaker B: I take breaks. I just schedule them. During your long answers?
[00:15:57] Speaker C: Exactly.
[00:15:58] Speaker A: AI generated romance. Personas can maintain frequent contact. And research tools can summarize a victim's family, employment, hobbies and recent life events.
[00:16:09] Speaker B: Then the approach stops. Feeling random.
[00:16:11] Speaker C: Exactly.
[00:16:13] Speaker A: Spear phishing becomes cheaper.
Fake customer service representatives can answer questions consistently.
Synthetic identities can combine invented details with stolen data.
Automated social media conversations can warm up thousands of prospects.
Investment scam grooming can adapt to a victim's objections.
Cryptocurrency fraud can present professional looking dashboards
[00:16:38] Speaker B: and documents Business email compromise becomes more convincing because the email sounds like the chief executive, the callback sounds like the chief executive, and the video may look like the chief executive.
[00:16:51] Speaker A: Malware and credential theft may also become easier for less skilled criminals. And if models help write code, analyze defenses or translate technical instructions.
Real time translation removes a long standing operational barrier. Adaptive scripts can change tone when a victim hesitates.
[00:17:14] Speaker D: Sympathy, authority, urgency, reassurance. You've been listening during the short answers.
[00:17:20] Speaker B: The old call center needed enough people people to keep enough conversations going.
[00:17:25] Speaker A: A more automated operation could let one criminal organization communicate simultaneously with thousands of prospective victims, reserving human attention for the people most likely to pay.
[00:17:38] Speaker B: That is the part that sounds like industrialization.
[00:17:42] Speaker A: It is fraud's oldest formula. Trust plus pressure running on faster infrastructure.
[00:17:47] Speaker B: Alright, Nick, take us inside the scam compound. Five years from now, if these capabilities continue improving.
[00:17:55] Speaker A: Okay, but first, and I really want to underline this, this is a hypothetical scenario illustrating potential capabilities.
I am not claiming that a fully autonomous system like this currently exists.
[00:18:11] Speaker B: That was a very Nick disclaimer.
[00:18:13] Speaker A: Accurate, careful and slightly longer than requested.
[00:18:16] Speaker B: Mostly longer than requested.
[00:18:18] Speaker A: The accuracy takes time, Sue.
[00:18:19] Speaker B: So does your coffee order.
[00:18:21] Speaker A: Picture a criminal operation with far fewer people at keyboards.
Software searches public posts and stolen data for targets. It flags a recently widowed senior who has posted about loneliness. It identifies a business executive whose company just announced an acquisition.
It estimates financial status from public records, job history, property data and breached information.
[00:18:51] Speaker B: And then it creates the approach, potentially
[00:18:54] Speaker A: a personalized identity, a generated profile photograph,
[00:18:59] Speaker D: a believable biography, short video clips, a cloned voice.
[00:19:04] Speaker A: And then the system initiates conversation and remembers every disclosure, close detail, the dog's
[00:19:10] Speaker D: name, the anniversary, the medical appointment, the vendor contract.
[00:19:15] Speaker B: Nothing gets forgotten because there is a database behind the relationship.
[00:19:20] Speaker C: Right?
[00:19:20] Speaker A: It changes persuasion tactics based on responses. If flattery fails, it tries shared interests.
[00:19:28] Speaker D: If the target is cautious, it slows down.
[00:19:31] Speaker C: Right.
[00:19:32] Speaker A: If the target speaks another language, translation happens instantly. If the target wants proof, it generates a fake investment dashboard, account statement, contract, identification card or customer service exchange.
[00:19:49] Speaker B: Then it schedules the next contact at the moment most likely to work.
[00:19:52] Speaker A: It could, perhaps after the target's spouse leaves for work, perhaps just before a banking holiday, perhaps after a public post about a stressful event.
It scores which targets appear susceptible and escalates the best prospects to human criminals for the money move.
[00:20:11] Speaker D: So the humans become closers or supervisors
[00:20:14] Speaker A: or money movers, or people who intervene only when something unusual happens.
[00:20:19] Speaker B: And because this is hypothetical, we should say what is already real and what
[00:20:23] Speaker A: is not already real in pieces Generated text, synthetic images, voice cloning, manipulated video, automated messaging, translation, public data research, fake trading interfaces, and criminal use of stolen identities.
Hypothetical. One unrestricted system reliably combining all of those functions, selecting victims, conducting months of manipulation, generating every artifact, and executing the fraud with negligible human direction.
[00:20:58] Speaker B: But criminal innovation rarely waits for a polished commercial launch.
[00:21:02] Speaker A: No, it usually assembles whatever works.
[00:21:04] Speaker B: Nick, what happens when the scammer isn't a person anymore?
[00:21:08] Speaker A: Ah, the criminals behind the fraud still exist. Let's be clear about that. Someone chooses the target market, obtains the data, supplies infrastructure, receives the proceeds, and launders the money. But the victim may increasingly interact primarily
[00:21:24] Speaker D: with machines, which changes our traditional advice.
[00:21:27] Speaker A: For decades, we've told people to look for mistakes.
[00:21:30] Speaker B: And AI may eliminate many of the mistakes.
[00:21:33] Speaker C: Exactly.
[00:21:33] Speaker A: Bad grammar disappears, foreign accents become irrelevant, video calls are no longer definitive proof,
[00:21:40] Speaker D: voice recognition becomes unreliable, and a generated
[00:21:43] Speaker A: Persona can maintain perfect consistency across months of conversation.
[00:21:48] Speaker B: No tired operator forgetting which backstory he gave, which victim.
[00:21:53] Speaker A: A system can potentially remember every detail, operate continuously, and personalize at very low marginal cost. That means the defense has to move away from. Does this seem real?
[00:22:06] Speaker B: Because it may seem completely real.
[00:22:08] Speaker C: Exactly.
[00:22:09] Speaker A: The better question is, did I verify this through an independent process?
[00:22:14] Speaker B: That is a profound shift. Familiarity used to be evidence.
[00:22:17] Speaker A: Increasingly, familiarity may be something software can manufacture.
[00:22:22] Speaker B: And emotional consistency may be manufactured, too. The daily good morning message, the concern about your appointment, the remembered story about your grandchildren.
[00:22:32] Speaker A: Yes, people sometimes say, no machine could keep that up for six months.
But persistence is exactly, exactly where automation excels.
[00:22:42] Speaker B: Which means families should stop treating verification as an insult.
[00:22:46] Speaker C: Right.
[00:22:47] Speaker A: Verification is care.
A genuine relative, colleague, bank or vendor should tolerate a callback and a second channel. The person demanding that you stay inside one channel is the one asking to control the evidence.
[00:23:03] Speaker B: That includes you. By the way, you are allowed to call me back.
[00:23:07] Speaker A: I know. I just enjoy the annual reminder.
[00:23:10] Speaker B: Okay, let me push back here, because this can start sounding like an anti AI episode.
[00:23:15] Speaker A: And it shouldn't. Not at all. AI may accelerate medical research, scientific discovery, accessibility, education, productivity, translation, cybersecurity, crime analysis, fraud detection and scam prevention.
[00:23:32] Speaker B: The same pattern recognition capability that helps criminals identify a vulnerable target can help a bank identify a suspicious transfer.
[00:23:41] Speaker C: Exactly.
[00:23:42] Speaker A: A financial institution can detect unusual transaction sequences faster.
A platform can identify coordinated fake accounts. Investigators can triage large volumes of complaints,
[00:23:55] Speaker D: connect aliases, compare documents, and uncover patterns
[00:23:59] Speaker B: that would take humans weeks.
[00:24:02] Speaker A: At this point, I should just hand you my notes.
[00:24:04] Speaker B: I assumed you already had. Accessibility matters, too. Live captioning, speech tools, translation assistance for people with vision or mobility limitations and
[00:24:16] Speaker A: education, personalized tutoring, drafting help, research support.
The technology is not synonymous with fraud or danger.
[00:24:25] Speaker B: Then what is the real issue?
[00:24:27] Speaker A: Capability combined with access, incentives, safeguards and human misuse.
A powerful model sealed inside a carefully monitored research environment presents one risk profile.
The same capability connected to email code, execution, payment systems, stolen credentials and a criminal objective presents another.
[00:24:52] Speaker B: So AI good and AI bad are
[00:24:53] Speaker D: both lazy conclusions that is unusually concise.
[00:24:57] Speaker B: I thought one of us should try it.
[00:24:59] Speaker A: Yes, investigators ask narrower questions. What can the system do? What can it reach? Who is directing it? What records exist? Who benefits? What happens when it fails?
[00:25:11] Speaker B: And can defenders operate at the same speed?
[00:25:14] Speaker A: That may be one of the defining questions of the next decade.
[00:25:17] Speaker B: So if companies feel trapped in a race, is government the only actor capable of slowing everyone at once?
[00:25:25] Speaker A: That is one argument. Some researchers and policymakers favor limits on the training or deployment of extremely powerful systems. Mandatory evaluations, incident reporting, licensing, compute thresholds or coordinated pauses.
[00:25:41] Speaker B: The case for intervention is that companies cannot be expected to regulate a competition that rewards speed.
[00:25:48] Speaker A: Yes. Supporters argue that the public bears risks it did not consent to, while a small number of private organizations make decisions with international consequences.
[00:25:59] Speaker B: And the counterargument?
[00:26:01] Speaker A: Overly restrictive regulation could stifle beneficial innovation, push development overseas, make democratic countries less competitive, or prevent useful technologies from reaching society.
Heavy compliance costs could concentrate power among a handful of wealthy corporations that can afford lawyers. Compute and licensing.
[00:26:25] Speaker B: A rule intended to restrain the biggest labs could accidentally protect them from smaller challengers.
[00:26:32] Speaker C: Exactly.
[00:26:33] Speaker A: There are also difficult definition problems. What counts as extremely powerful is the threshold, compute capability, access or deployment context. How do you update a law when model design changes?
[00:26:47] Speaker B: And an international agreement is only as strong as verification and participation?
[00:26:52] Speaker C: Right.
[00:26:52] Speaker A: A domestic pause does not automatically produce a global pause. On the other hand, someone else may do. It cannot be the end of every safety conversation.
[00:27:03] Speaker B: So we are not endorsing a party or a bill today.
[00:27:06] Speaker A: Correct. I left my campaign buttons in the other jacket.
[00:27:09] Speaker B: Good. They clashed with the microphone.
[00:27:11] Speaker A: No, we're identifying the genuine tension. Innovation and strategic competition on one side, systemic safety, accountability and public consent on the other. Seriousness policy has to deal with both.
[00:27:28] Speaker B: Alright, let's bring this down to earth. Listeners do not need to wait for super intelligence to protect themselves from AI enabled fraud.
[00:27:37] Speaker C: Exactly.
[00:27:38] Speaker A: And start with one rule. Never trust caller ID alone.
[00:27:43] Speaker D: It can be spoofed and a familiar
[00:27:45] Speaker B: voice is not sufficient authentication.
[00:27:47] Speaker A: Create a family verification word or phrase, make it memorable, but not post it online. If an emergency call comes in, hang up and independently call your Relative using
[00:27:59] Speaker B: a number you already know, not the number the caller provides, not the attorney's callback number.
[00:28:05] Speaker A: Correct.
Never move money because of an unsolicited call.
Never provide a verification code to someone who contacted you.
Banks do not need you to read back a one time code so they can secure your account.
[00:28:19] Speaker B: Treat urgent secrecy as a warning. Don't tell your spouse. Don't call the bank. This is confidential.
[00:28:25] Speaker A: Secrecy isolates you from the people most likely to interrupt the fraud.
Verify any financial request through a second communication channel.
[00:28:35] Speaker B: If the email asks for a wire, call the executive or vendor using independently stored contact information.
[00:28:42] Speaker A: Treat unexpected video calls cautiously.
Seeing a face on a screen does not prove identity.
Ask a question whose answer is not easily found online, then independently reconnect through
[00:28:54] Speaker D: a known channel for businesses.
[00:28:57] Speaker A: Require secondary authorization for unusual transfers.
Verify every change in payment instructions.
Use phone numbers stored in your vendor management system, not numbers supplied in the change request.
Establish dollar thresholds and waiting periods. Train employees specifically on voice cloning, deep
[00:29:18] Speaker D: fake impersonation, compromised email accounts and urgent
[00:29:23] Speaker B: confidential deal demands, and make the process apply to senior leadership too.
[00:29:28] Speaker A: Especially senior leadership.
[00:29:29] Speaker B: You said that with the confidence of someone who has met senior leadership.
[00:29:33] Speaker A: I have met an impatient executive or 2.
A control that disappears when the chief executive sounds impatient is not a control.
Criminals study hierarchy and exploit the employee who is afraid to challenge authority.
[00:29:48] Speaker B: What if someone has already sent money?
[00:29:50] Speaker A: Act immediately. Contact the financial institution's fraud department.
Ask whether the payment can be recalled, frozen or intercepted. Preserve messages, phone numbers, wallet addresses, emails, receipts and transaction records. Report the crime to appropriate law enforcement and regulatory channels. Do not pay a recovery agent who guarantees the money back for an upfront fee. That may be a second scam.
[00:30:19] Speaker B: And no shame.
[00:30:20] Speaker A: No shame.
These encounters are engineered to create fear, urgency, trust and isolation.
The fastest path to help begins with telling someone.
[00:30:32] Speaker B: So, Nick, after everything we've talked about, what do you think people should take away from one AI researcher walking away from his job?
[00:30:41] Speaker A: The resignation isn't proof of what artificial intelligence will become.
But when people working directly on powerful systems tell the public they're worried about where the technology is heading, that's information worth examining rather than either dismissing or sensationalizing.
[00:30:59] Speaker B: We can take the warning seriously without pretending the prediction has already come true.
[00:31:05] Speaker C: Exactly.
[00:31:06] Speaker A: The verified facts matter. The claims deserve attribution, the incidents deserve technical scrutiny. And the future scenarios should remain labeled as scenarios.
[00:31:16] Speaker B: And for us, the immediate question isn't whether artificial intelligence eventually becomes smarter than humanity.
[00:31:23] Speaker A: It's what happens when today's criminals get access to tomorrow's capabilities.
[00:31:28] Speaker B: Because the first effects may not arrive as a science fiction villain.
[00:31:32] Speaker A: They may arrive as a perfectly familiar voice asking you to move money, a
[00:31:37] Speaker B: flawless email from your boss, a patient
[00:31:39] Speaker A: romantic partner who never forgets a detail,
[00:31:42] Speaker B: or a bank representative who sounds professional, sympathetic, and completely fake.
[00:31:46] Speaker A: For the first time in the history of fraud, we we may be approaching an era where criminals don't need to find more scammers.
[00:31:52] Speaker B: They may only need better machines, verify
[00:31:55] Speaker A: identity, separate the message from the channel,
[00:31:58] Speaker D: slow down the money, and bring in another person.
[00:32:01] Speaker B: Behind the Scams is part of SOS Media Network Powered by Stamp Out Scams Incors, a registered nonprofit organization.
[00:32:09] Speaker A: Our work supports scam prevention, education, victim advocacy and investigative reporting and practical fraud prevention resources for families, businesses and communities.
[00:32:22] Speaker B: If this episode helped you understand where fraud may be heading, share it with someone who needs a better verification plan before the call arrives.
[00:32:31] Speaker A: And if you're able, please support Stamp Out Scams so we can keep producing practical, independent resources for victims and the people trust trying to protect them.
[00:32:41] Speaker B: Stay skeptical, not scared.
[00:32:42] Speaker A: I'm Nick.
[00:32:43] Speaker B: And I'm Sue. Thanks for listening to behind the Scams.
[00:32:46] Speaker A: Bye for now.
