Former Pakistani Prime Minister Imran Khan Addresses Supporters from Jail Via Deepfake Generative AI Voice Clone [Report]
Pakistan’s former Prime Minister Imran Khan managed to deliver a campaign speech from prison this week using a voice clone created using deepfake voice cloning, as reported by The Guardian. Khan’s team employed tools built by synthetic audio startup ElevenLabs to create the voice clone, debuting it during a virtual rally.
Synthetic Speaker
Khan provided text the outline of the text, which was fleshed out by members of his Pakistan Tehreek-e-Insaf (PTI) party into a final script. The team used archival samples of Khan’s past speeches and ElevenLab’s synthetic audio engine to create a model for Khan’s voice that could mimic the sound and rhetorical style of the former Prime Minister. The final audio was overlaid on historical footage in a virtual rally broadcast to Khan’s supporters. Viewers were notified through captions that the message utilized his digitally recreated voice.
Khan has been under a media blackout imposed since riots erupted following his previous arrest in May. The technology enabled Khan to circumvent that rule, with the virtual rally drawing more than 4.5 million views across social platforms despite internet censorship attempts. Though the reaction to the synthetic voice was mixed, according to the report, it still shows how AI impersonation can be employed to evade state suppression, albeit with many limits and caveats. Khan hopes to regain his position following Pakistan’s national elections in February using these kinds of tactics, where regulations are still fuzzy where they exist at all.
Khan’s move is a particularly high-profile example of the growing trend among politicians to employ generative AI and synthetic media for their campaigns as in-office projects. For instance, New York City Mayor Eric Adams recently deployed a generative AI voice clone of himself speaking multiple languages in a set of robocalls to voters. The Mayor’s synthetic voice speaks Spanish, Mandarin, Yiddish, and other languages and alerts people to events like job fairs. It’s enough of a concern that Google changed its rules for political ads ahead of the 2024 elections to require generative AI disclaimers.
Follow @voicebotaiFollow @erichschwartz
NYC Mayor Adams Uses AI Voice Clone for Multilingual Robocalls
Google Mandates Deepfake Generative AI Disclaimers on Political Ads Ahead of 2024 Election
