AI Voice Simulator Misused To Deepfake Emma Watson, Joe Rogan, Justin Roiland

ElevenLabs, a research lab exploring new frontiers of Voice AI, is exploring options to implement additional safeguards after its Prime Voice AI led to “an increasing number of voice cloning misuse cases.”

What Happened: ElevenLabs’ official Twitter handle posted several tweets informing users about the significant number of voice cloning abuse that has happened since its beta platform launched. 

See Also: Why Munster Thinks Timing Of Microsoft-Open AI’s Expanded Deal Is A ‘Smart Move’ — Hint: It Has To Do With Google

The company stated that while they can track “any generated audio back” to users who misused the technology, they’d address the issue by “implementing additional safeguards.”

The tweets came after it was reported that many users and trollers were using ElevenLabs’ Prime Voice AI to create hateful messages using deepfake celebrity voices, reported Gizmodo. 

Why It’s Important: The company first released its text-to-voice open Beta platform on Jan. 23, allowing users to clone voices from audio samples. 

According to Motherboard, one 4Chan user posted a clip of Harry Potter fame Emma Watson reading a passage from Mein Kampf — the 1925 autobiographical manifesto by Adolf Hitler. Other 4Chan users reportedly uploaded deepfake voices of celebrities like Joe Rogan and Robin Williams.  

“Rick and Morty” co-creator Justin Roiland‘s titular character Rick Sanchez’s voice was reportedly used to pose as him talking about beating his wife, the report noted. Roiland is currently facing domestic abuse charges.

Read Next: ChatGPT Is Doing for AI What Apple’s iPhone Did for Smartphones