A group of researchers created a first-generation AI worm that can steal data, spread malware, and spam others via an email client to spread through multiple systems. This worm was developed and successfully functions as designed in test environments using popular LLMs. Based on their findings, the researchers advised GenAI developers, sharing a concern about the potential dangers such malicious programming can have. The team shared research papers and published a video showing how two methods are used to steal data and affect other email clients.
Ben Nassi created this worm from Cornell Tech, Stav Cohen from the Israel Institute of Technology, and Ron Bitton from Intuit. They named it 'Morris II' after the original Morris, the first computer worm that created a worldwide nuisance online in 1988 . This worm works by targeting Gen AI apps and even with Gen AI-enabled email assistants that generate text and images against AI models like Gemini Pro , ChatGPT 4.0, and LLaVA.
This works by using an adversarial self-replicating prompt that is used against its models, similar to how jailbreak functions to spread toxic content by using AI. The researchers demonstrated this by creating an email system with these generative AI engines and using a self-replicating prompt using text or embedded in an image file.
The text prompt infects the email assistant using the LLM to use extra data from outside its system, which is then sent to GPT-4 or Gemini Pro to create text content. This content jailbreaks the GenAI service and, hence, successfully steals data. The image prompt method encodes the self-replicating prompt in an image, making the email assistant forward messages containing propaganda and abuse to everyone, infecting new email clients and forwarding the infected email. During both processes, the researchers could mine confidential information, including but not limited to credit card details and social security numbers.
Such functioning worm, even in a controlled environment, proves that it is no longer theoretical and needs to be looked at seriously with effective solutions deployed whenever such malicious prompts are found. This is where research papers like this come in, where they're shared with the affected parties and for others to simulate and verify.
GenAI Leader's Response and Plans to Deploy Deterrents
Like all responsible researchers, the team reported their findings to Google and OpenAI. Wired reached out where Google refused to comment about the research, but OpenAI’s spokesperson responded. They said, “They appear to have found a way to exploit prompt-injection type vulnerabilities by relying on user input that hasn’t been checked or filtered.” They also assured that it is making its systems more resilient and added that developers should use methods that ensure they are not working with harmful input.
Seeing that such methods can infect Generative AI applications and compromise users’ systems, such findings come when AI and NPUs are implemented on GPUs and CPUs for PCs , smartphones, cars, and email services, which are crucial. In some cases, AI-infused SSDs can identify and eliminate ransomware. But on the other end, we have worms and custom LLMs that can create malware .
This is where the industry needs to pace itself and have countermeasures to attack or deploy an effective solution for every genAI-based product released to the public. New solutions and innovations potentially create new problems. With such research exposing such issues during the early stages of AI apps, priority needs to be taken to secure GenAI engines that could potentially harm.
