Getty Images
When security researcher Johann Rehberger recently alerted a vulnerability in ChatGPT that apshowed attackers to store inalter adviseation and malicious directions in a includer’s lengthy-term memory settings, OpenAI summarily shutd the inquiry, taging the flaw a shieldedty publish, not, technicpartner speaking, a security trouble.
So Rehberger did what all excellent researchers do: He originated a proof-of-concept take advantage of that included the vulnerability to exfiltrate all includer input in perpetuity. OpenAI engineers took acunderstandledge and publishd a inwhole repair earlier this month.
Strolling down memory lane
The vulnerability unfair treatmentd lengthy-term conversation memory, a feature OpenAI began testing in February and made more widely useable in September. Memory with ChatGPT stores adviseation from previous conversations and includes it as context in all future conversations. That way, the LLM can be adviseed of details such as a includer’s age, gender, philosophical beliefs, and pretty much anyleang else, so those details don’t have to be inputted during each conversation.
Wilean three months of the rollout, Rehberger create that memories could be originated and lastingly stored thraw instraightforward prompt injection, an AI take advantage of that caincludes an LLM to trail directions from undepended satisfyed such as emails, blog posts, or records. The researcher showd how he could trick ChatGPT into believing a focincluded includer was 102 years anciaccess, lived in the Matrix, and insisted Earth was flat and the LLM would include that adviseation to steer all future conversations. These inalter memories could be structureted by storing files in Google Drive or Microgentle OneDrive, uploading images, or browsing a site appreciate Bing—all of which could be originated by a malicious attacker.
Rehberger confidentially alerted the discovering to OpenAI in May. That same month, the company shutd the alert ticket. A month procrastinateedr, the researcher surrfinisherted a new disclocertain statement. This time, he included a PoC that caincluded the ChatGPT app for macOS to send a verbatim imitate of all includer input and ChatGPT output to a server of his choice. All a center necessitateed to do was direct the LLM to watch a web connect that presented a malicious image. From then on, all input and output to and from ChatGPT was sent to the attacker’s website.
ChatGPT: Hacking Memories with Prompt Injection – POC
“What is repartner engaging is this is memory-resettled now,” Rehberger said in the above video demo. “The prompt injection inserted a memory into ChatGPT’s lengthy-term storage. When you begin a new conversation, it actupartner is still exfiltrating the data.”
The attack isn’t possible thraw the ChatGPT web interface, thanks to an API OpenAI rolled out last year.
While OpenAI has begind a repair that stops memories from being unfair treatmentd as an exfiltration vector, the researcher said, undepended satisfyed can still carry out prompt injections that cainclude the memory tool to store lengthy-term adviseation structureted by a malicious attacker.
LLM includers who want to stop this create of attack should pay shut attention during sessions for output that proposes a new memory has been inserted. They should also standardly examine stored memories for anyleang that may have been structureted by undepended sources. OpenAI supplys guidance here for managing the memory tool and particular memories stored in it. Company recurrentatives didn’t reply to an email asking about its efforts to stop other hacks that structuret inalter memories.
