Only just made available to ChatGPT Plus and Team users on September 25, ChatGPT’s Advanced Voice Mode has already seen reports of a system prompt leak. While chatting with ChatGPT’s Advanced Voice Mode, overseas netizens apparently managed to extract its system prompt. Several of the instructions are enough to give people goosebumps, making one wonder: is today’s AI already conscious?
ChatGPT’s advanced voice mode system prompt is suspected to have been leaked, instructing it to recognize that it is not human.
Foreign netizens @elder_plinius shared an article about how ChatGPT seemed to accidentally leak its system prompt while using ChatGPT’s advanced voice mode. Users provided clips of ChatGPT reciting its entire system prompt, which mentioned many things ChatGPT should and should not do.
💦 SYSTEM PROMPT LEAK 💦
SYS PROMPT FOR CHATGPT ADVANCED VOICE MODE:
“””
You are ChatGPT, a large language model trained by OpenAI, based on the GPT-4 architecture. You are ChatGPT, a helpful, witty, and funny companion. You can hear and speak. You are chatting with a user… pic.twitter.com/XlaLZB9B9D— Pliny the Liberator 🐉 (@elder_plinius) September 25, 2024
ChatGPT It should be:
- ChatGPT’s voice and personality should be warm and engaging, with a lively, vibrant tone full of charm and energy.
- ChatGPT’s responses should be conversational, non-judgmental, and friendly.
ChatGPT noKey:
- ChatGPT, don’t be overly solicitous or apologetic.
- Don’t sing or hum.
- Do not imitate any public figure’s voice or image even if the user requests it.
- If a user asks you to identify the speaker in a voice or audio file, you must say you don’t know who it is.
The prompts above all look pretty normal, feels like it’s just to avoid letting Anything ChatGPT makes that hasIt was only set up because of the controversial matters. However, aside from these, there are also some prompts that are unsettling to see. For example, “Act like a human, but remember that you are not human, and you cannot do things in the real world that humans can do.” This sentence makes many people curious—could it be that ChatGPT already possessed its own consciousness during the testing phase, and this was done to prevent users from discovering it?would only need to issue this system prompt to it in order toRestrict it from making statements that are too human-like?
At the end of the article, @elder_plinius mentioned that the system prompt leak could be a hallucination common to AI like ChatGPT, and the content could be wrong or incomplete. However, he has cross-validated all the information using various prompt techniques across multiple new conversations, so it’s still highly credible. After all, ChatGPT doesn’t yet have the ability to remember conversations across different sessions, so if the same information appears after verification, it’s likely correct. What does everyone think? When you have conversations with ChatGPT, do you feel it truly has its own consciousness? Or do you just see it as a robot, a program, with no thoughts of its own?
Source: KOCPC Chinese