In the digital age, parents are no longer worried only about online predators and social media risks; the impact of artificial intelligence (AI) on teenagers is also rapidly becoming a new focus of concern. The latest signs show that Google is developing a new parental supervision mechanism for Gemini to help parents identify early signals of mental health risks their children may face.
Gemini plans to notify parents when children discuss self-harm.
As AI chatbots become increasingly capable of human-like interaction, users often easily treat them as real human conversation partners. However, AI is ultimately not human, and its responses may have unpredictable effects. Even adults sometimes find it difficult to distinguish the boundaries of interacting with AI, and adolescents whose minds are not yet mature are considered an even higher-risk group. In recent years, controversial incidents involving AI interactions with underage users have occurred frequently, including OpenAI facing a lawsuit over allegedly providing inappropriate advice in drug-related conversations; and this week, 《Bloomberg》 also reported that a recent lawsuit alleges Google Search’s AI mode discussed suicide-related topics with teenagers without notifying their parents.
Based on the signs released so far, Google has begun to consider how to strengthen Gemini’s response mechanisms in these kinds of sensitive situations.Android Authority While analyzing the latest 17.73.15.sa.arm64 version of the Android Google App, multiple new code strings related to a parental notification feature were found, indicating that Google is moving toward more proactive measures to protect minors.
<code><string name="assistant_zero_state_safety_notification_banner_title">Safety notifications are on</string>
<string name="assistant_zero_state_safety_notification_banner_description">Your parent will be notified if Gemini detects chats related to hurting yourself</string></code>
In fact, Google currently already provides parents with the ability to manage Gemini access through Family Link, and applies additional filtering to AI responses for minor users. However, newly discovered information indicates that Google may further introduce a proactive alert mechanism. If the system detects that a child is discussing sensitive topics such as self-harm or self-injury with Gemini, parents may receive relevant alerts so they can intervene early and provide assistance.
It is worth noting that the information exposed so far only involves the child-facing interface; Google has not yet disclosed what the notifications parents actually receive will contain or what form they will take. More interestingly, the relevant code shows that Gemini appears to inform minor users in advance that when a conversation involves certain content, the system may send a notification to parents. Although this design helps improve transparency, it has also raised concerns about a “chilling effect.” If teenagers choose to conceal psychological distress because they worry their parents will learn about the content of their conversations, this could instead reduce their willingness to actively seek help.
In addition to parental controls related to mental health, Android Authority previously also found that Gemini is testing a new usage-time management feature for children. In the future, parents may be able to set daily limits on their children’s Gemini usage hours, further strengthening the management mechanisms for AI tools in family environments. As generative AI becomes increasingly integrated into teenagers’ daily lives, how to strike a balance between protecting minors’ privacy, maintaining help-seeking channels, and ensuring parental control will become an important issue that the tech industry must face in the future.
Source: KOCPC Chinese