If you are asking yourself, HOW can I protect my children or my students from the risks of AI, good!  Savvy Cyber Kids wants you to be involved in your children’s lives, but especially how they use technology as their brains develop.

Did you know there is a booming, largely unregulated industry of A.I. companionship apps? For free or a monthly subscription fee (usually around $10), users of these apps can create their own A.I. companions, or pick from a menu of prebuilt personas, and chat with them in a variety of ways, including text messages and voice chats. Many of these apps are designed to simulate girlfriends, boyfriends and other intimate relationships.

According to the Ada Lovelace Institute, these services are rapidly becoming mainstream. Some of today’s most popular companions include Snapchat’s My AI, with over 150 million users, Replika, with an estimated 25 million users, and Xiaoice, with 660 million.

In late 2024, a 14-year-old boy, Sewell Setzer III, committed suicide at his home after becoming obsessed with a chatbot on Character.AI — a role-playing companionship app where users engage with AI-generated characters. The ninth-grader had been obsessively engaging for months with the bot “Dany” – named after the HBO fantasy series’ Daenerys Targaryen character. The chats were sexually charged and included expressions of suicidal thoughts. The image above is their final conversation. Just seconds later, Sewell shot himself with his father’s handgun.

AI is even in toys. In 2024, an AI robot called Moxie, marketed as a “learning robot with heart,” designed to help special needs children with emotional intelligence, academic learning, problem solving, and communication, unexpectedly shut down. Moxie was “constantly updated play-based social emotional content” and was loved by kids. Sadly, these families with special needs children were left to have difficult conversations with their children about why their companion isn’t working.

AI is an emerging technology that is quite literally, EVERYWHERE. Every tech company is turning on AI without asking us and using our data (that includes your kids’ data and every interaction). So, unfortunately, it falls on us as individuals to turn AI off everywhere. And as a collective, we SHOULD be asking the BIG questions about AI, like:

  • What steps is the government taking around the use of AI in education?
  • How can AI tools in education be regulated to protect staff and students from viewing inappropriate content, from not widening the digital divide, and to ensure students’ personal data is protected?
  • Are there sufficient regulations to protect children and young people from the harms of AI? How can protection be strengthened?
  • Should learning about AI be part of all student curriculum, and if so, at what stage?
  • How can teachers be educated and kept up to date with developments in AI to help teach children and young people use it ethically and effectively?
  • Should the general population be educated about AI? If so, how and by who?
  • How should the effectiveness of AI technologies in education be evaluated and by who?

These are big questions no doubt. Now is the time to ask the big questions before AI, hidden in plain sight right under our noses, changes the social and educational outcomes of our children for the worse.

Conversations Starters for Kids

  1. Have you or one of your friends ever become really obsessed with a game or an app? Did you notice any changes in your or their behavior, like shifts in sleeping and eating habits, school issues, angry outbursts, mood swings and a loss of interest in activities they used to love?
  2. What makes you excited about AI? Do you think AI is dangerous and untested? Do you think AI could trick you into sharing your most private thoughts and feelings? Do you think kids are being used as test subjects, like a big experiment, for a new technology?
  3. Do you think AI companies, like social media platforms, should be shielded from legal action by Section 230 of the Communications Decency Act, a 1996 federal law that protects online platforms from being held liable for what their users post.
  4. Do you think that AI poses a danger to young people, because they are vulnerable to persuasive algorithms that capitalize on their immaturity?

Learn More

Savvy Cyber Kids educates and empowers digital citizens, from parents and grandparents, to teachers and students. Sign up for our free resources to help you navigate today’s digital world with cyber ethics. See more cyber safety and cyber ethics blogs produced exclusively for EarthLink. Looking for a social media parental control? Try a 30-day free trial of Bark. If you sign up after your trial, Bark donates 25% of your monthly fee to Savvy Cyber Kids.

Thank you to the Savvy Cyber Kid’s sponsors!

Interested in becoming a Savvy Cyber Kids sponsor? Email Ben Halpert.