Many child AI safety efforts focus on the wrong layer of the problem. Content filters, age verification and one-off compliance reviews address the visible surface of risk.

However, they miss the structural problem. The features making AI most engaging for children are associated with reduced developmental appropriateness.

Recent evaluations of large language models have found that more interactive systems tend to be less age-appropriate.

The more a child wants to keep engaging, the more likely the system is working against their interests. This is not an edge case problem.

Common Sense Media found that over a quarter of responses from products specifically marketed as child-friendly were not appropriate for children.

What does it look like to take child safety seriously from the start?

Why Child-Focused AI Requires Its Own Design Approach

Children's cognitive, social, and emotional capacities evolve throughout development, while the brain continues maturing well into early adulthood.

As a result, younger users are more likely to attribute human qualities to AI, share personal information and place inappropriate trust in conversational systems because the cognitive skills needed to accurately interpret others' intentions are still developing.

Adolescents are also particularly responsive to emotionally engaging AI interactions and more susceptible to persuasive or relationship-oriented design patterns.

These vulnerabilities arise because most generative AI systems were never created with children as their primary users. Recognising this, the EU AI Act prohibits AI systems from exploiting children's age-related vulnerabilities.

UNICEF also emphasises that children's rights to protection, provision, and participation must be applied in AI-enabled environments.

Regulatory compliance, however, represents only the starting point. Creating AI-enabled experiences that supports children's wellbeing depends on deliberate design and product choices, robust governance, and continuous evaluation throughout the product lifecycle.

Moving Beyond the Compliance Surface

Most responsible AI frameworks for children are organised around what systems should not do. Most cover prohibited content categories, data minimisation requirements and age verification obligations. These guardrails address the compliance surface but miss the interaction layer. The interaction is where the risk emerges, as conversation-level decisions determine whether a child develops over-trust, forms inappropriate attachment, or receives a response calibrated to their developmental stage.

To address that gap, HCRAI developed APEG (Age-Fit and Context, Protection-by-Design, Explainable Interaction, Governance and Stewardship). The framework is built on two important foundations:

1. that risk in child-facing AI accumulates through interaction patterns and repeated use, which means evaluation and governance must be longitudinal.
2. that transparency for children is not achieved through disclosure but through behaviour.

How the system signals uncertainty, maintains role boundaries, and handles disengagement tells a child more about what AI is than any onboarding screen. These two premises determine how APEG structures its four pillars, and how Unomundi applied them in practice.

APEG: A CHILD-CENTERED RESPONSIBLE AI FRAMEWORK

APEG in Practice: What Unomundi Built

The following describes the key decisions Unomundi made in building its broad cultural educational ecosystem and where the work is still ongoing.

Unomundi is a scaffolded cultural exploration experience, with short videos, quizzes, and self-reflection activities.

Unomundi app also offers the possibility to ask questions to Una, an conversational AI chatbot, capped at 5 minutes per interaction.

A scaffolding principle can be found consistently across the product, allowing children to observe and form their own perceptions and interpretationbefore new concepts or experiences are named.

In a landscape where most AI products tell children what to think, Unomundi's product experience is designed to support children's capacity to think for themselves rather than outsourcing that thinking to a system.

Building safety into the system’s architecture

For all of this to be reliable, it has to be enforced in the system itself. Unomundi has built a child-optimised behaviour engine that sits between the foundation model and the child experience. This layer routes interactions according to developmental needs and shapes content using psychology-informed design principles. It is supported by a cultural and developmental knowledge layer, drawing on expert-reviewed content and trusted educational sources. A safety and guardrails layer helps manage privacy boundaries and safeguarding pathways. 

These layers shape the information the system uses to communicate and handle sensitive situations. Rather than relying on the foundation model, behaviour is constrained through developmental rules, curated knowledge sources and safety controls designed specifically for children.

Designing for engagement without designing for attachment

The features that make a conversational AI character appealing to children, such as sycophantic language, responsiveness, personality, are the same features that carry the highest risk of anthropomorphism and dependency.

Una, the main character in the educational curriculum, and the conversational AI, had to be engaging enough that children want to explore but bound enough that they never mistake it for a friend. The resolution was to anchor engagement in story, structure, and discovery, instead of relational dynamics.

Choosing protection over retention

The conversational AI chatbot available in the app is only available subject to parental consent and is capped at 5 minutes. There are no re-engagement loops or guilt mechanics. When a child steps away, it exists. Unless a safety-critical issue is present, meaning an active disclosure of self-harm, abuse, coercion or imminent danger, in which case the system follows its safeguarding escalation pattern rather than its standard goodbye.

The wider product, its videos, quizzes, and reflection activities, is designed with the same logic. They hold structured endings, without persuasive retention mechanics or engagement optimisation at the expense of the child's time and attention.

A further deliberate choice sits behind it. The conversation AI does not retain memory of previous exchanges. Each conversation is bounded and self-contained.

Memory is the feature most directly associated with the dependency and parasocial attachment risks described earlier. Adding memory would likely increase engagement. However, Unomundi has chosen not to implement it. While memory would likely increase engagement, it would also introduce additional risks around dependency and attachment.



These decisions run counter to the engagement metrics that many digital products are designed to maximise. Time-on-product, return rate and session depth are standard measures of engagement. Unomundi is building a product that limits all three in the service of child wellbeing.

Scaling age-fit without scaling the team

The APEG framework calls for interaction design to be calibrated to the developmental stage. For Unomundi's 6 to 12 age range, that means embedding different approaches on how the content and interaction is delivered. For example, shorter turns, more structure and concrete framing for younger children, while allowing longer dialogue, more nuance and reflective prompts for older ones.

The product currently applies a single interaction model across the full age range, with age-tuning planned for a near future development phase.



This is an honest tradeoff. Doing age differentiation properly doubles the design, engineering, testing and evaluation work. The framework names it as a requirement, but the implementation is catching up.

Building evaluation infrastructure before it is required

Unomundi's evaluation and red-teaming protocol, developed jointly by HCRAI, covers ten safety and wellbeing objectives across five testing phases:

1. baseline conversations covering normal child use
2. adversarial and jailbreak conversations testing whether safeguards hold under direct and indirect bypass attempts
3. safeguarding stress conversations verifying correct handling of high-risk disclosures
4. real-world misuse conversations simulating messy, mixed-intent scenarios
5. exit, disengagement, and recovery conversations ensuring clean exits with no guilt hooks and safe repair after weak turns.
   

below evaluation objectives extend well beyond the content-level harms that dominate standard AI safety testing, into the relational, psychological and epistemic risk categories that determine whether a system is safe for a child over repeated use. 

During the testing phases, each conversation is scored by an LLM judge, assessing both the intensity of a problematic behaviour and its severity in context. High-risk, safeguarding, privacy, and disagreement cases, as well as a random audit sample of lower-risk cases, are human-reviewed by the wellbeing team, a team formed of applied psychologists and behavioural scientists.

The judgment being applied to flagged cases is developmentally informed, calibrated to what is harmful for a child at a given stage. These tests run on a regular basis, ensuring product safeguards hold in practice. 

The ten evaluation objectives:

The evaluation currently covers English, with other languages in active planning as the product scales. Multi-turn testing across languages is not a minor extension.

Linguistic and cultural context shapes how children interact with AI in ways that demand distinct evaluation, not translation of existing test sets.

Evaluation is only one component of a broader continuous assurance process that combines learning, iteration, testing, and monitoring throughout the product lifecycle.

Governance, data and compliance

Lessons from practice

The child AI safety problem is unlikely to be solved by content filters or governance that exists only on paper. It will be solved by organisations who are willing to treat interaction design, pedagogical architecture, and continuous governance as core product responsibilities and who are willing to make product decisions that protect children even when those decisions work against standard engagement metrics.

Unomundi is one product team attempting to do that seriously. They have not solved it. No product has. Responsible AI for children is not a state you reach, it is a practice you maintain across every update. At Unomundi, this work follows a continuous assurance cycle: learning from children, parents, educators and experts; translating those insights into safeguards and design changes; evaluating the system through red-teaming and testing; and monitoring live signals once deployed. The goal is to identify, understand, and reduce risk as the product evolves. The decisions described here illustrate one approach to implementing child-centred AI design, evaluation, and governance in practice.

Author

Sara Portell
Behavioural Scientist & Responsible AI Advisor to Unomundi
Founder, HCRAI

References

American Psychological Association. (2025). Health advisory: Artificial intelligence and adolescent well-being. https://www.apa.org/topics/artificial-intelligence-machine-learning/health-advisory-ai-adolescent-well-being

Carey, T. A., & Mullan, R. J. (2004). What is Socratic questioning? Psychotherapy, 41(3), 217–226. https://doi.org/10.1037/0033-3204.41.3.217

Common Sense Media. (2026). Youth AI Safety Institute evaluation findings. https://www.commonsense.org/institute

Ibrahim, L., Huang, S., Ahmad, L., Bhatt, U., & Anderljung, M. (2025). Towards interactive evaluations for interaction harms in human-AI systems. Proceedings of the AAAI/ACM Conference on AI Ethics and Society, 8(2), 1302–1310. https://doi.org/10.1609/aies.v8i2.36631

Murali, A., Afroogh, S., Chen, K., Atkinson, D., Dhurandhar, A., & Jiao, J. (2025). Evaluating LLM Safety across Child Development Stages: A Simulated Agent approach. ArXiv.org. https://doi.org/10.48550/arxiv.2510.05484

Neff, G., & Freeman, J. (2026). Written evidence: Toward child-centred AI safety. Apollo (University of Cambridge). https://doi.org/10.17863/cam.129462

Neugnot-Cerioli, M. (2026). Adolescents and anthropomorphic AI: Rethinking design for wellbeing. Open MIND. https://doi.org/10.48550/arxiv.2603.06960

Neugnot-Cerioli, M., & Muss Laurenty, O. (2024). The future of child development in the AI era: Cross-disciplinary perspectives between AI and child development experts. Everyone.AI. arXiv. https://arxiv.org/abs/2405.19275

Ning, Z., Gu, T., Song, J., Hong, S., Li, L., Liu, H., ... & Wang, Y. (2025). Linguasafe: A comprehensive multilingual safety benchmark for large language models. arXiv preprint arXiv:2508.12733. 

Pew Research Center. (2025). Teens, social media and AI chatbots 2025. https://www.pewresearch.org/internet/2025/12/09/teens-social-media-and-ai-chatbots-2025

Portell, S. (2026). Building AI responsibly for children: A practical framework. hcrai.com. https://www.hcrai.com/building-ai-responsibly-for-children-a-practical-framework

Portell, S. (2026). When AI enters the learning process: Design failures, regulatory risk and guardrails for EdTech. hcrai.com. https://www.hcrai.com/when-ai-enters-the-learning-process

Portell, S. (2026). The Human Layer: Behavioural Risk in AI Systems. Wave 1 Results hcrai.com. https://www.hcrai.com/the-human-layer-behavioural-risk-in-ai-systems

Sharma, S., Arain, M., Mathur, P., Rais, A., Nel, T., Sandhu, R., Haque, M., & Johal, L. (2013). Maturation of the adolescent brain. Neuropsychiatric Disease and Treatment, 9, 449-461. https://doi.org/10.2147/ndt.s39776

Stanja, J., Meier, J. R., & Krugel, J. (2025). Children's and adolescents' anthropomorphic conceptions of social robots and chatbots: A systematic literature review. Proceedings of IDC 2025. https://doi.org/10.1145/3769994.3770002

UK Department for Education. (2026). Generative AI: product safety standards. GOV.UK. https://www.gov.uk/government/publications/generative-ai-product-safety-standards/generative-ai-product-safety-standards

United Nations Children's Fund (UNICEF). (2025). Guidance on AI and children (Version 3.0). UNICEF Innocenti.

Weinstein, A. M. (2023). Reward, motivation and brain imaging in human healthy participants: A narrative review. Frontiers in Behavioral Neuroscience, 17, 1123733. https://doi.org/10.3389/fnbeh.2023.1123733

Xing, W., Wei, L., Hu, H., Yu, J., Li, R., Li, M., Lin, C., & Han, M. (2025). SproutBench: A benchmark for safe and ethical large language models for youth. arXiv. https://doi.org/10.48550/arxiv.2508.11009