By Hija Kamran

Scrolling through social media, there are high chances of coming across a video of a child just being a child, or an underage creator performing for the screen. Many parents who work as content creators set up their children’s Instagram accounts as soon as they decide on the name to “lock username and future digital identity”, as one famous Pakistani couple mentions on their child’s Instagram account. Kids like Zartasha Kashif, Muhammad Sheeraz and many of the same age are practically growing up on screen. Most of the time, their videos feel routine – cheerful glimpses into family life or their everyday creativity that appear harmless and funny, or even necessary in an economy where every extra stream of income counts, as for many parents, the proposition to turn engagement into earnings is understandable. So we, as audience, rarely pause to consider what sits beneath this constant stream of childhood on display. But something about the sheer volume, the ease of access, and the normalisation of it all points at a deeper shift in how children are being seen in digital spaces, and how in the long run, this exposes a child to exploitation, profiling, and the loss of control and agency over their own narrative. This shift deserves closer attention as the stakes may be far greater than we allow ourselves to admit.

Billionaires who have built their fortunes by expanding the reach of technology to previously disconnected geographies through often deceiving schemes, and by encouraging people to share their most intimate moments online, rarely show the same enthusiasm when it comes to their own families. Take Mark Zuckerberg, owner of Meta, for example. In July 2023, he shared a family photo with the faces of his children hidden with emojis. It is understandable, and honestly expected, for any parent to be cautious about their children’s privacy, even if in Pakistan that caution is often rooted in cultural beliefs about the evil eye. But the irony is hard to miss when it comes from the man who repeatedly claims that his platforms prioritise and respect user privacy.

I have always had a particular kind of dislike for tech CEOs who very regularly make sweeping commitments they never intend to keep. But this case goes beyond the usual cycle of hollow promises. It is not only about the privacy that platforms are supposed to guarantee for their users, but also the privacy that users should be expecting from one another. Even if Zuckerberg genuinely believes his platforms prioritise user privacy (which I highly doubt), his own caution around his children’s visibility suggests an instinct to protect them from the countless eyes that would linger on their faces. The dilemma sits in that odd space where corporate negligence and everyday human behaviour merge, making children an easy resulting target.

As a digital rights advocate, I find it relieving that in Pakistan, privacy is no longer the contested topic it once was among many people who use the internet in the country. A decade ago, when I was doing community engagements and trying to explain why the then-proposed Prevention of Electronic Crimes Act (PECA) should matter to them, the routine dismissal of online privacy as a right was mostly expected. Almost everyone would brush it aside, insisting that they have nothing to hide, while others would argue that it was unrelated to the realities of the people of Pakistan who are more concerned about “roti, kapra, makaan.”

However, with time, attitudes towards the right to privacy and its interconnected civil liberties have shifted, thanks to various factors that have evolved in the past decade. A younger, more informed generation is navigating online spaces with much more awareness, questioning the terms on which their interactions with technology are shaped. This shift has also been pushed by the many violations people have witnessed as successive governments passed draconian legislation, and as digital colonialism deepened. But it is equally driven by lived experience shaped by data leaks, increased surveillance, and the realisation that the loss of privacy is directly connected with how we experience and access all of our rights. It encompasses everything from political expression to personal safety, and people are increasingly beginning to recognise that.

So what is it about our interactions on the internet that makes us overlook children’s need for privacy? Parents often shape their online lives around their children’s happy moments, and understandably so as children define the day of a household. But content that is often just a simple memory for parents becomes engagement and exploitable data for tech companies that profit from a dataset they can get on any individual – adult or not.

Kidfluencers as business strategy

The phenomenon of “kidfluencers” has become increasingly common worldwide, with young parents curating dedicated social media accounts to document their children’s lives as they navigate modern parenting. A Netflix documentary, “Bad Influence: The Dark Side of Kidfluencing,” documents the experience of a group of children who were forced into content creation by parents because of the economic prospects attached with it. As a result, kids were shown to be consistently sexualised, forced to act like adults, maintain an unsustainable routine and lifestyle for their age, and carried the pressure of public image.

Brand deals, polished lifestyles and even body language that appears almost professionally coached can seem enticing when viewed through the lens of monetisation, especially in a global economy where many have built significant wealth through online visibility. However, this monestisation creates pressures that reshape childhood, and pushes them into continuous performance through curation of an online persona, repeating behaviours for views, and prioritising an audience’s expectations over private development. Researchers find that this adultification can harm mental health, normalise commodification of the self, and increase exposure to predatory attention. Frances Haugen, a whistleblower, revealed internal Meta documents in 2021 that showed that Instagram is worsening body image issues in teen girls and the company knew it. A 2023 lawsuit against Meta further claimed that Instagram is affecting young users’ mental health, stating, “Its motive is profit.”

It is true that platforms profit from attention, not childhood. Ad-driven business models and recommendation systems reward content that keeps viewers watching and clicking, and children’s videos perform well and therefore get amplified, generating revenue for the company, whether the clip is harmless or exploited. A 2022 analysis shows that major platforms made 11 billion USD from youth engagement alone, reminding that children are not an incidental audience but a lucrative business decision. Content that attracts attention becomes valuable, and that value can be monetised even when the material is abusive. Parents turning children into branded channels risk seeding data and images that can be weaponised by offenders or repurposed by bad actors using AI.

More sinister than imagined

The risks are not limited to a child’s biometric information sitting on a server in a data centre somewhere in the global North, being sold to the highest bidders as part of the collective audience metrics for targeted advertising. Instead, each post builds a traceable identity long before the child has a say in who they are or how they are represented. This premature digital identity is already being manipulated, stolen and weaponised in ways parents rarely imagine – from AI-generated deepfakes to long-term profiling that follows them into adulthood. As a result, the cost of casual sharing is far higher than a lost sense of privacy in an atmosphere where children’s data can fuel surveillance systems, discriminatory algorithms, or even facilitate targeted violence.

In Pakistan particularly, children’s safety is already compromised by sprawling networks of pedophile rings and the vast circulation of child sexual abuse material across the internet and the dark web. Every image and digital footprint feeds into an ecosystem that is far less innocent than it looks on the surface. And as tech companies continue to evade meaningful accountability, the onus of protection ends up shifting to those not always adequately equipped to understand the dangers their children are exposed to.

In addition, the commercialisation of AI is largely being seen as transformative in the content creation industry as many use these tools to generate bizarre outfit transitions or showing off a reality that doesn’t exist to an impressionable audience. However, there is a far more sinister use lurking beneath the surface. A multi-billion dollar underground economy exists around child sexual abuse material (CSAM) that is now employing AI for their business, thriving even as regulations struggle to keep up. According to the Internet Watch Foundation (IWF), there were 245 reports of hyperrealistic AI-generated child sexual abuse imagery in 2024, up 380% from 2023, amounting to 7,644 images, including deeply disturbing videos. Out of these images and videos, 6,945 or 98% showed sexual abuse of girls.

AI’s ability to generate hyperrealistic images has created a particularly alarming risk for young girls. Even innocuous photos can be manipulated to produce sexualised depictions that never existed in reality, turning a child’s likeness into a tool for abuse. These images are disturbingly convincing, often indistinguishable from real photographs, and can circulate widely before detection. In some cases, AI is used to amplify features or expressions in ways that sexualise children, contributing to online demand for exploitative content.

What’s worse is that AI tools are constantly being refined to produce on-demand outputs that seem hyperrealistic, evident from the latest models of Generative AI images that no longer display the familiar giveaways of AI creation, like distorted hands, airbrushed skin, or blurred or undefined hair. In some cases, known survivors of abuse or publicly-known children are being revictimised through “deepfake” videos, and these are not confined to hidden corners of the web. Almost all AI-CSAM flagged by the IWF in recent months has appeared on publicly accessible sites, not just the dark web.

The technology’s speed and scale are deeply worrying as a single AI model can generate tens of thousands of new abuse images in minutes, overwhelming existing systems that rely on “hash matching” to spot known content. Moreover, a Stanford Internet Observatory study found that one of the most widely used open-source AI training datasets, LAION-5B, contained over 3,200 images suspected to depict child sexual abuse, which may be embedded in models now widely in use, and has also been noted to produce nude or explicit content out of fully-clothed images.

Pakistan’s failure

In Pakistan, the threat of child sexual abuse is entrenched in systemic failures as witnessed over the recent years. The country was reported to be among the top three countries witnessing the surge in child sexual abuse material (CSAM) content. According to Sahil, a children’s rights organisation, 3,364 child abuse cases were reported across the country in 2024 alone. The Interior Ministry has also confirmed hundreds of child rape cases over recent years, noting 200 cases involving children between 2021 and 2025 in Islamabad, with only 12 convictions.

In light of this already fragile landscape, the rise of AI-enabled child exploitation and the connected risks significantly multiply. With the steady rise of CSAM being produced for a global audience – as seen in Pakistan’s largest known CSAM case, which involved the abuse of an estimated 300 children in Kasur – commercial AI has only made the situation more dangerous. The technology increases profitability for abusers by eliminating logistical barriers as in many cases, a single photo of a child is enough to generate hyperrealistic abuse images and videos. However, with the ability of AI to make up faces and identities that do not exist in reality, predators do not need real children's photos to produce CSAM for the market. It also accelerates production, allowing offenders to create vast quantities of material in minutes, all while exploiting the grey areas of legislation that has not yet caught up with the speed and scale of the AI advancement.

These threats are all the more dangerous in a cultural context where “honour” is enough to kill a girl. In Pakistan, the sexualisation of children, real or AI-generated, collides with deeply rooted norms around family reputation. This dynamic can have devastating consequences as children who are “seen” in ways their families didn’t intend or consent to may be shamed, punished, or even physically harmed in the name of preserving honour. Such exposure could trigger not just digital violence, but real-world violence for many unaware of how their images are being used in the dark corners of the internet.

Meanwhile, the law is trying to catch up, but with serious gaps. Pakistan’s Prevention of Electronic Crimes Act (PECA) was amended in 2023 to introduce the Criminal Law Amendment Act 2023 that more explicitly criminalises grooming, solicitation, and commercial sexual exploitation of children online. Yet challenges in the control of dissemination as well as enforcement of laws remains weak, which is coupled by the lack of digital literacy among parents, which is further exacerbated by limited trust in the justice system and reporting mechanisms. This is further evident in a worrying paradox that shows that even though online CSAM reports from Pakistan number in the millions, fewer than 350 cases were formally reported to Pakistan’s FIA over a five-year period.

In a country like Pakistan, where the protection of children is already undermined by weak institutions, weak convictions, and cultural violence, AI-generated imagery could become a tool for exploitation and harm, rather than just a technological worrying trend. It has become urgent to rethink how much of a child’s personhood is exposed online and whether parents still retain any real control over their children’s images, especially when those images contain sensitive data like biometric identifiers. In many contexts, including Pakistan, families often struggle to recognise children as individuals with their own identities and rights. But the principle is simple: just as adults would not want their photos circulating online without consent, children deserve the same respect for their privacy and agency. Protecting that boundary is not only a parental responsibility but a fundamental safeguard against the evolving risks of AI-driven misuse.

Way around

In light of this realisation, and with an acknowledgement that while parents may wish to continue sharing images of their children, it is possible to do so more responsibly in the current tech realities. Posting low-resolution images, limiting angles to a side profile, avoiding sharing videos and photos of children showing a wide range of emotions and facial expressions can reduce the likelihood that AI tools replicate a child’s likeness with high precision. It is important to note, however, that the digital security approach of today may not be relevant tomorrow as technology is constantly advancing at an accelerating rate. But the underlying principle remains critically important throughout – just as you wouldn’t leave your child unattended with a stranger on the street, you should exercise the same caution in managing your child’s digital identity.