‘In the end, you feel blank’: India’s female workers watching hours of abusive content to train AI

The unseen labour powering artificial intelligence is increasingly reliant on a workforce facing significant psychological strain. In India, particularly, women in rural and marginalized communities are taking on the role of content moderators, a job that involves reviewing disturbing material and training AI algorithms to recognise harmful content.

The Rise of “Ghost Workers”

Monsumi Murmu, 26, works from her village in Jharkhand state, classifying images, videos, and text flagged for potential violations of platform rules. She views up to 800 items daily. This work, performed by individuals often described as “ghost workers,” is fundamental to the advancements in machine learning, as AI’s effectiveness is directly tied to the quality of the data it’s trained on.

Did You Know? In 2021, the data annotation market in India was valued at approximately $250 million (£180m), with 60% of revenues originating from the United States.

The Psychological Toll

The nature of the work takes a heavy toll on content moderators. Murmu initially experienced sleepless nights and intrusive thoughts, haunted by the disturbing images she reviewed. While she reports a degree of emotional numbing over time, the experience continues to affect her, with dreams occasionally bringing back the disturbing content.

Researchers, including sociologist Milagros Miceli, emphasize the dangerous nature of this work. Miceli states that content moderation “belongs in the category of dangerous work, comparable to any lethal industry.” Studies indicate that it triggers lasting cognitive and emotional strain, leading to heightened vigilance, anxiety, and sleep disturbances. Traumatic stress was identified as the most pronounced psychological risk in a study of content moderators in India conducted last December.

The Vulnerable Workforce

Approximately 80% of data-annotation and content moderation workers in India come from rural, semi-rural, or marginalized backgrounds. Companies deliberately locate operations in smaller cities and towns to reduce costs and access a growing pool of first-generation graduates. Improvements in internet connectivity have facilitated this trend, allowing workers to participate in the global AI supply chain without relocating.

Women comprise more than half of this workforce, valued for their perceived reliability, attention to detail, and willingness to accept home-based or contract work. For many, particularly those from Dalit and Adivasi communities, this work represents an improvement over traditional employment options like agricultural labour or mining.

Expert Insight: The reliance on a vulnerable workforce to perform emotionally taxing work raises significant ethical concerns about the distribution of risks and benefits within the AI industry. The lack of adequate support and protections for these workers could have long-term consequences for both individuals and the development of responsible AI.

Challenges and Concerns

The “respectability” of the work and the need for income can discourage workers from questioning the psychological harm it causes. Raina Singh, a former data annotator, described being assigned to categorize pornographic content, including material involving child sexual abuse, after initially being hired for text-based tasks. She reported feeling disgusted and disconnected from her partner as a result.

Strict non-disclosure agreements (NDAs) further isolate workers, preventing them from discussing their experiences even with family and friends. The absence of legal recognition of psychological harm in India’s labour laws leaves workers with limited protections.

What Might Happen Next

If current conditions persist, it is likely that the psychological toll on content moderators in India will continue to rise. Companies may face increasing pressure to provide better mental health support and transparency regarding job responsibilities. A possible next step could involve the development of industry-wide standards for worker well-being and the enforcement of stricter regulations regarding NDAs. It is also conceivable that increased awareness of these issues could lead to greater advocacy for legal protections for data workers.

Frequently Asked Questions

What is the role of content moderators?

Content moderators classify images, videos, and text flagged by automated systems as potential violations of platform rules, training algorithms to recognise harmful content.

What percentage of content moderation and data annotation workers in India are women?

Women form half or more of this workforce.

What are some of the psychological effects experienced by content moderators?

Content moderators report lasting cognitive and emotional strain, including intrusive thoughts, anxiety, sleep disturbances, and traumatic stress.

As AI continues to evolve, how will the human cost of its development be addressed?

‘In the end, you feel blank’: India’s female workers watching hours of abusive content to train AI | Global development