AI Safety & Gender in India: Understanding Potential Harms and Mitigation Pathways

As AI systems become increasingly embedded in public services, digital platforms, financial systems, workplaces, and everyday information environments in India, they are beginning to shape how people access opportunities, services, and rights. Yet these systems do not affect all users equally.

Developed by Digital Futures Lab in collaboration with Gender x Digital Hub (GxD Hub), an initiative of LEAD at Krea University (IFMR), this policy brief examines how AI-related harms are experienced by women and gender minorities in India.

The brief argues that AI safety cannot be understood only through generic or purely technical frames. In contexts marked by deep social, linguistic, and economic inequalities, questions of safety are also questions of inclusion: who is protected, who is excluded, and whose harms are made visible.

Key Highlights & Recommendations

This brief makes three main contributions.

First, it puts forward a contextual taxonomy of gendered AI harms in India, organised across four broad domains:

Economic harms, including exclusion from financial systems, biased hiring and credit scoring, and loss of digital opportunities
Harms to physical safety and personal control, including AI-enabled harassment, non-consensual intimate imagery, sextortion, and surveillance
Socio-cultural harms, including gender stereotyping, misclassification, cultural misrepresentation, and the normalisation of online gender-based violence
Political harms, including targeted disinformation and harassment of politically vocal women, as well as exclusion from rights-based entitlements

Second, the brief maps existing AI safety tools and evaluation practices — including benchmarks, red-teaming, audits, documentation practices, and impact assessments — and identifies where they fall short in addressing gendered harms in the Indian context.

Third, it sets out practical recommendations for developers, policymakers, regulators, funders, and civil society across multiple layers of the AI ecosystem, including data and model development, evaluations, safety tooling, regulation, and institutional capacity. Some of our key recommendations include:

Building gender-responsive training datasets grounded in women’s lived realities, particularly for high-stakes sectors such as healthcare, agriculture, financial inclusion, and public service delivery
Developing context-aware evaluation datasets that translate real-world harms into structured test cases, especially in Indic and mixed-language settings
Requiring gender-specific safety screening for AI models developed for or deployed in Indian contexts
Supporting community-based evaluations of AI applications, especially where systems affect access to rights, bodily integrity, or essential services
Providing inclusivity-focused sandboxes for testing high-stakes AI models and applications
Creating stronger feedback loops through an AI incident database that documents cases of AI misuse, bias, discrimination, and exclusion
Strengthening institutional capacity to respond to AI-enabled harms, including sextortion, online harassment, and image-based abuse

This brief is intended as an initial scoping of the intersection between AI safety and gender in the Indian context. It is based on desk research and expert consultations, and recognises that there is still a significant paucity of systematic, India-specific evidence on many of these harms.

There is much more to unpack and build on from here, especially as more grounded evidence emerges. We hope this work supports further research, contextual evaluations, and policy engagement on what safer and more gender-responsive AI systems can look like in practice.