AI and Language
Natural Language Processing, a branch of Artificial Intelligence, is the ability of computers to process and analyze human language using computational techniques. Familiar applications of NLP are chatbots, voice assistants, grammar checkers. While it has been widely taken up by governments and industry, claims of ‘algorithmic injustice’ have also surfaced, which suggest that the machine is replicating the discriminatory social biases present in the historical data that it has been trained on (Brookings, 2021).
NLP in India
Work on NLP is gaining ground in India. State educational institutions like IIT Madras are collaborating with global corporations like Google and non-profits such as Pratham Books to create local open-source NLP for Indian languages called AI4Bharat (IndiaAI, 2021). To provide some real-life examples of the nature of intervention that is being visualised with regard to NLP in education – Pratham is designing AI-based recommendations of educational content for students, grading of numerical and reading skills are becoming automated as is assessment of short free text or voice answers. The requisite infrastructure for these operations is also being built through libraries of digitalised voice and textual data on Indian languages like INLTK, Indic NLP Library, StanfordNLP . Nevertheless, the corpus of annotated data remains inadequate as there is no data on non-scheduled languages which have no official recognition and often no written text, and even for those, there is very little representation of dialectical diversity (GIZ, 2020).
Further, many Indian languages have very distinct and different typographical (font) and morphological (word structure, tone, emphasis, etc) features. This diversity is at risk with the introduction of ‘foundation models’ like GPT3, BERT, RoBERTa, BART, T5 which form the ‘language agnostic’ base over which most NLP is built nowadays as they are known to have a homogenizing effect on the language (Bommasani, 2021). Additionally, the data that is available is not open source, and therefore expensive with restricted access for smaller organisations/individuals (GIZ, 2020). These data problems might lead to exclusion of people from the NLP applications, or marginalisation of certain forms of language. Given that Indian Language NLP currently has no ‘benchmarks’, or a set of tasks, by which the functionality of a software is evaluated, it becomes more difficult to detect and reverse these problems (Sambasivan, et al. 2021).
NLP in Education
In public conversations about the future of education, while there is a simultaneous “tech-evangelism” and a moral panic around teachers being made irrelevant by AI, it might be useful to think closely and speculatively about the equity effects that the use of NLP could have for students.
In India, resources in higher education are mostly available in English, which disadvantages the vast majority of students whose elementary education has been in regional-language medium. The National Translation Mission aims to revert these inequalities by providing translations of textbooks and supplementary material in the Scheduled Indian languages (PIB, 2019). Natural Language Processing software, by automating this process could support this mission by providing cheaper, swifter translations. This process is worth speeding up as this gap is currently filled by unregulated actors – individuals, companies – whose content is not standardised. However, automating these translations will require resources i.e a corpus of digitised and labelled text data that is unequally available for most Indian languages now. In fact, the level of resources available for more socially dominant languages like Hindi far outpace that of more marginalised languages like Assamese (Sambavisan, et al 2021 and GIZ, 2020). Thus, students from regionally and socio-economically marginalised groups, who tend to need the translations, are likely to receive either poorer quality translations or none at all, unless there is an urgent investment in building repositories for these low resource languages.
The other popular use of NLP in education is teaching through chatbots, particularly in aid of language teaching. As there is shame associated with lack of familiarity with ‘prestige languages’ like English and language learners might hesitate to speak for fear of making mistakes, app-based language learning has its benefits for equity. A recent instance of this is the app Professor Idiom which is an NLP English-teaching chatbot for Bengali speakers (IBM, 2021). But the problem of data persists, especially so for non-hegemonic dialects/versions of the language as they are not available in written form. The voice data that is available for the Scheduled Languages, usually covers only dominant dialects. Therefore the chatbot will be unable to recognise the language of other dialect speakers and be unusable for them. For instance, Mozilla’s Deepspeech performs worse for Indian English speakers (Nacimiento-García et al, 2021) and African American English (Martin and Tang, 2020). This problem of dialectical bias in speech recognition software exists outside of English, and evidence for it has been found in Arabic (Droua-Hamdani et al., 2012) as well as anecdotal evidence on Microsoft Kinect not recognising Castilian Spanish (Tatman, 2017).
NLP software which can provide formative feedback to English language writers, on questions of grammar, spelling, sentence construction, adherence to academic norms, can mitigate differences in fluency, and consequences on the grades received (Eg. Grammarly). Students thus have a better chance of being judged for their grasp of the content. Given the established linkages between class and access to quality English medium education, this is an important inequality to address. NLP software can also be used by staff for automated grading support (eg. ‘e-rater’, an automated scoring system for essays used by ETS). The reduction of this burden on teachers can also be beneficial indirectly to marginalised students. Institutions need not hire more staff to take on this work, which would have driven up costs and consequently user fees. Additionally, research shows that the teacher’s social biases on gender and race can shape who they perceive to have academic ability (Zimmerman and Kao, 2020), and in India these tendencies would likely translate to caste as well. Automating this process reduces the possibility that these arbitrary biases come into play. However, a Plain Language Movement is afoot in academia to democratise academic writing, with advocacy for greater accessibility in style and de-jargonisation (Clayton, 2015). Automated scoring that relies on historical data could be replicating higher-score patterns of higher scores to essays that display greater complexity in syntax and vocabulary. In addition, qualities that so far cannot be judged by machines are originality and creativity, which are of prime importance in high-stakes essays. Cathy O’Neill (2016) argues that automation tends to be first inflicted on the least advantaged as mass automation can often be cheaper than employing human labour. In this context that would imply – disadvantaged students being limited to a technocratic judgment of their abilities, and not be screened for higher-order abilities. This might result in talent among the less advantaged lying unidentified.
The Way Forward
Apart from the broader questions being raised about the compulsive need for automation or the future of human labour in education with the advent of AI, most of the critique of NLP-based technology can be traced to the existence of patchy data. A contributing factor for this is that it is largely created in commercial global north enterprises. This leads to poorer representation of global south voice and text data in their corpus, as well as a bias against underprivileged populations who do not constitute a relevant ‘market’ for most technology, or have no presence and access to digital spaces. As education is a Public Good, and NLP is being brought to intervene in removing barriers in access to learning, it is important for governments to invest in collating well-labeled, diverse and representative datasets, which are acquired with consent and compensation of data providers. Technology is often seen as a tool to build a new world. The work of those engaged in ‘futures thinking’ in education is to also attend to the ways in which the hierarchies and power relations of the past persist into these futures and creatively subvert them.
References
O’neil, C. (2016). Weapons of math destruction: How big data increases inequality and threatens democracy. Crown.
Zimmermann, C. R., & Kao, G. (2019). Unequal returns to children’s efforts: racial/ethnic and gender disparities in teachers’ evaluations of children’s noncognitive skills and academic ability. Du Bois Review: Social Science Research on Race, 16(2), 417-438.
Droua-Hamdani, G., Selouani, S. A., & Boudraa, M. (2012). Speaker-independent ASR for Modern Standard Arabic: effect of regional accents. International Journal of Speech Technology, 15(4), 487-493.
Tatman, R. (2017, April). Gender and dialect bias in YouTube’s automatic captions. In Proceedings of the First ACL Workshop on Ethics in Natural Language Processing (pp. 53-59).
Nacimiento-García, E., González-González, C. S., & Gutiérrez-Vela, F. L. (2021, September). Automatic captions on video calls, a must for the elderly: Using Mozilla DeepSpeech for the STT. In Proceedings of the XXI International Conference on Human Computer Interaction (pp. 1-7)
Martin, J. L., & Tang, K. (2020). Understanding Racial Disparities in Automatic Speech Recognition: The Case of Habitual” be”. In INTERSPEECH (pp. 626-630).
Sambasivan, N., Arnesen, E., Hutchinson, B., Doshi, T., & Prabhakaran, V. (2021, March). Re-imagining algorithmic fairness in india and beyond. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (pp. 315-328).
GIZ (2020) A study on open voice-data in Indian Languages.
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., … & Wang, W. (2021). On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258.
Brookings (2021) Detecting and mitigating bias in natural language processing. https://www.brookings.edu/research/detecting-and-mitigating-bias-in-natural-language-processing/
IndiaAI (2021) IIT Madras is building language technology for Indian Languages. https://indiaai.gov.in/case-study/iit-madras-is-building-language-technology-for-indian-languages
Press Information Bureau (2019). Various steps has been taken to promote Regional Languages in Higher Education: HRD Minister.
Agnihotri, R. K. (2019). Linguistic diversity and marginality in South Asia. Handbook of education systems in South Asia, 1-37.