The Dark Side of Language Models

Uwais Iqbal • 2023-02-01

There are a growing number of concerns around the sustainability, social impact and openness of language models that are seldom discussed. While language models like BERT and GPT-3 are all the rage, they aren’t without their vices. The current trend in NLP is one of brute force where ‘bigger is better’ - more data, more parameters and ever larger models. How sustainable is this approach and are there any drawbacks?

Bias

Language models are trained on massive amounts of internet text. While there is useful and beneficial textual information on the internet, there are endless troves of toxic xenophobic and racist content that also exists.

Language models can’t discriminate between socially positive material and socially problematic content. They end up being exposed to the whole lot during pre-training and learn from these skewed perspectives.

The extreme biases and racist associations perpetuated within the content end up being scaled and surfaced within the language models themselves.

BERT has a problem with gender bias where roles like nursing are automatically associated with the female gender [1], while GPT-3 is Islamaphobic and struggles to associate Muslims with anything other than violence [2].

Carbon Footprint

Language models require intensive computational resources to pre-train, deploy and maintain. These resources output carbon emissions and the overall carbon footprint of deep learning-based language models in particular is significant.

One study estimates that simply training a deep learning model can generate carbon emissions equivalent to the total lifetime carbon footprint of five cars [3].

As AI becomes more mainstream, ever more models will be trained and deployed but rarely do AI teams in industry or academia stop to think about the environmental impact of their work.

Privatisation

The sheer scale and magnitude of language models mean that university academics with limited budgets and resources are sidelined. The only players in the game are big tech companies.

They are the ones who can bear the financial costs of pre-training on massive amounts of data and have the engineering talent to pull off the feat of creating models with hundreds of billions of parameters.

AI is a powerful technology that will impact every sphere of society. Such a powerful technology being dominated by the interests of a handful of private companies is a dangerous road to follow.

Closing Thoughts

While language models have pushed forward the realm of what is possible with NLP there certainly are issues that need addressing.

Bias, environmental impact and equality of contribution and access are all growing issues that are present more widely in AI but are manifested together in language models.

The next time you hear someone singing the praises of BERT or GPT-3 gently remind them that there is a dark side too. A sustainable and ethical approach to AI requires new thinking, close engagement with subject matter experts and ultimately transparency.