Some followups on the politics of machine translation
Some thoughts on chatbots and immune systems
I got a lot of good feedback and responses on yesterday’s newsletter on the politics of machine translation.
ML expert Matt Post has a great thread that engages with a number of points in the newsletter, adding more color and insight on issues like how Google Translate works, as well as technical fixes and mitigations for some of the problems I raised.
Another point that came up in a few venues, including the comments of the newsletter, is the issue of “fine-tuning” LLMs. You can, actually, tweak and tune the models in a separate, one-off session after they are trained. There’s some discussion of that approach in this paper, which I’ve yet to fully finish.
However, the point I raised in the newsletter about how it’s cost-prohibitive to have an LLM that works on only one subdomain still applies. No one can afford to spend what it takes to train and run an LLM, only to narrow the model’s focus to serve a specific community.
It does strike me, though, that fine-tuning may provide an avenue for the kinds of activist tweaking and agenda-driven meddling I worried about in the Twitter threads linked at the end of that newsletter. I’ll need to drill down deeper on this, though.
Finally, there has been some work done on the problem of gender and machine translation. Here are a few examples I’m aware of:
Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphology
Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overview
Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem
You’ll notice that all four of these papers are from 2019 and 2020. Natural language processing (NLP) systems have only recently gotten as good as they now are, and good technical work on many of these linguistic bias and identity issues is just getting started in earnest.
If you have other examples of technical papers where people are working on solutions for these issues, please drop them in the comments.
Over-sanitizing and suppressed immune systems, but with chatbots
This really thoughtful essay, Fair ML Tools Require Problematic ML Models, did not come my way directly as a result of my MT post, but it does touch on some of the same themes of bias and toxicity in language models.
What I really like about Jacob Buckman’s piece is that it introduces a concept I had not considered for thinking through the problems of identity, community, and toxicity in language models. Specifically, Buckman lays out the chatbot case for what is essentially the LLM version of an immune system, arguing that a chatbot system cannot fight off toxic language without actively seeking out firsthand knowledge of such language and incorporating that knowledge into its model.
If you go to build a chatbot on top of an LLM where the LLM’s training data has been sanitized of anything and everything bad, then you will:
Fail at this, because language evolves, so “toxic” is a moving target, and
Produce a chatbot with no knowledge of toxicity, and therefore no way to filter such out to suit changes in context.
Here’s how Buckman puts it:
We can’t hard-code everything. Language is too broad in its scope, and cultural norms too subtle and varied to police effectively. (Not to mention that they change over time!) The last few years of progress in deep learning provide ample evidence that hand-engineering is terribly limited in its usefulness, and that language is best approached in a data-driven way. The only real solution is for our chatbot to have access to a strong model of harmful language, so it can identify why that description would be inappropriate, and avoid causing this harm.
All the approaches to this issue that I’ve seen so far involve sanitizing either the inputs or the outputs, or complaining about bias effects in the model and hand-waving about how the identity characteristics of the model-makers are somehow to blame for that. So as far as I know, this is a novel idea and one I really like.
Probably the main place where Buckman’s essay intersects with my post from yesterday is where he insists that the LLMs aren’t themselves toxic — they’re just reproducing (and in some cases, amplifying) patterns that are in the training data — but rather it’s specific uses of their output that are toxic.
In other words, the problem of building a non-toxic chatbot should be separated out from the problem of building an LLM. The LLM needs deep and broad exposure to all aspects of language, both the good and the bad, whereas it’s the chatbot maker’s responsibility to see that the bad parts aren’t exposed to the end-user (i.e. by using the LLM’s knowledge of both good and bad to do filtering).
This is similar to a point I raised about MT: right now, it’s properly the user’s responsibility to sanitize and otherwise correct an MT tool’s output. If you’re just publishing the raw output of MT for human consumption, then you’re probably not using the tool responsibly.
Despite all the caveats and concessions in yesterday’s post around this issue of blame, I do still hold to the idea that toxicity and bias problems in MT are currently the responsibility of the tool’s users, even in cases where MT is being used in place of an actual translator. If people use high-tech tools in production, then they need to be aware of their limitations and take measures to mitigate them. Maybe this means placing appropriate warnings on the machine-generated translations that are published, or otherwise positioning them for consumption.
At the point when it’s possible to correct these issues in production, then if a model builder doesn’t correct them, they are at fault. Until then, the parties that deploy these tools need to give some guidance on interpreting the output.