How machines absorb human associations between words

Waterfall of Meaning uses technology called ‘word embeddings’ to analyse millions of existing English sentences and map words’ meaning based on their use. When a word crosses one of the axes, its location shows where it falls on the spectrum: more or less male or female, for instance.

These dimensions help the machine represent some of the subtleties of language usage: in a sense, it's a way of transforming meaning into math. Certain directions in a word embedding map may reflect contrasts such as female vs. male, or good vs. bad. Understanding how these implicit dimensions form is currently a subject of great interest, both as a scientific question and as a type of transparency, helping us peer inside the black box of this type of AI. But we can also view the embedding a model of how humans have collectively used words, giving us a way to measure connotations quantitatively. In the end, this piece is not so much a portrait of a machine but a picture of how humans speak.

To this end, any biases that are found in the data will be reflected in the model as well. For this reason, it's paramount to ensure that these models are not being used in ways that enforce existing biases, and to continue developing ways to de-bias these models.

Exploring the Meaning of Human Word Usage

"We were motivated by the biases that exist within languages, and are culturally propagated through the written and spoken word. Our prior work in word embeddings had given us a way to quantify and visualize biases learned by models through word embeddings, and we wanted to find a way to share this with the world - invite people to discover, explore, play and reflect on them."

"We picked axes that were predominantly opposites across cultures - male and female; good and bad; cheap and expensive, to name a few. Surprisingly, we found that male and female were not consistently opposite across cultures."

"We started with some quick explorations to see how best to visualize specific words, answering simple questions like 'is the color 'red' more male or female?', and 'is 'potato' cheap or expensive'. Inversely, we also explored how many associated words behaved across our axes.

We found several commonly known biases - nurses was almost always more female than male; some unexpected ones - zombie and robot are more female but monster and demon are more male; and even some funny ones - why are 'bagels' weak?"

"What started to quickly catch our attention is how the same word was biased along different axes, and in relation to the words around it. We explored various animation styles to show how the words flowed through the axes along their own currents of meaning."

"To execute this idea, we precomputed thousands of word embeddings by sending words through a pretrained neural net and extracting the activations. We then visualized these embeddings and projected them on a large area (3m by 6m) to give the viewer the feeling of being immersed in these words and their meanings."

The team behind 'Waterfall of Meaning':

Nikhil Thorat, Martin Wattenberg, Lauren Hannah-Murphy, Emily Reif, Tolga Bolukbasi, Mahima Pushkarna, Fernanda Viegas.