Attention, Love And Diversity Is All We Need For AI

The story behind one of the most influential AI publications of recent years is one of the lead stories in the current issue of Wired magazine. In 2017, eight AI researchers published a paper on Google entitled Attention Is All You Need, alluding to the Beatles’ song All You Need Is Love.

The fateful eight researchers Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser and Illia Polosukhin described in their paper how so-called transformers with parallel calculations in neural networks instead of sequentially, as was previously the case with LSTM, and were thus able to process longer text passages more quickly, understand more context and maintain a greater overview. The responses and images generated surpassed the quality of previous transformerless approaches.

At the time, the people involved, most of whom have since left Google and founded their own companies, had only a limited idea of the impact their paper would have and how it would trigger an AI revolution. Because the T in ChatGPT‘s GPT stands for Transformer.

What is interesting for this article, however, is not so much how generative AIs are now on everyone’s lips and how they are influencing all areas of our lives, but rather the environment that made good ideas and innovation possible.

Not only were the authors all employed by Google, they also worked in the same offices. Encounters in the corridor and overheard conversations over lunch led to great moments. We also call this Return on Collision (ROC) where workplaces allow employees to encounter each other frequently and by chance. Be it at lunch, in the coffee corners, in the corridors or – yes – in the toilets. The star-shaped buildings at SAP with all the coffee machines and restrooms in the center or the Pixar building in Emeryville facilitate this ROC.

The eight Google researchers are also culturally diverse. Six of the eight authors were born outside the United States; two others are the children of two Germans with green cards who were temporarily residing in California and of a first-generation American whose family had fled persecution in his home country.

With these different cultural backgrounds and life experiences, this group brought a wide range of knowledge and approaches to the tricky problem, enabling them to find an innovative and influential solution. Of course, improvements could have been made. There was only one woman among the eight researchers. But not every dimension of diversity always has to be ticked in order to generate more ideas.

It paid off for us and for the “fateful eight”. We got our hands on a technology that can change all aspects of life for the better, and the researchers all started their own companies with the exception of one who now works at OpenAI, the beacon company of AI transformers. Now it is up to us to promote such a culture of innovation in our organizations.

Leave a Reply