The peanut butter jelly sandwich instructions for an AI

How do you prepare a sandwich spread with peanut butter and strawberry jam? Quite simply, you take two slices of bread, smear peanut butter and jam on it and you’re done? Not quite, if you follow these instructions exactly.

That’s exactly what Josh Darnit is doing with daughter Johnna and son Evan after he asks them to write him instructions on how to make a peanut butter and jelly sandwich. And he drives them to despair with it. Several times the children have to grab the instructions written on a piece of paper and rewrite them, because to their shock they have to watch again and again how their dad reinterprets the instructions again and again in unexpected ways.

Here is the entertaining video:

Evan, in one of his desperate moments, makes a statement that gives us a clue to the problem. “You’re not making any sense! You know how to make a sandwich!” he throws at his father, close to tears.

“You know it!” Yes, but what exactly does Dad know?

This is context and common sense, which, as it turns out, requires quite a bit of prior knowledge. It starts with the peanut butter jar, which, before you can take peanut butter out with a knife, has to be opened. The lid has to be unscrewed. Where to put the two spreads on the bread leaves a few options open. And Dad clearly chooses the one the kids didn’t mean.

Especially funny is the moment when their father sticks the butter knife with the handle into the peanut butter. The children’s faces show the shock.

This kind of lack of context and common sense guidance that Papa Josh so deliberately and gleefully lays out is exactly what the first artificial intelligence systems and, in modified form, today’s machine learning-based AI systems suffer from. The first systems were deterministic expert systems. They were given instructions that were very specific about what to do under what circumstances. If X has a value A1, then do Y, if X has a value A2, then do Z. But if X has a value A3, then there were no instructions and the system failed at that point. Even catch-all routines for these possibilities, such as “For all other values of X, do M” did not solve the problem. What if the result is more complex and a decision must be made from different equations at the same time? And what if Y, Z or M cannot be applied because they need other information first?

These decision trees can assume a very large and unmanageable scope. At the same time, however, they are always fragile. One unmapped variant, and the system disintegrates. No matter how much work is put in, system fragility remains and increases in systems that are not stable and change.

By the way, I write about this and many other topics around AI in my (German) book on AI: When Monkeys Learn from Monkeys. And also in a humorous way.

Wenn Affen von Affen lernen

Wenn Affen von Affen lernen

What is intelligence in the artificial and human sense?

This book can be purchased from the publisher, Amazon, or bookstores.

We see very quickly how deterministic system programming soon reaches its limits. And rapidly meant several decades for the AI experts who developed such deterministic systems. From the beginnings of AI with Turing in the 1930s/40s and Marvin Minsky and John McCarthy in 1959, they continued to hone it until the 1990s. But then, thanks to the Internet and faster processors, statistical methods took over…. Systems were left to learn on their own and we have reached a point where, thanks to big data, we let systems teach themselves, with the support of AI experts, how to interpret the data and make decisions.

However, this only solves one set of challenges that deterministic systems suffered from. Context and causality remain unsolved challenges that modern AI systems understand. A chess computer may play chess better than humans, but it does not know that it is a game. It doesn’t even know what can be meant by a board. Or what else a king, a pawn, or a knight on a horse mean. The chess computer also doesn’t celebrate its victories, it doesn’t know what joy is, or what a particularly beautiful move had been.

Josh makes this clear to his children in a humorous way. The fact that a knife has a handle and a blade is not obvious without prior knowledge. Sure, you can spread peanut butter with the blade, but you’re more likely to hurt yourself on the sharp blade. The computer doesn’t know that a cut to our fingers hurts and in the worst case can lead to our death and is therefore to be avoided at all costs for us humans. Why we don’t eat the plastic of the peanut butter can is not only a mystery to the computer, it doesn’t even come up with the question.

All of this shows us how far we still are from superintelligence, and that even young people like Johnna and Evan with their biological computational units – also known as brains – have already understood how information from different areas of knowledge must be combined, at which point which questions must be asked and the answers applied in a practical implementation. And how they sometimes have to start everything all over again because their father is being so stupid.

Leave a Reply