The perpetual amateur

Kids and Adults - A Contrast

From the balcony of my house, I can see the kids' playground. It is fun observing kids play for hours, seemingly achieving nothing, trying the same thing over and over and still having so much fun. It is instructive to contrast that behavior with the group of adults. Just imagine an office meeting. You will see, in front of your mental screen, a bunch of bored adults wishing they were someplace else. All these adults, mind you, are experts in their own fields. They have spent years mastering their domain, and would be Mr. Miyagis from Karate Kid, an image that comes to mind when you think of an expert. Why does this happen? Why a kid who does not know anything about a new game can enjoy more than an adult who is an expert? Or is it the expertise that is a problem? How could we retain the sense of fun and adventure as we grow up?

What is expertise?

Herbert Simon, the polymath, spent his life understanding the decision making of the experts. He postulates that experts organize their knowledge in chunks of information to reduce the load on the working memory. In this perceptive article, he says:
"In chess, for example, a grand master must be able to recognize and recall roughly 50,000 chunks, within a factor of two. It is quite reasonable to assume that experts in all walks of life require 50,000 chunks. Or more!"

Chunking is essentially the filtering process where the expert ignores certain information based on his or her past experience of the domain. Chunking allows the expert to draw on intuition or hunch as a means of decision making. When someone visualizes an expert, they think of a person solving a problem instantly.
It is essential for an expert though, to understand the nature of the domain one is working on. Domains like chess are relatively stable, meaning the rules of the game do not change. Chunking can work fabulously well in these domains. But what about real life where one does not have the luxury of a stable domain?

LTCM and other Spectacular Failures

A great example of expertise gone wrong in a volatile domain is Long Term Capital Management, a hedge fund that boasted about the number of Nobel Laureates it employed. It seemed like a good story based on the high returns it delivered, till they encountered a novel situation. The novel situation was two financial crises in the late 1990s. It drove them to become bankrupt.
What is it that drives experts to such a fate? Does it have any similarities to the comparison we drew in the beginning of the post?

Success, confidence and comfort

I think it does. The outcome of a successful decision is the resultant confidence it gives to the decision maker. The emotional content accompanied by a good decision is a feeling of having arrived, a sense of complacency and a sense of confidence. It is precisely this confidence that makes adults insensitive to the feedback they receive from the world. In stable domains, like chess, the feedback from the world does not matter. It is distraction to say the least. It is prudent to ignore the distractions in such domains. But in case of volatile domains that continue to change, the feedback is the life blood of the success of the decision.

Some researchers, like Kahneman, postulate that expertise in uncertain domains is just not possible. In an article that talks about this topic, Kahneman says:
“It’s very difficult to imagine from the psychological analysis of what expertise is that you can develop true expertise in, say, predicting the stock market,” he said. “You cannot because the world isn’t sufficiently regular for people to learn rules.”

If Kahneman is right, we are faced with a unique problem related to expertise. It is possible to become expert in toy domains, like chess, that may not have too much relevance to the real world and it is not possible to become experts in real life domains, like business, that have relevance to the world.

The case for a perpetual amateur

Let us now go back to the situation we portrayed in the first paragraph of the post. In case of the kids, the domain may be regular, but the expertise was lacking and hence the kids looked amateur. What is important though, is the emotional content of the kids' mind. The joy, thrill, the perseverance and the seemingly unbounded energy. These are the qualities of the amateur, also known as a hobbyist. A hobbyist is someone who keeps the fun alive and does something for the sake of it and not for some external goal. Alen Perlis talked about this spirit in his introduction to Structure and Interpretation of Computer Programs, a seminal introductory text about programming.
“I think that it's extraordinarily important that we in computer science keep fun in computing. When it started out, it was an awful lot of fun. Of course, the paying customers got shafted every now and then, and after a while we began to take their complaints seriously. We began to feel as if we really were responsible for the successful, error-free perfect use of these machines. I don't think we are. I think we're responsible for stretching them, setting them off in new directions, and keeping fun in the house. I hope the field of computer science never loses its sense of fun. Above all, I hope we don't become missionaries. Don't feel as if you're Bible salesmen. The world has too many of those already. What you know about computing other people will learn. Don't feel as if the key to successful computing is only in your hands. What's in your hands, I think and hope, is intelligence: the ability to see the machine as more than when you were first led up to it, that you can make it more.”
As we grow old and become experts in some domain, it is extremely important to understand the shortcomings of expertise and the dangers of thinking of oneself as an expert. Not only from a perspective of making some mistakes that would cost you financially(like the LTCM case), but also from the perspective of getting into a life of boredom, a life devoid of fun, a life that avoids failures, experimentation and novelty.
An alternative to this life would be to lead a life of a perpetual amateur, seeking expertise when it makes sense, being aware of the pros and cons of both the approaches and using them when they make sense.