Limits of Analytics

If you read many of my posts, you will discover that I am an analytic person. That is, I tend to look at situations from the outside and try to figure out what is actually going on through the use of observation and reason.

Despite (or perhaps because of) this natural tendency, I am well aware of the limits of this sort of analysis. There is a strong strain in my culture that believes rational analysis is the primary basis of human knowledge and that this sort of knowledge should underlie action. I think this is a pernicious view that cheapens us as humans. Our ever increasing scientific knowledge base can protect us from some of our cruel and irrational tendencies. For example, the existence of the well respected Centers for Disease Control in the United States will keep us (unlike the Egyptians) from slaughtering all our pigs because we are threatened with the "swine flu". In general though, analytic knowledge is too incomplete, too prone to error, and too slow to serve us in most situations. We have other, more effective tools to make our way through the world.

Violin construction is an excellent example of how humans actually learn and change. the golden age of violin construction occurred three or four centuries ago. To this day we do not understand why the Cremona violin construction produced what we feel to be superior instruments. In the intervening centuries we have developed mathematics and simulations that have improved our understanding of vibration and acoustics. We have used this and our knowledge of consistent manufacturing to create increasingly better musical instruments in increasingly large numbers. As with so many other things, we live in a golden age. It has never been easier or cheaper to own a fine musical instrument. That is, our analytic knowledge has raised the bar, but it has not allowed us to attain the level of mastery achieved by people without these formal analytic tools.

So how do we actually learn and progress in areas that cannot be understood by current analytic techniques. In short, trial and error. Humans rarely invent, but we have a marvelous capacity to vary what we do and to detect the effects of the variations. Most of what we call invention is really applying existing knowledge to a new area or amassing small sets of variations to achieve a shifting goal. Violin makers have rules accumulated through experimentation. These rules govern the choice of materials, their manufacture (strike here with your fingernail and if it sounds "dead" thin the material over in this area). These rules do not come from any kind of mathematical analysis and only rarely from a rigorous program of experimentation. Instead they come from the accretion of experience over generations. This experience is based on a deepening, visceral understanding of the desired end and the practical means to achieve it.

A generation ago there was much work on "expert systems" that could replace humans in various areas. The general notion was: figure out the rules people use to solve problems, encode those rules, and replace the human with a computational engine operating over the rules. The first step was to find a competent expert and extract the rules. Usually this failed. Most experts could not express the rules and those who could were often wrong. Observation of actual problem solving shows that it is a wonderfully complex and non-linear activity. People try one approach, and abandon it midway. They put down the problem altogether and work on unrelated things. They try different approaches in seemingly random order. Some approaches seem to not make much sense. They pick up abandoned work and add to it. After the problem is solved, they explain (and remember) what they did as a linear sequence that led to the solution, but that does not fit the actual activity.

There are limits to most of our analytic techniques. Many practical problems are frightfully complex and often chaotic in the mathematical sense. I have little doubt that we will continue to increase our analytic tools, but the end result may provide only statistical rather than direct guidance. For these situations we can make informed guesses, but having a ninety percent chance of success doesn't help much if you find yourself on the ten percent side.

One reasonable response to complex situations is to make them simpler. This underlies much of modern manufacturing and business. If materials used in manufacturing are not uniform, manufacturing techniques must adjust to the differences. A good solution is to improve the raw materials to eliminate the variability. On the human side, we invent personnel rules that constrain how employees are treated. This simplifies management and gives people a sense of fairness and boundaries. At the same time it depersonalizes and turns the most unique of animals into an industrial part. I was once on a conference call where someone started referring to "human capital". I told him that if he called me that I would hit him.

As biological animals we have evolved to be incredibly adept at a rapid, subconscious, and often amazingly accurate analysis. The trick is to harness this biologic capacity while using our analytic knowledge to inhibit our more destructive automatic tendencies.

Over time, analytic knowledge tends to increase faster, be more reliable, and be transmitted more easily than experiential knowledge. Serious inquiry requires rigorous analysis and careful, controlled experimentation. But in novel situations (almost every situation we encounter) and in areas where we are expanding our knowledge, purely rational techniques do not serve us well. There is some special non-analytic human capability that allows us to succeed.