2000 Column Index

From Data To Wisdom

January 03, 2000

By Bill Nicholls

Stepping Up to Information

We all understand what data is/are. The singular form refers to an android on Star Trek: The Next Generation. The plural form refers to the stuff we put into computers in the hope of getting something useful out. Most companies are hard a work turning data into information. Few even believe there is a next step, yet in my view there are two more: knowledge and wisdom. How do we get there from here?

Most every one instinctively knows the difference between data and information. Here is a simple example of what I mean:

Each line alone is simply a collection of nine data points. Taken alone, one is simply a clock record, the other a temperature record. Multiple data points like those records contain information called a trend. Taken together, more can be inferred -- that is new information can be deduced. Information, as opposed to data, comes from the juxtaposition of related data points. What is the information we can deduce? The temperature change in the morning was increasing, leveled off at noon, and decreased rapidly in the afternoon. Without using other facts, that is all that can be directly deduced

Using this second table we can see that the drop in temperature was accompanied by an increase in humidity. Again, the table does not directly point to the cause of the temperature drop -- that requires some outside facts. Without additional information of the subject area, a computer could not make any further deduction from that information. That additional information, related to but not part of the specific table, is called a Knowledge Subject Area (KSA).

Those additional facts are well known to us because we have experienced what happens when clouds cover the sun, the temperature drops, and rain falls. As things stand now, there is no way a computer could predict rain. What's missing is the knowledge of a subject area. Without that KSA, such as a weather forecasting program, no computer can deduce that, yet a human would recognize it almost instantly. But a computer with a weather forecasting program would not be able to deduce anything except weather.

The Critical Jump To Knowledge

Knowledge is more than a bunch of data points, trends, or information. It is an accumulation of facts or information that has meaning beyond the facts themselves. In the human sense, knowledge is the understanding of consequences from a group of facts. These consequences can be experienced or deduced, but in either case are known to us. Computers, on the other hand, cannot 'know' from experience, only from databases or direct programming. There is work under way to bridge this gap.

One approach is Artificial Intelligence (AI) programs. AI programs can emulate human deduction in very limited areas that are well understood by human practitioners. The classic AI example is the DEC program for medical analysis. Within its limited boundaries, it can diagnose as well or better than the best human experts. The research effort for the medical program was substantial over a 10- year period. A wide variety of current research on AI can be found at MIT's AI Lab

Another area where computers are beginning to simulate knowledge in a very limited way is the field of Agents. Agents are semi-autonomous programs which interact with a variety of data or information sources to yield a selection based on criteria which are usually entered by a human. A simple example of this is the pricing agent that is designed to find the best price for an item by searching Web stores and comparing prices, and possibly other information. This simulates, at the simplest level, the knowledge of a purchasing agent who shops suppliers for the best price.

The computer agent does not usually evaluate things like delivery, product quality, and customer service, but with the addition of XML and business to business links, this could be done. Rather than replacing purchasing agents, these computer agents will become their tools.

Knowledge, as used here, means a body of facts and deductions that are capable of being used to make decisions or take actions without human intelligence in the loop. An example of where this is essential is in the programming for an autonomous Mars Rover. The speed of light delay is so large that by the time a controller on earth saw a cliff, the rover would have gone over it 10 or more minutes before. This autonomous program is very difficult because we know so little of Mars at the level of detail needed. Except for Viking and the Mars Pathfinder landings, we have no knowledge of the surface texture. This is like planning a trip from New York to Los Angeles having only seen Central Park and the deep desert.

Computer Programs Don't Understand

One obvious and essential difference between people and computers is understanding gained from experience. Computers do not learn from experience. They can be programmed with facts, given routines to deduce other facts, and programmed to make decisions and take actions. But something as simple as a general block stacking program is still a difficult problem for computers, a task that a child does without special training.

This difference, while clear, is hard to accurately define. What is the basis for this difference? It is because humans have developed through experience an innate knowledge of how the world works. People understand the effects of weather and gravity as well as the strength of materials without needing to know the science behind the facts. Glass breaks, things fall when unsupported, rain makes things slippery, all known from experience.

Knowledge projects go beyond simple agent examples. One current project is described and demonstrated as the "IBM Knowledge Utility". You can view a conceptual overview with examples here. These knowledge projects and AI efforts still only attack the edges of the problem of programming a computer to 'know' some area of reality. Individual specific problems are being attacked, but the broader problem of a general learning program is still out of reach.

Wisdom Is Different Than Knowledge

Wisdom is making the best choices under limited circumstances taking both short and long term effects into consideration. This implies a judgement ability to select among alternative courses of action. This choice will be made on a set of criteria that may include fuzzy, qualitative and quantitative elements, resulting in a list of choices by preference. People do this evaluation, often without conscious effort, every day. Many times the choices they make might not be seen as 'wisdom.'

One of the great difficulties for computers is that the criteria may be incommensurable, that is they have no simple value comparisons. Such criteria as value and color are examples of this. When this happens to a person, they fall back automatically to higher-level considerations. If there are two good choices, one with best color, another with best value, human reasoning then considers a higher level to determine which should take precedence. Higher-level considerations could be budget impact, possible color matches with other items, personal preferences and alternative choices from a different source.

The hardest part of the hierarchy of evaluations is the weighing of intangibles. How much is your time worth now? In the future? Does this choice impact other choices favorably or unfavorably? Such dilemmas are enshrined in sayings like "I have champagne tastes and a beer pocketbook." This is a difficult problem for people and to my knowledge, no one has yet attempted it with a computer. But is it impossible?

The fictional HAL 9000 in Stanley Kubrick's 2001: A Space Odyssey showed judgement but not wisdom when he killed off the astronauts. Current AI products show judgement and wisdom only in so far as they reflect the judgement and wisdom of their creators. Just as AI has slowly evolved from limited experiments into useful tools for specialists, it might be possible for knowledge systems (KS) to gradually expand to include broader areas of knowledge.

The big hurdles that appear include the handling of intangibles and the broader area of evaluating long term effects. While we don't understand these issues in enough detail to turn into programs today, individually they are not intrinsically impossible. A KS that could show wisdom even in a limited area has two known obstacles that could prevent its achievement.

The first is the issue of complexity. As the areas covered expand, the number of options to be considered grows exponentially and each option takes longer to evaluate. This appears to be what theorists call an 'NP Complete' problem, one in which complexity overwhelms whatever computer power you have available.

Our second obstacle is software technology. A Knowledge System, which covered the areas needed, is beyond our current ability to construct the software and databases. Regrettably, software technology has lagged far behind hardware. In more than 35 years of programming, the tools I use have barely advanced the equivalent of one hardware generation. We still write code by hand, one line at a time, indeed one character at a time. Worse, adding people slows down a programming project beyond an optimum small number.

In short, we don't have the tools to build the tools that could make a KS capable of wisdom. This is also the subject of a saying: "The cobblers' children have no shoes." Programming is still a labor intensive, error-prone process. Until we make some big strides in this area, the realm of wisdom will remain theoretical for computers.

All content on this site is Copyright 2001 by Bill Nicholls