AI and Scientific Creativity

AI AND SCIENTIFIC CREATIVITY
Sakir Kocabas

Abstract : This article examines the elements of scientific creativity through a series of basic cognitive and computational concepts. Scientific creativity requires motivation, an access to a body of systematic knowledge, an ability to correctly formulate research problems and to define a comprehensive problem space. It also requires an ability to reduce the corresponding search space by using methodological knowledge, and rigour to conduct search in the constrained search space. The article discusses the types and the role of knowledge involved in scientific research, types of scientific creativity, and the dimensions of scientific research.

Introduction
Scientific discovery and creativity has recently become one of the special concerns of artificial intelligence. Within the last five years, a number of research papers and two important books have appeared on scientific discovery (see, Langley, Simon, Bradshaw, & Zytkow, 1987; Shrager & Langley, 1990). Closely related with the subject, two other books have appeared: one on the computational philosophy of science (Thagard, 1988), and another one on creativity (Boden, 1990).

Langley et al.'s (1987) work posed the first serious challenge to the conventional study of science by proposing that, far from being mysterious and unexplainable, scientific discovery (and by implication scientific creativity), can be explained in a series of processes. Their work also described several computational models in support of the authors' view. Shrager and Langley's (1990) later study introduced new methods for the study of scientific development, and explained how the methods of the computational study of science were superior to those of conventional philosophy of science. Boden's (1990) work on the other hand, extended some of these views and discussed, from a cognitive scientist's perspective, how creativity in arts and literature, as well as in science could be studied within a computational context in a more systematic way.

However, previous work leaves some of the important issues in discovery untouched, such as the elements of scientific creativity, the types of scientific discovery and creativity, and the dimensions of scientific research. In this article, we examine the basic cognitive concepts of creativity, and describe how these concepts are connected, and then discuss the role of background knowledge and the kinds of knowledge necessary for scientific research. Finally, we discuss the types of scientific discovery and the elements of scientific research.

Creativity in Science
Creativity and intelligence are closely linked concepts, so much so that the existence of one is the measure of the other. Therefore, any attempt that brings clarity to one concept will be helpful to define the other. Lenat and Feigenbaum (1987) define intelligence in terms of 'search", as the power to find a solution to a problem in an large search space. Later, Feigenbaum defined intelligence in terms of 'knowledge assembly" rather than "search" (see, Engelmore & Morgan, 1988, vii). According to this new definition, an intelligent system has the ability to assemble the neccessary body of knowledge to conduct a complex task.

A distinction can be made between scientific creativity and other types of creativity such as artistic, architectural, musical and literary creativity. The former may involve the discovery of a new substance, the invention of a new mechanism or method, or the construction of a new model of reality (a hypothesis or a theory). The latter usually manifests itself as a work of art or a new style, and the term 'creativity" is usually associated with this type. Therefore, when we talk about scientific creativity, this is to be understood within this perspective.
Even in historical terms, scientific creativity is distinguished from other forms of creativity such as in arts, music and literature, in its extensive reliance on background knowledge and experience. This may explain why we do not see child prodigies in creative science as we see in music and arts.

Scientific creativity can be investigated through five basic cognitive and computational concepts. These are:
1) Motivation for scientific research.2) Ability to correctly formulate research problems within a body of knowledge.3) Ability to create a comprehensive search space for the solution of a scientific problem.4) Ability to assemble (or induce) and implement a set of heuristics to reduce the search space.5) Patience and stamina for the exhaustive search for solving the scientific problem within the constrained search space.

Fig. 1 summarizes the links between these concepts. Any missing link between them, can hinder scientific creativity.
Motivation for Formulate Generate Reduce Conduct
Scientific --> Research --> Search --> Search --> Exhaustive
Research Problems Space Space Search
Fig. 1. Problem formulation and search in scientific discovery

As indicated in the above list, research motivation tops the requirements for scientific creativity. There seems to be a significant relationship between concept structures (or ontology) and scientific motivation. The three millennia of European history of thought indicates this, in its ups and downs in scientific activity. The social aspects of science has been extensively studied both by sociologists and some philosophers of science. Various types of human motivation have also been studied by psychologists in the last five decades, but there seems to be little work on the relationships between ontology and scientific motivation.

In modern scientific research, an access to a large and systematic body of knowledge is necessary for correctly formulating scientific problems, both in creating a comprehensive search space, and in reducing the search space to reach for a solution within acceptable limits of time and resources. The correct formulation of research problems requires a mastery of the conceptual structure of the field of science involved. The creative scientist can change this structure for reformulating a research problem, and in some cases, these changes may involve the most fundamental concepts and principles, such as time and measurability in physics.
Scientific creativity exhibits itself during the completion of a series of research tasks. Different types of knowledge is used for such tasks, as will be explained next.

The Role of Background Knowledge in Scientific Creativivity
Modern scientific research is one of the most complex human activities, requiring the use of different types of general and specific knowledge. Knowledge necessary for modern scientific research can be divided into four types asa) Commonsense Knowledge,b) Technical Knowledge,c) Theoretical Knowledge, andd) Methodological Knowledge.

Commonsense knowledge is simple, general and relatively unstructured knowledge about the world. Statements such as 'Water extinguishes fire,' 'Fire burns paper' are examples of commonsense knowledge. Technical knowledge can be defined as the knowledge about instruments, methods and processes. Knowledge about how to repair a TV set, how to control a chemical reactor, and how to fly an aeroplane can be considered as technical knowledge. Theoretical background is helpful, but not always essential, in acquiring this kind of knowledge. Technical knowledge can be descriptive as well as prescriptive.

Theoretical knowledge is structured, descriptive knowledge about the world, embodying classifications and numerous interrelated hypotheses. Typical examples of theoretical knowledge are the classical mechanics and electro-magnetism.

Methodological knowledge, on the other hand, is exclusively prescriptive; it can be represented as condition-action rules. Methodological knowledge includes knowledge about how to distinguish between scientifically interesting and uninteresting phenomena, how to choose between alternative goals, strategies and methods during scientific research, how to design experiments, how to propose new hypotheses, and how to generalize, test and evaluate them. It is mostly the extent of this type of knowledge that makes the difference between a research scientist and a nonscientist.

Unlike the inference rules in theoretical knowledge, many of the methodological rules rely on extralogical methods such as inductive generalizations, abduction, abstraction and analogy. Such rules are frequently used in formulating problem states, in constraining large search spaces, and in hypothesis formation during the activity of scientific research.

Theoretical knowledge helps in formulating constraints for domain-dependent reduction of search space in scientific research. The story of Edison's discovery of a durable ligth bulb filament illustrates the relationships between theoretical knowledge, motivation and search. In the background of an insufficient theoretical knowledge available at the time about materials, Edison is said to have tried thousands of filaments made from different elements, metals and alloys. In this way, he compensated the deficiency of theoretical knowledge with his endless motivation and meticulous search. Theoretical and empirical knowledge played a much more important role in the discovery of the high-temperature oxide superconductors in 1986 and 1987.

In some cases, one discovery facilitates other discoveries. This has been the case in the discovery of certain quantum properties in particle physics, and in the discovery of new oxide superconductors in high temperature superconductivity research. In particle physics, the discovery of lepton, spin, and strangeness properties, after the discovery of baryon number, would require less cognitive effort in terms of abstraction and abductions applied in the process. This is because, by the discovery of the baryon property, the abstraction from electrical charge to a class of such quantum properties, and the abduction from observed and unobservable particle reactions, had already been successfully made (see, Kocabas 1991).

Similarly, in oxide superconductivity, after the discovery of La-Ba-Cu-O superconductor by Bednorz and Mueller in 1986, physicists extended the application of the ideas and methods that were developed, and discovered other oxides with higher transition temperatures.
Some discoveries rely more heavily on analogical reasoning than abstraction and abduction. For example, the so far unsuccessful attempts by physicists on 'cold-fusion' relies on an analogy between extreme pressures obtainable in a plasma, and that within the crystal structure of a metal electrode. Another interesting analogy for research in this field could be a 'nuclear catalyst' induced from a well known concept 'chemical catalyst'. In chemical kinetics, chemical catalysts can initiate certain chemical reactions otherwise unrealizable under the same temperature and pressure conditions, due to constraints explained by activation energy levels. Could one find a 'nuclear catalyst' to similar effects for nuclear fusion?

Types of Scientific Discovery and Creativity
Scientific creativity can be examined in relation to the scope of the research in which a discovery takes place. Kocabas (1992c) introduces a classification of scientific discovery as follows: 1) Logico-Mathematical Discovery, 2) Formal Discovery, 3) Theoretical Discovery, and 4) Empirical Discovery. This classification is based on the categorization of descriptive knowledge by Kocabas(1992a), and reflects the types of knowledge used in scientific research, and the type of knowledge discovered. All these four types of discovery have been studied by computational models in AI.

According to this classification, logico-mathematical discovery takes place, as the name suggests, in the abstract domain of logic and mathematics. Some of the earliest AI systems such as Logic Theorist were logico-mathematical discovery models designed to prove theorems in logic. Among the more recent computational models, AM (Lenat, 1979) appears as a successful example for mathematical discovery. The distinguishing characteristic of logico-mathematical discovery is that, in principle, it does not require experimentation or observation. Nor does it need the knowledge of a physical domain par se, except for analogical transference in some cases.

Formal discovery takes place in a formal domain involving abstract entities, their classes and properties. Formal discovery requires logico-mathematical knowledge as background knowledge, for deductive inference on formal knowledge. Lenat's (1983) EURISKO, in its applications to Naval Fleet Design, Evolution, and 3-D circuit design, is a good example to formal discovery systems.

Theoretical discovery requires logico-mathematical, formal and theoretical knowledge, and in general, results from theoretical analysis and synthesis. Some computational models of theoretical discovery are systems such as PI (Thagard & Holyoak, 1985), ECHO (Thagard & Nowak, 1990), and GALILEO (Zytkow, 1990). The first two could better be characterized as concept discovery systems, and as such, are closer to formal discovery models. GALILEO on the other hand, is an interesting example of discovery by theoretical analysis. In the history of science there are rather important theoretical discoveries or inventions such as Maxwell's equations and the Einstein-Lorenz transformations.

Empirical discovery is an extensively studied area, and a number of computational models have been designed to investigate its various aspects. Empirical discovery requires experimental and observational data, as well as logico-mathematical and formal knowledge. Theoretical knowledge has not been a prerequisite in the early empirical discoveries in the history of science (e.g. in the 17th and 18th century chemistry), but in modern empirical research such as in oxide superconductivity and 'cold fusion' experiments, extensive theoretical domain knowledge is necessary.

Empirical discovery systems can be divided into two main classes as qualitative and quantitative models, although this distinction is sometimes irrelevant. Among the qualitative discovery systems, GLAUBER (Langley, et al., 1987), STAHL (Zytkow & Simon, 1986), STAHLp (Rose & Langley, 1986), BR-3 (Kocabas, 1991a), KEKADA (Kulkarni & Simon, 1988), AbE (O'Rorke, Morris & Schulenburg, 1990), and COAST (Rajamoney, 1990) can be cited. Some of rediscoveries of these systems can be identified as formal discovery, such as GLAUBER's classification of substances as 'acid', 'alkali' and 'salt'.

Among the quantitative discovery models BACON (Langley, et al., 1987) FAHRENHEIT (Zytkow, 1987) and IDS (Nordhausen & Langley, 1987) can be cited as prominent examples. BACON was the first successful model of quantitative discovery, which also has attracted the interest of philosophers of science(*). The IDS system on the other hand, integrates qualitative and quantitative methods.
--------------------------------------------* See, e.g. the special issue (Vol 19, No 4) of Social Studies of Science.
Dimensions of Scientific Research
Research in the computational study of science has revealed a number of important aspects of science that were overlooked by the conventional philosophical study. Shrager and Langley (1990) describe the basic differences between the computational and the conventional philosophical approaches to science as follows: Conventional philosophical tradition focuses on the structure of scientific knowledge and emphasizes the evaluation of laws and theories, while the computational approach focuses on the processes of scientific thought, and emphasizes scientific discovery including the activities of data evaluation, theory formation and experimentation.

The distinction can be extended even further. Computational study of science concerns not only with the issues of hypothesis formation, testing and verification, which have been the main concern of conventional study of science, but also a series of other related issues. Kocabas (1992b) names more than a dozen different major tasks involved in scientific research. These are: Formulating research goals, selecting research goals, defining research framework, gathering knowledge, organising knowledge, selecting research strategies, methods, tools and techniques, proposing experiments, designing experiments and selecting experiment materials, setting expectations, conducting experiments, data collection, data evaluation, hypothesis formation, theory formation, theory revision, goal satisfaction control, and producing explanations.

Any of these research tasks may concern activities dealing with a variety of planning, classification and evaluation problems. To provide an idea about the diversity of the activities involved in these research tasks, we will give some of the results of our study on the research in oxide superconductivity (Kocabas, 1992b) in terms of five of the research tasks listed above. These are: Formulation of scientific research goals, choosing between formulated goals, proposing strategies, proposing experiments, and hypothesis formation.

Research goals can be divided into two general forms that may overlap: Those that aim at explaining a phenomenon, and those that aim to study a penomenon. Creative scientists seem to utilize several general rules for formulating their research goals: They focus their attention to problems and phenomena that have not been explained or unexplainable within the current scientific framework. However, such problems must have some general and important implications to be worthy of investigation. For example, why the moon has more craters in one particular area than others may not be regarded as an interesting problem. On the other hand, the research for understanding why some elementary particle reactions have never been observed would be important, because the results would interest not only quantum physics but also cosmology. Some scientific research problems may be strongly related to important technological needs. Energy conversion, storage, and transfer are still major technological problems that motivate scientific research into such areas as 'cold fusion', oxide superconductivity, and electrochemistry. However, interestingness in itself is not a sufficient criterion for a phenomenon to attract the attention of the creative scientist. The research goals that are formulated must be achievable.

It is not unusual that, in relation to a certain phenomenon, a scientist formulates alternative research goals. In such cases, the selection of a research goal among alternatives is another research task. Scientists use several selection criteria in deciding which problem to primarily focus on. Interestingness, importance, the materials and technological tools needed, economic constraints, and acheivability within a timescale are some of the metrics that affect the decision. As can be seen, some of these constraints conflict with one another, so that the scientist may have to do some classification before deciding which goal to select.

Selecting research strategies is another important task for achieving a research goal. Strategy selection depends on the type of the research goal. If the goal is to explain a certain phenomenon, gathering knowledge by detailed literature survey and theoretical analysis may take precedence. On the other hand, if the goal is to study a phenomenon, then experimentation and observation needs to be considered. If experimentation is selected, then the types of experiments needs to be decided. For example, if the research goal is to study the possibility of improving a certain important physical property (e.g., electrical conductivity) of a substance, there may be a number of alternative strategies. The following are only a few of the strategy heuristics extracted from the research reports on oxide superconductivity in 1987.

If the goal is to improve a property P and a process S improves P, then propose experiments applying S.If the goal is to improve a property P and another property Q is positively related with P, and a process S improves Q, then propose experiments to apply S.If the goal is to improve a property P, and another property Q has a negative effect E on P, then propose experiments to reduce or eliminate E.

Once the experimentation strategy is selected, the scientist has to decide about the relevant processes and techniques for the current strategy. S/he also has to decide about the experiment materials, and has to classify these materials against a set of parameters such as availability, likeliness to yield success, cost and relative hazards (e.g., radioactivity, flammability and corrosiveness), and select the best materials for the experiments.

Scientific experiments need to be designed and conducted according to certain procedures. Experimental parameters are defined, tests are made and parameter values are measured, and in this way relevant data is collected. The data is evaluated to make sure if they reflect any violation of the experimental conditions. After data evaluation, hypotheses are formed.
Hypothesis formation is one of the most important tasks of scientific research. Despite the fact that it has been a primary concern of the conventional philosophy of science for a long time, it still remains to be an aspect of scientific discovery that needs a detailed investigation. In our study on oxide superconductivity research, we have identified over 40 hypothesis formation heuristics that were utilized by scientists working in this field. Some of these heuristics are as follows:

If the value of a property P changes with the value of another property Q, then hypothesize that P and Q are related.If a process does not change a set of experimental parameters P1,...,Pm, but changes other such parameters Q1,...,Qn, then hypothesize that P1,...,Pm and Q1,...,Qn are independent.If a process is expected to enhance a property P of a substance M, but the expected increase does not take place, then hypothesize that there is another property Q hindering the effect.

The majority of these heuristics are general, while some are domain specific. Two examples are as follows:
If a change in the crystal structure S1 sharply diminishes a property P (e.g. electrical or thermal conductivity), and the change is also accompanied by the disappearence of some substructure S2, then hypothesize that S1 plays an important role for P.If two compounds M1 and M2 have have very similar bonding and electronic structure over a wide range of temperature, then hypothesize that M1 and M2 have very similar conduction properties within the same range.
These are only some examples of hypothesis formation rules used in a rather specialized domain of physical science. We will not discuss the methods and rules used in hypothesis verification and theory revision here for reasons of space. However, by considering the rules and methods used in various fields of science, from physical to human sciences, and over a dozen research tasks in each of them, we can realize the dimensions of scientific research.

The diversity of interrelated research tasks is by itself sufficient to show that, scientific discovery is not a logical procedure or a process in itself, but the product of a series of complex processes called scientific research. Scientific creativity may be required in any of the research activities in these processes. History of physics has many examples of this. Although an extreme example, consider the design, construction and the operation of the CERN particle accelerator, where research involves proposing and designing experiments, setting expectations, conducting experiments, data collection, data evaluation, hypothesis formation and verification, and theory revision.

Conventional philosophical approach ignores the multiplicity of the tasks and activities involved in scientific inquiry. We believe that, a much more detailed and careful examination and analysis of science is needed than that is envisaged by the conventional study of science. The computational approach provides both the necessary concepts and methods for such a study.

Conclusion
Scientific creativity needs to be investigated within its natural environment, namely within the processes of scientific research and discovery. Conventional philosophy of science, probably due to the limitations of its scope, has ignored a number of issues about science. Scientific creativity displays itself in the form of scientific discovery, which in turn, is the product of a series of complex tasks called scientific research. Therefore, a comprehensive study of science and scientific discovery requires a sufficiently rich set of concepts for a detailed and systematic investigation. Recent developments in the computational study of science provides some of these concepts. Based on these concepts, we have introduced a more detailed definition of scientific creativity, classified scientific discovery and creativity, and examined the the role of background knowledge in discovery within the wider dimensions of scientific research. A systematic investigation of scientific creativity cannot be conducted without considering the multiplicity of research tasks that have to be carried out by scientists during their activities.

References
Boden, M. 1990. The creative mind. Sphere Books, London.Engelmore, R. and Morgan, T. 1988. Blackboard systems. Addison Wesley.
Kocabas, S. 1991. Conflict resolution as discovery in particle physics. Machine Learning, Vol 6, No 3, 277-309.
Kocabas, S. 1992a. Functional categorization of knowledge. AAAI Spring Symposium Series, 25-27 March 1992, Stanford, CA.
Kocabas, S. 1992b. Elements of scientific research: Modeling discoveries in oxide superconductivity. Proceedings of the ML92 Workshop on Machine Discovery, 63-70.
Kocabas, S. 1992c. Evaluation of discovery systems. Proceedings of the ML92 Workshop on Machine Discovery, 168-171.
Kulkarni, D. and Simon, H. 1988. The processes of scientific discovery. Cognitive Science, 12, 139-175.
Langley, P., Simon, H., Bradshaw, G., and Zykow, J. 1987. Scientific discovery: Exploration of the creative processes. MIT Press.
Lenat, D.B. 1979. On automated scientific theory formation: A case study using the AM program. In Hayes, J., Michie., D., and Mikulich, D.I. eds., Machine Intelligence, 9, 251-283, Halstead, New York.
Lenat, D.B. 1983. EURISKO: A program that learns new heuristics and domain concepts. Artificial Intelligence 21, 61-98.
Lenat, D.B. and Feigenbaum, E. 1987. On the thresholds of knowledge. Proceedings of the Tenth International Joint Conference on Artificial Intelligence, 1173-1182.
Nordhausen, B. and Langley, P. 1987. Towards an integrated discovery system. Proceedings of the Tenth International Joint Conference on Artificial Intelligence, 198-200.
O'Rorke, P., Morris, S. and Schulenburg, D. 1990. Theory formation by abstraction. In Shrager, J., and Langley P. eds. Computational models of scientific discovery and theory formation. Morgan Kaufmann, San Mateo, CA.
Rajamoney, S.A. 1990. A computational approach to theory revision. In Shrager, J., and Langley P., eds., Computational models of scientific discovery and theory formation. Morgan Kaufmann, San Mateo, CA.
Rose, D. and Langley, P. 1986. Chemical discovery as belief revision. Machine Learning, 1, 423-452.
Shrager, J., and Langley, P. Eds. 1990. Computational approaches to scientific discovery. In Shrager, J., and Langley P., eds., Computational models of scientific discovery and theory formation. Morgan Kaufmann, San Mateo, CA.
Thagard, P. 1988. Computational philosophy of science. The MIT Press, Cambridge, MA.
Thagard, P. and Holyoak, K. 1985. Discovering the wave theory of sound: inductive inference in the context of problem solving. Proceedings of the Ninth International Joint Conference on Artificial Intelligence, 610-612.
Thagard, P. and Nowak, G. 1990. The conceptual structure of the geological revolution. In Shrager, J., and Langley P., eds., Computational models of scientific discovery and theory formation. Morgan Kaufmann, San Mateo, CA.
Zytkow, J.M. 1990. Deriving laws through analysis of processes and equations. In Shrager, J., and Langley P., eds., Computational models of scientific discovery and theory formation. Morgan Kaufmann, San Mateo, CA.
Zytkow, J.M. and Simon, H. 1986. A theory of historical discovery: The construction of componential models. Machine Learning, 1, 107-137.