The Infant Designer Enterprise:
Learning design semantics from "good designs"

Team Members: Madan M. Dabbeeru and Amitabha Mukerjee

What is a symbol?

h-symbol: human symbols attach a meaning to each label f-symbol: formal computer symbols are just the labels

If you ask a designer what a "symbol" is, the answer will usually be that it is a sign that stands for something else. In design discourse, a symbol like gas flow triggers a rich set of associations, which constitutes its semantics. On the other hand, the symbol gas-flow used in computer programs (based in theory on formal logic) is a placeholder for syntactic operations and doesn't "stand for" anything else - any meaning they have is encoded in the relations it has with other symbols. We may consider this analogous to a blind man's knowledge of the colour "red", which may be quite rich - e.g. he may know that red is different from green or that crimson is a kind of red, but his knowledge is limited to such relations, and is not "grounded" in direct perceptual and other experience.

A second difficulty with formal symbols are that they are ultimately defined within the universe of symbols, which means that their definitions are in the end, circular.

At the same time, formalisms that deal with perception and other "modes" of cognition, often do not result in symbols. Although there is some who claim a place for "perceptual symbols" (e.g. Barsalou 99), purely perceptual symbols are hard to fix, they may continue to drift based on changing patterns of experience.

However, it is now becoming possible to use machine learning approaches to model the process of symbol formation as it happens in humans. Eventually, by deploying such symbols in a social process (e.g. design communication), one may anchor the meaning given to a "public" symbol, at least within a social group. The semantics of the symbol in each user may be somewhat different based on their own experience, but they would still have a large degree of overlap with each other. A major advantage of such grounded symbol systems is that it can make inferences that are not explicitly programmed - e.g. that gas-flow may be similar in some aspects to liquid-flow. More critically, the so-called brittle failures experienced in formal symbol ontologies is unlikely to occur if symbols are associated with rich sensorimotor linkages.

Infant Designer Enterprise: Starting small

In the "infant designer enterprise", our objective is to discover symbolic knowledge about the design domain via explorations in the design space. Initially, the infant designer is exposed only to problems that are well-understood. Thus problems it considers are those for which the set of design variables, as well as the measures of performance are given. Based on this input, it explores the space of "good designs", or the Functionally Feasible Regions (FFRs). In many situations, these spaces occupy only a small fraction of the design space (otherwise the design problem would be too simple). The infant designer seeks to find compact encodings for the FFRs, so that it can represent the design using fewer bits of information. Computational analogs of this process involve trying to reduce the dimensionality of the space in which the FFRs lie. The result is that the system builds up considerable amounts of domain knowledge, though at this stage this knowledge may be implicit (sub-symbolic).

We demonstrate this process with the infant designer address some extremely simple problems, such as learning the the notion of containment - rules like "a cavity can hold only objects that are smaller", and show how this can lead to the concepts such as clearance and tight fit. These grounded models constitute an abstraction in the design space, which may be thought of as an image schema or the semantic pole of an incipient symbol. We believe that in order to eventually build systems that can generalize and learn about more complex design situations, we must first build such an elementary symbol system, and then expose it to problems of increasingly complexity in order to build a symbol systems that are ultimately grounded in the design experience.

Learning containment. When can a latch of width t (y-axis) fit a slot of width w(x-axis)? The patterns of successful insertions (+) versus failures (◊), specifies the Functionally Feasible Region or FFR (blue area). With little experience (10,50 instances in a,b), the pattern learned is poor, but the w>t is quite well learned by (c) 200 instances.

This example shows a form of learning some basic properties of the domain. While this type of knowledge is fundamental in grounding symbols, the notion of "semantics" needs to be clarified further before we proceed.

What is semantics?

look after the sense, and the sounds will take care of themselves
- Lewis Carroll, Alice in Wonderland

What constitutes the semantics of a symbol has been the subject of extensive debate. We consider both the classical view, which forms the basis for most formal systems, and then the more recent, cognitive science view.

Classical semantics

Many design symbols - e.g. consider the symbol dent - often refer to things, but how can this be formalized? The semantics of dent may be simply the set of all possible entities dent can refer to, past or future or imagined. This explicit (but usually unbounded) set is the "extension" of the symbol. Alternately, the semantics of dent may consist of a specification of what these entities share - that a dent is an unintentional depression on a surface, say. This is the "intension" of the symbol.

Most views of meaning involve intension, which tries to determine if any new instance belongs to the class or not, but specifying such a definition is invariably problematic. Thus, it is extremely difficult to define dent, though we "know what it is". Some dents may be more prototypical than others, and some may violate some norm and still belong (e.g. some dents may be intentional). Furthermore, the meaning of a symbol like dent may not even have an independent definition - its meaning is affected by its relation to similar symbols like slit, slot, groove, etc. (this relational view goes back to the linguist de Saussure).

Semantics is relational. The meaning of "dent" depends on other symbols available to us. In a language system without close terms, the interpretation would be broader.

Semantics in Cognitive Science

Cognitive science suggests that human usage of symbols are not based on definitions or intensions, but involves a rich set of associations, which permit "partial matches, and the projection from one domain to another" [Fauconnier 2001]. Unlike the composition of formal symbols, where rules are applied to obtain new combinations of f-symbols in a syntactic process, cognitive symbols must compose both the labels, as well as the semantic components. In the latter, the focus is limited to some fragments of the constituent models.

Thus a concept like gas flow has two elements, gas, and flow, which encompass two rather different domains. To some extent the semantics of gas flow can be derived by merging partial structures present in gas flow and gas flow. But the meaning of gas flow would also have additional aspects which may include emergent phenomena, non-obvious inferences and patterns learned from personal experience.

In design practice, symbols are largely learned through usage. Even when a symbol is explained to us, as in an apprenticeship setting, it is interpreted in terms of a deep, personal understanding of the phenomenological domain of the design. This is why design pedagogy is invariably rooted in practice. Most rules are not explicitly told - e.g. one may be able to combine semantics of several domains to make dynamic inferences like moisture may collect in dents and cause rusting.

This merging of concepts also makes the interpretations quite stretchable. Every day, thousands of designers with very different backgrounds talk to each other, often using the same symbol very differently, but misunderstandings are rare - their shared phenomenological experience stretches their interpretation to fit the context (note that it is only because misunderstandings are so rare that some become legendary!)

In contrast, semantics of a f-symbol can only be defined in terms of other f-symbols, and all possible dynamic inferences (potentially unbounded) need to be specified. In practice, programmers limit these definitions so that the system works for a few example subspaces, resulting in brittleness even for very near-by domains.

Thus, in the long run, computer systems must base their understanding of design symbols by grounding the meaning in experience outside the f-symbol universe, e.g. in perception. In this work, we hope to present an initial approach towards this process, in the context of design explorations.

Dimensionality Reduction, Chunks, and Symbols

Many relations are observed and become part of the human sensori-motor experience, but are not encoded as symbols. For example, how the image changes as one moves one's head, is clearly very strongly encoded within us, but this is part of the vast subsymbolic computation in the brain, and is not accessible to us.

Symbols are expensive to maintain, owing to the memory and computational cost of maintaining their rich associations. Thus, each symbol must be information compressing - in terms of coding theory, they must permit representations of the same data in a smaller number of "chunks". The semantic component of symbols are often "chunks" which subsume a number of relationships among surface features.

As an example of this process, let us consider a slightly more complex design task, that of the universal motor, which can run on AC or DC. We can formulate the design problem in terms of a number of design variables (eight, say), and a number of design objectives - maximizing torque and efficiency while minimizing weight, say. Now, we can find the set of solutions which cannot be bettered on all counts. These non-dominated solutions lie along a 2-D surface in the 3D objective space (figure below, left). This is the space of "good designs" or the FFRs.

The question we would like to ask is if these "good designs" exhibit some pattern in the original eight-dimensional design space, that would permit dimensionality reduction and the emergence of chunks? In this particular example, even in the 8-D design space, a 2-D manifold appears to fits this data quite well. The pattern of solutions on this manifold can be captured in these two dimensions, which may be thought of as chunks. Each chunk or discovered "feature" represents a complex inter-relation in the original design space that determine where the meaningful designs lie.

Semantics and Dimensionality Reduction. Universal motor design: 8 design variables, three design objectives. (a) In the 3-D objective space, the "good designs", lie along a 2-D pareto surface. (b) It turns out that even in the 8-D design space, a 2D manifold fits the "good designs" (FFR) quite well.

[More on chunks in the Universal Motor design space.]

Symbol emergence

Most chunks do not become symbols - but most symbols may arise from chunks. These chunks already encode a number of constraints that result in good designs. If such chunks arise repeatedly in different problems, they may eventually become ingrained as implicit symbols. In the end, if they acquire a label, they would become true symbols, with both a phonological and a semantic pole.

Symbol emergence. The space of "good designs" or the Functionally Feasible Regions (FFR)s constitute narrow regions of the design space, often reflecting a much lower-dimensional embedding. The patterns inherent in this mapping are "chunks" - which may be reified (become real) if they are encountered frequently. Eventually, if they acquire a label (a "phonological pole") they become a true symbol.

This process results in the emergence of rich, perceptually grounded symbols that are flexible and can infer many of the rules and behaviours that need to be painstakingly programmed in present computational knowledge systems.

We believe dimensionality reduction may be an important aspect of the discovery of chunks, which ultimately form the core of design symbols. In our preliminary work, we have found empirical evidence of such dimensionality reduction in many constrained-objective and multi-objective situations. The discovery of such patterns, often in an implicit manner, may also underlie the evolution of design expertise. Clearly, this is only a start and much further work is needed.

In this research effort, we also investigate how these patterns of "good designs" may change with changes in embodiment or function, and also how these ideas may be applicable to problems such as product portfolio management.


  1. Amitabha Mukerjee and Madan M. Dabbeeru, The Birth of Symbols in Design, Proc. ASME 2009 IDETC Design Theory and Methodology Conference, DTM-09, August 31-September 2, 2009, San Diego, USA. [pdf]
  2. Amitabha Mukerjee and Madan M. Dabbeeru, Symbol Emergence in Design, IJCAI-09, 5th Workshop on Neural-Symbolic Learning and Reasoning (Nesy '09), July 11, 2009, Pasadena, California, USA. [pdf]
  3. Madan M. Dabbeeru and Amitabha Mukerjee Product Platform selection in Lower-Dimensional Manifold Spaces, Proceedings of the ASME 2009 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference, August 31- September 2, 2009, San Diego, USA. [pdf]
  4. Madan M. Dabbeeru and Amitabha Mukerjee, Functional Part Families and Design Change for Mechanical Assemblies, Proceedings of the ASME 2008 IDETC, August 3-6, 2008, New York, USA. [pdf]
  5. Madan M. Dabbeeru and Amitabha Mukerjee, Discovering Implicit Constraints in Design, Proceedings of the Third International Conference on Design Computing and Cognition, June 23-25, 2008, Atlanta, USA. [pdf]

a mukerjee jul 2009