Chapter 2 referred to the field of semiotics. It was first mentioned in Weinrich's definition of a language "different from other semiotic mechanisms" (Weinrich 1966, p.142). Although Weinrich's very strict definition of a language could not be taken to include GUIs, it could be that they form another type of semiotic mechanism. Also, when considering the evolution of the desktop metaphor, reference was made to Nadin's work on the Lisa, in which semiotics was used to structure the interface design (Nadin 1988). In this chapter I will follow this up with a consideration of whether semiotics provides an effective tool for the analysis of interface metaphors.
Semiotics began at the end of the last century, when de Saussure proposed the creation of a new study he called semiology (now more usually known as semiotics), "a science that studies the life of signs within society", of which "linguistics is only a part." (de Saussure 1974, p.16). Research into computer interfaces, including GUIs, inevitably centres on the study of 'signs within society' and it would thus seem that semiotics offers a potential discipline for achieving a better understanding of the way in which the user and the computer interact, including the role of metaphor in their interaction.
Although semiotics is comparatively recent, it rests on thousands of years of the study of rhetoric. Classical rhetoricians such as Aristotle and Quintilian categorised the methods and tools of description and argument within highly stylised forms of language such as poetry and drama. They identified a rhetorical discourse as consisting of "invention" (developing arguments), "disposition" (organising the subject), and "style" (the means of persuasion). Two types of stylistic device, or rhetorical figure, were identified:
Much attention was given in earlier research on computer languages to their schematic structure or syntax, comparing, for example, the postfix syntax of languages such as Forth with the use of the prefix syntax common in functional languages. In contrast, this thesis concentrates on tropes, principally metaphor. As discussed in Chapter 2, the choice of syntax can be influenced by the metaphor of interaction but much of the importance of metaphor lies in its status as a trope, which is independent of the syntax or even the mode of presentation.
Having been developed by ancient Greek and Latin philosophers, the application of rhetorical analysis was mainly restricted to those languages, particularly to formalised language such as poetry. Large numbers of schemes and tropes were identified by early rhetoricians but most subsequent work consisted of categorising the descriptive structures of poetry and other texts, rather than developing a deeper understanding of why or how they work. Rhetoricians and grammarians lost touch with contemporary language except in etymology, the derivation of words. Even this was heavily influenced by the desire to ensure 'correct' usage of spelling and syntax by reference to historical roots.
This approach was challenged by de Saussure who began to examine the structure of contemporary parole (language as actually spoken by people), rather than the formal langue. He defined his approach as synchronic, looking at the full structure of language at a given time, in contrast to the dominant diachronic approach of etymology, looking at small elements of language as they change over time (de Saussure 1974, p.81). In doing so, de Saussure is widely recognised as the first proponent of what became known as structuralism, the synchronic approach that dominated linguistics, and influenced many other studies, through much of this century. Piaget (1971), one of structuralism's leading exponents presents a good introduction to leading figures in the movement while Sturrock (1986) provides a more recent overview. While structuralism has been successfully applied to many social sciences and to mathematics (Piaget 1971, 17-36), Chomsky has proved the most important structuralist in developing de Saussure's work in linguistics - see Chomsky (1975; 1986) for overviews.
De Saussure saw his new approach to linguistics as part of a greater science, the new science of 'semiology' - the study of signs:
A science that studies the life of signs within society is conceivable; it would be part of social psychology and consequently of general psychology; I shall call it semiology..... Linguistics is only a part of the general science of semiology; the laws discovered by semiology will be applicable to linguistics, and the latter will circumscribe a well-defined area within the mass of anthropological facts.
(de Saussure 1974, p.16).
At the same time, the philosopher Peirce (1985) was also setting out the boundaries of a field of study that he termed 'semiotic'. Both adopted the prefix 'semio' from the Greek for sign, semeion (semeion). Whereas de Saussure saw 'semiology' as a branch of psychology, Peirce termed the field 'semiotic' which he saw as a branch of logic. It is now usually known as semiotics, analogous to its major component linguistics. Although each approached their studies from different viewpoints, one central issue united de Saussure and Peirce: a sign cannot be studied without considering what it signifies to a person.
The somewhat tautological definition of a sign in semiotics is 'anything that signifies something to someone'. The concept of a 'sign' can thus include a word, a sentence or an entire book. It also includes non-verbal signs: a statue, a diagram or a photograph. A sign does not have to be man-made: clouds on the horizon might signify rain; spots might signify measles. Various terminologies have been used to express the concepts involved. I have adopted de Saussure's original terminology (de Saussure 1974), with additions by Eco (1979). In this, the perception of spots forms the signifier and the concept of measles is the signified. The third element is the act of signification, which is context dependent: a modern doctor might see spots as signifying a disease, but other societies might interpret them in very different ways.
All three of these terms describe mental constructs: the spots might be seen quite differently by someone who is colour-blind, while a visual disorder might lead to someone seeing spots with no physical existence. Reserving the term sign to refer to the physical sign, I will use Eco's term sign-function to refer to the total conceptual system as shown in the following diagram based on that of Eco (1979, p.58):
Figure 3.1: The sign-function.
The original form of this diagram comes from Ogden and Richards (1938) although they used different terms. This and similar versions are discussed by Eco (1979, p. 58-59) and by Sebeok (1991, p.52), who strongly disagree on its value. Whereas Sebeok sees the triangle as "heuristically valuable", Eco attacks it as, "an over-simplified diagram which has rigidified the problem in an unfortunate way." It should be taken here as a simplified heuristic aid, rather than an all-encompassing, rigid definition of a sign. Martin (1975, p.26) uses similar terms but extends the triangle to a rectangle to include the physical sign.
As the triangle shows, the sign only carries meaning as part of the process of signification. It is an interactive act, depending on the way it is perceived rather than the intent in generating it. Semioticians usually express this as 'reading' meaning into the sign, referring to any related collection of signs as a 'text'. Despite the terminology, these terms are not limited to writing and semiotics is applied to 'reading' non-verbal 'texts' such as a film or the contents of a painting.
Semiotics also takes great note of what is termed 'intertextuality' (Chandler 1995). We can never read a text in isolation; it must be read in the context of other texts of the same type. In the case of computing, we understand a particular Macintosh wordprocessor in the context of other wordprocessors and other Macintosh programs we have used. Applying this to interface design implies that interfaces should be consistent across applications and across platforms. As this principle is already commonly held in HCI, it will not be pursued further. Instead I will look a little more closely at metaphor in semiotics.
Metaphor can be seen in two different ways. Either a new idea is created from the fusion of the two original ideas, or our understanding of the first idea, or tenor, is transformed by consideration of the vehicle. These can be represented symbolically as:
(1) T + V => C, or
(2) T + V => T(V),
where T is the tenor, V the vehicle and C is a new concept created by the use of the metaphor.
The role of metaphor, and that of metonymy, in the development of language and computer interfaces was discussed in Chapter 2. I hope to establish that this is through process (1) above and that this is also the process of most importance in computing. Much work on metaphor can be disregarded as concentrating on poetic metaphor which is represented by process (2), where the vehicle is introduced to change one's perception of an existing concept - the tenor - rather than create a new concept.
The most extreme case for the role of metaphor in our language is that presented by Lakoff and Johnson (1980), as introduced in the previous chapter. After stating that, "Our ordinary conceptual system, in terms of which we both think and act, is fundamentally metaphorical in nature," they continue:
The concepts that govern our thought are not just matters of the intellect. They also govern our everyday functioning, down to the most mundane details. Our concepts structure what we perceive, how we get around in the world, and how we relate to other people. Our conceptual system thus plays a central role in defining our everyday realities. If we are right in suggesting that our conceptual system is largely metaphorical, then the way we think, what we experience, and what we do every day is very much a matter of metaphor.
(Lakoff 1980, p.3).
There is some experimental evidence to support at least some of this viewpoint. If most of our language were literal, with metaphor as an 'add-on', this would imply that we would comprehend the literal meaning of an expression and then work out the metaphor, taking longer than the interpretation of a purely literal text. In fact, evidence generally supports the view that metaphors take no longer to comprehend than literal expressions. See Cornell Way (1991, p. 51-59) for summaries of a number of experiments which have shown equivalent comprehension time for metaphorical and literal expressions.
Some evidence has been presented to oppose this view which has shown that it is possible to 'force' a longer reaction time by leading the reader "down the literal path" (Gerrig 1983, p. 668). For example, it takes longer to interpret, "The concert hall was filled with sunshine by the orchestra," than, "The orchestra filled the concert hall with sunshine". Gerrig and Healy (1983) contend that we prefer to follow a literal reading first but that a literal interpretation of the vehicle is quickly truncated by the introduction of the tenor. However, their examples do more than their claim to "lead subjects down the literal path", they actually introduce greater ambiguity into a half-read sentence which the reader must interpret. Consider the number of potential literal mis-interpretations in the two examples they use:
[[[The concert hall was filled] ( i.e. a full house) with sunshine] (i.e. it was well-lit) by the orchestra] (i.e. the orchestra generated happiness).
[[The orchestra filled the concert hall] (i.e. occupied the concert hall) with sunshine] (i.e. the orchestra generated happiness)..
The first form of the sentence does more than guide the user to a literal interpretation; it introduces greater ambiguity that will take longer to resolve. It may also be argued that comprehension of the active form is faster than that of the less common passive form. It is therefore not surprising that reaction times are greater in this case. Other experiments have looked at the effect of context, finding that more contextual information, which reduces ambiguity, leads to no difference in comprehension time between metaphor and literal expressions (Ortony 1978). A summary of research in this area by Hoffman also appears to confirm Lakoff and Johnson's views:
In ordinary contexts, figurative language takes no longer to comprehend than ordinary communication, because figurative language is ordinary communication. It does not seem to require special comprehension processes, if to be "special" means "to take more time". (Hoffman 1984, p.154)
However, Levin (1988) has argued that Lakoff and Johnson's work is questionable in treating almost all language as metaphorical in that many of their examples have become entirely lexicalised and that we are no longer aware of their metaphorical nature. He argues that they are conventionalised metaphors that do not demand that we think of concepts in a new way. But, he claims, this is exactly what poetic metaphor does, causing us to 'imagine a metaphoric world in which trees actually do weep'.
Levin's claim that language is not full of poetic metaphor can be accepted but does not mean that language is not full of metaphor. The metaphors that Lakoff and Johnson or Eco deal with are not used for poetic effect but as part of the basic structure of language and thought. Poetic metaphors are used to modify existing concepts, whereas scientific metaphors and the metaphors employed in language development are used to create new concepts. To distinguish this type of metaphor from poetic metaphor, I will refer to it as generative metaphor.
When using computers, we are dealing with concepts not previously held in our language. Metaphor, here, applies a familiar vehicle to an unfamiliar tenor. Applying the vehicle within the context of the computer system, we generate an entirely new concept, as in process (1), the generative metaphor:
(1) T + V => C
Poetic metaphor differs from this in offering the transformation of one concept by linking it with another: a familiar tenor and a familiar vehicle. In picturing the two concepts together, the tenor is enriched by its association with the vehicle, the poetic metaphor:
(2) T + V => T(V)
Or, as Martin puts it:
When Lowell writes of 'yellow dinosaur steamshovels', the actual appearance of dinosaurs and steamshovels has to be contemplated and compared in imagination. We have to picture both tenor and vehicle, and fit them over each other, 'picturing' both at once.
(Martin 1975, p.209).
Another expression of this idea is provided by Hester (1967). He stresses the superimposition of ideas in the experiencing of metaphor, but uses Wittgenstein's discussion of seeing as in Philosophical Investigations to explain his views. Hester writes: "Metaphor involves . . . the intuitive relation of seeing as between parts of the description . . . [it] involves not only a tenor and vehicle, to use Richards' terms, thrown together in a sentence, but the positive relation of seeing as between tenor and vehicle."
Empirical evidence for this viewpoint is given by the work of Kelly and Keil (1987), who examined whether, "comprehension of a metaphor alters one's understanding of a domain over and above the concepts explicitly stated in the metaphor." They conducted a series of experiments in which they found that comprehension of the metaphors not only increased the similarities between the tenor and the vehicle but also increased similarity between other concepts from the same domain which could have formed different but appropriate metaphors if related.
By contrast, terms from the two domains that would form inappropriate metaphors if related tended to decrease in similarity. They concluded:
...first, that whole domains of concepts are implicated immediately in the process of comprehending individual metaphors. In addition, the conceptual domains interacting in metaphor are restructured, at least in terms of the similarity relations between concepts within the domains. Finally, this restructuring is asymmetric in that the tenor's domain undergoes greater change than the vehicle's domain.
(Kelly 1987, p.47).
This supports the case that, in the case of poetic metaphor, the complete mental model of the vehicle is applied to the tenor: the two are mentally 'superimposed', modifying the tenor rather than generating an independent concept.
Compared to the history of language, the history of computing is extremely short. Generative metaphor thus plays a major role in providing expressions for the new concepts which computing has introduced. By contrast, poetic metaphor depends on the application of a familiar vehicle to a familiar tenor. Few concepts in computing have yet become familiar enough for poetic metaphor to be used but an obvious exception is the computer itself. An example of poetic metaphor has been the naming of certain types of computer (and sometimes computer terminals) as 'workstations'. Chambers dictionary lists the principal meanings of 'work station' as "a position at which particular work is done." (Schwarz 1988). At the time that the term came into general use, computers were classified as 'mainframe' or 'mini' computers; rather than the obvious term 'micro computer' for still smaller machines, some manufacturers preferred to drop the term 'computer' altogether and use the new metaphor 'workstation' which drew attention to the new role they foresaw for this type of computer.
A second circumstance in which poetic metaphor can be used in computing is where an additional, poetic metaphor is applied to a concept which has already been identified by a generative metaphor. Thus, Windows 95 includes a special type of folder known as the 'My Briefcase'. The term 'folder' is an example of a generative metaphor with which it is assumed the user has become familiar but the user is then asked to consider one folder as if it is also a briefcase. My Briefcase behaves like a normal folder in most senses but includes an additional 'Briefcase' menu for performing specialist tasks such as updating data on a portable computer. In other words, the naming of the folder as a briefcase has transformed the perception of the folder, modifying the tenor rather than generating an original concept - the definition of a poetic metaphor.
At the simplest level, generative metaphors may appear to be what Goatly refers to as "lexical filling" (Goatly 1997, p. 27). This is a simple means by which children learn language. Goatly gives the example of his child talking of the "shell of the bread." He points out that this is not truly metaphor - the word 'crust' has not yet entered her vocabulary and so she uses the word with the 'best fit'. Similar lexical filling happens in adult language when the boundaries of a category are not understood. The 'hedge sparrow' looks like a sparrow but is actually a dunnock; similarly the 'sea anemone' looks similar to an anemone but is actually a sea animal. However, the processes involved in the two cases are quite different: 'sea anemone' is an example of generative metaphor, using a familiar word to name a new object which is clearly not an anemone, whereas 'hedge sparrow' represents an error in categorisation or lack of knowledge of the true name. To refer to a camel as a 'ship of the desert' is different again, involving poetic metaphor: both 'camel' and 'ship' must be understood before the metaphor works.
Generative metaphor plays its most important role when the object or concept is completely new to the culture. Most of the concepts used in computing fall into this category, having been developed within the last 30 years. Also, as explained in Chapter 2, computers have a unique ability to emulate other systems. This power extends to generative metaphors. With natural objects, we might use a metaphor to name them according to their natural appearance, as with the 'sea anemone'. In contrast, computer software and interfaces can be re-structured to fit almost any metaphor we choose. The leaves and branches of a tree will always have a tree-structure but computer 'objects' can be structured as a tree, a chain, a stack or a queue, according to the metaphor we choose to apply. Thus computer interfaces have a greater scope for the use of metaphor than anything previously encountered.
As explained above, Lakoff and Johnson placed metaphor as an essential ingredient of cognition. In a later work, Lakoff extends this to cover metonymy, which he sees as "one of the basic characteristics of cognition," in which people "take one well-understood or easy-to-perceive aspect of something and use it to stand either for the thing as a whole or for some other aspect or part of it." (Lakoff 1987, p.77).
As a generative device comparable to generative metaphor, metonymy is certainly very important in naming physical devices, as with the floppy disc or the keyboard, constructing a term from the properties of a device or naming it after its function. In computer interfaces, however, I would assert that metonymy must always be of secondary importance to metaphor, particularly in its ability to help the user understand new concepts. Lakoff writes of "one well-understood or easy-to-perceive aspect of something" but this depends on understanding something or easily perceiving its aspects. If a computing function is entirely new and a new term must be generated then it is unlikely to have any fixed aspects to name it by, unless these are introduced through the mechanism of metaphor.
Consider, for example, the DOS command 'PRINT'. Apart from actually printing by sending output to a printer, the command can be used in many other ways: sending output to a spooler, sending output to a file, sending output to the screen, and so forth. In the strict sense, the name of the command is metonymic: it applies to only one feature (the original function) of the total functionality, but this has happened through the extension of the command from its original function, a process based on metaphor, seeing the other functions as metaphorically 'printing' to a file, to the screen, and so forth. By naming something after a single aspect or feature, we tend to concentrate attention on that feature. New users of DOS are likely to think that the PRINT command can only be used for printing, limiting the signification of the sign to less than its total functionality.
It is difficult to see any circumstances where metonymy will significantly increase signification as metaphor can. This is because metaphor brings the vehicle - an additional concept - to the tenor whereas metonymy uses an existing aspect of the concept which is already "well-understood." The unique role of metaphor in increasing signification forms a central feature of my semiotic model of human-computer interaction.
Before describing the semiotic model, it is worth considering whether another approach might have provided a more effective route towards building a model of the metaphor process. Semiotics is an example of the linguistic approach considered as a possible approach in Chapter 1. Two other approaches mentioned were computer science and standard HCI techniques. As was explained, the former could provide useful tools in the formal specification of a model, while the latter offers useful techniques for testing the usability of a specific metaphor. However, neither approach provides a suitable theoretic framework for constructing a model of the metaphor process. The fourth approach considered in Chapter 1, that of cognitive psychology, appears to be more promising. It is therefore worth considering what cognitive psychology, or an associated approach, might offer and how this might be compared to the semiotic approach.
There is a well established body of research on computer interfaces in the fields of experimental psychology and software ergonomics. However, to date, this work has concentrated on testing the effectiveness of specific interfaces or interface features. The work of Carroll and others, reported in Chapter 2, provides valuable insight into how metaphors affect the usability of specific interfaces but more extensive research on metaphor depends on creating an underlying model of the process of metaphor and testing this model. I propose basing such a model on semiotics, but semiotics is not the only discipline which could be used to build such a model.
Whittock identifies two potential approaches to the study of metaphor:
Theories of metaphor are closely related to theories of imagination and to the processes and structures imagination employs. The study of metaphor leads off in one direction towards cognitive psychology with its interest in the mental processes underlying perception and mental categorization; in another direction towards rhetoric and strategies of communication. (Whittock 1990)
A third direction, not identified by Whittock, might be to look at metaphor as a part of society. For example, the work of Hutchins, described in this and the previous chapter, is based on his experience as a social anthropologist, but there is not yet any empirical basis for this approach. Research methods in anthropology, such as ethnology, could be used to test Hutchins' ideas but this work is yet to be done. Of the two directions proposed by Whittock, this chapter has thus far concentrated on 'rhetoric and strategies of communication' applied to verbal and non-verbal signs, i.e. semiotics. Before developing my semiotic model, it is necessary to look at ways in which cognitive psychology might provide an alternative route to understanding interface metaphors.
Mental models (Johnson-Laird 1983) have provided one of the most popular tools for cognitive scientists to develop models of human-computer interaction, of which Tauber (1991) introduces a number of examples. A particularly influential approach has been that of Norman (1986), further developed by Fischer (1991). Norman sees effective system design in terms of the mental models of the designer and the user: "The problem is to design the system so that, first, it follows a consistent, coherent conceptualization - a design model - and, second, so that the user can develop a mental model of that system - a user model - consistent with the design model." (Norman 1986, p.46). He then illustrates this with a diagram of their interaction:
Figure 3.2: The design model and the user's model
(based on Norman 1986, p.46).
Building on Norman's terminology, Fischer (1991) breaks down the user's model into three components, shown as D1, D2, and D3, in the following diagram:
Figure 3.3: Levels of system usage
(based on Fischer 1991, p7).
The different areas correspond to the following:
The important distinction introduced by Fischer is between the set of concepts the user knows, D1, and the set the user thinks exists, D3. When a user is first introduced to a system, only D4, the actual system, and D3 will be present. When the system is based on a metaphor, Hammond and Allinson (1987) and Rogers et al (1988) suggest that the user's initial mental model, D3, can be expected to correspond to the user's mental model of the metaphor vehicle.
The effectiveness of the metaphor can then be considered by comparing the features of the metaphor vehicle with the features of the system, the assumption being that a closer fit will mean a more effective metaphor. This can then be expressed by considering the features of the vehicle and tenor as intersecting sets:
Figure 3.4: A model of metaphor at the human-computer interface
(based on MITS 1994, p.11).
The first set represents the features of the vehicle, V, whilst the second represents the features of the tenor, in this case the computer system, S. This conceptualisation can be illustrated using examples taken from the Macintosh wastebasket:
~SUV An example of a real-world feature not present in the Macintosh system might be the ability to upturn a wastebasket and sit on it.
SUV This category includes being able to move documents from the desktop to the bin and retrieve them by taking them out of the bin, providing it has not been emptied.
SU~V This includes the ability to eject disks by dragging them to the wastebasket.
~SU~V Although these features are neither part of the system nor the vehicle, consideration of features that could be appropriate to the system will be important when making choices about what to use as a vehicle and what functionality to include. For example, users could be allowed to open documents in the wastebasket without taking them out (although this would not be possible in the real world).
It should be noted that, in practice, the set V should not be taken as the actual features of the vehicle but the set of what the user thinks are the features of the vehicle - the user's mental model of the vehicle - as this is what the user's behaviour will be based on. Based on these definitions, Anderson et al (1994) suggest a number of inter-related factors that determine the effectiveness of a metaphor, depending on the degree of overlap between the two sets of features. The most important of these is that users should make appropriate inferences about the functionality of the system from their understanding of the vehicle (Douglas 1983; Carroll 1988; Smyth 1993). Anderson et al propose that problems will occur whenever there is a high proportion of ~SUV features compared to SUV features. They describe this as 'conceptual baggage' brought to the interface by the vehicle - the features of a vehicle that are not utilised in a particular vehicle-system pairing.
This problem can be reduced either by selecting a vehicle with a restricted scope or by expanding the functionality of the system to include more features of the vehicle. However, as noted in Chapter 2, authors such as Johnson (1987) and Dieberger (1994a) have shown that blindly 'following the metaphor' can lead to restrictive systems in which the ability to do anything truly new is lost.
Anderson et al (1994; Smyth 1995b) carried out an experiment to look at the influence of metaphor choice on the user's mental model of the system. Three metaphor-based interfaces were developed, each supporting identical system functionality. The original system, 'Doorways', later shortened to 'Doors', was implemented at the Rank Xerox EuroPARC research centre, where it provided a test interface to the RAVE multimedia internal communication system (Buxton 1990; Gaver 1992). Office doors formed the main metaphor of the system, indicating the availability of another person. Three states of availability were supported: available (door open), unavailable (door closed) and 'busy' (door ajar). A 'busy' person could be 'glanced at' by a still video frame shown briefly to allow the user to decide whether to interrupt.
As reported in Chapter 2, metaphor is seen as "important because it provides us with a way of moving from known ideas and familiar concepts to new and unknown ones." (Cornell Way 1991, p.8). From the Star onwards, it has been assumed that interface metaphors should therefore be chosen from the existing working environment. To test this assumption, two metaphors were chosen which were familiar to users but did not form part of the office environment. 'Dogs' was chosen as a familiar concept irrelevant to office activity. 'Colours' was originally chosen as a metaphor-free indicator, simply using different colours to indicate the three states. However, the designers chose the three colours of traffic lights to emulate the three states, an implicit metaphor. Despite this, they were presented as simple blocks of colour and the underlying metaphor was not made more explicit. Each of the interfaces had a graphically presented equivalent for the three states of availability:
Table 3.1: States of availability.
| State | Doors | Dogs | Colours (Traffic lights) |
| Available | Open | Dog standing | Green (Go) |
| Busy | Part-open | Burying a bone | Orange (Caution) |
| Unavailable | Closed | Asleep in basket | Red (Stop) |
Subjects were presented with a task scenario which required them to act as a member of a small software development team. This task entailed attempting to contact various people and having to deal with each of the three states. The roles of other members of the software team were played by experimental stooges. All subjects completed a questionnaire to test how well they had understood the system, each statement being assigned to one of the sets outlined above.
As predicted by the experimenters' model, the greatest variation in the number correct responses occurred with the ~SUV set, i.e. the 'conceptual baggage':
Figure 3.5: Correct answers for each metaphor
(based on Anderson 1994).
The Doors interface metaphor caused the subjects to make significantly more incorrect assumptions about the underlying system functionality, the number of correct answers in the ~SUV condition being significantly less than in any other condition, or for any other metaphor. Despite this, subjects' confidence in their answers, which they also recorded, was as high as in the other sections. This misplaced confidence appeared to be due to the richness and contextual relevance of this vehicle, which had the effect of masking the boundary of the mapping between vehicle and system through a large amount of conceptual baggage. The effect of this baggage was exacerbated by the relative simplicity of the underlying system functionality.
In contrast to the Doors vehicle, it appeared that although Dogs provided a rich set of resources, these were largely inappropriate. Thus, while a degree of conceptual baggage was present, the lack of contextual relevance caused the effect to be reduced. In the case of Colours, the vehicle mapped only to a small part of the system functionality, apparently causing subjects to be aware of the boundary between the two. Despite this, subjects felt that the Doors interface was more intuitive, with far less of them asking for manuals than users of the Dogs and Colours interfaces. In fact, no manual or other assistance was provided, even when requested.
The findings of this experiment certainly appear to provide strong support for the mental model view of interface metaphors. However, the experiment supported a much lower level of functionality than most applications used in the real world. Also, the subjects used were computer science post-graduates, who could be expected to be far more computer-literate than most users. Whether this approach would be useful in a more realistic setting will therefore be further explored further on in this thesis.
Other attempts have been made to apply semiotics to computing. The most expansive is Andersen (1990), but he offers a much lower level view than the conceptual level this thesis is dealing with. For example, Andersen devotes less than five pages (from over 400) to metaphor (Andersen 1990, p.155-159), and none to other tropes. Also, much of his discussion of semiotics is concerned with the phoneme or its visual equivalent, the grapheme, which de Saussure (1974, p. 4; 18; 66) explicitly excluded from semiotic relevance.
The most relevant semiotic approach to user interface design is that developed by de Souza (1993) and extended by Prates et al (1997). This depends on Eco's concept of 'unlimited semiosis' and has much in common with my own approach. I will examine their work in greater depth below, when I introduce the concept of unlimited semiosis.
The literature of semiotics is full of different models and personal terminology that can obscure understanding. Although de Saussure (1974, p.66-67) and Peirce (1985, p.5) came to similar conclusions in seeing the sign as inseparable from what it signifies, there are important differences in their models of this. Speaking of parole (spoken language), de Saussure draws a dual relationship:
The linguistic sign unites, not a thing and a name, but a concept and a sound-image. The latter is not the material sound, a purely physical thing, but the psychological imprint of the sound, the impression that it makes on our senses. The sound-image is sensory, and if I happen to call it "material" it is only in that sense, and by way of opposing it to the other term of the association, the concept, which is generally more abstract....
The linguistic sign is then a two-sided psychological entity that can be represented by the drawing:
Figure 3.6: Concept and sound image
(de Saussure 1974, p.66).
He then introduces his favoured terminology for these:
Ambiguity would disappear if the ... notions here were designated by three names, each suggesting and opposing the others. I propose to retain the word sign [signe] to designate the whole and to replace concept and sound-image respectively by signified [signifé] and signifier [signifiant]; the last two terms have the advantage of indicating the opposition that separates them from each other and from the whole of which they are parts. (de Saussure 1974, p.67).
Placing these terms in the diagram, we get the following, with the arrows standing for "the opposition that separates them from each other" :
Figure 3.7: Signifier and signified.
Although this applies directly to spoken language, de Saussure is clear that linguistics forms only a part of 'semiology', and the signifier could be replaced by equivalent mental images for other types of sign. It is important to note that for de Saussure the sign is "a two-sided psychological entity"; the physical entity - the sound or the image - is not included as part of the sign.
In contrast to de Saussure's two part model, Peirce proposed a more complex model:
A sign, or representamen, is something which stands to somebody for something in some respect or capacity. It addresses somebody, that is, creates in the mind of that person an equivalent sign, or perhaps a more developed sign. That sign which it creates I call the interpretant of the first sign. The sign stands for something, its object. It stands for that object, not in all respects, but in reference to a sort of idea, which I have sometimes called the ground of the representamen.
(Peirce 1985, p.5).
Comparing the two models, the following equivalencies appear to have been made:
Table 3.2: Comparing de Saussure and Peirce's concepts.
| Concept | de Saussure | Peirce |
| The physical sign | - | Representamen |
| Perception of the sign | Signifier | Interpretant |
| What it signifies | Signified | Object |
Note that Peirce's term 'ground' is similar to, but not the same as, the 'ground' of a metaphor defined by Richards in Chapter 2. After its introduction, Peirce rarely refers to the ground, preferring to keep to a triadic model: the representamen, the interpretant and the object. In this context it is necessary to add a cautionary note to any assessment of Peirce's work. As Gardner (1992, p.65-66) explains, Peirce tried to express his mathematical and philosophical ideas in terms of triads, seeing this as a fundamental form, and it is possible that he simply down-played the importance of the ground as an unwelcome fourth component.
Barthes adapted de Saussure to also produce a triad, though this does not equate to Peirce's. Barthes created his three-part model by the addition of the 'sign' as a concept in addition to, rather than simply combining, the signifier and the signified:
[Any] semiology postulates a relation between two terms, a signifier and a signified. This relation concerns objects which belong to different categories, and this is why it is not one of equality but of equivalence. We must here be on our guard for despite common parlance which simply says that the signifier expresses the signified, we are dealing, in any semiological system, not with two, but with three different terms. For what we grasp is not at all one term after the other, but the correlation which unites them: there are, therefore, the signifier, the signified and the sign, which is the associative total of the first two terms. (Barthes 1973, p.112-13).
The "correlation which unites" the signifier and signified is not Peirce's representamen, the physical sign, although it may be closer to Peirce's term 'ground'. Unfortunately, the term 'sign' has now been used to refer to the physical sign (Peirce), the combination of two psychological concepts (de Saussure) and the correlation between these two concepts (Barthes). To avoid too much confusion, I will use sign in the commonest sense, the physical sign, but it should always be read as being in the context of the entire process. I will continue to use de Saussure's terms for the perception and cognition of the sign: the signifier and the signified, respectively (de Saussure 1974, p.67). For the correlation that unites these, in common with most semioticians, I use the term signification, as defined by Eco (1979, p.8). Eco draws a clear distinction between communication and signification:
So let us define a communicative process as the passage of a signal (not necessarily a sign) from a source (through a transmitter, along a channel) to a destination. In a machine-to-machine process the signal has no power to signify in so far as it may determine the destination sub specie stimuli. In this case we have no signification, but we do have the passage of some information.
When the destination is a human being, or 'addressee' (it is not necessary that the source or the transmitter be human, provided that they emit the signal following a system of rules known by the human addressee), we are on the contrary witnessing a process of signification - provided that the signal is not merely a stimulus but arouses an interpretive response in the addressee. This process is made possible by the existence of a code. (Eco 1979, p.8).
The code that Eco refers to is the set of rules that allows the process of signification to take place. Consider, for example, the handshake. We normally interpret a handshake according to a relatively simple code in which a firm handshake signifies a certain type of friendliness and trustworthiness. A Mason, however, has a quite separate code with which to interpret the same handshake, recognising certain arrangements of the digits to indicate common fellowship in the Masons. The two codes co-exist and both are culturally dependent: Japanese businessmen have to be taught the standard code of the handshake just as Westerners need to be taught the Japanese codes for bowing.
Peirce (1985, p.8) identified three classes of signs:
Note that the use of the term 'iconic' does not correspond to the wide use of the term in relation to user interfaces. A named icon in a GUI, or the name of program or file in DOS, is a very special type of sign that I will refer to as an interface sign. An interface sign functions in many ways and could actually be placed in any of Peirce's three classes. In the simplest sense, the choice of characters by which we choose to name a file is entirely arbitrary: a symbolic sign. The most important power of the interface sign, however, comes from it being a particular form of indexical sign. Barthes comes closest to expressing this in his example of a woodcutter:
Here we must go back to the distinction between language-object and metalanguage. If I am a woodcutter and I am led to name the tree I am felling, whatever the form of my sentence, I 'speak the tree', I do not speak about it. This means that my language is operational, transitively linked to its object; between the tree and myself, there is nothing but my labour, that is to say an action. This is political language: it represents nature for me only inasmuch as I am going to transform it, it is a language thanks to which I 'act the object'; the tree is not an image for me, it is simply the meaning of my action. But if I am not a woodcutter, I can no longer 'speak the tree', I can only speak about it, on it.... I no longer have anything but an intransitive relationship with the tree; this tree is no longer the meaning of reality as a human action, it is an image-at-one's-disposal.. (Barthes 1973, p.145-46).
For an interface sign, the signifier is existentially linked to its signified; a file is created by naming it; manipulating the interface sign manipulates the file it signifies (or an alias pointing to that file); 'deleting' the sign deletes the file or alias; neither can exist without the other. The language of our interaction with the computer is thus what Barthes terms 'political', 'speaking' the computer, 'acting the object,' with a direct impact on what the computer does, as distinct from, say, this thesis which speaks about the computer.
Much of the semiotic code with which we interact with the computer is concerned with this indexical nature of the interface sign. For example, it is possible to 'double-click' on icons in the Macintosh interface to open the document with its associated application. The action of double-clicking is unrelated to the desktop metaphor; it is unrelated to the interface sign itself; it is unrelated to the contents of the file; it is only related to the performance of the interface sign as an object in itself and its indexical link to the file.
Finally, there are some respects in which the interface sign is iconic, whether it is an actual icon in a GUI or a file name in a command language. Although an iconic sign 'resembles' its signified, this need not be a literal representation; it can be highly coded:
Particularly deserving of notice are icons in which the likeness is aided by conventional rules. Thus, an algebraic formula is an icon, rendered such by the rules of communication, associations, and distribution of the symbols. ... a great distinguishing property of the icon is that by the direct observation of it other truths concerning its object can be determined than those which suffice to determine its construction. (Peirce 1985, p.11).
One 'likeness' of an interface sign lies in its file type, whether expressed by the characters at the end of the file name (TXT, EXE, etc.) or by the type of interface icon used. This is determined by the construction of the interface sign but, like the algebraic formula, it conforms to logical rules from which the user can determine other truths. For example, a file created with SimpleText will have an icon type determined by SimpleText but the user can deduce that the file can be opened by Microsoft Word. Other icons generated by graphics programs are iconic in the most trivial sense, in that they generate 'thumbnail' forms of the picture itself.
Before leaving this, I should point out that Eco has described Peirce's categories as "an untenable trichotomy" (Eco 1976, p. 178), saying that the terms are too vague to be useful and proposing his own far more complex typology (Eco 1976, p.218) with four aspects of a sign under which it can be classified in up to twelve different ways. While accepting that Eco's argument has value, Peirce's simple division is sufficient to draw attention to the particularly powerful indexical nature of interface signs.
In the quotation above, concerning the woodcutter, Barthes considered the distinction between 'mythological' and 'political' speech or actions. This distinction deserves a brief explanation. As the quotation explains, 'political' language, or "language-object" changes things, whereas myth merely comments on it. Myth is "is a second-order semiological system" (Barthes 1973, p.114, italics in original) or, as he puts it in the quotation above, "a meta-language".
This distinction is not absolute. Every sign has both political and mythological elements, whether the text of which it is part is political or mythological. Every sign both 'speaks itself' and 'speaks about itself', two types of signification known as denotation and connotation. Consider the Apple logo as an example:
Figure 3.8: Apple logo.
At a simple level, this picture denotes an apple. But any picture of an apple could denote an apple. The combination of the rainbow colouring and the bite out of the side also says 'Apple Computer'. However, it has many connotations beyond this. Apple chose a picture, not a logo based on their name like IBM, DEC or most other competitors at the time. It is also coloured like a rainbow, a concept associated in America with California's ethnic mix of white, black, Hispanic and Asian and with the 'Rainbow Alliance': the name given to the loose cooperation between black activists, radical gays (pink) and the Greens.
The example of the Apple logo shows that a simple sign, created as a sign, can have many different significations among which it is not always easy to identify the denotation. Where a sign has not been created as a sign but simply exists as a sign, such as real apple, the distinction between denotation and connotation is easier. A real apple will have many connotations, heavily dependent on context. In a church it may become a symbol of the harvest festival or, in a slightly different context, of Original Sin. In a basket at the dentist or doctor it becomes a symbol of healthy eating or looking after one's teeth. However, an apple still maintains one simple denotation, that of an apple, whatever language we speak: an apple, pomme, or apfel.
Fiske claims that, "it is often easy to read connotative values as denotative facts; one of the main aims of semiotic analysis is to provide us with the analytical method and the frame of mind to guard against this sort of misreading" (Fiske 1982, p.92). According to this view, it might be argued that there is a single 'literal' denotation of the Apple logo - the picture denotes an apple, just as a physical apple itself denotes an apple - with all other meanings representing cultural connotations of the sign. It is possible to argue with this interpretation in that the denotation itself is not without cultural or ideological connotations; in another part of the world, the picture might be interpreted as a quite different fruit, perhaps an inedible one. It could therefore be argued that the true denotation of the Apple Computer logo is 'Apple Computer' as this is the most unambiguous interpretation of this specific portrayal of an apple.
In contrast to Fiske, Hall claims that there are dangers in seeing denotation and connotation as distinctly separate:
The term 'denotation' is widely equated with the literal meaning of a sign: because the literal meaning is almost universally recognised, especially when visual discourse is being employed, 'denotation' has often been confused with a literal transcription of 'reality' in language - and thus with a 'natural sign', one produced without the intervention of a code. 'Connotation', on the other hand, is employed simply to refer to less fixed and therefore more conventionalised and changeable, associative meanings, which clearly vary from instance to instance and therefore must depend on the intervention of codes...
But analytical distinctions must not be confused with distinctions in the real world. There will be very few instances in which signs organised in a discourse signify only their 'literal' (that is, nearly-universal) meaning. In actual discourse most signs will combine both the denotative and the connotative aspects (as redefined above)...
The terms 'denotation' and 'connotation', then, are merely useful analytic tools for distinguishing, in particular contexts, between not the presence/absence of ideology in language but the different levels at which ideologies and discourses intersect.
(Hall 1980, p.132-33)
Hall points out that ideologies and discourses can intersect at different levels. I will pursue this with a recursive model in which denotation is merely a 'seed', the first of the many layers of signification recursively formed.
Returning to Barthes' model of the sign, he illustrates the distinction between political language (denotation) and the meta-language of myth (connotation) in a simple diagram (Barthes 1973, p.115). I have reproduced this below, adapted to fit the terminology I am using:
Figure 3.9: Signification as a signifier
(Adapted from Barthes 1973, p.115).
Or, as put by Chandler (1995):
In semiotics there are different 'orders of signification' (levels of meaning). Semioticians distinguish (perhaps sometimes too tidily) between denotation - what a sign stands for - and connotation - its cultural associations. References to the signifier and the signified are sometimes described as the first order of signification - that of denotation, whilst connotation is described as a second-order signifying system.
In conventional semiotic terms, connotation uses the first sign (signifier and signified) as its signifier and attaches to it an additional signified. Connotations 'derive not from the sign itself, but from the way the society uses and values both the signifier and the signified.' (Chandler 1995, p.1).
We can see from his diagram that Barthes' concept of myth is as a second-order signifying system, taking the total sign-function and treating it as the signifier within a higher level sign-function. If we accept that this is possible, is there any reason not to continue the recursion indefinitely?
When Peirce proposed his triadic model of the sign, he appeared to see it as part of an unlimited recursive model of signification:
A Sign, or Representamen, is a First which stands in such genuine triadic relationship to a Second, called its Object, as to be capable of determining a Third, called its Interpretant, to assume the same triadic relation to its Object in which it stands itself to the same Object...
[The Third] must be capable of determining a Third of its own; but besides that, it must have a second triadic relation in which the Representamen, or rather the relation thereof to its Object, shall be its own (the Third's) Object, and must be capable of determining a Third to this relation. All this must equally be true of the Third's Thirds and so on endlessly; and this, and more, is involved in the familiar idea of a sign; and as the term Representamen is here used, nothing more is implied. A Sign is a Representamen with a mental Interpretant.
(Peirce 1985, p.6)
Peirce's writing style is somewhat opaque and this passage could be interpreted in a number of ways. I would suggest that Peirce sees the totality of the sign as a recursive process. "The Third... must be capable of determining a Third of its own" and "[the Third] must have a second triadic relation in which the Representamen, or rather the relation thereof to its Object, shall be its own (the Third's) Object, and must be capable of determining a Third to this relation." Placing this into the terminology I have been using:
1 The signification must determine a signification of itself
2 The relationship of the signifier to the signified also becomes a new signifier, forming a new signification.
Peirce's view does not stop here, "All this must equally be true of the Third's Thirds and so on endlessly; and this, and more, is involved in the familiar idea of a sign." In other words, the recursion is endless. Obviously, our minds have a finite capacity so that the recursion must remain finite, but in mathematical terms it is unbounded.
As an example, we can see destructive recursion build up in interpersonal relationships by reading higher signification into a sign. We might say, "He's being so nice, he must be after something." We distrust someone for being too 'nice', so they act more 'nicely' to overcome our distrust which increases, so they act even more 'nicely' and so on. An extended analysis of the manner in which these loops can occur can be found in Laing (1966) et al, while Laing also shows how this analysis can be used to examine specific examples (Laing 1970).
Another semiotics based approach to user interface design exists in the form of 'semiotic engineering'. Originally formulated by de Souza (1993) for the design of user interface languages, the approach has also been applied to multi-user systems (Prates 1997) and the gathering of user requirements (Pimenta 1997). Eco splits his Theory of Semiotics into two complementary parts, a Theory of Codes (Eco 1979, p. 48-150), relating to signification and a Theory of Sign Production (Eco 1979, p.151-313), relating to communication. Semiotic engineering is based on his Theory of Sign Production (de Souza 1993, p.754), whereas this thesis is based on his Theory of Codes and is thus complementary.
Signification deals with what a sign signifies to the user, not what has been done to generate it. The existence of a sign does not necessarily imply that there has been any conscious design process, as when red spots signify measles to a doctor. This thesis examines the signification of the metaphor to the user, not the designer's intentions. As demonstrated by the following diagram, in which she shows the relationship between cognitive engineering and semiotic engineering, de Souza is concerned with the design process, an important concern but complementary to mine:
Figure 3.10: Cognitive and semiotic engineering (de Souza 1993, p.761).
In taking this approach, de Souza looks at Kammersgard's (1988) classification of HCI into four perspectives: the systems perspective whereby users are seen as data entry components; the dialogue-partner perspective, where users and systems are seen as equal partners in conversation; the tool perspective whereby systems are tools to be employed by users; and the media perspective in which systems are viewed as a communication medium through which people pass messages. De Souza concentrates her discussion on the dialogue-partner and media perspectives, claiming that systems are "message senders and receivers at the immediate interface level, but they are also achieved messages, themselves, sent from designers to users through the computational medium." (de Souza 1993, p.753). This thesis does not depend on this perspective, looking only at what the computer interface signifies to the user, regardless of the designer's intention when constructing it.
The work on semiotic engineering raises a concern about one of Eco's concepts which at first appears similar to the layers of signification. Eco's concept is of 'unlimited semiosis' (Eco 1979, p.71), which he further develops as 'infinite semantic recursivity' (Eco 1979, p.121). Put simply, it appears that Eco is pointing out that a sign can only be defined through other signs (or parts of the sign-function), leading to a need to define those signs, thus introducing further signs, ad infinitum. Eco points out that "Semiosis explains itself by itself" (Eco 1979, p.71, italics in original), in the same manner as a dictionary defines words with the words themselves. This does not appear to entail the same form of recursion as in the model I am proposing, but this may be a difference in personal interpretation. Certainly semiotic engineering appears to interpret Eco's concept in just this way, as this diagram, taken from Prates et al (1997, p.29) shows:
Figure 3.11: Communications process.
Whether this interpretation of unlimited semiosis is correct does not, in fact, affect this thesis. Having introduced the concept, Prates et al do not investigate it but turn their attention towards the ways in which multi-user systems mediate interpersonal communications. They do not consider the intercommunication between the computer and the person in semiotic terms, only the intercommunication between the people as mediated by the computer system and the interfaces. The concern of the authors is to use semiotics to examine the communications between users, helping designers to create interfaces that will support the multiple layers of signification that exist in natural communication. By contrast, this thesis examines the layers of signification generated by the intercommunication between the user and the computer.
Peirce and Barthes' models of recursion suggest a mechanism by which multiple layers of signification could exist in any sign. The implication is that this signification will extend infinitely. If this is so, then the designer can never hope to be fully aware of the impact of the interface on the user. The problem is analogous to the more common question, "What is the longest sentence in the language". The answer to this question is that if one were to propose a 'longest sentence', and label it 'S', then one could create a longer sentence by saying "S is the longest sentence in the language," with the actual content of S substituted for the symbol. A more relevant question becomes, "What is the highest level of signification that matters?"
In the context of user interface design, I propose that this level will be much higher than the levels usually attended to in the design process. Computer companies are certainly aware of very high levels of signification in the promotion of their products. For example, the Macintosh was introduced with a television commercial, showing the Macintosh as a revolutionary device, able to smash a totalitarian 'Big Blue' state, while IBM's AS400 has been advertised with a picture of a slave breaking his chains. Messages like this may be beyond the immediate scope of the interface designer, but an interface is designed in the context of a company's advertising and signification at this level is certainly not beyond the scope of the advertiser. Even religious symbolism has not been ignored by computer manufacturers: Apple has placed much faith in its 'Evangelism' division (Kawasaki 1998).
It should not be surprising that the main commercial application of semiotics at these levels has been in advertising where the advertiser wishes signs to be associated with positive rather than negative metaphoric connotations. The use of semiotics in advertising and the analysis of advertising is widely discussed in various papers edited by Blonsky (1985), as is the use of semiotics in media such as television, cinema and political posters. The importance of semiotics in graphic design has also been recognised, with some attempts to apply these principles in computer interfaces such as that by Aaron Marcus (1983). More recently, industrial designers have been trying to adopt semiotic concepts, although they seem very divided as to what type of contribution semiotics can make: see Vihma (1989) for a wide range of examples. However, the central concern of this thesis is whether semiotics supplies effective tools for helping user interface designers, particularly in the use of interface metaphors.
The quotation from Weinrich in Chapter 2 referred to 'semiotic systems' but, as with other authors, offers no explicit definition of the term. For the purposes of this thesis, I therefore propose the following definition, based on the manner in which the term is commonly used in the literature:
As in the definition of a sign above, a semiotic system does not have to be intended as such. To take the example of spots signifying measles to a doctor, the presence of other signs such as a high temperature and a headache might change the signification to one of meningitis. Thus the symptoms of the body taken together form a semiotic system. As with other aspects of semiotics, the definition of 'related signs' is dependent on the observer, who 'reads' meaning into the signs: a practitioner of alternative medicine might see the positions of the planets as part of the same semiotic system as the spots in coming to a diagnosis. It is also important to note that other signs within the system need not be physically present to affect the signification. In the standard traffic light sequence, red and amber signify that the green light is about to come on, whereas an amber light alone signifies that the lights are about to turn to red. The lack of a red light in the latter case is as much of a sign as the presence of the red light in the former, adapting the signification of the amber light accordingly.
Some of the issues raised in Chapters 2 and 3 depend on underlying assumptions; others imply consequences that need to be tested. In this section, I will draw these out as explicit assertions and questions and introduce the analyses, experiments and studies that I propose to use to establish the validity of my approach. The subsequent chapters will then describe these tests in more detail, showing the results and conclusions that can be drawn from them.
The first assumption, raised in Chapter 2, concerns the complexity of user interfaces as semiotic systems:
| Assertion 1 | A user interface is a sufficiently complex semiotic system (even if not a true language) to develop through metaphor and metonymy, as natural languages do. |
This assertion is relatively easy to check, by looking at the degree to which an existing interface makes use of tropes, particularly metaphor and metonymy. As recent interfaces have been explicitly based around metaphor, my analysis was carried out on an older version of the MS-DOS command language.
The descriptions of the semiotic sign-function imply a multiplicity of signification, even in the simple signs that make up a user interface.
| Assertion 2 | Layers of signification are so numerous that it must be quite easy to uncover many of them in any interface. |
Before devising a more structured approach to semiotic analysis, I carried out an ad hoc analysis of the Macintosh user interface. Starting with a small part of the interface, I looked at what it appeared to signify in as many ways as possible, what these significations implied, and so on. This is highly subjective but demonstrates that such an analysis can be carried out.
Although the work in cognitive psychology described in Sections 3.4.3 and 3.4.4 appears to confirm the usefulness of an approach based on the user's mental model of the metaphor vehicle, it raises a number of important questions:
| Question 1 | Does it matter whether users form accurate models of the system? |
| Question 2 | Would 'real world' users behave differently? |
| Question 3 | Are the results valid for more complex computer systems and interfaces? |
These questions are answerable by echoing the original experiment with a more realistically complex system and subjects taken from the likely user group for the system. Performance at the tasks set can then be compared with the apparent accuracy of the user models of the system.
It has been suggested that, by bringing in wider aspects such as social factors, semiotics offers a richer view of metaphor in HCI than is provided by other approaches. This also suggests the possibility that the choice of metaphor might influence the levels of signification that users naturally adopt:
| Assertion 3 | Interface metaphors create many different forms of signification not accounted for by the mental model approach. |
| Question 4 | Does the type of metaphor affect the forms and levels of signification? |
The simple classification of interface metaphors types given in Section 2.4.4 was used to choose three metaphors that might be expected to carry very different signification. These were then used as a basis for the experiment. Signification was looked at by examining the open-ended questions in the questionnaire completed by the subjects, looking at the terms they used when describing the system. The results, and the complete experiment, are described in Chapter 4.
Earlier in this chapter, a recursive model of signification was put forward that forms the basis for practical application of semiotics to user interface design. This is because the act of signification itself forms a sign: the fact that a signifier is associated with a specific signified by a particular person is significant in itself:
| Assertion 4 | The recursive nature of signification leads to a structured model of multiple layers of signification. |
A simple mechanism is suggested to test this:
| Assertion 5 | Further layers of signification can be uncovered by asking of each layer, "What does that signify?" or, more simply, "What for?" |
The simplest way to test this assertion is by interviewing users of computer interfaces, continually asking, "What is that for?" in response to their answers. This will also lead to the answer to a question posed earlier in this chapter:
| Question 5 | What is the highest level of signification that matters to the user? |
It is suggested that this technique could be used by interface designers to discover aspects of the user interface they might otherwise have overlooked. The interviews with users are described in Chapters 5 and 6.