《英语听说》课程教学资源（参考资料）Alignment and task success in spoken dialogue（2014）.pdf

Alignment and task success in spoken dialogue David Reitter a,⇑ , Johanna D. Moore b a College of Information Sciences and Technology, The Pennsylvania State University, United States b School of Informatics, University of Edinburgh, United Kingdom article info Article history: Received 12 June 2013 revision received 26 May 2014 Keywords: Dialogue Interactive alignment Syntactic priming Structural priming Entrainment Task success abstract Task-solving in dialogue depends on the convergence of the situation models held by the dialogue partners. The Interactive Alignment Model (Pickering & Garrod, 2004) suggests that this convergence is the result of an interactive alignment process, which is based on mechanistic repetition at a number of linguistic levels. In this paper, we develop two predictions arising from the theory, along with two methods to quantify the known structural priming effects in the full inventory of syntactic choices found in text and speech corpora. (a) Under a rational perspective, we expect increased repetition in task-oriented dialogue compared to spontaneous conversation. We find within- and between-speaker priming in a corpus of spontaneous conversations, but stronger priming in task-oriented dialogue. (b) The Interactive Alignment Model predicts linguistic adaptation to be correlated with task success. We show this effect in a corpus of task-oriented dialogue, where we find a positive correlation of long-term adaptation and a quantifiable task success measure. We argue that the repetition tendency relevant for the high-level alignment of situation models is based on slow adaptation rather than short-term priming. We demonstrate that lexical and syntactic repetition are reliable and computationally exploitable predictors of task success. 2014 Elsevier Inc. All rights reserved. Introduction Humans appear to be remarkably efficient communicators in light of the computational complexity of natural language. Dialogue poses many challenges: interlocutors have different viewpoints, linguistic preferences and knowledge states. What may help is that we are copy cats rather than creators; we prefer to adapt our language rather than to go against the grain. The Interactive Alignment Model (IAM, Pickering & Garrod, 2004) posits that such mutual adaptation is easier than careful selection of information and targeting of the message in dialogue. The IAM suggests that basic priming effects at lower processing levels (lexical, syntactic) reinforce alignment at higher ones (e.g., semantic, pragmatic), leading to linguistic adaptation and grounding of situation models during speaker interaction. Priming occurs when memory retrieval is biased by previous context; in this case, priming refers to a tendency to choose linguistic constructions that have been used shortly beforehand. The IAM assumes that this repetition of linguistic choices is not just an artifact of general memory retrieval properties, but instead is a mechanism (alignment) by which interlocutors build a common understanding of the situation, enabling them to successfully communicate without keeping track of one another’s linguistic idiosyncrasies. According to the IAM, repetition is a heuristic that helps establish common ground unless the situation requires more careful monitoring and modeling of one’s interlocutor’s state of knowledge. The success of our interactions varies. The success of task-oriented dialogue depends on communication and is quantifiable, allowing us to test the IAM by linking it to http://dx.doi.org/10.1016/j.jml.2014.05.008 0749-596X/ 2014 Elsevier Inc. All rights reserved. ⇑ Corresponding author. Address: IST Building, Penn State, University Park, PA 16802, USA. E-mail addresses: reitter@psu.edu (D. Reitter), J.Moore@ed.ac.uk (J.D. Moore). Journal of Memory and Language 76 (2014) 29–46 Contents lists available at ScienceDirect Journal of Memory and Language journal homepage: www.el sevier.com/locate/jml

alignment. In this paper, we correlate priming at levels of sentence structure (syntax) and word choice, the problem-solving objective of the dialogue, and success. Hypotheses Humans align their linguistic choices at several representational levels. At a low level, phonetic reductions occur in jointly understood words (Bard et al., 2000). An example of adaptation at a higher level of representation involves dialogue partners that develop coherent situation models, as in Garrod and Anderson’s (1987) Maze Game study. The task was designed to elicit a coordinated communication system between participants. They found that speakers tended to make the same semantic and pragmatic choices as in the utterances they had just heard. As the games proceeded, participants developed a common description scheme for positions in the maze. However, the full causal cascade from lower-level priming to high-level alignment has not yet been observed. Specifically, the hypothesized correlation between the two, and ultimately successful communication, has eluded empirical verification. In this paper, we focus on implicit linguistic decisions: the basic mechanics of communication implemented in syntactic structure, as opposed to the high-level strategies speakers use to describe aspects of a task, or the more explicitly controlled lexical choices. Syntactic priming occurs when speakers show a tendency to prefer one phrase structure over an available alternative shortly after having used this structure or having heard an interlocutor use it (Bock, 1986). Verbatim, lexical repetition is known to increase the strength of priming (Gries, 2005; Hartsuiker, Bernolet, Schoonbaert, Speybroeck, & Vanderelst, 2008; Pickering & Branigan, 1998). This lexical boost is a crucial effect for the IAM, as it shows propagation of alignment from lower to higher levels of representation. Thus far, there is only limited evidence for the occurrence of structural adaptation outside of carefully controlled laboratory settings. As we will see, speakers also adapt in situated, realistic dialogue. For example, consider this excerpt from the Map Task corpus (Anderson et al., 1991; McKelvie, 1998), a dataset that we will use extensively in this study. One speaker (g) is giving directions for another one (f) to follow on a map: f: from the mill wheel and up to the abandoned cottage to the right like a tick shape it’d be s– [the shape of a tick] from the g: no g: [the shape of a] [like an oval shape] from the caravan park you start just above the caravans Here, g first sets out to repeat the latest syntactic construction (the shape of an oval), but proceeds to use an alternative one (like an oval shape) in its repair, mirroring his interlocutor’s first syntactic choice (like a tick shape). The spontaneous syntactic choice is a direct repetition, but would be ungrammatical if completed (the shape of a oval). Both of g’s expressions reflect structural repetitions rather than plausible alternatives to describe an ovalshaped path. This example of repetition reflects not only syntactic, but also lexical choices. A quantitative model of priming should cover such cases, but also repetitions that occur outside of lexically or semantically similar contexts. In our study, we are concerned with implicit (syntactic) effects. We therefore measure priming of syntactic phrase-structure rules, whereby word-by-word repetition (topicality effects, parroting) is explicitly excluded. We examine the IAM from a functional perspective, and derive two groups of testable hypotheses. The first examines syntactic priming in task-oriented dialogue, while the second adds a functional perspective by showing a correlation between adaptation and task success. Our first hypothesis concerns the mechanisms of priming. Syntactic priming is claimed to be a mechanistic effect, though this does not necessarily mean that it is automatic and agnostic to contextual influence. According to some cognitive architectures (Anderson & Lebiere, 1998), priming effects are the result of working memory activity. From a functional and rationalist point of view, the enhancement of communication by priming suggested by the IAM could have led to an architectural configuration where the demands of the dialogue situation drive syntactic priming. For instance, syntactic representations may be temporarily associated with semantic ones. Topics determine semantics held in working memory, and so, meaning is typically clustered rather than randomly mixed. In line with this, theories of dialogue have suggested clustering of topics, and coherence of topic structure (Grosz, Joshi, & Weinstein, 1995; Grosz & Sidner, 1986). Given any syntactic-semantic associations, syntactic structure may tend to cluster as well. We hypothesize that there is a tendency for dialogue partners to repeat syntactic structure within brief time windows, and that they do more so in task-oriented dialogue than in spontaneous conversation. Regardless of the underlying mechanisms, the IAM seems incompatible with the inverse hypothesis: less priming in task-oriented dialogue. In the first set of experiments (1–2), we look at shortterm priming effects and whether speakers implicitly use increased short-term adaptation in situations where they may benefit from it. The second hypothesis is derived from the IAM’s core idea connecting low-level priming to high-level mutual understanding and task success. Adaptation itself is diffi- cult to manipulate in naturalistic human–human dialogue. However, we expect observable variation in adaptation levels. The IAM predicts that task-oriented dialogues that exhibit more syntactic adaptation between the interaction partners will ultimately yield more task success. We test this prediction in Experiments 3–4. We conclude with an experiment that uses machine learning techniques to demonstrate that both syntactic and lexical alignment can be exploited to predict task success (Experiment 5). We will refer to several different variants of syntactic adaptation. Adaptation denotes an increased amount of re-use of decisions compared to expected repetition occurring by chance. Short-term priming is short-lived adaptation, which disappears after a few seconds. Long-term adaptation is adaptation that is enhanced by repeated 30 D. Reitter, J.D. Moore / Journal of Memory and Language 76 (2014) 29–46

exposure, persistently increasing the availability of syntactic structures. Alignment is a cascade of adaptation processes between speakers at different linguistic levels postulated by the IAM. Alignment culminates in assimilated situation models and established ad hoc conventions between speakers. Interactive alignment and structural priming in dialogue Structural priming is a special case of adaptation, either between or within speakers. Language production and comprehension are biased by recent experience, regardless of whether the structures were observed while comprehending language, or whether they were used in one’s own speech. Alignment at the syntactic level is well-documented and known to occur in a variety of contexts: between questions and answers (Levelt & Kelter, 1982), in comprehension and production. It can be specific to dialogue partners (Brennan & Hanna, 2009) or to the perceived abilities of an interlocutor (Branigan, Pickering, Pearson, McLean, & Brown, 2011). Bock (1986) established the experimental paradigm that uncovered structural priming in speech. Bock and Loebell (1990) demonstrated evidence for priming of syntactic structure independent of semantics and metrical or event structure. Pickering and Branigan (1998) found syntactic priming in written language production using scripted situations and a sentence completion task. Branigan, Pickering, and Cleland (2000) found clear evidence for syntactic alignment in dialogue-like lab interactions. Their experimental design is prototypical of much of the experimental work in structural priming. In their experiments, dialogue partners took turns describing pictures to one another to enable their partner to identify the card containing the described picture from a set of cards laid out in front of them. One of the speakers was a confederate and produced descriptions based on a script that manipulated syntactic choice, in particular whether a double object or a prepositional object construction was used (e.g., the cowboy giving the clown a balloon vs. the cowboy giving a balloon to the clown). The syntactic structure of the confederate’s description strongly influenced the syntactic structure of the subject’s description in the turn immediately following. Two adaptation effects occur: (a) fast, short-term and short-lived priming, and (b) slow, long-term adaptation that persists and is likely to be a result of implicit learning (see Ferreira & Bock (2006) and Pickering & Ferreira (2008) for reviews). Long-term adaptation is a learning effect that can persist over several days (Bock, Dell, Chang, & Onishi, 2007; Kaschak, Kutta, & Schatschneider, 2011). Recent work has proposed models that explain the mechanisms of the effects (Bock & Griffin, 2000; Kaschak, Kutta, & Jones, 2011) within the context of language acquisition (Chang, Dell, & Bock, 2006) and general memory retrieval (Reitter, Keller, & Moore, 2011). The remainder of this article will address short-term syntactic priming first, and then discuss experiments with long-term syntactic and lexical alignment. Most of the results on priming and alignment come from controlled experiments. We caution that designs in which subjects do a task constructed to elicit linguistic target constructions many times may not be a true reflection of linguistic choices made by participants in natural, spontaneous real-life dialogue. For instance, findings regarding verb-argument preferences in experimental conditions do not always correlate well with corpus studies (Roland & Jurafsky, 2002). One reason why some linguistic laboratory experiments fail to faithfully reproduce real-world language use may be the complexity of linguistic choice as evidenced by models derived from corpora. Gries (2005) argues that experimental designs may effectively control only some confounds, but not the variety of factors that influence linguistic decision-making. Such criticisms are addressed by work on language elicited outside of artificially created situations, often in the context of spoken dialogue (Bock & Kroch, 1989; Dubey, Keller, & Sturt, 2005; Estival, 1985; Gries, 2005; Levelt & Kelter, 1982; Szmrecsanyi, 2006, 2005). These studies corroborate the laboratory experiments and also show that structural priming occurs in spontaneously produced language. However, these studies employ a design pattern that contrasts the use of alternative syntactic choices sharing the same semantics (e.g., She picks up the book vs. She picks the book up). Typically, such use of explicit alternations limits corpus studies as well as lab experiments to a small set of predetermined syntactic rules or constructions, such as particle placement as in the example, active vs. passive voice, or double object (DO) vs. prepositional object (PO) use for arguments to verbs. This design also hinges on a very simple notion of semantics. One could object that active and passive constructions, for instance, are not semantically equivalent and carry different connotations and information statuses (Steedman, 2000). Syntactic alternations mark syntactic choice points, i.e., where a speaker must choose a construction to use. The corpusbased approach we follow refers to syntactic choices, but does not require alternations to define or even measure priming. Pickering and Garrod (2004) argue that if the main reason that priming effects occur is to facilitate alignment, they will be particularly strong during natural interactions. Corpora provide an opportunity to quantify and contrast spontaneous processes and the interaction between linguistic choices and cognitive tasks. The next section will describe this methodology in detail. Methodology: measuring short-term priming in corpora What we describe in the following is a method to quantify and contrast priming levels in datasets. They contain language spontaneously produced in contexts not designed to elicit syntactic priming or to test the IAM. The Switchboard corpus (Marcus et al., 1994) is a set of spontaneous telephone conversations; the HCRC Map Task corpus (Anderson et al., 1991) contains task-oriented dialogues. Consider the following example. If we were to detect priming of passive constructions, we can do so with a range of different verbs and semantics by counting occurrences of passives, and contrasting the counts under two D. Reitter, J.D. Moore / Journal of Memory and Language 76 (2014) 29–46 31

conditions: a repetition case (where a passive occurred shortly before), and a control case (where the passive has not occurred recently). Priming is the result of the difference between the normalized counts. Under this view, priming is not repetition, but the increase in probability caused by a preceding occurrence. Our technique is similar, but extends this method by looking at all syntactic constructions rather than just passives, and by using regression for greater sensitivity. In this and other corpus studies, the concept of adding predictors as controls replaces the strict control of semantics in the laboratory experiment. We see a high degree of variance in speakers’ choices of syntactic forms, which is natural, as the underlying semantics largely dictate how to construct the sentences. However, examining a large number of data points allows us to treat semantic variation as noise. Corpus processing To examine ‘‘all kinds of syntactic constructions’’, we analyze our datasets in terms of their syntactic phrase structure. Both of the corpora have been annotated with phrase structure trees through automatic and manual processes that included extensive verification (Anderson et al., 1991; Marcus et al., 1994). From the trees, we identify the syntactic rules used to construct them. We see the rules as a proxy for memory items that a speaker has to retrieve to produce or comprehend a sentence. For example, the tree yields the six phrase structure rule instances shown in Table 1. 1 The conversion from syntactic trees to rule instances is unambiguous. Decay-based model of short-term priming The amount of rule repetition can now be quantified. Structural priming predicts that a rule (target) occurs more often closely after a potential prime of the same rule (stimulus) than further away. Therefore, we correlate the probability of repetition with the distance between prime and target. For example, if a sentence-level conjunction leads to the rule S ? S cc S, and such a conjunction appears in utterances 3 and 11, we would observe a repetition, noting its distance d ¼ 8 utterances. We sample repetitions and non-repetitions within 1-s or 1-utterance windows at different distances (lnðDistÞ, up to 25 utterances or 15 s). Thus, a rule occurrence in the dialogue will normally lead to up to 25 or 15 data points for the various distances, with a binary response variable indicating repetition vs. nonrepetition. Memory effects generally decay non-linearly. Analysis of the repetition probabilities over increasing d confirmed this distribution. lnðDistÞ is therefore log-transformed in our models. Unlike in controlled experimentation where specific syntactic constructions are elicited, every rule may be biased by a prior prime in this paradigm. The example shown in Fig. 1 shows a subset of the rules appearing in the text. Repetitions a and b are both at distance 2, because the occurrences (prime and target) are two utterances apart, or 4.6 and 3.2 s, respectively. To facilitate the computation, we also drop all hapax rules (frequency f ¼ 1). We exclude cases where syntactic repetition is a mere consequence of verbatim lexical repetition (c). The reason for this is that speakers may merely repeat such phrases without analyzing them syntactically. Lexical repetition is likely to result in syntactic repetition, which would possibly inflate results. The basic statistical model compares the probability of a rule occurrence in situations when it was and was not primed. The null hypothesis is that this probability should be unaffected by the prime. Our statistical model is a sensitive variant of this idea. We predict the probability of repetition as a function of the time between prime and target. Priming effects decay over time or are subject to interference in working memory, so the effect assumes a decline of repetition probability with increasing distance between prime and target. The slope of this decline is the basis for comparison of priming strength under different conditions. The logistic regression model is specified in the appendix. The effect of distance on syntactic repetition has been shown in related studies on corpora. Gries (2005) demonstrated a correlation of distance with the repetition Table 1 Syntactic rules and additional information extracted from the Map Task corpus. The speaker here is the direction follower, as opposed to the direction giver. This is a simplified example compared to the actual annotation. Onset time (s) Speaker Syntactic rule Yield 185.105 Follower VP ! VBG PP Keeping on the edge of the page 185.363 Follower PP ! IN NP On the edge of the page 185.490 Follower NP ! AT NN The edge 185.490 Follower NP !NP PP The edge of the page 185.692 follower PP ! IN NP Of the page 185.729 follower NP ! AT NN The page 1 The analysis uses the Brown Corpus part-of-speech tags Kucera and Francis (1967). IN: preposition, AT: determiner, VBG: verb, present participle/gerund. CC: sentence-level coordinating conjunction. 32 D. Reitter, J.D. Moore / Journal of Memory and Language 76 (2014) 29–46

probability of selected syntactic alternations in a corpus of spoken and written English. Gries found no effect of distances greater than one parsing unit (a unit similar to an utterance). Similarly, in our data, we see a strong decay only during the initial 5 s. In our method, unlike that of Gries, we take the distance effect on repetition within the short initial time period as a measure of short-term priming and determine how it interacts with other variables. How repetition probability is modeled depends on assumptions about the underlying cognitive mechanisms. The first of two common views, temporal decay, implies a diminishing of repetition probability or priming effects over time. This assumes a form of decision-making that is influenced by decaying activation. The alternative view assumes interference of other material, resulting in a similar reduction in repetition probability. In this case, the selection of syntactic rules is influenced by interference from more recent syntactic structures even if they are inappropriate in light of contextual or semantic constraints (see Jonides et al., 2008, for a review of the two views of short-term memory). The latter may also suggest the influence of working memory on syntactic decisions, where working memory provides cues that aid in retrieval of memory. Short-term priming can be modeled as a combination of rapid temporal decay of syntactic information, and cue-based memory retrieval subject to interfering and facilitating semantic and other information in working memory (Reitter et al., 2011). The interaction of multiple activation mechanisms is a common assumption of ACTR (Anderson & Lebiere, 1998). An additional difficulty in modeling such phenomena arises from the fact that one mechanism (e.g., temporal) may act as a proxy for the other (e.g., interference-based). So, although the rational analysis of memory retrieval needed in typical environments or text corpora may suggest temporal decay at the computational level (Marr, 1982), the underlying cognitive processes and neural implementation may be different (Lewandowsky, Duncan, & Brown, 2004). The initial experiment 1 models distance between stimulus and target (DIST) in terms of utterances, while the following experiments model it in seconds.2 Cumulative priming by a stimulus that is repeated several times is not captured by this statistical model. We distinguish comprehension-production (CP) priming, where the speaker first comprehends the prime (uttered by his/her interlocutor) and then produces the target, and production-production (PP) priming, where both the prime and the target are produced by the same speaker. This distinction is encoded in the factor CP, which is coded as 1 for between-speaker CP priming, and 0 (base case) for within-speaker PP priming. A predictor lnðFreqÞ is included to control for the frequency of the repeated syntactic rule in the corpus, as the log-transformed rule frequency normalized by corpus size. Frequency is an important covariate in many Fig. 1. Arrows on the right illustrate two instances of syntactic repetitions (a; b) and a lexical-syntactic one (c) from Map Task. c is not counted as it is also a lexical repetition. Arrows on the left show three samples (out of up to 15 per rule instance) connecting a rule instance of PP ! IN NP (at bottom) with onesecond time windows at varying distances d prior to the rule. The window at distance 3 contains repetition case b, yielding a positive sample (marked ‘‘Yes’’). In the other two windows, there is no repetition, yielding negative samples. 2 We aim to show broad applicability of the method, but see time as the most reliable and neutral basis for decay. Reitter (2008) contains further experiments varying this metric. D. Reitter, J.D. Moore / Journal of Memory and Language 76 (2014) 29–46 33