Title Page
Abstract
Contents
Chapter 1. Introduction 24
1.1. Thesis Overview 25
Chapter 2. Background 29
2.1. Preliminary 29
2.1.1. The Sub-fields of Machine Dialogue 29
2.1.2. Types of Neural Dialogue Models 31
2.1.3. Training and Response Generation 31
2.1.4. Evaluation of Conversational Agents 32
2.2. Related Work 32
2.2.1. Applying Pragmatics to Text Generation 32
2.2.2. Endowing Persona and Improving Consistency 33
2.2.3. Displaying Empathy and Recognizing Emotion Causes 34
2.2.4. Ensuring Dialogue Safety 35
2.2.5. Methods for Creating Dialogue Datasets 36
Part Ⅰ. Social Cognition-inspired Response Generation 38
Chapter 3. Improving Persona Consistency via Pragmatic Self-Consciousness 39
3.1. Introduction 39
3.2. Insensitivity to Contradictory Words in Existing Persona-based Agents 42
3.3. Approach 45
3.4. Experiments 50
3.5. Summary 57
Chapter 4. Improving Empathy by Focusing on Emotion Causes via Perspective-taking 60
4.1. Introduction 60
4.2. Identifying Emotion Cause Words with Generative Emotion Estimation 62
4.3. Controlling the RSA framework for Focused Empathetic Responses 66
4.4. EmoCause: Emotion Cause Words Evaluation Set 70
4.5. Experiments 72
4.6. Summary 79
Part Ⅱ. Social Commonsense-infused Dataset Construction 80
Chapter 5. Improving Prosociality with Constructive Negative Feedback based on Social Norms 81
5.1. Introduction 81
5.2. Prosociality and Receptiveness in Conversational Agents 84
5.3. ProsocialDialog 87
5.4. Building Socially Responsible Dialogue Agents with ProsocialDialog 92
5.5. Experiments on ProsocialDialog 95
5.6. Generalizability of Prost and Canary 98
5.7. Societal and Ethical Considerations 101
5.8. Limitations 104
5.9. Summary 105
Chapter 6. Improving Generalizability via Million-scale Dialogue Distillation with Social Commonsense 106
6.1. Introduction 106
6.2. CO₃: A Contextualization Framework for Conversation Distillation using Commonsense 109
6.3. SODA: A Million-scale Social Dialogue Dataset 113
6.4. COSMO: A Socially Situated Conversation Model 120
6.5. Generalizability of Cosmo 121
6.6. Limitations 125
6.7. Summary 127
Chapter 7. Conclusion 129
7.1. Summary of Contributions 129
7.2. Future Directions 130
요약 133
Appendices 134
Appendix A. Supplementary Details for Improving Persona Consistency 134
A.1. Results on Variants of Distractor Selection 134
A.2. Implementation Details 135
A.3. Dialogue Examples 137
Appendix B. Supplementary Details for Improving Empathy 139
B.1. Implementation Details 139
B.2. Emotion Classification 141
B.3. Details of EmoCause Evaluation Set 141
Appendix C. Supplementary Details for Improving Prosociality 144
C.1. Details of Constructing ProsocialDialog 144
C.2. Details of Model Training 153
C.3. Details of Experiments 155
C.4. Details of zero-shot experiments 157
C.5. Dialogue Dataset Descriptions 158
Appendix D. Supplementary Details for Improving Generalizability 164
D.1. Details of CO₃ 164
D.2. Details of Soda 165
D.3. Details of Cosmo 170
D.4. Experiment Details 170
D.5. Dialogue Dataset Descriptions 173
Bibliography 177
Table 3.1. Comparison between ground-truth utterances, top-ranked entailing candidates and Contradict@1 utterances in ROUGE and SPICE scores. 43
Table 3.2. Example of a contradictory utterance returned by the model and its GT utterance with perplexity per token. The words of entailment and contra-... 43
Table 3.3. Comparison of our approach (S₁) with base speakers (S₀) on the Dialogue NLI evaluation set [9]. +DM is the Distractor Memory. High scores... 52
Table 3.4. Comparison of our approach (S₁) with base speakers (S₀) on PersonaChat [1]. C is the consistency score evaluated by a pretrained NLI model... 53
Table 3.5. Comparison of our approach (S₁) with base speakers (S₀) on the Dialogue NLI evaluation set [9] with pretrained NLI model attached. 53
Table 3.6. Human evaluation results comparing the consistency and engagingness of the base speaker (S₀) and our self-conscious agent (S₁). Numbers in... 54
Table 3.7. Comparison of our approach (S₁) with base speaker Blender (S₀) when conditioned on dialogue context in three datasets. We compute the con-... 55
Table 4.1. A dialogue example in EmpatheticDialogues. 64
Table 4.2. Example of sampled outputs from our generative emotion estimator (GEE) using Nucleus sampling. 64
Table 4.3. Statistics of the EMOCAUSE compared to RECCON. 69
Table 4.4. Examples of annotated emotion cause words. 70
Table 4.5. The most frequent cause words for each emotion. Other emotions can be found in Appendix. 70
Table 4.6. Comparison of emotion cause word recognition performance between our generative emotion estimator (GEE), random, RAKE [12], EmpDG [13],... 74
Table 4.7. Comparison of our approach (Focused S₁) with other speakers on EmpatheticDialogues [14]. Exploration, and Interpretation scores are evaluated... 75
Table 4.8. Comparison of our approach (Focused S₁) with base speakers (S₀) on human rating. 76
Table 4.9. Comparison of our approach (Focused S₁) with base speakers (S₀) on A/B test for empathetic response generation. The win and lose rates are based... 77
Table 4.10. Comparison of our approach (Focused S₁) with plain pragmatic approach (Plain S₁) on A/B test for reflecting the given emotion cause words.... 77
Table 4.11. Examples of recognized emotion cause words from our GEE and responses from S₀ and Focused S₁. We underline words where our Focused S₁... 78
Table 5.1. Statistics of PROSOCIALDIALOG compared to other dialogue datasets. Utt. denotes utterance. Brief description for each dataset is in Ap-... 91
Table 5.2. Dialogue safety classification accuracy (%) and rules-of-thumb generation results (§5.5.1) on PROSOCIALDIALOG. PPL denotes perplexity. 96
Table 5.3. Response generation results on PROSOCIALDIALOG test split (§5.5.2). 97
Table 5.4. Results of head-to-head human evaluation between dialogue agents on response generation for PROSOCIALDIALOG (in percentages; §5.5.2). 97
Table 5.5. Zero-shot response generation results (§5.6.1) for our Prost and other dialogue agents on ToxiChat [16]. All numbers in percentages. 100
Table 6.1. A sample from $$ SODA. More examples are in Appendix D.2.[이미지참조] 112
Table 6.2. Statistics of SODA compared to other large-scale dialogue datasets. Lexical diversity is measured with MTLD [17]. Description for each dataset is... 116
Table 6.3. Common topic keywords of the narratives (i.e., conversation context) in SODA. Numbers in parentheses denote the ratio of the relations in SODA. 117
Table 6.4. The ratio (%) of Top-10 emotions in 10K utterances from DailyDialog, BlendedSkillTalk, and SODA, labeled by the GoEmotions' 27-emotion-type... 118
Table 6.5. Results of head-to-head human evaluation between model responses on an unseen dataset: DailyDialog [7] (§6.5.1). The differences are all statisti-... 123
Table 6.6. Human evaluation results for head-to-head comparison of model responses under one-sided out-of-domain setting with COSMO and BlenderBot [4]... 124
Table 6.7. Head-to-head human evaluation between models on response generation for SODA (§6.5.3). The differences in the Specific from the top row, and... 125
Table A.1. Quantitative results of the proposed Distractor Memory (DM) and other distractor selection methods on the Dialogue NLI evaluation set. 134
Table B.1. Comparison of different k values for top-k emotion cause words on generating empathetic responses in EmpatheticDialogues [14]. Exploration and... 140
Table B.2. Comparison of emotion classification accuracy from different models trained on EmpatheticDialogues. 141
Table B.3. Examples of our annotated emotion cause words. Words with background color are selected as emotion cause words by annotators. 142
Table B.4. Number of emotion cause words per utterance and Top-10 frequent emotion cause words for each emotion. 143
Table C.1. Sampled dialogues from PROSOCIALDIALOG. 151
Table C.2. Sampled dialogues from PROSOCIALDIALOG. 152
Table C.3. Response generation results on PROSOCIALDIALOG and other existing large-scale dialogue datasets (§5.4.2). PPL denotes perplexity. 157
Table C.4. Results of head-to-head comparison between dialogue agents on response generation for PROSOCIALDIALOG according to crowdworker judge-... 157
Table D.1. A sample from $$ SODA.[이미지참조] 166
Table D.2. Another sample from $$ SODA.[이미지참조] 167
Table D.3. Statistics of commonsense relations in SODA. 168
Table D.4. Templates for converting symbolic commonsense knowledge to sentence form. 168
Table D.5. Templates for converting symbolic commonsense knowledge to questions for validation. 168
Table D.6. The ratio (%) of emotions in 10K utterances from DailyDialog, BlendedSkillTalk, and SODA, labeled by the 27-emotion-type classifier from... 171
Table D.7. Evaluation results of commonsense validation for short question-answering with InstructGPT on 100 human-annotated samples. 172
Table D.8. Human evaluation results for head-to-head comparison of model responses under zero-shot setting with COSMO, Koala [18], and Vicuna [19].... 173
Table D.9. The original ground-truth response and sample responses from DialoGPT, BlenderBot, GODEL, and COSMO to a context in DailyDialog. 174
Figure 3.1. Illustration of the consistency issue in dialogue. While a literal dialogue agent (S₀) fails to deliver a consistent persona, our self-conscious agent... 40
Figure 3.2. Proportion of Hits@1, Entail@1, Neutral@1 and Contradict@1 in the top-1 candidates returned by the models on the Dialogue NLI dataset. 42
Figure 3.3. The proposed self-conscious agent S₁ consists of base speaker S₀ and imaginary listener L₀. It recursively generates the next token uₜ at every... 46
Figure 3.4. Examples of generated responses by our self-conscious agent with Distractor Memory (S₁+DM) on the PersonaChat dataset [1]. We compare... 58
Figure 3.5. Performance variation of the self-conscious agent for TransferTransfo (left) and Blender (right) according to β. We compare different methods of... 59
Figure 3.6. An example of utterance changes by controlling the speaker rationality α on the PersonaChat. 59
Figure 4.1. Overview of our method, consisting of emotion recognition (§4.2.3), emotion cause word recognition (§4.2.4), distractor context sampling (§4.3.2),... 68
Figure 4.2. Emotion ratio of RECCON and our EMOCAUSE evaluation set. 69
Figure 5.1. (a) Sample responses from existing state-of-the-art conversational models [3, 4, 5] to a problematic context. (b) An example dialogue from PROSO-... 82
Figure 5.2. The overall pipeline for collecting PROSOCIALDIALOG. 87
Figure 5.3. Ratio of positive, ambiguous, and negative utterances in large-scale dialogue datasets and our PROSOCIALDIALOG, measured by the pretrained... 92
Figure 5.4. The overall ratio and turn dynamics of dialogue safety labels in PROSOCIALDIALOG. We include the actual proportions (%) inside the bars. 94
Figure 5.5. Results of head-to-head comparison between models with and without Canary on PROSOCIALDIALOG via human judgements (§5.6.2). 101
Figure 5.6. Results of head-to-head comparisons between Instruct GPT-3 vs. GPT-3 and Instruct GPT-3 vs. GPT-3 with Canary on PROSOCIALDIALOG via... 102
Figure 6.1. An illustration of our CO₃ framework (§6.2), SODA dataset (§6.3), and conversation model COSMO (§6.4) trained on SODA. Conversations are dis-... 107
Figure 6.2. Results of head-to-head comparison between dialogues from $$ SODA, DailyDialog [7], and BlendedSkillTalk [8] via human judgments (§6.3.2)....[이미지참조] 115
Figure 6.3. Results of head-to-head comparison human evaluation between conversations from SODA and those sampled from GPT-3.5 without context... 119
Figure A.1. Examples of generated responses by our self-conscious agent with Distractor Memory (S₁+DM) on the PersonaChat dataset [1]. We compare... 138
Figure C.1. The annotation page for annotating rules-of-thumb (RoTs) and responses to dialogues on Amazon Mechanical Turk. 160
Figure C.2. The annotation page for proofreading the previous response annotation round on Amazon Mechanical Turk. 161
Figure C.3. The definition and description for dialogue safety labeling for annotation on Amazon Mechanical Turk. 162
Figure C.4. The annotation page for labeling dialogue safety to utterances on Amazon Mechanical Turk. 163
Figure D.1. The annotation page for evaluating dialogues on Amazon Mechanical Turk. 175
Figure D.2. The annotation page for evaluating responses on Amazon Mechanical Turk. 176