Title Page
Contents
ABSTRACT 7
초록 9
Symbols 13
Introduction 14
2. Relatesd Research 17
2.1. Text Summarization 17
2.1.1. Extract Summarization 17
2.1.2. Abstractive Summarization 18
2.2. Summarization model 19
2.2.1. Seq2Seq model 19
2.2.2. Transformer 21
2.2.3. Pretrained Language Model 21
2.2.4. BART 23
2.4. Limitation 24
3. Methodology 26
3.1. Proposed Model 26
3.1.1. Candidate Generation 27
3.1.2. Triplet Data Mining 28
3.1.3. Triplet Loss 28
3.2. Evaluation Metrics 30
3.2.1. ROUGE Score 30
4. Experiments 32
4.1. Dataset 32
Original News Data 33
Predict Summary 33
4.2. Base Model 33
4.3. Candidate Generation 33
4.3.1. Beam Search 34
4.3.2. margin 35
5. RESULT 37
6. Conclusion 40
Reference 42
TABLE 1. Contents of Dataset 32
TABLE 2. Example of original text and its abstract 33
Figure 1. Architecture of Seq2Seq Model 19
Figure 2. Architecture of Pretrained Language Model based on Transformers. 23
Figure 3. Proposed model Architecture 26
Figure 4. ROUGE scores achieved by using different beam search methods on the base model. 34
Figure 5. Variation in Loss during Training with Different Margin Values 36
Figure 6. ROUGE score between Based and Proposed 37
Figure 7. Example of summarization generated by based and proposed model 38
Figure 8. Distribution Table of Word Length between Reference and Prediction 38