1 GPT 与 GPT-2 模型的差别就在于 GPT-2 使用了更多的训练数据，增加了模型参数，在具体结构上并无较大差异。
2 BERT 是基于双向 Transformer 结构构建，而 GPT-2 是基于单向 Transformer，这里的双向与单向，是指在进行注意力计算时，BERT会同时考虑被遮蔽词左右的词对其的影响，而 GPT-2 只会考虑在待预测词位置左侧的词对待预测词的影响。
3 GPT-2 就是一个语言模型，能够根据上文预测下一个单词，所以它就可以利用预训练已经学到的知识来生成文本，如生成新闻。也可以使用另一些数据进行微调，生成有特定格式或者主题的文本，如诗歌、戏剧。
4 本模型是通过预训练模型，输入"guobing, you are so fat"，得到：guobing, you are so fat, that it takes a bit. So, I am very lucky that this year has had this big and wonderful opportunity, which means so I can take the risk with you." The first few hours on the bike are hard and hard but the second is just too long so it doesnnt feel like there has even been anything on. "When a big mountain is coming to a place, we need to go. We don d go out with no bikes at the top, we just want it there。
5 再通过138150字的戏剧文本《罗密欧与朱丽叶》进行微调，再次输入"guobing, you are so fat"，得到：guobing, you are so fat, O me. Come now to my chamber; where is Romeo to lie! O Juliet--Hush; she will give thee her rest: go to Montag: O Montignac; there lies Romeo. O mad Juliet, what art's her here-- Where art these poor little friars? 'Alas!' quears the friAR! I see you must have an eye, O Romeo; I see that Juliet bears such fruit, O mad Paris!' Come! what an ill humour this Romeo wakes you! Come thou now hither: come I now, nurse-- O, my head is heavy--Come now; come hither again:' Come thou again: come, go! ' CAPulete cries out again--CAPULEt Where have the Montserrel come to die? come now again, and stay: Juliet, I must kill myself --I'll, kill you all--but, my heart--Go. Juliet rushes in: Juliet cries And comes, O good nurse. 'Come now! my heart!' and 'O happy day is upon her; go to her. I see--she will look after thee,-- Herself, my love--'Ah! what is her name again'?--Hail me, Romeo!' O Juliet,' say ye: I am so ill-wisher--'Al as ill! O my head--I must die!' I am gone--I come again. What is Romeo--Ah, mad Montagueso!--O Juliet, My Romeo's man--What an aeeling wag thou shalt see: Romeo!' 'Come now: look thou back:' and I come! Juliet comes: Ah!' Ah,' what! Romeo is in his shroud!-- I say--here he be: come forth and kill the man: O! what, my son!' CAPUMEAUUE falls Nurse? what a wretch are yoked in! JUNE JULIS is in heaven,--Nursed, Hence the earth tremans and throws forth her fiery wings like lightning To pounce forth her holy fiery wheels to rip down our saintful and delectated idol. I am Romeo!' What dut'er art thou there Where I once saw so fair, That with thy help she may make thee an everlasting saint? O! my child--I have known this child since childhood, my。可以看出，得到的结果就是类似于戏剧文本。
--by liuguobing