Deep Learning with Python

[TOC]

8.1 Text generation with LSTM

采样策略

• 贪婪采样(greedy sampling)：始终选择可能性最大的下一个字符。这个方法很可能得到重复的、可预测的字符串，而且可能意思不连贯。（输入法联想）
• 纯随机采样：从均匀概率分布中抽取下一个字符，其中每个字符的概率相同。这样随机性太高，几乎不会生成出有趣的内容。（就是胡乱输出字符的组合）
• 随机采样(stochastic sampling)：根据语言模型的结果，如果下一个字符是 e 的概率为 0.3，那么你会有 30% 的概率选择它。有一点的随机性，让生成的内容更随意富有变化，但又不是完全随机，输出可以比较有意思。

• 更高的温度：熵更大的采样分布，会生成更加出人意料、更加无结构的数据；
• 更低的温度：对应更小的随机性，会生成更加可预测的数据。

字符级 LSTM 文本生成实现

数据准备

/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject
return f(*args, **kwds)
/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject
return f(*args, **kwds)

606208/600901 [==============================] - 430s 709us/step
Corpus length: 600893


Number of sequences: 200278
Unique characters: 57
Vectorization...


训练语言模型并从中采样

1. 给定目前已有文本，从模型中得到下一个字符的概率分布；
2. 根据某个温度对分布进行重新加权；
3. 根据重新加权后的分布对下一个字符进行随机采样；
4. 将新字符添加到文本末尾。

👉 epoch 1
1565/1565 [==============================] - 170s 108ms/step - loss: 1.4089
📖 Generating with seed: "ary!--will at least be entitled to demand in return that
psy"

🌡️ temperature: 0.2
ary!--will at least be entitled to demand in return that
psychological senses of the the most comprehensed that is a sense of a perhaps the experience of the heart the present that the profound that the experience of the exploition and present that is a self and the present that is a more of the senses of a more and the sense of a more art and the the exploition and self-contempt of a perhaps the superiom and perhaps the contempt of the superiom and all th
🌡️ temperature: 0.5
superiom and perhaps the contempt of the superiom and all the instance and plays place of comprehensed and so in morals and all the auther to present mettoral of the senses and fellines to have the conceive that the
possibility of the expendeness, the hasters as how to really expendened and
all that the forenies; and all the most consequently comes that the most constanter to constantering only all the expearated the expendeness is in the delight and one w
🌡️ temperature: 1.0
l the expearated the expendeness is in the delight and one would be persists more and world, others which in the utility of the fellows
among and deceeth, virtuery, mode. one
soul the most conterrites and ly under that conservate to mapt of the own toweration here old taste inilt, the "frew-well
to openhanss"--and the way could easily hamint; becoming being, thrien himself, very deteruence who usked slaver and own scientification, of the eever," wand
undou
🌡️ temperature: 1.2
ed slaver and own scientification, of the eever," wand
undoupible "befost suldeces." it weim even astreated drwadged,
owing parits, word of hister"
enocest.-
sychea, that that words in the savery:
y allverowed", when and liqksis felling, them soperation of clentous
and blendiers.
the pleasrvation humiturring, likeford of feit-rum. i must "nrightenmy," beneveral man." the goods-
the cerses is christrantss and lightence--not man goemin love,
frro-akem.y tix
--------------------
...
👉 epoch 30
1565/1565 [==============================] - 452s 289ms/step - loss: 1.2692
📖 Generating with seed: "y system of morals, is that it is a
long constraint. in orde"

🌡️ temperature: 0.2
y system of morals, is that it is a
long constraint. in order to be stronger and according to the same the standand, as a men and sanctity, and all the sense of the strength to the sense of the strong the striving of the strong the striving of the sense of the same this consequently and something and in a man with the sense of the sense of the same the strong of the same the sense of the strength to the sense of the same the artist the strong more and the
🌡️ temperature: 0.5
to the sense of the same the artist the strong more and the intellectually in the called bet of the more of such as something of standand, the experience and profound, and in the delusions of the morality of the propeted and the more of a good and one has a sorts of the constitute in the same the religion
in the same the human condition, the old proper all this taste which seeming and the standand, and are to the person of the strength and the latten more
🌡️ temperature: 1.0
, and are to the person of the strength and the latten more matter, fith, niquet of nature of rove seemsion and proxible is itself, it has
final "genion" in coptent,
and un so larmor, romant.

bicident, with which fur one remain echo of
called in this acqueis forceek of consciate convoled,
asceticilies
invaluable demardinatid.

he in respons himons, which, a engliches bad hope feels these see,
fermines "only he will"n--as stapty pullens of bad a
🌡️ temperature: 1.2
se see,
fermines "only he will"n--as stapty pullens of bad adaval inexcepning at the unvery in all, whose in--that is a power be many trainitg prtunce--by the crueled him is every
noble,
as he society of this
sneke viged, has to telless all juscallers. we megnant cominghik, as gray illow and of holy este, "the knowing" how videly upon the adventive. there is discived and it attine to jesubuc'--the .=--what
lived
a! iapower, profoundc: of himself, it is lie
--------------------
...
👉 epoch 59
1565/1565 [==============================] - 173s 110ms/step - loss: 1.2279
📖 Generating with seed: "inward self-contempt, seek to get out of
the business, no ma"

🌡️ temperature: 0.2
inward self-contempt, seek to get out of
the business, no may be only and the ascetiably sense of the real and the sense of the more sense of the community of the moral and profound the way there is the sense of the most contempt of the spirit in a perceive, and the interpreting the sense of the world as it is the soul is the artist of the sense and the world as in the entire
complication of the world as it is the strong of the most profound to the world a
🌡️ temperature: 0.5
orld as it is the strong of the most profound to the world as the self nowadays to me that all the artifure in the mother of significance of the roman and we are this from the strength of existing and faith enthurable to the discover of the contrain to
religious were and self-grades the world with the way to the present think there are also all the plato-pit firels made a weart for the person of the rank on a way of the ethical prombates, to reason of the
🌡️ temperature: 1.0
he rank on a way of the ethical prombates, to reason of the mortand: that pureon good lack as has refree, in a herore proposition is
pronounce
the last the reasonably book of will calued and vexeme, and no means mentary honour pertimate our new, conversations. it is that
the sense ciertably with our distins of the remare of counter that german "miswome centurially instinct" in course,.
hsyaulity he in their minds and matter of mystimalsing of itplupetin
🌡️ temperature: 1.2
y he in their minds and matter of mystimalsing of itplupeting.t his asserated approysely to special,
are devial.; the finerevings
in orieicalous also mistakeng time, a :     this
benurion and
european voightry
ands centuries: it problem as place into the errords for triubtly etsibints, vetter after distrinitionn, where it lacking sole .=-vow accorded.

12u borany utmansishe and diffire being savest his
cardij(l qualition. neeked upon this
essentially, his
--------------------


附：基于词嵌入的文本生成

By(“CDFMLR”, “2020-08-20”);