Adaptive Prediction for Diversity Promoting Text Generation
Abstract - Current text generating models are suffering from dull and repeated generation. They are biased to generate frequently used and easily predictable words. This facts mainly contribute to the quality degradation and they hamper applications of text generating models.
We propose Humelo GPT to improve the diversity and thus the quality of generated texts. It has two main characteristics:
- Humelo GPT is trained not only to predict the target word, but also the degree of predictability of the target word
- Humelo GPT has its unique decoding strategy to generate diverse and semantically rich texts
Humelo GPT can be applied to many applications including KPOP music production and celebrity chat-bot
- Example Demo (Compared with GPT)
- Quantitative Results
Our Humelo GPT outperforms OpenAI GPT models both in quality and diversity.
‘Perplexity’ and ‘KL-Divergence’ : measure quality, lower the better
‘Self-BLEU’ : evaluates diversity, lower the better