Essentials of Generative AI

The first part explains the foundational technologies and architectures that support the realization of generative models. It covers evolved and deepened elements, word embeddings as a representative example of representation learning, and the Transformer as a network foundation, along with its underlying attention mechanism. Reinforcement learning, which became essential for elevating large-scale language models to language generation models, is also discussed in detail, focusing on essential aspects.

The second part deals with language generation. It starts by elucidating language models

and introduces large-scale language models with broad applications as the foundational architecture of language processing, further discussing language generation models as their evolution. Though not common terminology, in this book, models such as ChatGPT and Llama 2, which are large-scale language models fine-tuned using reinforcement learning, are referred to as generative language models.

The third part addresses image generation, discussing variational autoencoders and the remarkable diffusion models. Additionally, it explains Generative Adversarial Networks(GAN). Although GAN poses challenges due to unstable learning, their conceptual framework is widely applicable, especially Wasserstein GAN seems suitable for introducing optimal trans- port distance, which is utilized in various scenarios.

This book primarily serves as a companion for researchers or graduate students in machine learning, aiming to help them understand the essence of generative AI and lay the groundwork for advancing their own research.