Large Language Models

 

As we know, the most popular AI system today is ChatGPT.

The large language model that forms the foundation of ChatGPT is GPT, which stands for “Generative Pre-trained Transformer.” Let us break down this term to gain an intuition for what GPT is:

• Generative: Simply put, a language model can predict which words are likely to come next after a sequence of words has been entered. Therefore, GPT can generate complete texts by adding the predicted words to a text.
• Pre-trained: “Trained” refers to the learning process of the model, during which it learns from a large amount of data to predict new words that complete an input text. “Pre” means that after pre-training, the model can continue to learn, for example, through fine-tuning.
• Transformer: In the previous module, we learned about a simple artificial neural network architecture. Over the years, more advanced ANN architectures have been developed to perform complex tasks such as machine translation, image recognition, and text and image generation more effectively. The transformer is the artificial neural network architecture that has become the standard for building large language models and is the architecture behind GPT.

In summary, GPT is a large language model that learns to generate texts through a pre-training phase using a large amount of data and the transformer artificial neural network architecture.

The training process of the transformer network can be divided into two phases: