AI Lexicon

Alignment

The broader goal of ensuring AI systems behave in ways that are beneficial and aligned with human values and intentions. This includes technical approaches like RLHF, but also encompasses broader considerations about safety, ethics, and the responsible development of AI systems.

Distillation

A technique where a smaller model (student) is trained to mimic the behavior of a larger model (teacher). The student learns to approximate the teacher's capabilities while being more efficient in terms of size and computational requirements. This makes deployment more practical and cost-effective.

Fine-Tuning

The process of further training a pre-trained LLM on a specific dataset for a particular task or domain. This allows the model to adapt its knowledge and capabilities to specialized use cases while maintaining its general capabilities. It's like giving additional specialized training to an already educated model.

GPT (Generative Pre-trained Transformer)

A specific architecture of AI model that's trained to predict the next token (word/character) in a sequence. It's "generative" because it can create new content, "pre-trained" because it's first trained on a broad dataset before any specific fine-tuning, and uses the "transformer" architecture as its foundation.

Hallucination

Refers to when language models generate content that is false, inaccurate, or completely fabricated, despite appearing confident and fluent. This can happen because LLMs are trained to predict statistically likely patterns in text rather than to represent verified truth, and they may fill in gaps in their knowledge with plausible-sounding but incorrect information.

LLM (Large Language Model)

An AI model trained on vast amounts of text data to understand and generate human language. These models process and generate text by predicting the most probable next words in a sequence, based on their training. Examples include GPT-4, Claude, and LLaMA. They can perform tasks like writing, analysis, coding, and conversation.

LMM (Large Multimodal Model)

A more recent evolution that can understand and work with multiple types of input (modes) - not just text, but also images, audio, and sometimes video. These models can process different types of media simultaneously and generate responses across modes. For example, they can describe images, generate images from text descriptions, or understand the context of a conversation that includes both text and images.

RAG (Retrieval Augmented Generation)

A technique that combines an LLM with an external knowledge base. When given a query, the system retrieves relevant information from its database and uses it to generate more accurate, up-to-date responses. This helps address LLMs' hallucination issues and knowledge cutoff limitations.

RLHF (Reinforcement Learning from Human Feedback)

A training method where human feedback is used to refine an LLM's outputs. The model is rewarded for generating responses that align with human preferences and penalized for undesirable outputs. This helps the model learn to be more helpful, truthful, and safe.

Transformer

The fundamental neural network architecture that revolutionized natural language processing. Its key innovation is the "attention mechanism," which allows the model to weigh the importance of different parts of the input when generating each part of the output. It can process all elements of a sequence in parallel, making it more efficient and better at capturing long-range dependencies in data.