🚀 Essential LLM Development Glossary: A Complete Guide from vLLM and RAG to Inference Endpoints

🚀 Essential LLM Development Glossary: A Complete Guide from vLLM and RAG to Inference Endpoints

"I want to build a service with an LLM, but… what is vLLM? How does RAG work? Why do I need an Inference Endpoint?"

The pace of progress in AI lately feels like the speed of light. Encountering large language models (LLMs) like ChatGPT makes you exclaim “Wow, that’s amazing!” over and over—but once you actually start building, you’re easily overwhelmed by a flood of unfamiliar terms like PagedAttention, Quantization, and Inference Endpoint.

If you’ve been trying to keep up with the latest IT trends, you’ve probably felt like technical jargon is pouring out like an alien language. Simply memorizing definitions won’t help. You need to approach each term from the angle of “What problem does this actually solve in real development?”—that’s how you build real skill.

This post goes beyond simply “using” an LLM. It focuses on building and optimizing a production service, and it walks through the core terms every working developer needs to know—explained thoroughly and kindly, as if a senior engineer were sitting next to you.

💡 1. The Core of Running LLMs: Understanding Inference Optimization Frameworks

Running an LLM means more than just calling an API. The core challenge is running a massive model with billions of parameters on limited GPU resources quickly and cheaply. That’s where specialized frameworks come in.

🚀 vLLM: The Innovator in Speed and Efficiency

vLLM is an open-source framework created to dramatically improve LLM inference speed. It goes beyond simply running a model and focuses on maximizing throughput.

• Core principle: vLLM’s biggest strength is PagedAttention. Traditional approaches allocated memory inefficiently during token generation. PagedAttention manages memory in blocks, like an operating system’s virtual memory, maximizing GPU memory utilization.
• Advantage: It lets you handle more concurrent user requests on the same GPU resources. In other words, throughput improves dramatically.
• When to use it: Essential for real-time chatbots and API backends that expect large user traffic.

⚙️ TensorRT-LLM: Precision Optimized for Hardware

TensorRT-LLM is an optimization library from NVIDIA. If vLLM’s strength is efficient memory management, TensorRT-LLM’s strength is extreme optimization for a specific hardware architecture.

• Core principle: It reconstructs the model graph itself for the target hardware (e.g., a specific GPU generation) and reorders operations to minimize overhead.
• Difference from vLLM: While vLLM pursues general-purpose high efficiency, TensorRT-LLM is specialized in extracting 100% of a specific vendor’s hardware performance. You choose based on where you need the highest performance.

💡 Developer insight: Both technologies are about speed, but vLLM emphasizes system-level efficiency while TensorRT-LLM emphasizes hardware-level optimization. Choose the right tool based on your project’s performance KPIs.

📉 Quantization: The Magic of Shrinking Model Size

LLM models are huge files that can reach tens of GB. To deploy them on client devices or low-spec servers, you need to shrink them. That’s where quantization comes in.

• Concept: The process of “compressing” the precision used to represent model weights (e.g., 32-bit floating point, FP32) down to fewer bits (e.g., 8-bit integers, INT8).
• Effect: The model file gets smaller, memory bandwidth usage drops, inference gets faster, and deployment costs go down.
• Trade-off: Higher compression can cause a slight accuracy drop, so finding the right balance is important.

🌐 2. Understanding Service Deployment: APIs and Endpoints

Training and optimizing a model is a completely different domain from actually serving it to users. The concept you must understand in this “productization” process is the Inference Endpoint.

📡 Inference Endpoint: The Window That Turns a Model into a Product

Definition: An Inference Endpoint is a stable, managed API address that a real service can call to use a specific LLM.

• Why do you need it? Exposing the model file itself is a security risk, and you need complex infrastructure for traffic control by usage, load balancing, authentication/authorization, and more. An endpoint encapsulates all of that so developers only have to make a simple API call.
• From a productization perspective: Think of it as a restaurant’s order counter. No matter how great the kitchen (the model) is, you can’t serve customers without a clear window (the endpoint) where they can place orders.
• Related concept (API Gateway): You often put an API Gateway in front of the endpoint. The gateway acts as a front-line defense, handling rate limiting, traffic monitoring, security checks, and more when requests come in.

🧠 3. Knowledge Augmentation and Structuring: Core Terms for Data Integration

LLMs have learned vast amounts of knowledge, but that knowledge is frozen at training time. They don’t know today’s news or your company’s internal confidential manuals. Knowledge augmentation is how you overcome that limitation.

📚 RAG (Retrieval-Augmented Generation): Injecting External Knowledge

RAG is currently the most standard and essential architecture for building enterprise LLMs.

• The problem: LLMs can hallucinate—they tend to invent plausible but untrue information. That’s because the model doesn’t know the source of its knowledge.
• How RAG works (3 steps):
  1. Indexing: Take external documents (PDFs, DBs, etc.), split them into small chunks, and store them in a Vector DB.
  2. Retrieval: When a user question comes in, convert it to a vector and retrieve the most relevant document chunks (context) from the Vector DB.
  3. Generation: Put the retrieved context and the original question together in the prompt and send them to the LLM. (“Answer the question using the following [Context].”)
• Result: The LLM now answers based on “the material you just gave it” rather than “what it learned during training,” so reliability and freshness of answers are maximized.

💾 Vector DB: A Modern Database That Stores Meaning

A Vector DB is different from a typical key-value database. The data stored here is in vector form.

• What is a vector? Unstructured data such as text, images, or audio converted into mathematical coordinates (arrays of numbers). In this coordinate space, vectors that are close together represent semantically similar information.
• Role: Its core function in RAG is performing similarity search between the question vector and document vectors at very high speed. (Typical examples: Pinecone, ChromaDB, etc.)

🚀 Summary and Practical Application Roadmap

Understanding and combining these technology stacks is the core of LLM application development today. Going beyond simply calling an API, what matters is the ability to design how you prepare data (RAG/Vector DB) and how you optimize the model (Quantization/PagedAttention).