Week 2: Adapting LLMs

Adaptation Options

Here’s the problem statement: Adapt a general-purpose LLM so it can accurately answer questions in a specific domain using additional documents.

There are three general approaches: fine-tuning, prompt engineering, and retrieval-augmented generation (RAG).

Let’s go through each of these approaches in turn, starting with fine-tuning.

Fine-tuning

This is where you train the LLM on domain specific data to improve its performance on a specific task. Remember: the LLM has been trained on most of the Internet text but it doesn’t know you’re business or use case!

But training the algorithm means changing the parameters in the original model. There are really two choices here:

Update all the parameters: Train the entire model from scratch or fine-tune a pre-trained model on your specific task.

This is not at all practical because LLMs have billions of parameters, which can take days or weeks to train, just to include our extra context data. As such, it’s not something an AI Engineer would consider unless you work for a large organization with significant resources.

Parameter Efficient Fine Turning (PEFT) Model adaptation is what most modern LLMs do because it only retrains a tiny percentage of the model, thus being faster, using less compute, and being cheaper.

There are several types of PEFT available to us:

Adapters and LoRA are by far the most popular. Papers for most of the others are listed below.

Adapters

This works by adding small trainable adapter layers to these transformer blocks, typically targeting the attention mechanisms. This means instead of updating billions of parameters throughout the entire model, you only train a few million parameters in the adapters.

The red Adapter modules are small, trainable layers added to each block. During PEFT training, the original model weights (Attention and MLP a.k.a Feed-Forward) remain frozen, while only the adapter weights are updated. This dramatically reduces:

• Training time
• Computational resources needed
• Storage requirements (you only save the small adapter weights, not the entire model)

LoRA

LoRA (Low-Rank Adaptation of Large Language Models) is a PEFT method that works by adding small trainable adapter layers to these transformer blocks, typically targeting the liner layers within the Feed Forward mechanism of transformer blocks. This is different from Adapters, which adapt a subset of existing parameters.

As a refresher, here’s what the FFN / MLP for each transformer looks like.

The Pretrained Weights are just a matrix of size d_out × d_in.

Now let’s see how LoRA adds a layer to this:

What LoRA does is introduce a low-rank approximation of the weights matrix, which allows for efficient fine-tuning of the model without needing to retrain the entire model from scratch.

Those two new matrices are learnable meaning that they can be trained without requiring the entire model to be retrained.

Prompt Engineering

• Few-shot Prompting. Showing the LLM some representative examples of the task at hand, and including the specific prompt we want an answer to. It basically guides the generation process.

• Zero-shot Prompting. Providing the LLM with a prompt that it can use to generate a response without any examples. For example, changing “How can I learn ML?” to “Q: How can I learn ML? A: “ is enough to structure the response. Or adding “Return only JSON like {“years_experience”: } for the resume.”

• Chain of Thought (COT). COT significantly improves results for logical or mathematical reasoning. A subsequent paper showed that just by adding “Let’s think step by step” to the prompt, even single-shot prompting would work better.

  

Putting this all together, we could add a system prompt for a customer support chat bot that reads the support policies first and then asks the user question:

Retrieval Augmented Generation (RAG)

The problem with our prior example (system prompt) is that reading the support policies could easily exceed the maximum number of input tokens supported by the LLM. Basically, the knowledge is too big to read into a prompt. That’s where retrieval augmented generation (RAG) comes in.

Retrieval is a search problem:

• Pre-process. Build a searchable index from documents.
• Runtime. Search from the index at runtime.

Build the searchable index

• Document parsing. From structured or unstructed to the text that the LLM can understand. Parsing is mostly done using AI these days. dedoc is a popular tool for document parsing.
• Document Chunking. If one source is a book, it will exceed the context window of LLMs (max tokens) and will be too complicate. By chunking using length, regex, or HTML/Markdown splitting, can get more manageable and even intelligent splitting of long documents. HTML/Markdown splitters tend to preserve the semantic structure of the document, like headers, etc. Check out text spliters by LangChain.
• Indexing. Vector-based is the most common way today. KB-based indexing is specific some certain domains. Text-based methods - they are simple and easy to implement using something like ElasticSearch. However, they lose the semantic meaning and synonyms. Vector-based methods - they are more complex and require more resources, but they preserve the semantic meaning and synonyms.

LLM Resources: Organized by Topic

The first part is all about parameter-efficient fine-tuning (PEFT)and model adaptation. Adapters and LoRA are both parameter-specific fine-tuning methods.

Fine-Tuning & Model Adaptation

Foundational Papers:

• Parameter-Efficient Transfer Learning for NLP
• LoRA: Low-Rank Adaptation of Large Language Models

Implementation Tools:

• HuggingFace PEFT (Parameter-Efficient Fine-Tuning)

Prompting Techniques

Research Papers:

• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
• Large Language Models are Zero-Shot Reasoners

Guides & Resources:

• Prompt Engineering Guide

Document Processing & Text Preparation

Document Parsing:

• Dedoc: Universal PDF/document extractor
• Layout-Parser: Document layout analysis

Text Chunking:

• Langchain Text Splitters

Embeddings & Vector Representations

Foundational Research:

• Efficient Estimation of Word Representations in Vector Space (Word2Vec)

Embedding APIs & Models:

• OpenAI’s Embeddings Guide
• The Best Embedding Models for Information Retrieval in 2025

Multimodal Embeddings:

• OpenAI’s CLIP: Connecting text and images

Vector Search & Retrieval

Vector Databases & Search:

• Faiss: Library for efficient similarity search

RAG Techniques:

• RAFT: Retrieval Augmented Fine-Tuning

Appendix - Quick Refresher on Transformers

Modern LLMs are built from stacked transformer blocks, where each block contains two main components:

• Attention: The mechanism that allows the model to focus on different parts of the input when processing each token
• MLP (Multi-Layer Perceptron): Also called the feed-forward network (FFN), this processes each token independently using learned transformations. It’s where the ‘knowledge’ is stored.

Each MLP layer consists of two linear transformations with a non-linear activation function in between. The MLP takes input vectors and transforms them through learned weight matrices:

The MLP typically expands the dimensionality in the first layer (Linear 1) and then projects it back down in the second layer (Linear 2). This is where much of the model’s learned knowledge is stored.

See Also

• Week 1: LLM Foundations