How to Build a Stable and Efficient QLoRA Fine‑Tuning Pipeline with Unsloth

3.1 Environment Setup

Start with a clean Colab runtime to avoid library conflicts. The following shell commands reinstall PyTorch with the correct CUDA version and install the exact versions of the supporting packages.

pip install -U pip
pip uninstall -y torch torchvision torchaudio
pip install --no-cache-dir torch==2.4.1 torchvision==0.19.1 torchaudio==2.4.1 \
  --index-url https://download.pytorch.org/whl/cu121
pip install -U transformers==4.45.2 accelerate==0.34.2 datasets==2.21.0 \
  trl==0.11.4 sentencepiece safetensors evaluate
pip install -U unsloth

After the installation, the notebook must restart. Unsloth detects the restart automatically and re‑imports itself, guaranteeing a stable environment.

3.2 Package Verification

Verify GPU availability and enable TensorFloat‑32 (TF32) for faster matrix multiplications:

import torch, gc
assert torch.cuda.is_available(), "GPU not detected"
print("GPU:", torch.cuda.get_device_name(0))
torch.backends.cuda.matmul.allow_tf32 = True
torch.backends.cudnn.allow_tf32 = True

def clean():
    gc.collect()
    torch.cuda.empty_cache()

3.3 Model Loading (4‑bit) & LoRA Adapter Configuration

Unsloth’s FastLanguageModel class loads a quantized model in a single line. Below we use the 1.5 B Qwen2.5 instruction‑tuned checkpoint, but any model on the Hugging Face hub works.

from unsloth import FastLanguageModel
from transformers import TextStreamer
from trl import SFTTrainer, SFTConfig

max_seq_length = 768
model_name = "unsloth/Qwen2.5-1.5B-Instruct-bnb-4bit"

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name=model_name,
    max_seq_length=max_seq_length,
    dtype=None,
    load_in_4bit=True,
)

model = FastLanguageModel.get_peft_model(
    model,
    r=8,
    target_modules=["q_proj", "k_proj"],
    lora_alpha=16,
    lora_dropout=0.0,
    bias="none",
    use_gradient_checkpointing="unsloth",
    random_state=42,
    max_seq_length=max_seq_length,
)

The r=8 setting keeps the adapter size under 2 MB, while use_gradient_checkpointing="unsloth" halves the memory needed for back‑propagation.

3.4 Dataset Preparation

We demonstrate with the AI SEO Analyzer dataset (a public instruction‑following collection). The pipeline converts multi‑turn dialogues into a single text field compatible with supervised fine‑tuning.

from datasets import load_dataset

raw_ds = load_dataset("trl-lib/Capybara", split="train").shuffle(seed=42).select(range(1200))

def to_text(example):
    example["text"] = tokenizer.apply_chat_template(
        example["messages"], tokenize=False, add_generation_prompt=False
    )
    return example

ds = raw_ds.map(to_text, remove_columns=[c for c in raw_ds.column_names if c != "messages"])
ds = ds.remove_columns(["messages"])
train_ds, eval_ds = ds.train_test_split(test_size=0.02, seed=42).values()

3.5 Training Configuration

The SFTConfig object mirrors the TrainingArguments API but is streamlined for LoRA‑based fine‑tuning.

cfg = SFTConfig(
    output_dir="unsloth_sft_out",
    dataset_text_field="text",
    max_seq_length=max_seq_length,
    packing=False,
    per_device_train_batch_size=1,
    gradient_accumulation_steps=8,
    max_steps=150,
    learning_rate=2e-4,
    warmup_ratio=0.03,
    lr_scheduler_type="cosine",
    logging_steps=10,
    eval_strategy="no",
    save_steps=0,
    fp16=True,
    optim="adamw_8bit",
    report_to="none",
    seed=42,
)

trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    train_dataset=train_ds,
    eval_dataset=eval_ds,
    args=cfg,
)

3.6 Training Loop & Inference

Run the training, then switch the model to inference mode for a quick sanity check.

clean()
trainer.train()
FastLanguageModel.for_inference(model)

def chat(prompt, max_new_tokens=160):
    messages = [{"role":"user","content":prompt}]
    text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    inputs = tokenizer([text], return_tensors="pt").to("cuda")
    streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
    with torch.inference_mode():
        model.generate(**inputs,
                       max_new_tokens=max_new_tokens,
                       temperature=0.7,
                       top_p=0.9,
                       do_sample=True,
                       streamer=streamer)

chat("Summarize the benefits of 4‑bit quantization for LLM fine‑tuning.")

Finally, persist the adapters for later deployment:

save_dir = "unsloth_lora_adapters"
model.save_pretrained(save_dir)
tokenizer.save_pretrained(save_dir)