Model Training, Pre-Training, Fine-Tuning and Serving workspace, scaling from a single RTX 3090 (24GB) up to multi-GPU.
- training
- Ongoing
- fine-tuning
- Fine-tune Qwen2.5-3B-Instruct on CNN/DailyMail dataset
- Fine-tune Vicuna-7b-v1.5 on CNN/DailyMail dataset
- pre-training
- OCR PNG pages with Surya OCR
- Summarize OCR text with (unsloth/gemma-3-4b-it)
- Describe page layout/structure image-grounded with (unsloth/gemma-3-4b-it)
- Generate synthetic QA pairs from OCR text with (unsloth/gemma-3-4b-it)
Text Datasets:
- abisee/cnn_dailymail
- text/summary (312k rows)
- The CNN / DailyMail Dataset is an English-language dataset containing just over 300k unique news articles as written by journalists at CNN and the Daily Mail. The current version supports both extractive and abstractive summarization.
# --- pre-training: PDF corpus -> OCR/summary/layout/QA CSVs ---
pre-training\exec_1.bat
# --- fine-tuning/vicuna-7b-lora: Vicuna-7B LoRA (transformers + peft) ---
# text summarization LoRA, loads lmsys/vicuna-7b-v1.5 directly
uv run --directory fine-tuning/vicuna-7b-lora python build_vicuna7b_dataset.py --cnn-dailymail-dir "C:\path\to\cnn_dailymail\3.0.0" --max-samples 2000
uv run --directory fine-tuning/vicuna-7b-lora python train_vicuna7b_lora.py --num-epochs 2 --output-dir runs/vicuna7b_lora
uv run --directory fine-tuning/vicuna-7b-lora python generate_vicuna7b_lora.py --adapter-dir runs/vicuna7b_lora/final_adapter --jsonl-eval data/vicuna7b_train.jsonl --num-samples 5
# --- fine-tuning/qwen25-3b-lora: Qwen2.5-3B LoRA (transformers + peft) ---
# same pattern as vicuna-7b-lora, ChatML prompt format
uv run --directory fine-tuning/qwen25-3b-lora python build_qwen3b_dataset.py --cnn-dailymail-dir "C:\path\to\cnn_dailymail\3.0.0" --max-samples 2000
uv run --directory fine-tuning/qwen25-3b-lora python train_qwen3b_lora.py --num-epochs 2 --output-dir runs/qwen3b_lora
uv run --directory fine-tuning/qwen25-3b-lora python generate_qwen3b_lora.py --adapter-dir runs/qwen3b_lora/final_adapter --jsonl-eval data/qwen3b_train.jsonl --num-samples 5Licensed under the MIT License.