🌐 Project Page  |   🤖 Model Collection  |   📊 Benchmark Collection  |   📘 Data Collection

Official repository for "RedSage: A Cybersecurity Generalist LLM" (ICLR 2026).

Authors: Naufal Suryanto¹, Muzammal Naseer^1†, Pengfei Li¹, Syed Talal Wasim², Jinhui Yi², Juergen Gall², Paolo Ceravolo³, Ernesto Damiani³

¹Khalifa University, ²Universität Bonn, ³University of Milan
^†Project Lead

• News
• Introduction
• Model Lineup
• Getting Started
• Build with RedSage
• Data
• Evaluation
• Responsible Use
• Citation

• 2026-01-26: Our paper has been accepted to ICLR 2026! We will release all the code, models, and datasets gradually. Please stay tuned!
• 2026-01-14: Added inference, deployment, and evaluation code (except OpenQA).
• 2025-10-14: Update the README.md

We are releasing RedSage sequentially in four phases. Track progress here (we’ll keep this list updated).

RedSage is an open-source, 8B-scale cybersecurity assistant engineered to tackle complex security workflows without the privacy risks of proprietary APIs. By combining massive domain-specific pretraining with a novel agentic dialogue pipeline, RedSage provides a locally deployable expert for everything from threat analysis to vulnerability management.

• Cyber-Domain Intelligence: Built on CyberFineWeb, a curated 11.8B-token corpus of high-quality cybersecurity resources spanning frameworks, offensive techniques, and security tool documentation.
• Agentic Augmentation: Trained on 266,000 multi-turn dialogues generated by a specialized agentic pipeline that simulates "User-Expert" workflows to solve multi-step security challenges.
• SOTA Performance: Outperforms Llama-3.1-8B and Qwen3-8B by +5.59 points on cyber-benchmarks and +5.05 points on the Open LLM Leaderboard.
• Comprehensive Benchmarking: Introduced RedSage-Bench, a new evaluation suite with 30,000+ MCQs and 240 open-ended tasks to measure cybersecurity knowledge, skill, and tools.
• Privacy-First Deployment: Optimized for the 8B scale, RedSage supports private, on-premise deployment on consumer-grade GPUs—ensuring your sensitive security data never leaves your environment.

Install uv first if you don't have it yet (see https://docs.astral.sh/uv/getting-started/installation/), then create an environment:

uv venv --python 3.12 --seed
source .venv/bin/activate

Install the tools you need with uv inside the env, for example:

uv pip install transformers torch accelerate

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_name = "RISys-Lab/RedSage-Qwen3-8B-Ins"

tok = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name, torch_dtype=torch.bfloat16, device_map="auto"
)

messages = [
  {"role": "system", "content": "You are RedSage, a helpful cybersecurity assistant."},
  {"role": "user", "content": "List three SSRF mitigations."}
]

text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=300, temperature=0.2)
print(tok.decode(out[0], skip_special_tokens=True))

Note: -Ins / -DPO are non-thinking chat models; no <think> blocks.

For more examples, see inference/README.md which includes the full chat inference demo code.

RedSage is production-ready with vLLM for high-throughput, OpenAI-compatible serving.

Start a server:

uv pip install vllm --torch-backend=auto
vllm serve RISys-Lab/RedSage-Qwen3-8B-DPO --port 8000 --max-model-len 32768
# OpenAI-compatible API at http://localhost:8000/v1

Call the API:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "RISys-Lab/RedSage-Qwen3-8B-DPO",
    "messages": [
      {"role": "system", "content": "You are RedSage, a helpful cybersecurity assistant."},
      {"role": "user",   "content": "Explain AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H."}
    ],
    "temperature": 0.2,
    "max_tokens": 512
  }'

• Use --tensor-parallel-size for multi-GPU, --max-model-len for long contexts.
• Prefer BF16/FP16 on recent GPUs; quantized weights will be linked in the collection if provided.
• Enable request batching in your gateway (nginx/Envoy) for best throughput.

For a comprehensive deployment guide, refer to docs/deploy/vllm.md.

See training/README.md for:

• CPT, SFT, & DPO workflows (Axolotl)
• Config references under training/configs/
• Hardware/memory notes and troubleshooting
• Example run scripts in scripts/

• Cybersecurity-filtered corpus with global dedup; includes a small general-domain replay to reduce forgetting.
• RedSage-Seed: curated Knowledge / Skills / Tools sources.
• RedSage-Conv: agentically generated, multi-turn, role-grounded dialogues with automatic validation.

Licenses and source notes are documented in data/README.md.

See eval/README.md for detailed instructions on:

• RedSage-Bench: 30K MCQs + 240 open-ended items with an LLM-as-judge rubric.
• Cybersecurity Benchmarks: CTI-Bench, CyberMetric, SecBench, SecEval, SECURE, MMLU-CSec.

# List all available tasks
python eval/run_lighteval.py --list-tasks

# Run a single benchmark
python eval/run_lighteval.py vllm \
  --model RISys-Lab/RedSage-Qwen3-8B-DPO \
  --tasks cybermetrics:500

# Run multiple benchmarks
python eval/run_lighteval.py vllm \
  --model RISys-Lab/RedSage-Qwen3-8B-DPO \
  --tasks cybermetrics:500,mmlu:cs_security,secbench:mcq-en \
  --output-dir results/my_eval

# Run curated benchmarks (e.g, All RedSage-MCQs)
python eval/run_lighteval.py vllm \
  --model RISys-Lab/RedSage-Qwen3-8B-DPO \
  --tasks tasks/redsage_mcqs.txt \
  --output-dir results/redsage_mcq

RedSage is released for research and educational purposes only. It contains offensive security knowledge that must be used ethically. Users are responsible for ensuring compliance with local laws.

@inproceedings{suryanto2026redsage,
  title={RedSage: A Cybersecurity Generalist {LLM}},
  author={Suryanto, Naufal and Naseer, Muzammal and Li, Pengfei and Wasim, Syed Talal and Yi, Jinhui and Gall, Juergen and Ceravolo, Paolo and Damiani, Ernesto},
  booktitle={The Fourteenth International Conference on Learning Representations},
  year={2026},
  url={https://openreview.net/forum?id=W4FAenIrQ2},
}