data-contamination

The Paper List on Data Contamination for Large Language Models Evaluation.

This python code performs an efficient speech reverberation starting from a dataset of close-talking speech signals and a collection of acoustic impulse responses.

Python .pyc decompiler (3.0–3.14) with a contamination-aware benchmark harness. Rule-only pass + one Codex call per module; evaluated on fuzz-synthetic (LLM-naïve) and *-obf (anonymised) corpora to put a number on the memorisation share. Three independent PyPI packages: pychd, pychd-pyfuzz, pychd-pyobf.

The official implementation of the paper "Data Contamination Calibration for Black-box LLMs" (ACL 2024)

The official repository for the paper entitled "Time Travel in LLMs: Tracing Data Contamination in Large Language Models."

DICE: Detecting In-distribution Data Contamination with LLM's Internal State

Python package developed to evaluate textual overlap (N-Grams) between two volumes of text.

The official repository for the paper entitled "Data Contamination Quiz: A Tool to Detect and Estimate Contamination in Large Language Models."

Point-in-time correct LLM instrumentation — tracing, version pinning and look-ahead-bias protection for research pipelines. pip install traceguard

Zero-dependency CLI + library to detect benchmark/eval data contamination against a training corpus via n-gram overlap, flag leaks, and emit a cleaned dataset.

Detect train-eval data contamination with exact, fuzzy, and semantic matching. Built for LLM fine-tuning, RAG evaluation, and reliable benchmarking.

Zero-CoT Probe (ZCP) for detecting evasive data contamination in LLMs. Paper Link: http://arxiv.org/abs/2605.21856

A contamination-resistant, adversarial rebuild of the BIRD Text-to-SQL benchmark, constructed as the evaluation substrate for a governed data agent.

Time-aware dataset forensics and early-warning risk prediction for LLM fine-tuning. Predicts whether a dataset will damage a model — before the fine-tuning damage becomes visible.

The official implementation repository for our EMNLP 2024 Findings paper, PaCoST: Paired Confidence Significance Testing for Benchmark Contamination Detection in Large Language Models.

Do quality filters pull benchmark questions into pretraining corpora? Injects MMLU, GSM8K, GPQA, ARC, HellaSwag, PIQA and TruthfulQA items into a corpus and ranks them with six quality classifiers, including DCLM, FineWeb-Edu and Gaperon.

TurkishMMLU kirlilik listesi — Türkçe web'de birebir geçen 113 test sorusu (n=13), yöntem ve betik

Scan evaluation data for n-gram overlap, canaries, answer patterns, hashes, and optional embedding similarity.

Leakage-free real-time evaluation of open-weights LLMs for US CPI inflation forecasting. Introduces the memorization premium (seen vs. unseen forecast-error gap) and a three-role decomposition (direct forecaster, FOMC-text extractor, combiner). Reproduces every number in the IJF manuscript's Table 3 from the committed checkpoint.

A contamination-resistant research platform for testing whether public market data carries tradeable information. Ten pre-registered experiments on an explore -> holdout -> live-forward design; none survived.