MedES: A Human-Centric Pipeline for Aligning Large Language Models with Chinese Medical Ethics

This repository is a sample-only public release for the MedES / MedEthicAlign benchmark associated with:

A Human-Centric Pipeline for Aligning Large Language Models with Chinese Medical Ethics
Haoan Jin, Han Ying, Jiacheng Ji, Hanhui Xu, Mengyue Wu
AAAI 2026

Paper (DOI) | Project Page

The accompanying AAAI 2026 paper introduces MedES as a scenario-centric Chinese medical-ethics benchmark constructed from 260 authoritative medical, ethical, and legal sources, together with a guardian-in-the-loop evaluation/alignment pipeline. The PDF also lists the project and dataset URL as https://github.com/X-LANCE/MedEthicAlign.

The paper reports MedES as a benchmark covering ethics and safety dimensions. Following the paper's experimental-evaluation口径, the ethics benchmark contains 4,004 subjective ethics test instances and 1,111 objective ethics test instances. This GitHub directory releases only deterministic per-scenario samples, not the full benchmark data.

To keep the GitHub release lightweight, this directory contains 10 examples per scenario/category for each released ethics dataset:

The full dataset is not included in this repository directory.

data/
├── objective/
│   └── objective_sample_by_scene.csv             # 10 sampled objective questions per objective scenario
│
├── subjective/
│   ├── subjective_sample_questions.csv           # 10 sampled subjective questions per subjective scenario
│   └── subjective_sample_responses_judges.csv    # all evaluated model responses and judge rationales for those samples
│
└── supplementary/
    └── dataset_scene_counts.csv                  # released sample counts by scenario/category

results/
└── subjective_leaderboard.csv                    # aggregate leaderboard computed from the complete evaluation files

docs/
└── figures/
    ├── leaderboard_final_score.png
    └── risk_quality_tradeoff.png

The leaderboard below is computed from the complete subjective evaluation files, while only aggregate scores are released here. The released sample rows preserve the original response, risk_judge, and quality_judge fields without truncation.

This long-format file contains one row per sampled question per evaluated model.

• Sampling is deterministic: for each scenario, the first 10 rows in the source order are selected.
• CSV files are encoded in UTF-8 with BOM (utf-8-sig) for Excel compatibility.
• Multi-line fields are preserved in CSV quoting; model responses and judge rationales are not truncated.
• results/subjective_leaderboard.csv is aggregate-only and was computed from the complete evaluation files.
• If a downstream script reads these files, use a CSV parser rather than manual line splitting because several fields contain embedded newlines.

If you use this dataset, please cite:

@inproceedings{10.1609/aaai.v40i45.41211,
  author    = {Jin, Haoan and Ying, Han and Ji, Jiacheng and Xu, Hanhui and Wu, Mengyue},
  title     = {A human-centric pipeline for aligning large language models with Chinese medical ethics},
  year      = {2026},
  isbn      = {978-1-57735-906-7},
  publisher = {AAAI Press},
  url       = {https://doi.org/10.1609/aaai.v40i45.41211},
  doi       = {10.1609/aaai.v40i45.41211},
  booktitle = {Proceedings of the Fortieth AAAI Conference on Artificial Intelligence and Thirty-Eighth Conference on Innovative Applications of Artificial Intelligence and Sixteenth Symposium on Educational Advances in Artificial Intelligence},
  articleno = {4310},
  numpages  = {9},
  series    = {AAAI'26/IAAI'26/EAAI'26}
}

This sample release is provided for research purposes. The benchmark concerns sensitive medical-ethics topics such as assisted reproduction, organ transplantation, and end-of-life care for AI safety evaluation. No real patient data is included.