Llama-3-Meditron: An Open-Weight Suite of Medical LLMs
A. Sallinen, A. Solergibert, M. Zhang, G. Boyé, M. Dupont-Roc, X. Theimer-Lienhard,
E. Boisson, B. Bernath, H. Hadhri, A. Tran, et al.
AAAI 2025, Gen AI for health
OpenReview
We finetune Llama-3.1 with the Meditron mixture using SFT and ORPO. Meditron outperforms its base model by 3% on medical benchmarks, and our Meditree inference method allows us to reach GPT4-Base performance of 80% accuracy on medical benchmarks, a 5% gain over Meditron alone.