BIBLIOS

Document type
Conference papers

Document subtype
Full paper

Title
LLM Fine-Tuning With Biomedical Open-Source Data

Participants in the publication
Christopher Anaya (Author)
LASIGE
Maria Fernandes (Author)
FACULDADE DE CIÊNCIAS DA UNIVERSIDADE DE LISBOA
University of Copenhagen
Francisco M Couto (Author)
Dep. Informática
LASIGE

Summary
In BioASQ Task 12b, we explored the potential of enhancing Large Language Models (LLMs) with external biomedical data. We fine-tuned Mistral-7B-Instruct v0.1 using open-source data and efficient techniques like QLoRA. To further enrich the model’s knowledge, we incorporated manually curated biomedical data alongside open-source resources. During the competition, our model tackled three question types: yes/no, factoid, and summary. While the results weren’t competitive, the process identified key areas for improvement, including data augmentation, hyperparameter tuning, and automation—aspects we intend to address in future iterations. The data is available at our group’s GitHub: https://github.com/lasigeBioTM.


Event
CLEF 2024: Conference and Labs of the Evaluation Forum

Publication Identifiers