BIBLIOS

Tipo
Artigos em Conferência

Tipo de Documento
Artigo Completo

Título
LLM Fine-Tuning With Biomedical Open-Source Data

Participantes na publicação
Christopher Anaya (Author)
LASIGE
Maria Fernandes (Author)
FACULDADE DE CIÊNCIAS DA UNIVERSIDADE DE LISBOA
University of Copenhagen
Francisco M Couto (Author)
Dep. Informática
LASIGE

Resumo
In BioASQ Task 12b, we explored the potential of enhancing Large Language Models (LLMs) with external biomedical data. We fine-tuned Mistral-7B-Instruct v0.1 using open-source data and efficient techniques like QLoRA. To further enrich the model’s knowledge, we incorporated manually curated biomedical data alongside open-source resources. During the competition, our model tackled three question types: yes/no, factoid, and summary. While the results weren’t competitive, the process identified key areas for improvement, including data augmentation, hyperparameter tuning, and automation—aspects we intend to address in future iterations. The data is available at our group’s GitHub: https://github.com/lasigeBioTM.


Evento
CLEF 2024: Conference and Labs of the Evaluation Forum

Identificadores da Publicação