BIBLIOS

Document type
Journal articles

Document subtype
Full paper

Title
ExpertosLF: dynamic late fusion of CBIR systems using online learning with relevance feedback

Participants in the publication
Soraia M. Alarcão (Author)
Dep. Informática
LASIGE
Vânia Mendonça (Author)
Dep. Informática
Dep. Informática
LASIGE
Carolina Maruta (Author)
Manuel J. Fonseca (Author)
Dep. Informática
LASIGE

Summary
One of the main challenges in CBIR systems is to choose discriminative and compact features, among dozens, to represent the images under comparison. Over the years, a great effort has been made to combine multiple features, mainly using early, late, and hierarchical fusion techniques. Unveiling the perfect combination of features is highly domain-specific and dependent on the type of image. Thus, the process of designing a CBIR system for new datasets or domains involves a huge experimentation overhead, leading to multiple fine tuned CBIR systems. It would be desirable to dynamically find the best combination of CBIR systems without needing to go through such extensive experimentation and without requiring previous domain knowledge. In this paper, we propose ExpertosLF, a model agnostic interpretable late fusion technique based on online learning with expert advice, which dynamically combines CBIR systems without knowing a priori which ones are the best for a given domain. At each query, ExpertosLF takes advantage of user’s feedback to determine each CBIR contribution in the ensemble for the following queries. ExpertosLF produces an interpretable ensemble that is independent of the dataset and domain. Moreover, ExpertosLF is designed to be modular, and scalable. Experiments on 13 benchmark datasets from the Biomedical, Real, and Sketch domains revealed that: (i) ExpertosLF surpasses the performance of state of the art late-fusion techniques; (ii) it successfully and quickly converges to the performance of the best CBIR sets across domains without any previous domain knowledge (in most cases, fewer than 25 queries need to receive human feedback).


Institution
FACULDADE DE CIÊNCIAS DA UNIVERSIDADE DE LISBOA

Where published
Multimedia Tools and Applications

Publication Identifiers
ISSN - 1380-7501

Publisher
Springer Science and Business Media LLC



Document Identifiers
DOI - https://doi.org/10.1007/s11042-022-13119-0
URL - http://dx.doi.org/10.1007/s11042-022-13119-0

Rankings
Google Metrics Q1 (2022) - 87 - Multimedia
SCIMAGO Q1 (2022) - 0.72 - Media Technology

Keywords
Content-based image retrieval Late fusion Prediction with expert advice Online Learning Relevance feedback