Fusion and combination methods for multimodal content based medical image retrieval

In this study, weinvestigate the impact of different fusion methods of modalities for performance improvement of Content-based Image Retrieval (CBIR) systems. We first evaluated the performance of low-level features to determine the suitable one. Then we provided a comparison on effect of different distance functions such as Euclidean distance and Cosine distance on multimodal content based medical image retrieval. Then we presented an in depth investigation on different combination methods for Multimodal CBIR systems. In this way, we show how overall system performance can be improved with combination of multimodality approach and how modalities should be combined in this manner. Furthermore, we suggest a new combination approach which is based on integrating multimodal retrieval and outperforms any other fusion techniques. For evaluation, we set up a series of experiments using ImageCLEF 2011 medical image retrieval track dataset. The results show that our combination approach improves the effectiveness of whole system ever and clearly outperforms over fusion techniques for performance of multimodal CBIR systems.