Structural and semantic analysis of audio content for content-based querying and browsing

viSTRUCTURAL AND SEMANTIC ANALYSIS OF AUDIO CONTENT FORCONTENT-BASED QUERYING AND BROWSING(Ph.D. Thesis)Mustafa SERTGAZİ UNIVERSITYINSTITUTE OF SCIENCE AND TECHNOLOGYJuly 2006ABSTRACTAutomatic audio content analysis is a general research area in whichalgorithms are developed to allow computer systems to understand thecontent of digital audio information for further exploitations. The main taskof automatic audio analysis is to discover some valuable structures of audiosignals in order to facilitate a better handling of the current explosivelyexpanding amounts of audio data available in digital collections. Thepractical applications such as automatic labeling, efficient indexing,browsing, or content-based retrieval of audio data benefit from thesestructures.In this research work, our investigation relies on two areas that areparticularly important for content-based audio management systems.Firstly, we propose a new method for structural analysis of audio signals inorder to detect repetitive patterns that are suitable for content-based audioinformation retrieval systems. These patterns provide valuable informationabout the content of audio, such as a chorus or a key concept for music andspeech, respectively. The proposed method aims to detect the structuralchanges in music and speech based on the Audio Spectrum Flatness (ASF)and the MFCC feature sets in order to provide a way to extract the mostsalient information of an audio signal. Contrary to existing approaches, weviiconsider the applicability of image processing techniques in audio contentanalysis. A database of approximately 5-hours of audio clip is prepared forthe evaluation of the proposed approach. The experimental resultsdemonstrate that, all the repetitive patterns and their locations are obtainedwith the highest recognition rates of 86% and 87% for music and speech,respectively.In this thesis, we also present a framework for flexible querying andbrowsing of audio data, which benefits from the structural analysis results.The proposed framework provides a wide range of opportunities to queryand browse an audio data by content, such as querying and browsing for achorus section, querying by sound effects, and query-by-example (QBE). Inaddition, the clients can express their queries in the form of point, range, ork-nearest neighbor, which are particularly significant in the multimediadomain.Science Code : 702.3.006Key Words : Audio segmentation and analysis, audio summary, audioinformation retrieval, audio spectrum flatness, MPEG-7Page Number : 64Adviser : Prof. Dr. Buyurman BAYKAL