NUS researchers work on making it easier to sift through multimedia info

by Asha Srinivasan

 – PHOTO: AGENCE FRANCE-PRESSE 

There are massive amounts of multimedia content comprising text, audio, images, videos and more available, making it difficult to find a specific piece of information when searching through the data. Multimedia information has become more pervasive and now permeates many aspects of our personal and professional lives. 

For example, new mobile devices allow us to easily record pictures and videos, and share them with friends across social networking sites on the Internet. 

Companies routinely integrate multimedia information into the development of products and delivery of services to their customers.Such developments have created an unprecedented amount of multimedia content comprising text, audio, images, videos and more. 

With so much information available, the challenge is how to efficiently sift through it and find a specific piece of information.This has led to the need for more effective data mining and search tools, which can process, analyse and fuse information. 

Researchers from the School of Computing at the National University of Singapore (NUS) are pioneering the next generation of technologies that will form the basis of media search tools of the future. 

Web-scale media search 

Search engines such as Google, Bing and Yahoo find information by sieving through their index of Internet webpages and documents, and returning those that are relevant to the user’s search. 

To cope with the explosion of multimedia content online, researchers are developing search tools that can handle real-time content too, generated, for instance, by social networking sites or surveillance systems. 

These new tools also allow users to make queries in their own language and to receive more precise answers, instead of ranked lists of documents provided by current search engines. 

Music search engine 

In the nascent field of sound and music computing, exciting applications are emerging, particularly in health care and edutainment. NUS researchers have created a suite of technologies to automatically analyse music content for characteristics, such as tempo, cultural and beat strength features. 

They have applied it to a search engine that helps therapists locate music pieces from free online music databases.These pieces have applications in music therapy to help Parkinson’s disease patients. 

Geo-referenced video search 

Online street maps such as GoThere.com and Google Maps have a ‘Street View’ function that provides panoramic but static views from various positions along the street. 

Researchers have enhanced this experience, through developing a Web-based search engine that allows users to search for geo-referenced videos, by specifying a location of interest on a map. 

To allow for community-driven data contributions to the search engine, the researchers have developed tools that allow individuals to easily capture the video and its associated location information. 

Multimedia analysis 

In security surveillance, the proper analysis of ‘live’ data (for example, video feeds from a network of surveillance cameras) is a challenging task, especially if the data has to be acted upon rapidly. 

To help surveillance system designers, researchers have developed a novel technique that allows the designer to explore various configurations of sensor types, sensor numbers and their placements, in order to meet specific surveillance goals. They have also developed a robust digital signature scheme for images and a unique authentication approach for videos, to ensure that individual rights of content owners, distributors and users are safeguarded. 

Document analysis 

Advances in printing, scanning, photocopying and digital photography have led to a proliferation of document images in our daily lives.Such document images are easily understood by humans, but pose great challenges to computers.Document analysis aims to examine document images, in order to acquire an understanding of their contents.NUS research in this area includes the extraction and recognition of text, graphics and symbols in document images. 

Natural language processing 

Today’s Internet users are increasingly using online machine translation systems to gain access to information available in foreign languages. NUS researchers have created technologies that will enable future translation systems to determine the correct meaning of a word in a specific context. Other methods have been created to improve the quality of translating resource-poor languages, which are less common languages for which few digital resources are available.

Extreme live media search

To advance the frontiers of search technologies, NUS recently partnered with China’s Tsinghua University to set up the NUS-Tsinghua Extreme Search Centre, funded by the Media Development Authority. 

This centre aims to extract situational information and events from live media sources arising from thousands of sensors, mobile phones, live forums and tweets to help realise a smart living environment for all users. The centre will also explore commercialisation of these technologies to companies in Singapore, China and elsewhere.Asha Srinivasan is a manager at the National University of Singapore Industry Liaison Office (ILO).