Ad-hoc DAta Grids Environments (ADAGE)

The project's title is "Efficient Management of Information Resources Over Ad-Hoc Data Grids". It is funded by the DEST-ISL Competitive Grants Programme (Round 10), The aim of the project is to investigate how to efficiently gather, store, retrieve and process ad-hoc data grids from both a manager and an end-user perspective. These techniques can benefit various applications including in e-science and e-research.

Please visit the new ADAGE collaborative platform at http://adage.unsw.edu.au

than click on New User and follow the instructions

Summary

This project is primarily concerned with data grids which is a new term used to refer to huge repositories of data which are distributed across several heterogeneous platforms and systems. In particular, the project deals with ad-hoc data grids which commonly refer to unstructured data (i.e. data that is not formatted according to standards) or “dirty” structured data (i.e. data that still needs cleaning due to errors introduced during data generation). Ad-hoc data is becoming increasingly important in all walks of life. It does not only include data generated by humans but by machines as well. For example, many sensors generate incomplete data, some market feeds also generate data with missing information or gaps etc.

The aim of the project is to investigate how to efficiently gather, store, retrieve and process ad-hoc data grids from both a manager and an end-user perspective. These techniques can benefit various applications including in e-science and e-research (e.g., flexible sharing of data among researchers, analysis of information for security purposes, analysis of sensed data, etc).

The project will involve researching and applying a wide spectrum of ICT technologies in areas as diverse as databases, query-processing, data mining, visualization and grid computing. Work will be focused on a number of applications (provided by a large organization) that involve analyzing important events across several ad-hoc data sources (news, announcements, market data etc.) particularly when it is not possible/practical for users to process all information locally. Examples of such applications include conducting business analysis, auditing and fighting crime. The project will be closely linked to a newly launched European Union funded project (called SORMA) aimed at developing methods, tools and software for efficiently managing grid computing resources.

Benefits

The first benefit of this project is to develop a centre of expertise in Australia about the requirements of ad-hoc data grids and their needs in terms of ICT infrastructure and management software. This knowledge is critical in dealing with a number of important applications such as business or security applications. The second benefit is the provision of a grid-based infrastructure that will act as a showcase for the technologies involved as well be leveraged to provide an operational platform for the storage, handling and high performance processing of huge amounts of ad-hoc data. The third benefit is a set of high level data analysis, modelling and process management techniques and tools intended for both managers and end-users.