Ten Years of Maintaining and Expanding a Microbial Genome and Metagenome Analysis System

Launched in March 2005, the Integrated Microbial Genomes (IMG) system is a comprehensive data management system that supports multidimensional comparative analysis of genomic data. At the core of the IMG system is a data warehouse that contains genome and metagenome datasets sequenced at the Joint Genome Institute or provided by scientific users, as well as public genome datasets available at the National Center for Biotechnology Information Genbank sequence data archive. Genomes and metagenome datasets are processed using IMG's microbial genome and metagenome sequence data processing pipelines and are integrated into the data warehouse using IMG's data integration toolkits. Microbial genome and metagenome application specific data marts and user interfaces provide access to different subsets of IMG's data and analysis toolkits. This review article revisits IMG's original aims, highlights key milestones reached by the system during the past 10 years, and discusses the main challenges faced by a rapidly expanding system, in particular the complexity of maintaining such a system in an academic setting with limited budgets and computing and data management infrastructure.

TrendsThe value of a (meta)genome comparative analysis system depends on its ability to rapidly incorporate new datasets and revise dataset annotations as functional resources and annotation methods evolve.Effective comparative analysis requires a consistent and complete view of biological data, therefore support for reviewing the quality of functional annotations is critical.Comparative analysis results depend on the breadth of available datasets and the strength of correlation of data items that may represent the same underlying biological objects across these datasets, that is, the depth of their integration.Comparative analysis typically involves a composition of steps, whereby the output of an individual step serves as input for the next step. Seamless flow of analysis steps and presentation clarity for the results they generate are critical for analysis efficiency.