J-TEXT distributed data storage and management system

As fusion experiment goes to steady state and high-performance data acquisition system been developed, traditional storage solutions cannot cope with the needs of fusion experiment data storage. Distributed file systems such as Lustre are being adopted by more and more facilities. The scaling-out of performance and capacity features addressed the main issue of traditional SAN (Storage Area Network) based storage. However, traditional file system based software solutions are still being used with the distributed file system. They lack modern data manage functions and limiting the performance of parallel storage. J-TEXT Cloud Database (JCDB) is a software stack that uses distributed database to provide fusion experiment data storage and management services. It ships with a storage engine powered by Cassandra database. This storage engine is designed for fusion experiment data and provides great performance. Data is divided into chunks when written and stored in a specially designed distributed database across the cluster. It has a MongoDB powered metadata management system which works seamlessly with the storage engine. JCDB is fully modular, handling different data type and metadata management functions are integrated as plugins. Even the storage engine can be changed. Though J-TEXT not supports long pulse experiments, the design of JCDB is aiming to meet long pulse discharge requirements, bring distributed database technology for future fusion devices such as China fusion engineering test reactor (CFETR). JCDB has the benefit of distributed file systems, provides complex metadata manage functions and comes with great performance.