Intelligent problem solving in process control of an event filter cluster for a particle physics experiment

Modern particle physics experiments observing collisions of particle beams generate large amounts of data. Complex trigger and data acquisition systems are built to select on-line the most interesting events and write them to persistent storage. The final stage of this selection process nowadays often happens on large computer clusters. The stable and reliable operation of such event filter clusters is critical for the success of these experiments. Operating the event filter cluster must ensure dead time free processing of large amount of data, requiring 24-h continuous status monitoring of each processing node, and fast detection and problem solving. Ideally, problems should be recognized before they deteriorate the system performance. The process control of the event filter cluster is performed exclusively by a human operator, placing high demands difficult to accomplish. In this paper, a hybrid system based on expert systems technology and statistical tools and methods is proposed to address this issue. The system is built upon a scalable modular architecture and a design overview is given. The proposed hybrid system is designed and tested in a real environment, with an event filter cluster prototype based on the architecture of the Compact Muon Solenoid experiment at CERN. The system test results with an analysis are provided. Finally, the future possibilities are discussed.

► Expert systems technology and statistical tools and methods applied in process control.
► Monitoring and detection of known problems and unfamiliar system behavior.
► Prediction of system performance deterioration.
► Decision support and recovery procedures.
► Application of the proposed problem solving architecture to a wider range of systems.