High Performance Computing (HPC), which is also known as supercomputing, refers to the computing systems with massive computational power that can solve complex problems [1]. Weather forecast, simulation of nuclear reaction or car crashes, which requires extremely high-speed computation, are the typical use cases for HPC systems. Supercomputers are also actively involved in the quest of COVID19, for instance, analysis of the virus genetic combinations out of thousands of samples [2].
Artificial Intelligence (AI) is playing an important role in every corner of our daily lives, such as smart cities, credit decision-making for banks. The EU-funded research project CYBELE adopts AI in Agriculture to assist in optimising agricultural and livestock production and minimising the wastes and costs, such as prediction of good sea fishing spots, crop yield prediction, monitor pig heath to improve meat quality. Analysis of such massive amount of data, e.g. satellite images of crops and the pig monitoring videos, not only require big data storage but also high processing speed. HPC systems that often incorporate thousands of cores have therefore paved the way for the development of such computation-intensive and data-demanding AI applications. Users can run hundreds of simulations simultaneously on HPC systems.
AI applications still face chanllenges when moving to HPC systems, such as portability and scalability. AI applications often need a complex stack of software and dependencies. Users often hold limited privileges to customise their application execution environments on HPC systems, as it can alter working environment of the other existing users and even raise system-wide security concerns. In the CYBELE project, the issues are addressed by application containerisation. More specifically, the AI applications are encapsulated in the containers that can provide hosting environments, such as software dependencies and configurations, which enables direct application deployments on HPC clusters. More details are presented in [3,4].
The HLRS supercomputer (Copyright: Ben Derzian for HLRS)
[3] Naweiluo Zhou, Yiannis Georgiou, Li Zhong, Huan Zhou, Marcin Pospieszny. Container Orchestration on HPC Systems. 2020 IEEE International Conference on Cloud Computing (CLOUD).
[4] Yiannis Georgiou, Naweiluo Zhou, Li Zhong, Dennis Hoppe, Marcin Pospieszny, Nikela Papadopoulou, Kostis Nikas, Orestis Lagkas Nikolos, Pavlo Kranas, Sophia Karagiorgou, Eric Pascolo, Michael Mercier, Pedro Velho. Converging HPC, Big Data and Cloud technologies for precision agriculture data analytics on supercomputers. 15th Workshop on Virtualization in High-Performance Cloud Computing (VHPC'20) (2020).
Comments