PUZZLE aims to create a highly accessible cybersecurity, privacy and data protection management framework that enables SMEs & MEs to monitor, forecast, assess and manage their cyber risks through targeted cybersecurity services. To enable heterogeneous information processing, knowledge sharing between SMEs & MEs and extract insights based on advanced analytics, PUZZLE leverages a multi-faceted real-time monitoring framework for collecting health, performance and logging data, accompanied by a Complex Event Processing (CEP) mechanism, formulating a pipeline that is used by the PUZZLE security orchestrator.
Health monitoring is essential to the PUZZLE monitoring subsystem. PUZZLE utilises a set of agents to support real-time health monitoring of the provided services and applications. These agents are based on current state-of-the-art solutions followed by the industry to achieve the project objectives, with a particular focus being placed on Kubernetes-related tools. Health monitoring is performed on two different layers: on the application/service graph layer, as well as on the security layer.
In the context of PUZZLE, a performance monitoring subsystem is employed to collect real-time performance metrics from physical and virtual machines, such as CPU usage, RAM statistics etc. The performance is analysed and potential remediation actions are triggered utilizing a Complex Event Processing engine, namely, Drools.
The security mechanisms developed in PUZZLE require a sophisticated logging and auditing subsystem. PUZZLE builds this subsystem by introducing various programmable log and metadata extractors to its architecture; to this end, log extraction and collection are performed by specialised agents deployed in each virtual machine. Those agents help PUZZLE monitor the behaviour of containers and VMs and assess potential threats. We note that different logging components can be activated or deactivated, while the PUZZLE logging fabric can be easily enhanced with additional agents and tools. By adopting this approach, PUZZLE offers increased flexibility, as the logging mechanism can be tailored to each user’s requirements.
CEP consists of a set of concepts and techniques for processing real-time events and extracting information from event streams as they arrive. PUZZLE uses the Streaming approach, supported by Offline analysis. This choice provides several benefits: On one hand, the streaming approach provides real-time insights into our apps and overall system and increases PUZZLE’s responsiveness in case of malfunctions/failures and security incidents. An inference rule will be evaluated practically in real time, as several streams can be combined together and assessed by the inference engine. On the other hand, streaming is also supported by offline analysis, in which data is indexed and available for future inspection (e.g., using Machine Learning techniques) to analyse incidents and patterns in a non-real-time fashion. The information obtained by the offline analysis is also used to enrich our CEP logic and enhance the PUZZLE security platform.
In Figure 1, some monitoring metrics for all the user applications that are deployed on a Kubernetes cluster are depicted. Upon the database application deployment the corresponding agent automatically discovers it and gathers the available monitoring data.
Author: Uni Systems
