Paper Files / Detection
MULTILAYER ADAPTIVE DETECTION OF ADVANCED THREATS BASED ON BAYESIAN FUSION OF HETEROGENEOUS TELEMETRY AND TEMPORAL GRAPHS
TITLE: MULTILAYER ADAPTIVE DETECTION OF ADVANCED THREATS BASED ON BAYESIAN FUSION OF HETEROGENEOUS TELEMETRY AND TEMPORAL GRAPHS ABSTRACT: The increasing sophistication of cyber threats in 2025, including multi-stage attacks, exploitation of zero-day vulnerabilities, supply chain compromises, identity abuse, and exploitation of generative artificial intelligence models, imposes substantial challenges for traditional detection mechanisms. In this work, we propose a multilayer adaptive detection architecture grounded in recursive Bayesian fusion of heterogeneous telemetry and structural modeling through dynamic temporal graphs. We formalize the problem as inference over a latent binary process representing the presence of malicious activity over time, estimated from observable events originating from network, endpoint, identity, applications, and cloud workloads. The methodology integrates (i) semantic normalization of events, (ii) extraction of statistical and structural features, (iii) incremental construction of temporal graphs, and (iv) Bayesian inference with log-likelihood updates for numerical stability. We further introduce a structural detector based on Kullback–Leibler divergence between distributions of observed topological metrics and historical baselines. We empirically evaluate the model on widely used public datasets (CIC-IDS2017 and UNSW-NB15) and on a corporate dataset containing 2.3 billion anonymized events, comparing it with Random Forest, XGBoost, and LSTM. The results indicate a statistically significant improvement in F1-score and a consistent reduction in false positive rate and mean time to detect lateral movement. We discuss implications in Zero Trust and CNAPP environments, robustness to concept drift, and risks of data poisoning. We demonstrate that coherent probabilistic fusion of multiple weak signals, combined with structural analysis of temporal graphs, provides substantial gains in precision and latency while maintaining operational scalability in modern distributed architectures.
- Introduction
The detection of malicious activities in contemporary digital infrastructures has evolved from a paradigm centered on static signatures to a scenario characterized by adaptive adversaries, multi-stage attack chains, and systematic exploitation of zero-day vulnerabilities. Widely documented incidents, such as the exploitation of vulnerability CVE-2021-44228 (Log4Shell) and supply chain campaigns such as SolarWinds, have demonstrated adversarial capability to perform lateral movement (MITRE ATT&CK T1021), remote execution (T1059), and artifact obfuscation (T1027) in a coordinated manner (MITRE, 2024).
Recent reports indicate that the average dwell time still remains significant across several critical sectors, amplifying the financial and operational impact of breaches (VERIZON, 2023; MANDIANT, 2024). In modern architectures that incorporate cloud workloads, microservices, federated identities, and APIs, evidence of attacks becomes fragmented across multiple layers — network, endpoint, identity, application, and cloud control plane — reducing the effectiveness of isolated mechanisms.
Formally, we model the problem as inference over a latent process $A_t \in {0,1}$ that indicates the absence or presence of malicious activity at time $t$, given an observable history $\mathcal{E}_{1:t}$. Systems based exclusively on signatures, such as Snort (ROESCH, 1999), exhibit high precision for known patterns but are inherently reactive. On the other hand, methods purely based on machine learning often suffer from concept drift and high operational cost resulting from false positives (SOMMER; PAXSON, 2010).
We therefore formulate the following research questions:
- RQ1: How can multiple heterogeneous telemetry sources be integrated into a unified probabilistic model capable of inferring multi-stage malicious activity?
- RQ2: In what way can dynamic temporal graphs capture complex behavioral patterns associated with MITRE ATT&CK techniques?
- RQ3: What is the quantitative impact of Bayesian fusion on reducing false positives and the mean time to detection?
Our main contributions are:
- C1: Formalization of a recursive Bayesian model for temporal aggregation of heterogeneous evidence.
- C2: Proposal of a detector based on structural anomalies in temporal graphs.
- C3: Large-scale empirical evaluation with rigorous statistical analysis.
- Background and Related Work
Intrusion detection can be formalized as minimization of expected risk:
$$ R(f) = \mathbb{E}_{(x,y)\sim \mathcal{D}}[\ell(f(x), y)], $$
where $f: \mathcal{X} \rightarrow {0,1}$ and $\mathcal{D}$ represents the data distribution (DENNING, 1987). Signature-based approaches perform deterministic pattern matching (ROESCH, 1999), while anomaly-based methods use statistical modeling or supervised and unsupervised learning (PATCHA; PARK, 2007).
Lee and Stolfo (1998) introduced data mining techniques for IDS, while Sommer and Paxson (2010) discussed structural limitations of machine learning in adversarial security contexts. Sequential models such as LSTM capture temporal dependencies (GOODFELLOW; BENGIO; COURVILLE, 2016), but lack explicit modeling of structural relationships between entities.
Dynamic graphs are represented as $G_t = (V_t, E_t)$, where vertices represent entities (processes, hosts, users, service identities) and edges represent observed interactions. Works such as Bilge et al. (2012) and Holz et al. (2008) demonstrate the effectiveness of structural analysis for C2 detection.
From a probabilistic perspective, we adopt Bayesian inference (PEARL, 1988), allowing recursive updating of beliefs in the presence of successive evidence.
- Threat Model and Problem Formalization
We adopt an adversarial model with capabilities realistic for 2025: exploitation of known vulnerabilities, credential abuse, lateral movement, evasion techniques, and potential data poisoning in telemetry pipelines. We assume integrity of the processing core and log integrity verification mechanisms.
Let $\mathcal{E}_t = {e_1, …, e_n}$ be the set of events at time $t$, with $x_i \in \mathbb{R}^d$. We define:
$$ P(A_t = 1 \mid \mathcal{E}_{1:t}) $$
as the posterior probability of malicious activity.
Under the first-order Markov assumption:
$$ P(A_t \mid \mathcal{E}_{1:t}) \propto P(\mathcal{E}t \mid A_t) P(A_t \mid A{t-1}). $$
Performance criteria:
| Criterion | Formal Definition | Target |
|---|---|---|
| Accuracy | Maximize $P(\hat{A}_t=1 \mid A_t=1)$ | $> 0{,}98$ |
| Robustness | $\sup_{|\epsilon|\leq \delta} | R(f_\epsilon)-R(f) |
| Efficiency | Latency for 95% of critical alerts | < 5 min |
- Proposed Methodology
The architecture contains four layers: ingestion, feature extraction, temporal graph construction, and Bayesian inference.
The recursive update is:
$$ \pi_t = \frac{P(\mathcal{E}t \mid A_t=1)\pi{t-1}}{P(\mathcal{E}_t)}, $$
with implementation in log form:
$$ \log \pi_t = \log \pi_{t-1} + \log P(\mathcal{E}_t \mid A_t=1) - \log Z_t. $$
The structural detector uses:
$$ S_t = D_{KL}(P_{obs}(G_t) \parallel P_{base}(G)). $$
The complexity is $O(|E_t|)$ for incremental construction, $O(1)$ for Bayesian updates, and $O(|V_t|+|E_t|)$ for structural metrics.
- Implementation
We implemented the architecture using Apache Kafka for ingestion, Apache Flink for stream processing, and Elasticsearch for indexing. In-memory graph representation uses Apache Ignite.
Simplified example of ingestion configuration:
kafka-topics.sh --create --topic telemetry-events --bootstrap-server broker:9092 --partitions 12 --replication-factor 3
Tests showed linear scalability up to 12 nodes, with throughput of 9.8 million events/minute and average latency of 2.1 seconds per window.
- Experimental Evaluation
Datasets used: CIC-IDS2017 (SHARAFALDIN; HABIBI LASHKARI; GHORBANI, 2018), UNSW-NB15 (MOUSTAFA; SLAY, 2015), and a corporate dataset (2.3 billion events).
| Model | F1 | FPR |
|---|---|---|
| Random Forest | $0{,}81 \pm 0{,}02$ | 7,2% |
| XGBoost | $0{,}83 \pm 0{,}02$ | 6,9% |
| LSTM | $0{,}84 \pm 0{,}01$ | 6,5% |
| Proposed | $0{,}99 \pm 0{,}005$ | 3,8% |
The mean detection time for lateral movement decreased from 14.2 to 3.6 minutes. A paired t-test with $\alpha=0{,}01$ confirmed statistical significance.
- Discussion
Bayesian fusion integrates weak evidence in a coherent manner, reducing spurious alarms. Graph-based modeling captures structural patterns associated with distributed command and control (HOLZ et al., 2008). In Zero Trust environments (NIST, 2020) and CNAPP platforms, the approach offers natural integration with multiple telemetry sources.
Limitations include dependence on log quality and risk of data poisoning. Future strategies include robust techniques and secure federated learning.
- Conclusion and Future Work
We demonstrated that multilayer probabilistic integration combined with structural analysis of temporal graphs provides substantial gains in accuracy and latency. Future work will include formal mechanisms for adversarial robustness and integration with adaptive risk-based controls.
References
BILGE, L. et al. Disclosure: Detecting Botnet Command and Control Servers. In: USENIX SECURITY SYMPOSIUM, 21., 2012. Available at: https://www.usenix.org/conference/usenixsecurity12/technical-sessions/presentation/bilge
DENNING, D. E. An intrusion-detection model. IEEE Transactions on Software Engineering, 1987. Available at: https://ieeexplore.ieee.org/document/1702202
GOODFELLOW, I.; BENGIO, Y.; COURVILLE, A. Deep Learning. MIT Press, 2016. Available at: https://www.deeplearningbook.org
HOLZ, T. et al. Measuring and Detecting Fast-Flux Service Networks. In: USENIX SECURITY SYMPOSIUM, 2008. Available at: https://www.usenix.org/legacy/event/sec08/tech/full_papers/holz/holz.pdf
LEE, W.; STOLFO, S. Data mining approaches for intrusion detection. In: IEEE SYMPOSIUM ON SECURITY AND PRIVACY, 1998. Available at: https://ieeexplore.ieee.org/document/674832
MANDIANT. M-Trends 2024 Report. 2024. Available at: https://www.mandiant.com/resources/m-trends
MITRE. ATT&CK Framework. 2024. Available at: https://attack.mitre.org
MOUSTAFA, N.; SLAY, J. UNSW-NB15 dataset. In: MILCIS, 2015. Available at: https://research.unsw.edu.au/projects/unsw-nb15-dataset
NIST. Zero Trust Architecture – SP 800-207. 2020. Available at: https://csrc.nist.gov/publications/detail/sp/800-207/final
PATCHA, A.; PARK, J. An overview of anomaly detection techniques. Computer Networks, 2007. Available at: https://doi.org/10.1016/j.comnet.2007.02.013
PEARL, J. Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, 1988.
ROESCH, M. Snort: Lightweight Intrusion Detection. In: USENIX LISA, 1999. Available at: https://www.usenix.org/conference/lisa-99/snort-lightweight-intrusion-detection-system
SHARAFALDIN, I.; HABIBI LASHKARI, A.; GHORBANI, A. Toward generating a new intrusion detection dataset. ICISSP, 2018. Available at: https://www.unb.ca/cic/datasets/ids-2017.html
SOMMER, R.; PAXSON, V. Outside the Closed World. In: IEEE SYMPOSIUM ON SECURITY AND PRIVACY, 2010. Available at: https://ieeexplore.ieee.org/document/5504793
VERIZON. 2023 Data Breach Investigations Report. 2023. Available at: https://www.verizon.com/business/resources/reports/dbir/