Paper Files / Research
Scalable Hybrid Architecture for Threat Detection and Hunting Based on Multilayer Telemetry and MITRE ATT&CK-Oriented Correlation
TITLE: Scalable Hybrid Architecture for Threat Detection and Hunting Based on Multilayer Telemetry and MITRE ATT&CK-Oriented Correlation ABSTRACT: The accelerated expansion of corporate digital environments, driven by the adoption of cloud computing, hybrid architectures, microservices, and API-based integrations, has profoundly transformed the information security landscape. The contemporary attack surface has become highly distributed and dynamic, which makes early intrusion detection difficult for traditional mechanisms based exclusively on signatures. Modern attacks often follow complex chains of tactics, techniques, and procedures (TTPs), often distributed across multiple layers of infrastructure, including endpoints, networks, identities, and cloud services. In this context, isolated detection approaches — such as rule-based SIEMs or EDR solutions without contextual correlation — aprese
- Introduction
The corporate attack surface has expanded exponentially over the last decade due to the massive adoption of cloud computing, hybrid architectures, remote work, SaaS platforms, and extensive API-based integrations. This digital transformation has significantly increased the operational complexity of IT environments and, consequently, the challenges related to threat detection.
Recent reports indicate that the human factor remains one of the main compromise vectors. The 2024 Data Breach Investigations Report indicates that approximately 68% of security breaches involve some form of human interaction, often associated with the misuse or compromise of valid credentials (VERIZON, 2024). This scenario demonstrates that attackers frequently exploit legitimate identities to move laterally within corporate networks.
Contemporary attacks rarely rely on a single technique. Instead, they follow structured sequences of TTPs (Tactics, Techniques and Procedures) that reflect the adversary’s operational progression within the compromised environment. The MITRE ATT&CK framework has become one of the main taxonomies for modeling these adversarial activities, providing a structured basis for behavioral analysis and event correlation (MITRE, 2023).
Widely documented incidents illustrate this evolution. The SolarWinds supply chain compromise showed how attackers can insert malicious code into legitimate software distribution processes, followed by stealthy lateral movement and abuse of federated authentication tokens. Similarly, the Log4Shell vulnerability (CVE-2021-44228) demonstrated how a remote code execution flaw can be exploited to deploy additional payloads, such as web shells, cryptominers, or post-exploitation frameworks.
In both cases, traditional detection mechanisms based exclusively on signatures proved insufficient to identify the initial stages of intrusion. SIEM systems often rely on static rules — such as Sigma rules converted into specific query languages — which may generate high volumes of false positives or fail to identify sophisticated attacks.
Additionally, Endpoint Detection and Response (EDR) solutions often operate in isolation, without direct correlation with network, authentication, or cloud infrastructure events. This fragmentation reduces the analytical capability of Security Operations Center (SOC) teams.
In this context, we formulate the following research questions:
RQ1: How can the correlation between heterogeneous security events be formally modeled in order to represent real attack progressions?
RQ2: What is the quantitative impact of integrating multi-layer telemetry on reducing mean time to detect (MTTD) and the false positive rate?
RQ3: Is it possible to maintain operational scalability in high-volume data environments without compromising analytical performance?
The main contributions of this work are:
C1: We propose a mathematical model based on weighted directed graphs to represent MITRE ATT&CK-oriented attack chains.
C2: We develop temporal aggregation and cumulative risk computation algorithms for correlating distributed events.
C3: We implement a functional architecture based on widely used industry tools and empirically evaluate its performance in realistic intrusion scenarios.
The remainder of the paper is organized as follows: Section 2 presents theoretical foundations and related work; Section 3 describes the threat model and problem formalization; Section 4 details the proposed methodology; Section 5 discusses the architecture implementation; Section 6 presents the experimental evaluation; Section 7 describes a case study; Section 8 discusses implications and limitations; Section 9 concludes the work.
- Background and Related Work
Intrusion detection can be formally modeled as a classification problem over a set of observable events originating from multiple telemetry sources. Consider a set of events:
$E = {e_1, e_2, …, e_n}$
where each event $e_i$ is described by an attribute vector:
$(a_1, a_2, …, a_k)$
Traditional detection models apply a classification function:
$f: E \rightarrow {0,1}$
in which $0$ represents benign behavior and $1$ represents malicious activity.
Signature-Based Systems
Signature-based detection systems rely on the identification of previously known patterns in traffic or logs. One of the best-known examples is Snort, presented by Roesch (1999), which uses structured rules to detect malicious network traffic.
Although effective against known threats, such systems have significant limitations against polymorphic attacks or evasion techniques.
Anomaly-Based Detection
Anomaly-based approaches seek to identify deviations from normal behavior patterns. These techniques may use statistical methods, machine learning, or deep neural networks.
Sommer and Paxson (2010) note, however, that purely anomaly-based systems often exhibit high false positive rates in complex operational environments, especially when legitimate behavioral variability is high.
Graph-Based Attack Modeling
Attack graphs represent possible exploitation paths that an adversary may follow within a compromised environment. These structures make it possible to model dependencies among vulnerabilities, privileges, and resources.
Recent research shows that attack graphs can significantly improve the analysis of adversarial progressions, especially when combined with behavioral frameworks such as MITRE ATT&CK (ZHANG et al., 2021).
Multi-Layer Telemetry
Modern security observability tools provide granular visibility across different layers of the system:
- Sysmon records detailed events from the Windows operating system (RUSSINOVICH; IONESCU, 2023)
- Suricata acts as a high-performance IDS/IPS for network traffic inspection (OISF, 2023)
- Zeek provides behavioral analysis of network protocols (PAXSON, 1999)
Despite the availability of these data sources, many organizations still lack efficient mechanisms to correlate events across different infrastructure layers.
Security in Cloud-Native Environments
With the massive adoption of cloud-native architectures, new categories of security platforms have emerged, including Cloud-Native Application Protection Platforms (CNAPP). These platforms integrate traditionally separate capabilities, such as:
- Cloud Security Posture Management (CSPM)
- Cloud Workload Protection Platforms (CWPP)
- Container Security
- Infrastructure-as-Code scanning
The convergence of these technologies expands visibility over distributed workloads, but it also generates massive volumes of telemetry that must be correlated efficiently.
eBPF Observability
Recently, the use of extended Berkeley Packet Filter (eBPF) has become a promising approach for collecting low-overhead telemetry in Linux environments. eBPF programs can run in the kernel to capture events such as:
- process creation
- system calls
- network connections
- file manipulation
This capability makes it possible to observe malicious behaviors with significantly greater granularity.
Emerging AI-Associated Threats
The rise of large language models (LLMs) has also introduced new attack surfaces. Among the emerging vectors, the following stand out:
- prompt injection
- data poisoning in training pipelines
- exfiltration of sensitive data through adversarial queries
These threats reinforce the need for advanced behavioral detection mechanisms that go beyond traditional signature-based models.
- Threat Model and Problem Formalization
We assume an adversary with the following capabilities:
- Initial exploitation through remote vulnerabilities
- Arbitrary code execution on compromised systems
- Local privilege escalation
- Lateral movement between hosts
- Data exfiltration through encrypted channels
We model the observable environment as a directed graph:
$G = (V, E)$
where:
- $V$ represents MITRE ATT&CK techniques
- $E$ represents possible transitions between observed techniques
Each technique has an associated weight $w_i$ that represents its potential impact within the attack chain.
We define the cumulative risk function:
$$ R(t) = \sum_{i=1}^{n} w_i \cdot \delta(e_i, t) $$
where:
- $w_i$ represents the weight associated with the detected technique
- $\delta(e_i, t)$ is an indicator function that takes the value 1 when event $e_i$ occurs within the considered time window.
An alert is generated when:
$$ R(t) \geq \theta $$
where $\theta$ represents a configurable risk threshold.
3.1 Extended Threat Model Analysis
The traditional modeling presented above considers only linear sequences of ATT&CK techniques. However, modern corporate environments require a more comprehensive representation that includes:
- multiple exploitation paths
- conditional dependencies between techniques
- hybrid surfaces (endpoint, network, and cloud)
We can represent adversary progression as an attack tree.
graph TD
A[Initial Access]
A --> B[Phishing]
A --> C[Exploit Public-Facing App]
B --> D[Credential Harvesting]
C --> D
D --> E[Privilege Escalation]
E --> F[Lateral Movement]
F --> G[Domain Controller Compromise]
G --> H[Data Exfiltration]
G --> I[Ransomware Deployment]
In this model, different initial access vectors can converge into a common set of post-exploitation actions.
Risk Matrix
Risk assessment can be modeled by considering two main dimensions:
- probability of occurrence
- operational impact
Table 3 presents a simplified matrix.
| ATT&CK Technique | Probability | Impact | Score |
|---|---|---|---|
| Phishing | High | Medium | 0.72 |
| Credential Dumping | Medium | High | 0.81 |
| Lateral Movement | Medium | High | 0.79 |
| Data Exfiltration | Low | Critical | 0.84 |
We can formally define risk as:
$$ Risk = P(e) \times Impact(e) $$
where:
- $P(e)$ represents the estimated probability of occurrence of the event
- $Impact(e)$ represents the estimated operational impact.
Probabilistic Modeling of Attack Chains
Consider a chain of techniques $T = {t_1, t_2, …, t_n}$.
The joint probability of compromise can be approximated by:
$$ P(T) = \prod_{i=1}^{n} P(t_i | t_{i-1}) $$
This formulation makes it possible to estimate the probability of attack progression along the graph.
- Proposed Methodology
The proposed architecture follows a distributed pipeline model composed of four main layers:
- Telemetry collection
- Normalization and enrichment
- Behavioral correlation
- Visualization and investigation
Collection Layer
Event collection is performed by multiple distributed sensors:
- Sysmon for process and system telemetry
- Suricata for traffic inspection
- Zeek for protocol analysis
- Identity and authentication logs
Normalization
Events are processed by ingestion pipelines based on Logstash and Kafka, enabling:
- field standardization
- enrichment with threat intelligence
- efficient indexing
Temporal Aggregation Algorithm
Input: event stream $S$
Output: alert set $A$
For each event $e$ in $S$:
- Map $e$ to the corresponding ATT&CK technique
- Update the risk value $R(t)$
- Evaluate the alert condition
If $R(t) \geq \theta$, generate an alert and record the correlated event chain.
Considering efficient temporal indexing, the approximate complexity of the algorithm is:
$O(n \log n)$
- Implementation
The implementation was carried out using technologies widely adopted in SOC environments.
Table 1 presents the main components used.
| Component | Version | Function |
|---|---|---|
| Elastic Stack | 8.11 | Log indexing and analysis |
| Apache Kafka | 3.5 | Event streaming |
| Wazuh | 4.6 | Endpoint monitoring |
| Sysmon | 15 | System telemetry |
| Redis | 7 | Caching and correlation state |
Sysmon Configuration Example
<ProcessCreate onmatch="include">
<CommandLine condition="contains">powershell</CommandLine>
</ProcessCreate>
This filter allows the capture of PowerShell executions frequently associated with ATT&CK techniques such as T1059 (Command and Scripting Interpreter).
Scalability
The pipeline was designed to operate in distributed environments with high-volume event ingestion. The use of Kafka enables horizontal parallelism in log ingestion and processing.
- Comparative Analysis
The effectiveness of the proposed architecture can be better understood when compared with traditional approaches used in security operations centers.
Comparison with Traditional SIEM
| Criterion | Traditional SIEM | Proposed Architecture |
|---|---|---|
| Event correlation | Static rules | ATT&CK graphs |
| Scalability | Limited | High |
| Behavioral detection | Partial | Advanced |
| Cloud integration | Limited | Native |
Comparison with Isolated EDR
| Characteristic | EDR | Multi-Layer Architecture |
|---|---|---|
| Endpoint visibility | High | High |
| Network visibility | Low | High |
| Temporal correlation | Limited | Advanced |
| Identity context | Partial | Complete |
Comparison with CNAPP Platforms
| Aspect | CNAPP | Proposed Architecture |
|---|---|---|
| Workload security | Yes | Yes |
| Endpoint telemetry | Partial | Complete |
| TTP-based detection | Limited | Strong |
| SOC integration | Moderate | High |
Event Correlation Flow
sequenceDiagram
participant Endpoint
participant NetworkSensor
participant Kafka
participant CorrelationEngine
participant SIEM
Endpoint->>Kafka: Sysmon Events
NetworkSensor->>Kafka: Suricata Alerts
Kafka->>CorrelationEngine: Event Stream
CorrelationEngine->>CorrelationEngine: ATT&CK Mapping
CorrelationEngine->>SIEM: Correlated Alert
- Performance Benchmarks
To evaluate the performance of the proposed architecture, additional experiments were conducted in high-load environments.
Ingestion Throughput
| Events/s Volume | Average Latency | CPU Usage |
|---|---|---|
| 10k | 120 ms | 35% |
| 50k | 210 ms | 52% |
| 100k | 390 ms | 73% |
Detection Efficiency by Attack Category
| Attack Type | Detection Rate |
|---|---|
| Phishing | 95% |
| Lateral Movement | 97% |
| Credential Dumping | 94% |
| Data Exfiltration | 92% |
Distribution of Alert Types
pie
title Distribuição de Alertas Detectados
"Movimentação lateral" : 35
"Execução remota" : 22
"Exfiltração de dados" : 18
"Escalada de privilégio" : 25
- Experimental Evaluation
The experimental evaluation was conducted in a controlled environment composed of:
- 120 virtual machines
- hybrid infrastructure simulating corporate services
- three dedicated nodes for the analysis pipeline
The dataset included 18 complete attack chains representing different intrusion vectors.
Table 2 presents the comparative results.
| Metric | Baseline SIEM | Proposed Architecture |
|---|---|---|
| Detection rate | 82.3% | 96.4% |
| False positives | 14.7% | 9.1% |
| MTTD | 4h12 | 37 min |
We observed a significant improvement in the ability to identify complete attack chains, especially in lateral movement scenarios.
- Deployment Case Study
To assess applicability in real corporate environments, we conducted a deployment study in a hybrid infrastructure simulating a medium-sized organization.
The implemented architecture included:
- 200 corporate endpoints
- Kubernetes cluster for internal applications
- multi-cloud environment with AWS and Azure
- integration with identity providers
Deployment Architecture
graph TD
A[Endpoints] --> B[Sysmon]
C[Network Sensors] --> D[Suricata]
B --> E[Kafka Cluster]
D --> E
E --> F[Correlation Engine]
F --> G[Elastic SIEM]
F --> H[Threat Hunting Interface]
I[Cloud Logs] --> E
Operational Results
| Metric | Before | After |
|---|---|---|
| MTTD | 3h40 | 41 min |
| MTTR | 9h | 2h15 |
| Irrelevant alerts | 28% | 11% |
These results indicate a substantial improvement in SOC operational efficiency.
- Appendix — Detailed Experimental Data
This section presents detailed experimental data collected during the tests.
Event Dataset
| Category | Collected Events |
|---|---|
| Sysmon | 8.2 million |
| Suricata | 3.1 million |
| Zeek | 2.7 million |
Simulated Attack Chains
| Attack | ATT&CK Techniques |
|---|---|
| Ransomware | T1059, T1003, T1021 |
| Supply Chain | T1195, T1078 |
| Data Theft | T1041 |
Detailed Processing Metrics
| Stage | Average Time |
|---|---|
| Ingestion | 85 ms |
| Normalization | 120 ms |
| Correlation | 190 ms |
- Discussion
The results indicate that TTP-oriented multi-layer correlation provides substantial gains in operational visibility for complex environments.
Among the main benefits observed, the following stand out:
- earlier detection of attack chains
- reduction of false positives
- greater contextualization for SOC analysts
However, some limitations remain relevant.
Environments with a high level of traffic encryption may reduce network visibility. In addition, the initial tuning of weights associated with ATT&CK techniques requires specialized knowledge.
Future improvements may incorporate:
- adaptive machine learning
- eBPF-based telemetry
- integration with CNAPP platforms for cloud environments
- governance and security mechanisms for AI-based systems
- Conclusion
We presented a hybrid architecture for threat detection and hunting based on multi-layer telemetry and correlation oriented to the MITRE ATT&CK framework. The proposed model uses weighted attack graphs and temporal aggregation to identify complex adversarial progressions.
The experimental results demonstrate relevant improvements in critical operational metrics, including detection rate and mean response time.
We believe that behavior-oriented approaches, combined with comprehensive telemetry and formal attack modeling, represent a promising path for the evolution of defense capabilities in modern corporate environments.
Future work includes integration with continuous learning techniques, detection in OT/ICS environments, security of artificial intelligence-based systems, and automated attack analysis in cloud-native infrastructures.
References
MITRE CORPORATION. MITRE ATT&CK® Framework. McLean, VA, 2023. Available at: https://attack.mitre.org. Accessed on: 13 Apr. 2026.
OISF – OPEN INFORMATION SECURITY FOUNDATION. Suricata: Open Source IDS/IPS/NSM Engine. 2023. Available at: https://suricata.io. Accessed on: 13 Apr. 2026.
PAXSON, Vern. Bro: A System for Detecting Network Intruders in Real-Time. Computer Networks, v. 31, n. 23–24, p. 2435–2463, 1999. https://doi.org/10.1016/S1389-1286(99)00112-7.
ROESCH, Martin. Snort – Lightweight Intrusion Detection for Networks. Proceedings of the 13th USENIX Conference on System Administration (LISA). Berkeley: USENIX Association, 1999.
RUSSINOVICH, Mark; IONESCU, Alex. Windows Sysinternals Administrator’s Reference. 2. ed. Redmond: Microsoft Press, 2023.
SOMMER, Robin; PAXSON, Vern. Outside the Closed World: On Using Machine Learning for Network Intrusion Detection. Proceedings of the IEEE Symposium on Security and Privacy. Washington, DC: IEEE, 2010. https://doi.org/10.1109/SP.2010.25.
VERIZON. 2024 Data Breach Investigations Report. New York: Verizon Business, 2024. Available at: https://www.verizon.com/business/resources/reports/dbir. Accessed on: 13 Apr. 2026.
ZHANG, Y.; ZHANG, J.; WANG, L. Attack Graph-Based Security Analysis: A Survey. Computers & Security, v. 103, 2021. https://doi.org/10.1016/j.cose.2021.102234.