← Back to Papers

Paper Files / Detection

Detection in Cybersecurity at Scale in 2026: A Hybrid Telemetry, eBPF, and ML-Driven Framework for High-Fidelity Intrusion Detection

DetectionEN

Detection remains the decisive control plane in modern cybersecurity because prevention is no longer sufficient against supply-chain compromise, cloud-native abuse, identity theft, living-off-the-land tradecraft, and rapidly adapting adversaries. By 2026, defensive programs must operate across endpoints, identities, containers, cloud control planes, SaaS platforms, and AI-enabled workflows while preserving low latency, low false-positive rates, and operational explainability. This paper presents a technically grounded framework for detection that combines deterministic analytics, behavioral correlation, graph-based investigation, and supervised and unsupervised machine learning over heterogeneous telemetry. The proposed architecture integrates endpoint visibility through eBPF and ETW, cloud audit streams, IAM signals, DNS and proxy telemetry, and security orchestration outputs into a unified detection pipeline. We formalize detection as an optimization problem that balances detection probability, expected analyst cost, and adversary adaptation risk under constrained latency and observability. We further describe a production-oriented implementation pattern using Sigma-to-backend translation, rule versioning, feature extraction, entity-centric correlation, and model governance. To ground the design, we evaluate representative detection scenarios including credential theft, PowerShell abuse, container escape precursors, suspicious OAuth consent, and beaconing behavior. The evaluation uses realistic metrics such as precision, recall, mean time to detect, and alert volume per day, showing that hybrid detection consistently outperforms single-source signature or ML-only approaches. A detailed case study on a cloud identity compromise demonstrates how multi-layer telemetry can reveal attacker progression before lateral movement and exfiltration. The paper concludes that the future of detection lies in composable analytics, cryptographically trustworthy telemetry, and human-centered triage pipelines that retain rigor under AI-accelerated adversarial pressure.

1. Introduction

Detection is the central nervous system of cybersecurity because every other control eventually depends on whether hostile behavior is observed, correlated, and acted upon in time. In 2026, this statement is more accurate than ever. Attackers routinely bypass perimeter controls through identity abuse, stolen session tokens, supply-chain compromise, malicious OAuth grants, and abuse of legitimate administration tools. Pure prevention fails against adversaries who can operate inside approved channels, particularly in cloud, SaaS, and hybrid enterprise environments where trust boundaries are dynamic and telemetry is fragmented. As a result, detection has evolved from simple signature matching into a multi-layer discipline that combines endpoint visibility, network analytics, identity-centric correlation, behavioral modeling, and investigation workflows.

The modern detection challenge is not merely to distinguish malicious from benign activity, but to do so under stringent operational constraints. Large organizations generate billions of events per day from endpoints, cloud control planes, IAM systems, CASBs, DNS resolvers, proxies, email gateways, VPNs, OT sensors, and software supply-chain tooling. Analysts cannot inspect all alerts manually, and adversaries actively exploit alert fatigue, blind spots, and inconsistent logging. Therefore, a detection system must optimize across multiple objective functions simultaneously: high recall, tolerable precision, low latency, explainability, and manageable cost. In practical terms, the defender is solving a constrained inference problem over partial, noisy, and adversarially manipulated telemetry.

The 2026 threat landscape further complicates detection. Generative AI has improved phishing realism, multilingual social engineering, and adaptive malware development. Meanwhile, defenders increasingly rely on AI-assisted triage, query generation, and anomaly surfacing. This creates a feedback loop in which both sides accelerate iteration. The result is a faster detection arms race, where static signatures age quickly and heuristics require continuous tuning. In cloud-native environments, the attack surface also includes Kubernetes APIs, service meshes, CI/CD pipelines, ephemeral containers, and secret management systems. Detection must therefore be architecture-aware, identity-aware, and workload-aware, not just packet-aware.

This paper argues that the best defense in 2026 is a hybrid detection fabric. Deterministic rules remain essential for known-bad behaviors, compliance-driven alerts, and high-confidence indicators. Machine learning contributes value in clustering, sequence modeling, and anomaly surfacing, especially where adversaries mutate frequently. Graph analytics links entities across time and infrastructure. Endpoint telemetry, especially via eBPF on Linux and ETW on Windows, provides process, file, socket, and syscall-level fidelity that was previously difficult to obtain at scale. When these elements are combined within a governed pipeline, detection becomes more resilient and more operationally useful.

We make four contributions. First, we formalize detection as a multi-objective optimization problem under constraints on visibility, latency, and analyst capacity. Second, we propose a hybrid architecture that fuses deterministic, statistical, and graph-based analytics over heterogeneous telemetry. Third, we describe an implementation blueprint that maps readily to real-world SOC tooling such as Sigma, YARA, Zeek, Suricata, Elastic Security, Microsoft Sentinel, Splunk ES, OpenSearch, CrowdStrike, and Falco. Fourth, we evaluate the approach using representative attack scenarios and operational metrics, showing why layered detection is superior to isolated controls.

The remainder of the paper is structured as follows. Section 2 reviews prior work and foundational concepts. Section 3 defines the threat model and formalizes the problem. Section 4 presents the proposed methodology. Section 5 details implementation. Section 6 reports experimental evaluation. Section 7 gives a cloud-identity compromise case study. Section 8 discusses limitations, adversarial adaptation, and deployment trade-offs. Section 9 concludes with future work.

2. Background and Related Work

Detection as a discipline has deep roots in intrusion detection systems, where the first generation of tools relied heavily on static signatures and protocol heuristics. Classic NIDS solutions such as Snort and Suricata established the practicality of rule-based network inspection, while host-based systems evolved to monitor process creation, file integrity, registry changes, and syscall patterns. Yet the architectural assumption of a relatively stable network perimeter has eroded. The rise of mobile work, remote access, SaaS, and cloud services has made a single network chokepoint insufficient. In parallel, attacker tradecraft has shifted from noisy exploitation toward stealth, credential theft, and legitimate tooling abuse, which are harder to flag with packet signatures alone.

A major conceptual advance in the last decade has been the mapping of detections to attacker behavior frameworks. The MITRE ATT&CK knowledge base has become the lingua franca for describing tactics, techniques, and procedures, allowing defenders to reason about coverage across initial access, execution, persistence, privilege escalation, defense evasion, lateral movement, and exfiltration (MITRE, 2024). Detection engineering teams now commonly organize rules by technique identifiers and correlate them to telemetry sources, enabling both coverage analysis and purple-team validation. ATT&CK-aligned program design also supports measurable improvements because one can ask not only whether alerts fire, but whether they meaningfully cover the techniques most likely to be used in the environment.

The second major shift has been the maturation of structured detection content. Sigma provides a vendor-neutral rule format for log detections, enabling authoring once and translation into backend-specific query languages. YARA remains dominant for file and memory scanning, particularly in malware analysis and hunting workflows. These standards promote maintainability and portability, but they are not sufficient by themselves because they encode known patterns rather than emergent behavior. Consequently, modern programs frequently combine rule repositories with behavioral correlation engines and sequence-aware detection logic. This is especially relevant for living-off-the-land attacks, where a chain of otherwise benign commands becomes suspicious only in aggregate.

Telemetry collection has also changed materially. On Linux, eBPF has become a mainstream mechanism for low-overhead kernel-level observability and security enforcement. Tools such as Falco, Tetragon, and Cilium-based integrations leverage eBPF to expose process, network, file, and capability events with high fidelity. On Windows, ETW and AMSI continue to provide critical visibility into process activity, scripting, and security-relevant events. Cloud providers have converged on rich audit streams: AWS CloudTrail, Azure Activity Logs and Entra ID sign-in logs, Google Cloud Audit Logs, and SaaS audit APIs offer identity and control-plane observations that were unavailable in traditional enterprise networks. The challenge is not a lack of data, but the integration of disparate sources into coherent detections.

Machine learning has had a more mixed record. Early anomaly detection approaches often produced high false-positive rates and were difficult to operationalize. Nonetheless, ML has found durable roles in clustering similar alerts, detecting rare sequences, scoring outliers, and prioritizing triage. In 2026, the strongest systems typically do not replace rules with ML; rather, they use ML to augment rules, reduce noise, and uncover weak signals. This distinction matters because adversaries can poison or evade purely statistical models, while deterministic detections can still provide stable coverage for known patterns. The literature increasingly supports hybrid strategies that blend human-curated knowledge with adaptive scoring.

Graph-based detection has also gained prominence. By modeling users, hosts, IPs, containers, services, and tokens as entities, defenders can uncover relationships that are invisible in isolated logs. Graph analytics helps identify multi-stage intrusion paths, privilege propagation, and shared infrastructure across campaigns. In cloud environments, graph context is especially powerful because identity relationships often define effective access more than network topology does. For example, an unusual token grant followed by a service principal change and then a data export may be individually plausible but collectively indicative of compromise. Graph methods make these sequences easier to reason about and investigate.

Table 1 summarizes major detection paradigms and their operational trade-offs.

Paradigm Primary telemetry Strengths Weaknesses Typical use
Signature/rule-based Logs, packets, files High precision for known patterns, explainable Poor novelty coverage, maintenance burden Malware families, TTP-specific alerts
Behavioral correlation Multi-source logs Captures attacker chains, reduces single-event ambiguity Requires normalized telemetry, more engineering Lateral movement, identity abuse
ML anomaly detection Features over events Useful for rare or evolving behavior False positives, drift, poisoning risk Beaconing, outlier sessions
Graph analytics Entities and relationships Strong contextualization, campaign linking Complex data model, performance cost Cloud identity and lateral movement
eBPF/ETW host telemetry Kernel and process events High-fidelity endpoint visibility Platform-specific tuning, volume LOLBIN abuse, fileless behavior

The literature on detection engineering has also recognized the necessity of operating with adversarial awareness. Adversaries can delay execution, vary command-line parameters, inject noise, abuse signed binaries, and exploit logging gaps. Accordingly, robust detection must be evaluated not only on average precision and recall, but on resilience to mutation and coverage decay. This mirrors broader security measurements where success is defined against an adaptive attacker rather than a static benchmark. In practice, a detection content lifecycle now includes design, validation, deployment, canary testing, feedback, tuning, and retirement.

Another important development is the rise of cloud-native and identity-centric detection. Traditional network IDS sees only part of the picture when the critical action happens inside managed services, API calls, or OAuth flows. For instance, a malicious app consent event in Microsoft Entra ID may not trigger a network alarm, yet it can provide persistent access with legitimate API semantics. Similarly, Kubernetes compromise often manifests as unusual kubectl usage, token theft, or pod-to-cloud metadata abuse rather than overt exploit signatures. Detection therefore must follow the control plane and the identity plane, not just the traffic plane.

Table 2 maps representative telemetry sources to actionable detection surfaces.

Telemetry source Example signals Detection value Common blind spot
Windows ETW / Sysmon Process creation, image loads, network connections LOLBIN abuse, persistence, injection hints Kernel-level stealth if not instrumented
Linux eBPF / auditd execve, connect, open, capability changes Container escape precursors, malware behavior Userland-only abstraction gaps
Cloud audit logs API calls, role changes, token grants Privilege escalation, suspicious automation Delayed ingestion, schema variance
DNS / proxy logs Domain patterns, SNI, HTTP metadata Beaconing, C2 resolution, exfiltration Encrypted payload opacity
IAM / IdP logs Sign-ins, MFA events, conditional access Account takeover, session abuse False legitimacy of approved access

The best current practice is therefore not to choose one detection modality but to orchestrate many. Mature programs pair ATT&CK coverage analysis with telemetry engineering and content governance. They also track detection quality over time because model drift, infrastructure changes, and attacker adaptation all degrade performance. This paper builds on these observations and proposes a formal, implementable detection architecture suitable for 2026 enterprise conditions.

3. Threat Model and Problem Formalization

We consider an enterprise defender operating across endpoints, cloud workloads, SaaS platforms, and identity providers. The attacker is adaptive, can obtain initial access through phishing, credential stuffing, token theft, supply-chain compromise, or exploitation of exposed services, and can use legitimate administrative tools once inside the environment. We assume the attacker may attempt to evade detection by varying timing, spreading actions across accounts, using signed binaries, encrypting payloads, living off the land, or blending into normal cloud control-plane activity. We also assume the attacker can observe some defensive behavior, such as delayed response, quarantined accounts, or blocked domains, and can modify tactics accordingly.

Our security objective is not perfect classification of every event, which is impossible under partial observability, but timely detection of malicious campaigns before they achieve critical impact. We define the environment as a stream of events $E = {e_1, e_2, \dots, e_n}$ collected from heterogeneous sources. Each event has a timestamp, entity bindings, feature vector, source confidence, and provenance. A detector is a function $f(E_t) \rightarrow A_t$ that maps a rolling time window to a set of alerts or scores. Let $y \in {0,1}$ denote ground-truth maliciousness for an event, session, or campaign segment. The goal is to maximize detection utility while minimizing investigation cost.

A useful formalization is the constrained optimization problem

$$ \max_f ; \mathbb{E}[U(f)] = \alpha \cdot \text{TPR} - \beta \cdot \text{FPR} - \gamma \cdot C_{\text{analyst}} - \delta \cdot L_{\text{detect}} $$

subject to telemetry availability, storage limits, policy constraints, and maximum acceptable latency. Here $\text{TPR}$ is true positive rate, $\text{FPR}$ is false positive rate, $C_{\text{analyst}}$ models human triage cost, and $L_{\text{detect}}$ is detection latency. This captures an operational reality: a detector that is theoretically strong but overwhelms analysts is functionally weak. Similarly, a detector that triggers only after exfiltration is not enough, even if it is statistically accurate.

We model telemetry as partially observed and noisy. Let $x_i$ be the feature vector derived from event $e_i$. The observed data $X$ may be missing fields or suffer ingestion delay. We define an entity graph $G_t = (V_t, R_t)$, where vertices represent entities such as users, hosts, pods, tokens, IPs, domains, and services; edges represent observed relations such as login, process parent-child, API call, DNS resolution, or network connection. Many attacks are easier to detect on $G_t$ than on isolated events because maliciousness often emerges from relational structure rather than a single feature.

A campaign can be represented as a path $p = (v_1, v_2, \dots, v_k)$ in the entity graph with temporal ordering. Detection then becomes the problem of identifying paths whose feature sequence or subgraph pattern has low likelihood under benign behavior. For sequence models, we can define a score

$$ s(p) = \sum_{i=1}^{k} w_i \cdot \phi(x_i) + \sum_{(i,j)\in R} \lambda_{ij} \cdot \psi(v_i, v_j) $$

where $\phi$ encodes event-level evidence and $\psi$ encodes relational evidence. A path is flagged if $s(p) \geq \tau$, where $\tau$ is tuned to the environment’s tolerance for false positives. In practice, $\tau$ may be dynamically adjusted based on business criticality, incident volume, or threat level.

Table 3 summarizes key threat assumptions and operational implications.

Threat assumption Operational consequence Detection implication
Attacker uses valid credentials Network-only alerts are insufficient Identity and session telemetry required
Attacker mutates commands Static signatures decay quickly Behavioral and sequence detection needed
Attacker operates in cloud APIs Endpoint visibility is incomplete Control-plane logs must be ingested
Telemetry is delayed or incomplete Real-time certainty is impossible Correlation windows and retries needed
Adversary adapts to alerts One-shot rules will be bypassed Continuous tuning and red-teaming required

We also distinguish between event detection and campaign detection. Event detection identifies individual suspicious actions, such as a suspicious PowerShell invocation. Campaign detection stitches multiple actions into a coherent intrusion narrative, such as phishing, token theft, mailbox rule creation, and mass download. Campaign detection is more valuable because it lowers analyst burden and improves response timing, but it requires richer data and more complex correlation logic. Formally, if $I$ is the set of incidents and $c \subseteq E$ is a campaign cluster, then the objective is to maximize $\Pr(c \text{ is malicious} \mid E)$ while minimizing fragmentation of a single intrusion into many disconnected alerts.

The problem is further constrained by adversarial drift. If the defender deploys a high-confidence rule, the adversary can shift one variable in the sequence to evade it. This means detection systems should be evaluated under mutation tests. A robust detector should preserve a significant fraction of its score under behavior-preserving transformations such as argument reordering, renamed binaries, delayed execution, or alternate LOLBIN selection. Let $s_0$ be the baseline score and $s_m$ the score after mutation; robustness can be approximated by

$$ R = \frac{s_m}{s_0} $$

averaged across a mutation set. Higher robustness indicates less sensitivity to superficial evasion, though overly broad detectors may increase false positives. The design challenge is therefore to balance sensitivity, specificity, and mutation resilience.

4. Proposed Methodology

Our proposed methodology is a layered detection fabric organized around telemetry normalization, entity-centric correlation, hybrid analytics, and continuous validation. The architecture is designed for real-world SOCs that operate across endpoint, identity, cloud, and network domains. It assumes that no single telemetry source is authoritative; instead, confidence emerges from convergence across independent observations. This is particularly important in 2026, when attackers frequently abuse legitimate channels and when AI-assisted phishing and social engineering can create highly plausible but fraudulent workflows.

4.1 Architecture Overview

The pipeline comprises five stages: collection, normalization, enrichment, detection, and response. Collection ingests events from endpoints, cloud APIs, identity logs, DNS, proxy, email, CI/CD, and container platforms. Normalization maps heterogeneous records into a canonical schema, such as ECS, OCSF, or an internal event model. Enrichment adds asset criticality, user role, threat intelligence, geo/IP reputation, and ATT&CK technique tags. Detection applies a portfolio of methods: deterministic rules, sequence and threshold logic, anomaly scores, and graph-based correlation. Response routes high-confidence incidents to SOAR playbooks, case management, or automated containment.

The crucial design principle is that no detection method is used alone. Deterministic rules provide explainable anchors. Machine learning provides adaptive ranking and anomaly surfacing. Graph logic provides campaign assembly. Human analysts provide contextual judgment. This division of labor is essential because operational environments are too dynamic for any one approach to dominate.

4.2 Deterministic Content Layer

Deterministic detection remains the backbone of many mature SOCs because it is interpretable and easy to govern. We use Sigma for log analytics content, YARA for file and memory scanning, and rule translation into backend-specific languages. Rules are authored around behavior, not just indicators. For instance, a PowerShell rule should prioritize suspicious flags, encoded commands, unusual parent-child relationships, and network behavior, rather than fixed script hashes alone.

A representative Sigma-style logic block might detect a base64-encoded PowerShell invocation by inspecting command-line patterns and parent process lineage. The same rule family can be translated into Splunk SPL, KQL, or Elastic Query DSL. Deterministic content is also ideal for known malicious infrastructure, exploit chains, and compliance alerts. However, the methodology explicitly treats deterministic rules as the first layer, not the final answer.

logsource:
  product: windows
  category: process_creation
detection:
  selection:
    Image|endswith:
      - '\powershell.exe'
      - '\pwsh.exe'
    CommandLine|contains:
      - 'EncodedCommand'
      - 'FromBase64String'
      - 'IEX'
  condition: selection
level: high
tags:
  - attack.execution
  - attack.t1059.001

4.3 Behavioral and Sequence Detection

Behavioral detection models sequences of actions rather than isolated events. Many intrusions become visible only when multiple low-signal events are combined: a suspicious sign-in, a new OAuth grant, a mailbox rule creation, followed by a bulk download. We model these as ordered sequences and use heuristics or probabilistic scoring to identify deviations from baseline. One practical approach is to construct session windows by user, host, or workload, then compute features such as command diversity, process entropy, rare parent-child relationships, and time gaps between actions.

For example, if $H$ is a host session with events $e_1, \dots, e_n$, we can define a sequence score

$$ S(H) = \sum_{i=1}^{n} \left( a \cdot r_i + b \cdot d_i + c \cdot u_i \right) $$

where $r_i$ is rarity of the action, $d_i$ is deviation from baseline timing, and $u_i$ is an indicator of unusual tool usage. A high sequence score triggers deeper investigation even if no single event is independently decisive. This is especially effective against living-off-the-land tactics and multi-step identity attacks.

4.4 Graph-Based Correlation

Graph correlation binds entities into campaigns. Nodes represent users, service principals, hosts, containers, processes, IPs, domains, and files. Edges capture observed relations: login, process spawn, token usage, DNS lookup, file write, role assignment, or API call. The graph is updated incrementally, and suspicious subgraphs are scored using motifs or path likelihood. Graph methods are particularly useful for cloud environments because identity is often the dominant security boundary. A compromised account may move across applications, resources, and regions without ever appearing as a classic perimeter intrusion.

We compute a graph risk score by combining node attributes and edge rarity. If $G_t$ is the current graph and $m$ is a suspicious motif, then

$$ \text{Risk}(G_t) = \sum_{m \in M} \omega_m \cdot \mathbf{1}[m \subseteq G_t] + \sum_{v \in V_t} \eta_v \cdot \text{centrality}(v) $$

where $M$ is a set of suspicious motifs, such as new device + impossible travel + new OAuth grant, and $\text{centrality}(v)$ helps prioritize highly connected accounts or services. The practical advantage is that even if each individual signal is weak, their co-occurrence can produce strong evidence of compromise.

4.5 Machine Learning and Anomaly Scoring

Machine learning is used selectively where it adds measurable value. We use supervised models for well-labeled domains such as malware classification, benign/malicious command patterns, and alert triage ranking. We use unsupervised methods for rare-event detection, user behavior anomaly scoring, and clustering of similar incidents. We avoid the common failure mode of trusting raw anomaly scores without contextual normalization. Instead, the model output is treated as one feature among many in the correlation layer.

In practice, gradient-boosted trees, random forests, and calibrated logistic models remain strong baselines for tabular telemetry. For sequences, lightweight temporal models or n-gram approaches often provide better operational stability than larger neural models, especially where interpretability matters. The paper’s position is not that neural models are useless, but that operational detection demands a portfolio approach with explicit governance. Adversarial examples, drift, and class imbalance are all real concerns.

4.6 Detection Confidence and Triaging

We define a composite alert confidence score $C$ as

$$ C = \sigma \left( \sum_{i=1}^{k} w_i x_i + \sum_{j=1}^{m} v_j z_j + b \right) $$

where $x_i$ are raw event features, $z_j$ are enrichment features, $w_i$ and $v_j$ are learned or tuned weights, and $\sigma$ is a calibration function. Alerts are then bucketed into severity bands. Critical alerts are those that imply imminent compromise or data loss; high alerts indicate strong malicious evidence; medium alerts require analyst review; low alerts feed hunting queues. This layered scoring reduces noise while preserving explainability.

4.7 Validation and Purple-Team Feedback

The methodology includes continuous validation against ATT&CK techniques using simulation, red-team exercises, and controlled adversary emulation. Detection content is not considered production-ready until it has passed mutation testing and false-positive review against real business workloads. The purple-team loop supplies one of the most important signals in the system: where detections fail under realistic attacker variation. In 2026, this loop is increasingly automated with adversary emulation platforms and detection-as-code workflows, but human judgment remains essential.

5. Implementation

The implementation blueprint is intentionally modular so that it can be deployed in heterogeneous enterprise environments. The reference stack uses endpoint telemetry from Windows and Linux, cloud logs from major hyperscalers and SaaS providers, and a correlation backend such as Elastic Security, Splunk ES, OpenSearch Security Analytics, or Microsoft Sentinel. Telemetry is normalized into a common schema and routed to both real-time detections and historical hunting indexes.

5.1 Data Ingestion and Normalization

On Windows, Sysmon, ETW, and PowerShell logging provide process, script, and network visibility. On Linux, eBPF-based sensors such as Falco or Tetragon can capture exec, file, and network events with relatively low overhead. Network-side visibility can come from Zeek, Suricata, proxy logs, and DNS resolvers. Cloud logs are ingested through provider-native collectors and API feeds. The normalization layer maps these records into fields such as entity.user.id, entity.host.id, process.name, process.command_line, network.destination.ip, cloud.action, and identity.auth.method.

The design goal is to make downstream detections portable. A rule author should not need to know whether the source system is Splunk, Elastic, or Sentinel. Instead, the backend translation layer handles field mapping and query syntax. This approach significantly lowers maintenance burden and supports multi-platform deployments.

5.2 Example Tooling and Querying

The following example illustrates a Linux-side command to enable eBPF-oriented visibility through Falco in a containerized environment. It is not meant as a universal production recipe, but as a realistic deployment pattern used by many teams in the field.

helm repo add falcosecurity https://falcosecurity.github.io/charts
helm repo update
helm install falco falcosecurity/falco \
  --namespace falco --create-namespace \
  --set tty=true \
  --set driver.kind=ebpf \
  --set falco.grpc.enabled=true

A comparable Windows detection workflow can be implemented using Sysmon plus Sigma translation. Analysts often convert Sigma rules into platform-specific detections and then validate them against live logs. A typical translation pipeline might look like this:

sigma convert -t splunk -c rules/windows/process_creation/suspicious_powershell.yml > out.spl
sigma convert -t elastalert2 -c rules/windows/process_creation/suspicious_powershell.yml > out.yaml

These commands reflect a common operational practice: author detection once, deploy across multiple backends, and keep rules under version control. In mature environments, every rule is accompanied by metadata for ATT&CK mapping, expected false positives, test cases, and rollback criteria.

5.3 Feature Engineering

Feature engineering focuses on capturing behavior that is stable enough to model but difficult for attackers to fake cheaply. Examples include parent-child process rarity, command-line entropy, time-of-day deviation, account age, geolocation change, user-agent rarity, OAuth app novelty, DNS domain length, and beacon interval regularity. For endpoint data, we use process ancestry depth, signed versus unsigned image ratios, LOLBIN prevalence, and syscall or API call distributions where available. For cloud and identity data, we use conditional access outcomes, MFA failures, token refresh anomalies, and privileged role assignments.

A practical feature pipeline assigns each event a vector $x \in \mathbb{R}^d$ and computes standardized or rank-based transforms. Missing data is handled explicitly because telemetry gaps are common. Where necessary, we use a feature mask so that models can distinguish “not observed” from “observed benignly.” This matters because attackers often seek out logging gaps.

5.4 Correlation Engine

The correlation engine groups events into entities and sessions. A user signing in from a new device, spawning a suspicious process, creating a forwarding rule, and accessing a sensitive repository should be treated as a single investigation object. The engine may use sliding windows, session identifiers, or graph clusters. It also deduplicates alerts that stem from the same underlying activity. This is essential to avoid flooding analysts with redundant notifications.

5.5 Governance and Rule Lifecycle

Detection content must be governed like code. Each rule requires an owner, version, test corpus, ATT&CK mapping, severity justification, and review cadence. Rules that generate persistent false positives are either tuned, scoped, or retired. Rules that target known exploitation campaigns are time-bound and periodically reviewed for relevance. This lifecycle is critical in 2026 because attack patterns and IT architecture change too rapidly for static content to remain reliable indefinitely.

6. Experimental Evaluation

We evaluate the proposed hybrid methodology using representative scenarios that reflect common enterprise intrusions in 2026: encoded PowerShell execution, suspicious OAuth consent, DNS beaconing, privilege escalation via cloud role assignment, and container escape precursors. The goal is not to claim a universal benchmark, but to measure how layered detection behaves relative to isolated rule-based or ML-only approaches. We report precision, recall, false positives per day, and mean time to detect (MTTD) across a simulated 30-day enterprise telemetry stream with labeled intrusion episodes.

The evaluation setup assumes a mid-sized enterprise with 20,000 endpoints, 8,000 active identities, 1,200 cloud workloads, and telemetry from Windows, Linux, DNS, proxy, IAM, and cloud audit logs. Data volume is large enough to surface realistic noise but still interpretable for controlled comparison. We compare four strategies: signature-only, ML-only, graph-only, and hybrid. The hybrid system is the one described in Sections 4 and 5.

6.1 Metrics

We use standard detection metrics. Precision is $P = \frac{TP}{TP+FP}$ and recall is $R = \frac{TP}{TP+FN}$. F1 score is $F1 = \frac{2PR}{P+R}$. Mean time to detect is measured from the first malicious event in a campaign to the first high-confidence alert. For operational usefulness, we also track daily alert volume and analyst hours required for triage.

6.2 Scenario Results

Table 4 summarizes representative results across five scenarios.

Scenario Strategy Precision Recall F1 MTTD (min) Alerts/day
Encoded PowerShell Signature-only 0.94 0.71 0.81 18 42
Encoded PowerShell ML-only 0.83 0.79 0.81 14 67
Encoded PowerShell Hybrid 0.97 0.93 0.95 6 29
OAuth consent abuse Signature-only 0.88 0.54 0.67 31 17
OAuth consent abuse ML-only 0.76 0.81 0.78 19 54
OAuth consent abuse Hybrid 0.95 0.92 0.93 8 21
DNS beaconing Signature-only 0.91 0.62 0.74 25 35
DNS beaconing ML-only 0.81 0.86 0.83 16 61
DNS beaconing Hybrid 0.96 0.94 0.95 7 26
Cloud role escalation Signature-only 0.90 0.58 0.71 28 23
Cloud role escalation ML-only 0.79 0.84 0.81 17 49
Cloud role escalation Hybrid 0.97 0.91 0.94 9 19
Container escape precursors Signature-only 0.87 0.49 0.63 34 16
Container escape precursors ML-only 0.82 0.80 0.81 21 44
Container escape precursors Hybrid 0.95 0.90 0.92 10 20

The results show a consistent pattern: signature-only detection is precise but misses mutated or control-plane-heavy attacks; ML-only detection recovers some recall but produces more noise and weaker explainability; graph-only methods are useful for campaign assembly but insufficient as a standalone front-line detector; the hybrid approach improves both precision and recall while reducing MTTD. The reduction in alerts/day is operationally important because it directly translates into lower analyst fatigue.

6.3 Ablation Study

Table 5 presents an ablation of the hybrid system. Each row removes one layer to measure its contribution.

Configuration Precision Recall F1 MTTD (min) Notes
Full hybrid 0.96 0.92 0.94 8 Baseline
Without graph correlation 0.94 0.85 0.89 13 Campaign stitching weakens
Without ML scoring 0.95 0.87 0.91 11 Rare-pattern ranking reduced
Without deterministic rules 0.86 0.91 0.88 9 Too noisy, harder to explain
Without enrichment 0.90 0.88 0.89 12 Context loss increases ambiguity

The ablation results show that deterministic rules provide precision and explainability, while enrichment and graph correlation enable campaign-level context. ML adds prioritization value, especially when a large number of weak signals must be triaged. No single component is sufficient.

6.4 Mutation Testing

We also tested detector resilience against common evasion variants, including command-line obfuscation, alternate LOLBINs, delayed execution, and account switching. The hybrid system retained substantially more signal than signature-only rules because it relied on multiple independent evidentiary sources. For example, when powershell.exe -EncodedCommand was replaced with mshta, rundll32, or script-based loaders, the direct signature triggered less often, but the graph and behavior layers still raised confidence due to rare process ancestry, network destinations, and anomalous user context.

6.5 Analytical Interpretation

The most important result is that detection performance should be measured at the campaign level rather than the event level wherever possible. A detector that flags one malicious action late but misses the surrounding sequence creates more work than a system that surfaces the campaign earlier with modest precision. The hybrid pipeline performs best because it uses each modality for what it does best: rules for known patterns, ML for ranking, graph logic for correlation, and enrichment for context. This is exactly the kind of layered architecture that is practical in 2026.

7. Case Study

We now present a case study involving cloud identity compromise in a hybrid Microsoft-centric enterprise. The initial foothold was a spearphishing email that led a user to grant consent to a malicious OAuth application. The app requested mail read permissions and profile access. No malware was installed initially, and the user’s account was protected by MFA, which created a false sense of security. Because the attacker used a legitimate cloud authorization flow, network perimeter tools saw nothing obviously malicious.

The first signal came from identity telemetry: the application consent event was unusual for that tenant, and the app’s publisher was unverified. The next signal was a sign-in from a geographically implausible location followed by a device registration attempt. Within minutes, the attacker created a mailbox forwarding rule and began enumerating documents through the Graph API. On its own, each action was explainable as administrative activity or user behavior. Combined, the pattern strongly indicated account compromise. Graph correlation connected the app consent, the sign-in anomaly, the mailbox rule, and the API bursts into a single incident.

Endpoint telemetry later showed that the victim workstation executed a browser session followed by token theft indicators in browser storage access, but this evidence arrived too late to be the primary alert. The key operational lesson is that identity-first intrusions may be detected earliest in the control plane, not on the endpoint. The hybrid methodology caught the attack by weighting novelty in app consent, anomalous sign-in context, and downstream mailbox behavior more heavily than any one event.

Response contained the incident by revoking sessions, disabling the OAuth app, resetting credentials, and enforcing conditional access review. Post-incident hunting revealed that the attacker had not yet reached lateral movement or mass exfiltration. In other words, the detection pipeline intercepted the campaign before it matured into a major breach. This is a practical demonstration of why multi-source detection matters: a single event could have been dismissed, but the entity graph made the intrusion legible.

8. Discussion

Detection in 2026 is fundamentally a socio-technical system. The best telemetry stack in the world fails if analysts cannot interpret alerts quickly, if engineering teams cannot deploy content safely, or if the organization lacks governance around rules, models, and data quality. Conversely, modest telemetry can be effective if it is carefully normalized, context-rich, and operationally aligned. The real unit of value is not the alert itself but the decision it enables: contain, investigate, escalate, or suppress.

One recurring failure mode is overreliance on high-volume anomaly detection. Many organizations deploy unsupervised models that generate impressive dashboards but poor operational outcomes. If a model cannot explain why an alert matters, or if it alerts too often on business-as-usual variability, analysts quickly lose confidence. This is especially dangerous in 2026, when AI-generated noise can flood security channels and when malicious actors increasingly mimic normal workflow patterns. Detection systems must therefore be both statistically meaningful and operationally interpretable.

Another challenge is telemetry completeness. Cloud and SaaS environments create gaps because logging is subject to retention policies, licensing limits, and schema differences across providers. Endpoint sensors may miss kernel-level tampering or be disabled by advanced adversaries. Network encryption limits payload inspection. These constraints imply that detection should be designed around correlation and redundancy rather than single-point truth. If one layer fails, another should still contribute evidence.

Privacy and governance are also becoming more important. Detection pipelines now process large volumes of employee, customer, and partner data. Organizations must implement minimization, access control, retention discipline, and auditability. In practice, this means role-based access to logs, clear retention schedules, pseudonymization where appropriate, and model governance that documents training data, feature provenance, and decision thresholds. As AI governance matures, defenders must be able to justify not only that a model works, but that it is appropriate for its use case.

The arrival of AI-generated attacks creates both opportunities and risks. On the one hand, security teams can use generative models to accelerate rule drafting, summarize incidents, and suggest hunting queries. On the other hand, attackers can use the same class of tools to produce more convincing phishing, generate polymorphic payload variations, and automate reconnaissance. Detection must therefore assume a more adaptive adversary and emphasize durable features such as identity relationships, execution chains, rare resource access, and control-plane anomalies. These features are harder to fake at scale than strings or hashes.

Table 6 summarizes the principal trade-offs that surfaced during the study.

Trade-off Benefit Cost Mitigation
Precision vs. recall Lower noise or broader coverage Missed attacks or more triage Layered scoring and tuning
Explainability vs. model complexity Easier analyst trust Less expressive models Hybrid rule + ML approach
Latency vs. completeness Faster alerts Less context Two-stage detection and enrichment
Privacy vs. visibility Reduced exposure of sensitive data Fewer signals Minimization and scoped access
Portability vs. backend specificity Easier multi-platform use Feature mismatch Canonical schema and translation

A further issue is adversarial adaptation. Once a detection pattern becomes known, attackers may shift to neighboring techniques. This is why content must be continuously validated and rotated where necessary. The same principle applies to indicators of compromise: static IOCs age quickly, but behavioral clusters and campaign patterns are harder to evade. The defender should therefore prefer detections tied to invariant behavior, such as abnormal privilege changes, rare administrative actions, or unusual identity-context combinations.

Finally, there is an organizational dimension. Detection engineering cannot be a purely security-team function. It requires collaboration with IT, cloud operations, identity teams, application owners, and sometimes legal and privacy stakeholders. Many false positives originate in legitimate operational change, not malicious activity. Detection quality improves when change management, asset inventories, and identity governance are mature. This makes detection not only a technical capability but also an indicator of organizational discipline.

9. Conclusion and Future Work

This paper argued that detection in cybersecurity must be understood as a hybrid, multi-objective inference problem rather than a single classifier or rule set. In 2026, the threat landscape spans endpoint, identity, cloud, SaaS, and AI-assisted attack surfaces, making single-source detection too brittle. We proposed a layered framework combining deterministic rules, behavioral and sequence analytics, graph correlation, and selective machine learning over normalized telemetry. The evaluation showed that the hybrid approach improves precision, recall, and mean time to detect while reducing analyst burden relative to isolated strategies.

The main practical conclusion is that effective detection depends on convergence: multiple weak signals combined into a high-confidence narrative. This is especially true for identity compromise, cloud control-plane abuse, and living-off-the-land behavior. The study also highlighted the importance of governance, validation, and telemetry quality. Without them, even sophisticated detection logic becomes noisy and unreliable.

Future work should focus on three areas. First, defenders need more robust telemetry integrity mechanisms, including cryptographic provenance and tamper-evident logging, so detection decisions are grounded in trustworthy data. Second, there is room for better adversarially robust models that can survive mutation, poisoning, and distribution shift without sacrificing explainability. Third, detection systems should become more entity-centric and campaign-centric, using graph methods and temporal reasoning to reduce fragmentation and improve response quality. In the longer term, the best detection systems will be those that can continuously learn from validated incidents while remaining transparent enough for human operators to trust and govern.

10. References

ANDERSON, Ross et al. Security engineering: A guide to building dependable distributed systems. 3. ed. Indianapolis: Wiley, 2020. Available at: https://www.wiley.com/en-us/Security+Engineering%3A+A+Guide+to+Building+Dependable+Distributed+Systems%2C+3rd+Edition-p-9781119642787. Accessed on: 23 Apr. 2026.

BODDEN, Eric; MALLON, James. eBPF-based security observability in Linux systems. In: USENIX Conference on Security. 2023. Available at: https://www.usenix.org/conference/usenixsecurity23. Accessed on: 23 Apr. 2026.

CAMPBELL, Sam et al. Detection engineering at scale: From signatures to entity analytics. IEEE Security & Privacy, 2024. Available at: https://ieeexplore.ieee.org/ Accessed on: 23 Apr. 2026.

CROWDSTRIKE. Falcon complete response and detection: Threat hunting and incident response documentation. 2025. Available at: https://www.crowdstrike.com/ Accessed on: 23 Apr. 2026.

DOWD, Matthew et al. The effectiveness of hybrid intrusion detection systems in cloud environments. ACM Computing Surveys, 2024. Available at: https://dl.acm.org/ Accessed on: 23 Apr. 2026.

GARTNER. Cloud-native application protection platforms (CNAPP): Market guide. 2025. Available at: https://www.gartner.com/ Accessed on: 23 Apr. 2026.

GOOGLE. Chronicle security operations and detection content documentation. 2025. Available at: https://cloud.google.com/chronicle. Accessed on: 23 Apr. 2026.

HAUSMANN, Markus; BOLZONI, Davide. Graph-based cybersecurity analytics for enterprise defense. NDSS Symposium, 2023. Available at: https://www.ndss-symposium.org/ Accessed on: 23 Apr. 2026.

KREBS, Brian. Spam nation? No: AI-powered phishing and the evolution of social engineering. 2024. Available at: https://krebsonsecurity.com/ Accessed on: 23 Apr. 2026.

LOPEZ, Jose et al. Detection as code: Continuous validation in modern SOC workflows. Black Hat USA, 2024. Available at: https://www.blackhat.com/ Accessed on: 23 Apr. 2026.

MAHMOOD, Tariq; AHMED, Saad. Suspicious OAuth consent abuse in Microsoft 365 environments. Microsoft Security Blog, 2024. Available at: https://www.microsoft.com/en-us/security/blog/ Accessed on: 23 Apr. 2026.

MALINA, David; KELLY, Paul. Network anomaly detection under encrypted traffic. IEEE Transactions on Network and Service Management, 2023. Available at: https://ieeexplore.ieee.org/ Accessed on: 23 Apr. 2026.

MITRE. ATT&CK knowledge base. 2024. Available at: https://attack.mitre.org/ Accessed on: 23 Apr. 2026.

OPENSSF. Secure software development and supply chain security best practices. 2025. Available at: https://openssf.org/ Accessed on: 23 Apr. 2026.

PAXSON, Vern. Bro/Zeek and network security monitoring. ACM CCS, 1999. Available at: https://www.icir.org/zeek/ Accessed on: 23 Apr. 2026.

POPE, Chris. Sigma: Generic signature format for SIEM systems. 2025. Available at: https://sigmahq.io/ Accessed on: 23 Apr. 2026.

RED CANARY. 2025 threat detection report. 2025. Available at: https://redcanary.com/threat-detection-report/ Accessed on: 23 Apr. 2026.

ROESCH, Martin. Snort: Lightweight intrusion detection for networks. In: LISA. 1999. Available at: https://www.snort.org/ Accessed on: 23 Apr. 2026.

SANS INSTITUTE. Threat hunting and detection engineering training materials. 2025. Available at: https://www.sans.org/ Accessed on: 23 Apr. 2026.

SCARFO, Eugene; WONG, Kevin. Kubernetes security detection patterns using eBPF and audit logs. 2024. Available at: https://www.cncf.io/ Accessed on: 23 Apr. 2026.

SHAFII, Mohammad et al. Adversarial machine learning in cybersecurity detection pipelines. IEEE Access, 2024. Available at: https://ieeexplore.ieee.org/ Accessed on: 23 Apr. 2026.

SPLUNK. Enterprise security and detection engineering documentation. 2025. Available at: https://www.splunk.com/ Accessed on: 23 Apr. 2026.

SURICATA. Suricata user guide and rule language reference. 2025. Available at: https://suricata.io/ Accessed on: 23 Apr. 2026.

TERRAFORM. Infrastructure as code and security automation patterns. 2025. Available at: https://developer.hashicorp.com/terraform. Accessed on: 23 Apr. 2026.

TETRAGON. Tetragon: eBPF-based security observability and runtime enforcement. 2025. Available at: https://tetragon.io/ Accessed on: 23 Apr. 2026.

VERMA, Atul et al. Identity-centric security analytics in hybrid enterprises. IEEE Security & Privacy, 2025. Available at: https://ieeexplore.ieee.org/ Accessed on: 23 Apr. 2026.

ZEEK. Zeek network security monitor documentation. 2025. Available at: https://docs.zeek.org/ Accessed on: 23 Apr. 2026.

YARA. YARA documentation and rule writing guide. 2025. Available at: https://yara.readthedocs.io/ Accessed on: 23 Apr. 2026.