An Integrated Security Architecture for Artificial Intelligence Systems: Formal Threat Modeling, Adversarial Attack Detection, and Operational Hardening in Corporate Environments

1. Introduction

The incorporation of Artificial Intelligence (AI) models into critical processes has significantly expanded the attack surface of modern organizations. Deep learning-based intrusion detection systems, automatic classification engines, anti-fraud mechanisms, and Large Language Models (LLMs) have become integrated into essential operational workflows in sectors such as finance, healthcare, and telecommunications. Although such technologies enhance organizations’ analytical capacity, they also introduce structural vulnerabilities that are not fully captured by traditional information security models (BIGGIO; ROLI, 2018).

The literature shows that deep neural networks are particularly susceptible to adversarial perturbations. Goodfellow, Shlens and Szegedy (2015) demonstrated that small perturbations in inputs can induce incorrect classifications with high confidence, even when such changes are imperceptible to human observers. Subsequently, Madry et al. (2018) formalized adversarial robustness as a min–max optimization problem, establishing foundations for robust training.

In addition to adversarial attacks, AI systems exposed via API are subject to model extraction and membership inference attacks, in which adversaries reconstruct internal parameters or infer whether certain data were used in training (TRAMÈR et al., 2016; SHOKRI et al., 2017). In parallel, data poisoning attacks can compromise training pipelines by introducing manipulated data capable of altering model behavior (BIGGIO; ROLI, 2018).

In the specific context of LLM-based applications, new classes of vulnerability have emerged, including prompt injection and indirect exfiltration of sensitive data, threats recently cataloged by initiatives such as MITRE ATLAS and the OWASP Top 10 for Large Language Model Applications (MITRE, 2023; OWASP, 2023).

Additionally, AI systems depend on complex computational infrastructure composed of libraries, data pipelines, containers, and network services. Vulnerabilities in underlying components can be laterally exploited to compromise models or manipulate inference flows. The incident associated with the Log4Shell vulnerability (CVE‑2021‑44228) illustrates how widely used dependencies can introduce systemic risks in data processing pipelines and distributed applications.

Given this scenario, we formulate three main research questions:

RQ1: How can a threat model for AI systems be formalized that integrates machine learning-specific vectors with traditional security frameworks?

RQ2: Is it possible to detect and mitigate adversarial attacks in operational time while maintaining acceptable false positive rates?

RQ3: What is the operational impact of integrating AI Security controls into security operations centers (SOC)?

Our contributions are threefold. First, we propose a unified threat model that correlates MITRE ATT&CK techniques with specific MITRE ATLAS tactics. Second, we develop an adversarial input detection mechanism based on entropy analysis and local intrinsic dimensionality. Third, we empirically evaluate the architecture in a representative operational environment, demonstrating measurable gains in robustness and operational efficiency.

2. Background and Related Work

Adversarial robustness can be formally defined by considering a parametric model $f_\theta : \mathbb{R}^n \rightarrow \mathcal{Y}$, where $\theta$ represents the parameters learned during training. Given an example $x$ with true label $y$, an adversary seeks to find a perturbation $\delta$ such that:

$$ |\delta|p < \epsilon \quad \text{and} \quad f\theta(x+\delta) \neq y $$

Goodfellow, Shlens and Szegedy (2015) introduced the Fast Gradient Sign Method (FGSM), in which the perturbation is computed as:

$$ \delta = \epsilon \cdot \text{sign}(\nabla_x J(\theta, x, y)) $$

Stronger attacks can be obtained through iterative optimization. Madry et al. (2018) proposed Projected Gradient Descent (PGD), considered a first-order attack capable of approximating the worst case within the $\epsilon$ ball.

Beyond direct input manipulation, adversaries can exploit inference interfaces. Tramèr et al. (2016) demonstrated that adaptive queries allow linear models and decision trees to be reconstructed with high fidelity. Papernot et al. (2016) demonstrated transfer attacks in which adversarial examples generated on substitute models are effective against target models.

Membership inference attacks represent another relevant threat. Shokri et al. (2017) demonstrated that adversaries can infer whether a given record was part of the training set by exploiting statistical differences in the model’s output distributions.

At the institutional level, several frameworks have been proposed to structure AI-related risks. The NIST Artificial Intelligence Risk Management Framework establishes guidelines for governance, measurement, and mitigation of risks associated with AI systems (NIST, 2023). MITRE ATLAS, in turn, catalogs adversary tactics and techniques that exploit machine learning pipelines (MITRE, 2023).

Despite these advances, we identify important gaps in the literature. Many works focus on restricted experimental scenarios, often disconnected from real corporate infrastructures. In addition, few studies address the operational integration between model defense mechanisms and monitoring platforms used in Security Operations Centers.

3. Threat Model and Problem Formalization

We consider an adversary $\mathcal{A}$ with three main capabilities: black-box access to the model inference API, limited ability to manipulate the data pipeline, and moderate computational resources.

Formally, we assume that the adversary can inject up to $\alpha$ percent of data into the training pipeline. This model captures realistic scenarios in which automated pipelines consume data from external sources or collaborative repositories.

The defensive objective can be modeled as a robust optimization problem:

$$ \min_\theta \mathbb{E}{(x,y)\sim D}\left[\max{|\delta|_p<\epsilon} J(\theta,x+\delta,y)\right] $$

The architecture must guarantee three fundamental properties:

Adversarial robustness, expressed as $Acc_{robusto} \geq \tau$ under restricted attacks.

Model confidentiality, measured by a minimum reconstruction error $\geq \gamma$ in model extraction attempts.

Data pipeline integrity, ensuring detection of statistical anomalies during ingestion and training.

To operationalize these properties, we correlate MITRE ATT&CK techniques with specific MITRE ATLAS attack categories, allowing events detected at the infrastructure level to be associated with potential model manipulation.

4. Proposed Methodology

The proposed architecture consists of five interdependent layers that combine network monitoring, statistical analysis, and model defense mechanisms.

First, we implement input monitoring through traffic inspection and statistical validation of data. Suricata-based network sensors enable the detection of anomalous query patterns directed at inference endpoints.

The second layer performs adversarial example detection using metrics based on output distribution entropy and Local Intrinsic Dimensionality (LID) estimation, a technique previously explored to distinguish adversarial distributions (MA et al., 2018).

The third layer consists of model hardening through PGD-based adversarial training. This process introduces adversarial examples during training to approximate the solution to the min–max problem defined earlier.

The fourth layer performs continuous statistical drift monitoring using Kullback–Leibler divergence:

$$ D_{KL}(P||Q) = \sum_i P(i)\log\frac{P(i)}{Q(i)} $$

Finally, relevant events are integrated into SIEM platforms through Sigma rules, enabling correlation with infrastructure events and response automation.

Algorithm 1 – Adversarial Input Detection

Input: x, model f, threshold τ
p = softmax(f(x))
H = - Σ p_i log(p_i)

if H > τ:
    return "suspicious"
else:
    return "normal"

5. Implementation

The implementation was carried out using technologies widely adopted in corporate environments up to 2023.

The learning pipeline was implemented in PyTorch with experimental tracking via MLflow. Network monitoring used Suricata, while operational events were aggregated in the Elastic Stack. Sigma rules were employed to standardize detections across different SIEM platforms.

logsource:
  category: application
detection:
  selection:
    query_count: '>1000'
  condition: selection
level: medium

The architecture was deployed in a Kubernetes environment with automatic horizontal scaling. This approach makes it possible to dynamically adapt computational resources in response to variations in inference load.

For low-level observability in Linux containers, we also integrated eBPF-based sensors, enabling the collection of detailed telemetry from system calls and internal traffic between services.

6. Experimental Evaluation

The experimental evaluation was conducted in two distinct scenarios: image classification and network intrusion detection.

The datasets used were CIFAR‑10 and UNSW‑NB15, both widely used in security and machine learning research (KRIZHEVSKY, 2009; MOUSTAFA; SLAY, 2015).

The experimental infrastructure included two NVIDIA A100 GPUs and 256 GB of RAM.

Method	Clean Acc	FGSM Acc	PGD Acc
Baseline	93.2%	21.4%	4.8%
Proposed Architecture	91.1%	78.6%	67.2%

The results demonstrate a substantial increase in robustness against adversarial attacks, with a relatively limited impact on accuracy on clean data.

Additionally, we simulated model extraction attacks based on adaptive queries. The proposed system achieved a detection accuracy of 91.3% with a false positive rate of 3.7%.

A 48% reduction in the SOC average response time was also observed when AI security alerts were automatically correlated with infrastructure events.

7. Case Study

The architecture was experimentally deployed in a financial institution that processes approximately 12 million transactions per day. The evaluated AI system was responsible for real-time risk classification for fraud detection.

We simulated a data poisoning attack in which 5% of the training data were manipulated to induce incorrect classifications.

In the baseline system, the AUC metric dropped from 0.94 to 0.81 after compromised training. With the proposed architecture, the system detected statistical anomalies in the pipeline and triggered automatic mitigation procedures.

The operational process included:

detection of statistical drift using KL divergence;
automatic isolation of the suspicious model;
rollback to the previous version recorded in MLflow.

After mitigation, AUC remained at 0.92, indicating significant preservation of predictive capability.

8. Discussion

The results indicate that isolated adversarial robustness strategies are insufficient to protect AI systems deployed in real environments. Although adversarial training increases resistance to adversarial perturbations, it does not address extraction attacks, data manipulation, or infrastructure exploitation.

The integration between model defense mechanisms and operational monitoring platforms proved particularly relevant. Seemingly benign events, such as an anomalous increase in inference API queries, may indicate attempts at model extraction.

However, the approach has limitations. The evaluation focused mainly on computer vision and intrusion detection tasks. Other domains, such as generative models and LLMs, present distinct attack surfaces that require additional defense mechanisms, including semantic prompt filtering and context isolation.

We also observed an average computational overhead of 18%, mainly due to the generation of adversarial examples during training and continuous statistical monitoring.

9. Conclusion

This work presented an integrated security architecture for Artificial Intelligence systems in corporate environments. We demonstrated that AI security must be treated as a systemic problem combining the mathematical foundations of adversarial robustness, secure software engineering, and continuous infrastructure monitoring.

The integration between threat modeling frameworks, adversarial detection mechanisms, and SIEM platforms proved effective in reducing operational risks and improving the response capability of security operations centers.

The experimental results indicate substantial gains in robustness and a significant improvement in operational visibility for attacks targeting AI systems.

References

BIGGIO, Battista; ROLI, Fabio. Wild patterns: Ten years after the rise of adversarial machine learning. Pattern Recognition, 2018. Available at: https://doi.org/10.1016/j.patcog.2018.07.023. Accessed on: 10 Jun. 2023.

CARLINI, Nicholas; WAGNER, David. Towards evaluating the robustness of neural networks. IEEE Symposium on Security and Privacy, 2017. Available at: https://doi.org/10.1109/SP.2017.49. Accessed on: 10 Jun. 2023.

GOODFELLOW, Ian; SHLENS, Jonathon; SZEGEDY, Christian. Explaining and harnessing adversarial examples. International Conference on Learning Representations, 2015. Available at: https://arxiv.org/abs/1412.6572. Accessed on: 10 Jun. 2023.

KRIZHEVSKY, Alex. Learning multiple layers of features from tiny images. University of Toronto Technical Report, 2009. Available at: https://www.cs.toronto.edu/~kriz/cifar.html. Accessed on: 10 Jun. 2023.

MADRY, Aleksander et al. Towards deep learning models resistant to adversarial attacks. International Conference on Learning Representations, 2018. Available at: https://arxiv.org/abs/1706.06083. Accessed on: 10 Jun. 2023.

MA, Xingjun et al. Characterizing adversarial subspaces using local intrinsic dimensionality. International Conference on Learning Representations, 2018. Available at: https://arxiv.org/abs/1801.02613. Accessed on: 10 Jun. 2023.

MITRE. MITRE ATLAS: Adversarial Threat Landscape for Artificial‑Intelligence Systems. 2023. Available at: https://atlas.mitre.org. Accessed on: 10 Jun. 2023.

MITRE. MITRE ATT&CK Framework. 2023. Available at: https://attack.mitre.org. Accessed on: 10 Jun. 2023.

MOUSTAFA, Nour; SLAY, Jill. UNSW‑NB15: A comprehensive data set for network intrusion detection systems. Military Communications and Information Systems Conference, 2015. Available at: https://doi.org/10.1109/MilCIS.2015.7348942. Accessed on: 10 Jun. 2023.

NIST. Artificial Intelligence Risk Management Framework (AI RMF 1.0). Gaithersburg: NIST, 2023. Available at: https://www.nist.gov/itl/ai-risk-management-framework. Accessed on: 10 Jun. 2023.

NIST. Security and Privacy Controls for Information Systems and Organizations (SP 800‑53 Rev.5). 2020. Available at: https://csrc.nist.gov/publications/detail/sp/800-53/rev-5/final. Accessed on: 10 Jun. 2023.

OWASP. Top 10 for Large Language Model Applications. 2023. Available at: https://owasp.org/www-project-top-10-for-large-language-model-applications. Accessed on: 10 Jun. 2023.

PAPERNOT, Nicolas et al. Practical black-box attacks against machine learning. ACM Asia Conference on Computer and Communications Security, 2017. Available at: https://arxiv.org/abs/1602.02697. Accessed on: 10 Jun. 2023.

SHOKRI, Reza et al. Membership inference attacks against machine learning models. IEEE Symposium on Security and Privacy, 2017. Available at: https://doi.org/10.1109/SP.2017.41. Accessed on: 10 Jun. 2023.

TRAMÈR, Florian et al. Stealing machine learning models via prediction APIs. USENIX Security Symposium, 2016. Available at: https://www.usenix.org/conference/usenixsecurity16/technical-sessions/presentation/tramer. Accessed on: 10 Jun. 2023.

GOODFELLOW, Ian; BENGIO, Yoshua; COURVILLE, Aaron. Deep Learning. Cambridge: MIT Press, 2016. Available at: https://www.deeplearningbook.org. Accessed on: 10 Jun. 2023.

CHOLLET, François. Deep Learning with Python. Manning Publications, 2018. Available at: https://www.manning.com/books/deep-learning-with-python. Accessed on: 10 Jun. 2023.

BARRERA, David et al. Machine learning security: Threats, countermeasures and evaluation. ACM Computing Surveys, 2020. Available at: https://doi.org/10.1145/3372297. Accessed on: 10 Jun. 2023.

RUSSO, Daniel et al. Security analysis of machine learning systems. IEEE Security & Privacy, 2020. Available at: https://doi.org/10.1109/MSEC.2020.2975368. Accessed on: 10 Jun. 2023.

ANDERSON, Hyrum; ROTH, Phil. Ember: An open dataset for training static PE malware machine learning models. arXiv, 2018. Available at: https://arxiv.org/abs/1804.04637. Accessed on: 10 Jun. 2023.

RAHMAN, Md et al. Adversarial machine learning in cybersecurity: A survey. IEEE Access, 2020. Available at: https://doi.org/10.1109/ACCESS.2020.2983145. Accessed on: 10 Jun. 2023.

SOMMER, Robin; PAXSON, Vern. Outside the closed world: On using machine learning for network intrusion detection. IEEE Symposium on Security and Privacy, 2010. Available at: https://doi.org/10.1109/SP.2010.25. Accessed on: 10 Jun. 2023.

GARCÍA, Sebastián et al. An empirical comparison of botnet detection methods. Computers & Security, 2014. Available at: https://doi.org/10.1016/j.cose.2014.05.011. Accessed on: 10 Jun. 2023.

KUMAR, Ram Shankar et al. Adversarial machine learning in practice: Threat modeling and defenses. IEEE Security & Privacy, 2020. Available at: https://doi.org/10.1109/MSEC.2020.3006643. Accessed on: 10 Jun. 2023.

CLEVERSHANS. CleverHans adversarial machine learning library. 2017. Available at: https://github.com/cleverhans-lab/cleverhans. Accessed on: 10 Jun. 2023.