* [Blog](https://www2.paloaltonetworks.com/blog)
* [Palo Alto Networks](https://www2.paloaltonetworks.com/blog/corporate/)
* [Must-Read Articles](https://www2.paloaltonetworks.com/blog/security-operations/category/must-read-articles/)
* Data --- The Lifeblood of S...

# Data --- The Lifeblood of Security and Detection Engineering

[](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fwww2.paloaltonetworks.com%2Fblog%2F2023%2F09%2Fsecurity-and-detection-engineering%2F)  
[](https://twitter.com/share?text=Data+%E2%80%94+The+Lifeblood+of+Security+and+Detection+Engineering&url=https%3A%2F%2Fwww2.paloaltonetworks.com%2Fblog%2F2023%2F09%2Fsecurity-and-detection-engineering%2F)  
[](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Fwww2.paloaltonetworks.com%2Fblog%2F2023%2F09%2Fsecurity-and-detection-engineering%2F&title=Data+%E2%80%94+The+Lifeblood+of+Security+and+Detection+Engineering&summary=&source=)  
[](https://www.paloaltonetworks.com//www.reddit.com/submit?url=https://www2.paloaltonetworks.com/blog/2023/09/security-and-detection-engineering/&ts=markdown)  
\[\](mailto:?subject=Data — The Lifeblood of Security and Detection Engineering)  
Link copied  
By [Dena De Angelo](https://www.paloaltonetworks.com/blog/author/ddeangelo/?ts=markdown "Posts by Dena De Angelo")  
Sep 12, 2023  
4 minutes  
[Must-Read Articles](https://www.paloaltonetworks.com/blog/security-operations/category/must-read-articles/?ts=markdown)  
[Points of View](https://www.paloaltonetworks.com/blog/category/points-of-view/?ts=markdown)  
[Product Features](https://www.paloaltonetworks.com/blog/security-operations/category/product-features/?ts=markdown)  
[Products and Services](https://www.paloaltonetworks.com/blog/category/products-and-services/?ts=markdown)  
[Cortex](https://www.paloaltonetworks.com/blog/tag/cortex/?ts=markdown)  
[Cortex XSIAM](https://www.paloaltonetworks.com/blog/tag/cortex-xsiam/?ts=markdown)  
[SOC](https://www.paloaltonetworks.com/blog/tag/soc/?ts=markdown)  
[This is how we do it](https://www.paloaltonetworks.com/blog/tag/this-is-how-we-do-it/?ts=markdown)

In episode 5 of ["*This Is How We Do It,*"](https://www.paloaltonetworks.com/blog/tag/this-is-how-we-do-it/) Peter Havens from Cortex product marketing and Isaac Krzywanowski, staff security engineer at Palo Alto Networks, discuss data pipelining, operational assurance and the importance of monitoring the health of data sources. Join their conversation as they dig into the complexity behind data ingestion and how it is managed in the Palo Alto Networks security operation center (SOC).

In an era where digital landscapes are under siege, *data* -- where it's from and how it's used -- emerges as the unsung hero of cybersecurity. Isaac is entrusted with the responsibility of data pipelining: "Data is the lifeblood of security and detection engineering." Within his purview lies the critical task of data ingestion and operational assurance, which are pivotal to arming the SOC with timely and relevant information to counter potential threats.

The process of data ingestion is no trivial matter. It includes the intricate orchestration of collecting data from multiple sources, ensuring its relevance to the SOC's needs, and then mapping it onto the Cortex Data Model (XDM). Isaac explains the significance of this data model, as it "standardizes the data, allowing analysts to search through multiple tables effortlessly, using a unified framework."

The palette of data sources at Palo Alto Networks is expansive and diverse. It includes firewall data, endpoint detection and response (EDR) data from Cortex XDR, cloud platform data from Google, Amazon and Azure, and even unconventional sources, such as source code repositories (like GitHub and GitLab). The inclusion of source code data proves insightful as it gathers audit logs to monitor user access and potential security breaches.

## Cortex XSIAM and Data Collection

With an astounding influx of 50 terabytes to 75 terabytes of data *each day*, the complexity of data collection in the Palo Alto Networks SOC cannot be underestimated. This torrent of data arrives in myriad formats from different vendors and partners, requiring careful consideration. XSIAM helps facilitate the collection of data regardless of where it's from. Isaac underscores the importance of selecting pivotal data sources, which comprise security data, identity-related information, access logs and the "crown jewels" of company-sensitive data that is vital to any organization:

"It's really important because you never know when a data source might drop for whatever reason, and you need to know about that as fast as possible. That's critical because the entire SOC is waiting for that log, so operational assurance is really important. Making sure that data is coming in, and coming in within a reasonable amount of time, is super important. First of all, we have an out-of-the-box table that basically holds what we call 'data freshness.' This is really helpful in determining whether your flow of data is starting to lag behind, or whether it's dropped altogether."

Our XSIAM platform serves as the backbone of data management, streamlining the process of data collection through API integrations, XDR collectors and broker virtual machines (VMs). Through these mechanisms, XSIAM not only simplifies data gathering but also standardizes it. Issac says that this allows data to be accessible and analyzable across the board:

## Operational Assurance --- Nurturing Data Health

A large part of the data ingestion process is the critical practice of operational assurance. It ensures the reliability of incoming data streams and whether the system's technical features are being bypassed, have vulnerabilities, and whether required procedures are being followed. Operational assurance has been fortified through real-time alerts. By integrating with the Slack communication platform, Isaac and his team receive instantaneous notifications about potential data ingestion issues. This real-time collaboration minimizes mean time to response (MTTR), fortifying the defense against potential data interruptions.

In cybersecurity, data mastery emerges as a fundamental pillar of protection. Isaac Krzywanowski's insights provide a window into the nuances of data pipelining and operational assurance at Palo Alto Networks. With data as the bedrock of detection, response and AI-driven analysis, it's evident that the processes discussed above are not just technical undertakings, but essential safeguards against digital threats.

**Watch their** [**full interview**](https://www.youtube.com/watch?v=k7BvV4rJuck)**on our Cortex YouTube channel.**

*** ** * ** ***

## Related Blogs

### [Must-Read Articles](https://www.paloaltonetworks.com/blog/security-operations/category/must-read-articles/?ts=markdown), [Points of View](https://www.paloaltonetworks.com/blog/category/points-of-view/?ts=markdown), [Product Features](https://www.paloaltonetworks.com/blog/security-operations/category/product-features/?ts=markdown), [Products and Services](https://www.paloaltonetworks.com/blog/category/products-and-services/?ts=markdown)

[#### Artificial Intelligence --- Beyond the Algorithms](https://www2.paloaltonetworks.com/blog/2023/09/artificial-intelligence-beyond-the-algorithms/)

### [Must-Read Articles](https://www.paloaltonetworks.com/blog/security-operations/category/must-read-articles/?ts=markdown), [Product Features](https://www.paloaltonetworks.com/blog/security-operations/category/product-features/?ts=markdown), [Products and Services](https://www.paloaltonetworks.com/blog/category/products-and-services/?ts=markdown)

[#### Threat Hunting to Find the Good Stuff](https://www2.paloaltonetworks.com/blog/2023/10/threat-hunting-to-find-the-good-stuff/)

### [Must-Read Articles](https://www.paloaltonetworks.com/blog/security-operations/category/must-read-articles/?ts=markdown), [Points of View](https://www.paloaltonetworks.com/blog/category/points-of-view/?ts=markdown), [Product Features](https://www.paloaltonetworks.com/blog/security-operations/category/product-features/?ts=markdown), [Products and Services](https://www.paloaltonetworks.com/blog/category/products-and-services/?ts=markdown)

[#### Multiplying Force with Automation --- Reducing the Soul Crushing Work](https://www2.paloaltonetworks.com/blog/2023/08/multiplying-force-with-automation/)

### [Company \& Culture](https://www.paloaltonetworks.com/blog/category/company-culture/?ts=markdown), [Must-Read Articles](https://www.paloaltonetworks.com/blog/security-operations/category/must-read-articles/?ts=markdown), [Points of View](https://www.paloaltonetworks.com/blog/category/points-of-view/?ts=markdown), [Product Features](https://www.paloaltonetworks.com/blog/security-operations/category/product-features/?ts=markdown), [Products and Services](https://www.paloaltonetworks.com/blog/category/products-and-services/?ts=markdown)

[#### Hasta La Vista Human Powers --- Automating the Automation](https://www2.paloaltonetworks.com/blog/2023/05/automating-the-automation/)

### [Must-Read Articles](https://www.paloaltonetworks.com/blog/security-operations/category/must-read-articles/?ts=markdown), [Points of View](https://www.paloaltonetworks.com/blog/category/points-of-view/?ts=markdown), [Products and Services](https://www.paloaltonetworks.com/blog/category/products-and-services/?ts=markdown)

[#### 4 Steps to Future-Forward SOC Transformation](https://www2.paloaltonetworks.com/blog/2021/10/steps-to-soc-transformation/)

### [Product Features](https://www.paloaltonetworks.com/blog/security-operations/category/product-features/?ts=markdown), [Products and Services](https://www.paloaltonetworks.com/blog/category/products-and-services/?ts=markdown), [Unit 42](https://unit42-dev2.paloaltonetworks.com)

[#### Prowling the Wilds --- Upgrade Your SOC and Hunt Down Threats](https://www2.paloaltonetworks.com/blog/2024/05/upgrade-your-soc-and-hunt-down-threats/)

### Subscribe to the Blog!

Sign up to receive must-read articles, Playbooks of the Week, new feature announcements, and more.
![spinner](https://www2.paloaltonetworks.com/blog/wp-content/themes/panwblog2023/dist/images/ajax-loader.gif) Sign up  
Please enter a valid email.  
By submitting this form, you agree to our [Terms of Use](https://www.paloaltonetworks.com/legal-notices/terms-of-use?ts=markdown) and acknowledge our [Privacy Statement](https://www.paloaltonetworks.com/legal-notices/privacy?ts=markdown). Please look for a confirmation email from us. If you don't receive it in the next 10 minutes, please check your spam folder.  
This site is protected by reCAPTCHA and the Google [Privacy Policy](https://policies.google.com/privacy) and [Terms of Service](https://policies.google.com/terms) apply.  
{#footer} {#footer}

## Products and Services

* [AI-Powered Network Security Platform](https://www.paloaltonetworks.com/network-security?ts=markdown)

* [Secure AI by Design](https://www.paloaltonetworks.com/precision-ai-security/secure-ai-by-design?ts=markdown)

* [Prisma AIRS](https://www.paloaltonetworks.com/prisma/prisma-ai-runtime-security?ts=markdown)

* [AI Access Security](https://www.paloaltonetworks.com/sase/ai-access-security?ts=markdown)

* [Cloud Delivered Security Services](https://www.paloaltonetworks.com/network-security/security-subscriptions?ts=markdown)

* [Advanced Threat Prevention](https://www.paloaltonetworks.com/network-security/advanced-threat-prevention?ts=markdown)

* [Advanced URL Filtering](https://www.paloaltonetworks.com/network-security/advanced-url-filtering?ts=markdown)

* [Advanced WildFire](https://www.paloaltonetworks.com/network-security/advanced-wildfire?ts=markdown)

* [Advanced DNS Security](https://www.paloaltonetworks.com/network-security/advanced-dns-security?ts=markdown)

* [Enterprise Data Loss Prevention](https://www.paloaltonetworks.com/sase/enterprise-data-loss-prevention?ts=markdown)

* [Enterprise IoT Security](https://www.paloaltonetworks.com/network-security/enterprise-device-security?ts=markdown)

* [Medical IoT Security](https://www.paloaltonetworks.com/network-security/medical-device-security?ts=markdown)

* [Industrial OT Security](https://www.paloaltonetworks.com/network-security/medical-device-security?ts=markdown)

* [SaaS Security](https://www.paloaltonetworks.com/sase/saas-security?ts=markdown)

* [Next-Generation Firewalls](https://www.paloaltonetworks.com/network-security/next-generation-firewall?ts=markdown)

* [Hardware Firewalls](https://www.paloaltonetworks.com/network-security/hardware-firewall-innovations?ts=markdown)

* [Software Firewalls](https://www.paloaltonetworks.com/network-security/software-firewalls?ts=markdown)

* [Strata Cloud Manager](https://www.paloaltonetworks.com/network-security/strata-cloud-manager?ts=markdown)

* [SD-WAN for NGFW](https://www.paloaltonetworks.com/network-security/sd-wan-subscription?ts=markdown)

* [PAN-OS](https://www.paloaltonetworks.com/network-security/pan-os?ts=markdown)

* [Panorama](https://www.paloaltonetworks.com/network-security/panorama?ts=markdown)

* [Secure Access Service Edge](https://www.paloaltonetworks.com/sase?ts=markdown)

* [Prisma SASE](https://www.paloaltonetworks.com/sase?ts=markdown)

* [Application Acceleration](https://www.paloaltonetworks.com/sase/app-acceleration?ts=markdown)

* [Autonomous Digital Experience Management](https://www.paloaltonetworks.com/sase/adem?ts=markdown)

* [Enterprise DLP](https://www.paloaltonetworks.com/sase/enterprise-data-loss-prevention?ts=markdown)

* [Prisma Access](https://www.paloaltonetworks.com/sase/access?ts=markdown)

* [Prisma Browser](https://www.paloaltonetworks.com/sase/prisma-browser?ts=markdown)

* [Prisma SD-WAN](https://www.paloaltonetworks.com/sase/sd-wan?ts=markdown)

* [Remote Browser Isolation](https://www.paloaltonetworks.com/sase/remote-browser-isolation?ts=markdown)

* [SaaS Security](https://www.paloaltonetworks.com/sase/saas-security?ts=markdown)

* [AI-Driven Security Operations Platform](https://www.paloaltonetworks.com/cortex?ts=markdown)

* [Cloud Security](https://www.paloaltonetworks.com/cortex/cloud?ts=markdown)

* [Cortex Cloud](https://www.paloaltonetworks.com/cortex/cloud?ts=markdown)

* [Application Security](https://www.paloaltonetworks.com/cortex/cloud/application-security?ts=markdown)

* [Cloud Posture Security](https://www.paloaltonetworks.com/cortex/cloud/cloud-posture-security?ts=markdown)

* [Cloud Runtime Security](https://www.paloaltonetworks.com/cortex/cloud/runtime-security?ts=markdown)

* [Prisma Cloud](https://www.paloaltonetworks.com/prisma/cloud?ts=markdown)

* [AI-Driven SOC](https://www.paloaltonetworks.com/cortex?ts=markdown)

* [Cortex XSIAM](https://www.paloaltonetworks.com/cortex/cortex-xsiam?ts=markdown)

* [Cortex XDR](https://www.paloaltonetworks.com/cortex/cortex-xdr?ts=markdown)

* [Cortex XSOAR](https://www.paloaltonetworks.com/cortex/cortex-xsoar?ts=markdown)

* [Cortex Xpanse](https://www.paloaltonetworks.com/cortex/cortex-xpanse?ts=markdown)

* [Unit 42 Managed Detection \& Response](https://www.paloaltonetworks.com/cortex/managed-detection-and-response?ts=markdown)

* [Managed XSIAM](https://www.paloaltonetworks.com/cortex/managed-xsiam?ts=markdown)

* [Threat Intel and Incident Response Services](https://www.paloaltonetworks.com/unit42?ts=markdown)

* [Proactive Assessments](https://www.paloaltonetworks.com/unit42/assess?ts=markdown)

* [Incident Response](https://www.paloaltonetworks.com/unit42/respond?ts=markdown)

* [Transform Your Security Strategy](https://www.paloaltonetworks.com/unit42/transform?ts=markdown)

* [Discover Threat Intelligence](https://www.paloaltonetworks.com/unit42/threat-intelligence-partners?ts=markdown)

## Company

* [About Us](https://www.paloaltonetworks.com/about-us?ts=markdown)
* [Careers](https://jobs.paloaltonetworks.com/en/)
* [Contact Us](https://www.paloaltonetworks.com/company/contact-sales?ts=markdown)
* [Corporate Responsibility](https://www.paloaltonetworks.com/about-us/corporate-responsibility?ts=markdown)
* [Customers](https://www.paloaltonetworks.com/customers?ts=markdown)
* [Investor Relations](https://investors.paloaltonetworks.com/)
* [Location](https://www.paloaltonetworks.com/about-us/locations?ts=markdown)
* [Newsroom](https://www.paloaltonetworks.com/company/newsroom?ts=markdown)

## Popular Links

* [Blog](https://www.paloaltonetworks.com/blog/?ts=markdown)
* [Communities](https://www.paloaltonetworks.com/communities?ts=markdown)
* [Content Library](https://www.paloaltonetworks.com/resources?ts=markdown)
* [Cyberpedia](https://www.paloaltonetworks.com/cyberpedia?ts=markdown)
* [Event Center](https://events.paloaltonetworks.com/)
* [Manage Email Preferences](https://start.paloaltonetworks.com/preference-center)
* [Products A-Z](https://www.paloaltonetworks.com/products/products-a-z?ts=markdown)
* [Product Certifications](https://www.paloaltonetworks.com/legal-notices/trust-center/compliance?ts=markdown)
* [Report a Vulnerability](https://www.paloaltonetworks.com/security-disclosure?ts=markdown)
* [Sitemap](https://www.paloaltonetworks.com/sitemap?ts=markdown)
* [Tech Docs](https://docs.paloaltonetworks.com/)
* [Unit 42](https://unit42.paloaltonetworks.com/)
* [Do Not Sell or Share My Personal Information](https://panwedd.exterro.net/portal/dsar.htm?target=panwedd)
  ![PAN logo](https://www.paloaltonetworks.com/etc/clientlibs/clean/imgs/pan-logo-dark.svg)
* [Privacy](https://www.paloaltonetworks.com/legal-notices/privacy?ts=markdown)
* [Trust Center](https://www.paloaltonetworks.com/legal-notices/trust-center?ts=markdown)
* [Terms of Use](https://www.paloaltonetworks.com/legal-notices/terms-of-use?ts=markdown)
* [Documents](https://www.paloaltonetworks.com/legal?ts=markdown)

Copyright © 2026 Palo Alto Networks. All Rights Reserved

* [![Youtube](https://www.paloaltonetworks.com/etc/clientlibs/clean/imgs/social/youtube-black.svg)](https://www.youtube.com/user/paloaltonetworks)
* [![Podcast](https://www.paloaltonetworks.com/content/dam/pan/en_US/images/icons/podcast.svg)](https://www.paloaltonetworks.com/podcasts/threat-vector?ts=markdown)
* [![Facebook](https://www.paloaltonetworks.com/etc/clientlibs/clean/imgs/social/facebook-black.svg)](https://www.facebook.com/PaloAltoNetworks/)
* [![LinkedIn](https://www.paloaltonetworks.com/etc/clientlibs/clean/imgs/social/linkedin-black.svg)](https://www.linkedin.com/company/palo-alto-networks)
* [![Twitter](https://www.paloaltonetworks.com/etc/clientlibs/clean/imgs/social/twitter-x-black.svg)](https://twitter.com/PaloAltoNtwks)
* EN  
  Select your language