Amazon Security Lake Debuts

November 30, 2022

Amazon Security Lake is a service that automatically centralizes an organization’s security data from cloud and on-premises sources into a purpose-built data lake in a customer’s AWS account so customers can act on security data faster. Amazon Security Lake manages data throughout its lifecycle with customizable data retention settings, converts incoming security data to the efficient Apache Parquet format, and conforms it to the Open Cybersecurity Schema Framework (OCSF) open standard to make it easier to automatically normalize security data from AWS and combine it with dozens of pre-integrated third-party enterprise security data sources. Security analysts and engineers can use Amazon Security Lake to aggregate, manage, and optimize large volumes of disparate log and event data to enable faster threat detection, investigation, and incident response to effectively address potential issues quickly, while continuing to utilize their preferred analytics tools.

Customers want greater visibility into security activity across their entire organizations to proactively identify potential threats and vulnerabilities, assess security alerts, respond accordingly, and help prevent future security events. To do this, most organizations rely on log and event data from many different sources (e.g., applications, firewalls, and identity systems) running in the cloud and on premises, each using a unique and often incompatible data format. To uncover security-related insights, like spotting unauthorized external data transfers for sensitive information or identifying the installation of malware across employee devices, organizations must first aggregate and normalize all this data into a consistent format. Once the data is formatted consistently, customers can analyze it and understand the current level of vulnerability, and then correlate and monitor threats for improved observability. Customers typically use different security solutions to address specific use cases, such as incident response and security analytics, which often means they duplicate and process the same data multiple times because each solution has its own data stores and format. This is time consuming and costly, slowing down security teams' ability to detect and respond to issues. As customers add new users, tools, and data sources, security teams must also spend time managing a complex set of data-access rules and security policies to track how data is used and ensure people can get the information they need. Some security teams create a central repository for all their security data in a data lake, but these systems require specialized skills and can take months to build due to the large amount of log data from different sources, which can run into petabyte scale.

Amazon Security Lake is a purpose-built security data lake that can be created in just a few clicks and enables customers to aggregate, normalize, and store data so they can respond to security events faster using their preferred tools. After setup and connections to selected data sources, Amazon Security Lake automatically builds a security data lake in a customer-selected region, which can help customers meet regional data compliance requirements. After customers choose their data sources, Amazon Security Lake automatically aggregates and normalizes data from AWS, combines it with third-party sources that support OCSF (an open standard), and optimizes it into a format that is easy to store and query. Amazon Security Lake automatically orchestrates the end-to-end process from data lake creation and data aggregation to normalization and integration. The new service builds the security data lake using Amazon Simple Storage Service (Amazon S3) and AWS Lake Formation to automatically set up security data lake infrastructure in a customer’s AWS account, providing full control and ownership over security data. Once ingested and normalized, customers can use their preferred security and analytics tools, including Amazon Athena, Amazon OpenSearch, and Amazon SageMaker, along with leading third-party solutions (e.g., IBM, Splunk, or Sumo Logic) to make it faster and easier to capture broader and deeper analytics from AWS and more than 50 third-party (e.g., Cisco, CrowdStrike, and Palo Alto Networks) and customer data sources. As a result, Amazon Security Lake helps customers improve their overall security posture, provide greater visibility for security teams to identify and understand events, and reduce the time to resolve security issues.

“Customers must be able to quickly detect and respond to security risks so they can take swift action to secure data and networks, but the data they need for analysis is often spread across multiple sources and stored in a variety of formats. Customers tell us they want to take action on this data faster to improve their security posture, but the process of collecting, normalizing, storing, and managing this data is complex and time consuming,” said Jon Ramsey, vice president for Security Services at AWS. “Amazon Security Lake lets customers of all sizes securely set up a security data lake with just a few clicks to aggregate logs and event data from dozens of sources, normalize it to conform with the OCSF standard, and make it more broadly usable so customers can take action quickly using their security tools of choice. With Amazon Security Lake, customers get superior visibility and control, with help from the largest ecosystem of security partners and solutions.”

Amazon Security Lake is available in preview today in US East (N. Virginia), US East (Ohio), US West (Oregon), Asia Pacific (Sydney), Asia Pacific (Tokyo), Europe (Frankfurt), and Europe (Ireland), with availability in additional AWS Regions coming soon.

FINRA is a government-authorized not-for-profit organization that oversees U.S. broker-dealers to protect investors and ensure the market’s integrity. “Every investor in America relies on fair financial markets. FINRA enables investors and firms to participate in the market with confidence by safeguarding its integrity. To do this successfully, we use a wide variety of the best security tools to secure our AWS environment and ensure the security of market data,” said Eric Pickersgill, chief information security officer at FINRA. “Amazon Security Lake makes it easier to gather all of our security data in the OCSF format, saving our security engineers substantial time and effort in deriving value from log and event data.”

Salesforce, the global CRM leader, empowers companies of every size and industry to digitally transform and create a 360° view of their customers. “Salesforce builds security into everything we do. As we scale to support the growth of our global customer base, our Detection and Response teams analyze petabytes of security logs to catch malicious activity and protect customer data,” said Vikram Rao, chief trust officer at Salesforce. “Amazon Security Lake streamlines that work by unifying security logs and events from AWS and other cloud providers—reducing time spent on log onboarding and coverage so that our engineers can focus on proactive prevention and incident response.”

Tinder is the world’s most popular app for meeting new people. Available in 190 countries and more than 40 languages, it’s been downloaded more than 530 million times and led to more than 75 billion matches. “Because our users entrust Tinder with their information, the security of our application and the privacy of our customers’ data is our top priority. Ensuring that we maintain a robust, transparent, and accountable security program is core to our commitment to our customers,” said Jonathan Walker, DevSecOps manager II at Tinder. “Amazon Security Lake has drastically reduced time and money in our efforts to query security events at scale across regions, sources, and events. This has allowed our team to shift our focus away from data engineering to analyzing security events within the cloud.”