Snowpark for Python GA

November 10, 2022

Snowflake introduced new innovations that disrupt application development and enable developers, data engineers, and data scientists to build directly in the Data Cloud. Snowflake’s latest advancements empower users to do more with their data, enhancing productivity and unlocking new ways to develop applications, pipelines, and machine learning models with Snowflake’s single data platform.

Snowflake Brings Python-Based App Development Directly to the Data Cloud

Streamlit, acquired by Snowflake in March 2022, enables tens of thousands of data scientists and other developers to easily build data applications with Python using its open source framework. Following the acquisition, Snowflake is now advancing its Streamlit integration (in development), so developers can bring their data and machine learning (ML) models to life as secure, interactive applications — all within Snowflake.

Snowflake’s Streamlit integration will bring together Streamlit’s ease of use and flexibility, with Snowflake’s scalability, governed data coverage, and security, so developers can build powerful applications without the traditional complexity involved with building and deploying web applications. The integration will allow developers to create applications with Python using their data in Snowflake, deploy and run these applications on Snowflake’s secure and governed platform, and share their applications with business teams to further unlock the value of data and ML models.

“Streamlit serves as the interaction engine for the vast majority of our Data Science & Machine Learning models today, actively transforming how our teams build, deploy, and collaborate on powerful applications with other stakeholders across the business,” said Sai Ravuru, GM Data Science & Analytics, JetBlue. “With Snowflake’s Streamlit integration, we can go from data to ML-insights all within the Snowflake ecosystem, where our data is already present, making it easier and more secure for us to create impactful applications to further mitigate the negative impact of flight disruptions, provide more predictability to our operational planning teams, and more customer personalization to give our customers the best possible experience.”

Snowflake Extends the Power of Python to All Users with Expansive Snowpark Ecosystem

With Python as the most1 popular language for data scientists and the third2 most popular language among all developers, Snowflake is now making Python and its rich ecosystem of open source libraries available to all users and teams with the general availability of Snowpark for Python. In the months since its public preview announcement and expanded Anaconda integration at Snowflake Summit 2022, Snowpark for Python has seen 6x growth in adoption, with hundreds of customers including Charter Communications, EDF, NerdWallet, Northern Trust, Sophos, and more building with their data using Snowpark.

With Snowpark as Snowflake’s developer framework, developers gain a streamlined architecture that natively supports users’ programming languages of choice including Java, Scala, SQL, and now Python. Snowpark for Python is part of the wider Snowpark ecosystem, bringing teams together so that they can collaborate and build on one unified platform with a highly secure Python sandbox, providing developers with the same scalability, elasticity, security, and compliance benefits they’ve come to expect when building with Snowflake. In addition, developers can eliminate data security and compliance roadblocks that have previously prevented projects from going into production. Snowflake is also releasing Snowpark-optimized warehouses (public preview in AWS), so Python developers can run large scale ML training and other memory-intensive operations directly in Snowflake, and Python Worksheets (private preview) to develop applications, data pipelines, and ML models inside Snowflake.

Partners like Anaconda, dbt Labs, and more have been instrumental in accelerating the adoption of Snowpark for Python and allowing developers to build with confidence. These advancements include Anaconda’s integration with Snowpark for Python, which make Anaconda’s open-source Python libraries seamlessly accessible to Snowflake users by eliminating the need for manual installs and package dependency management. In addition, dbt’s new Snowpark for Python support effortlessly combines the power of SQL and Python for modern analytics, enabling customers to further bridge the gap between analytics and data science teams.

“With Snowpark for Python, we can build a fully integrated data science ecosystem that empowers financial engineers, data scientists, and business analysts to work with the full wealth of their data to build and deliver custom analytics,” said William Wu, Head of Quant Analytics, FOA, Northern Trust. “Snowpark enables us to collaborate on the same data as the rest of the business, so we can uncover and visualize new insights, and drive dynamic fact-based and data-driven investment decision-making for our customers.”

Snowflake Simplifies Streaming Pipelines and Drives Increased Automation and Observability for Developers

Snowflake is also reimagining how users build data pipelines, making it easier to work with streaming data within a single platform, and further eliminating silos for customers. To do so, Snowflake is equipping them with the capabilities needed to eliminate complexity while leveraging core software development principles. Users can now improve productivity by onboarding data faster with Schema Inference (private preview), and execute pipelines effortlessly with Serverless Tasks (general availability) natively in Snowflake’s platform. Additionally, Snowflake is unveiling enhanced tools that further empower developers to build in the Data Cloud including:

Dynamic Tables (private preview): Formerly introduced as Materialized Tables, Snowflake is removing the boundaries between streaming and batch pipelines by automating incremental processing through declarative data pipelines development for coding efficacy and ease. This also simplifies use cases including change data capture and snapshot isolation, and is native to Snowflake so it can be shared across all Snowflake accounts with full security and governance.

Observability & Experiences: To further meet the needs of developers, Snowflake is investing in native observability and developer experience features so they can build, test, debug, deploy, and monitor data pipelines with increased productivity through alerting (private preview), logging (private preview), event tracing (private preview), task graphs and history (public preview), and more.

“As we continue to disrupt application development, we’re giving builders the data access and tools they need to accelerate their pace of innovation securely under Snowflake’s one unified platform,” said Torsten Grabs, Director of Product Management, Snowflake. “Snowflake’s advancements provide developers with the capabilities to build powerful applications, pipelines, and models with the utmost confidence, and eliminate complexity so they can drive value across their organizations with the Data Cloud.”

Snowflake also announced new innovations to Snowflake’s industry-leading data platform that will further drive economic value for customers, enhancements to its extensive partner ecosystem, and more at Snowday 2022.

“Snowpark for Python has created new opportunities and use cases for our team to build and deploy secure and compliant data pipelines on Snowflake, so we can more efficiently provide our customers with the tools needed to handle every aspect of their finance journey,” said Sathish Balakrishnan, Director of Data Engineering, NerdWallet. “Snowflake’s continued investments in Python allow us the flexibility to code in our programming language of choice, and accelerate the speed of innovation for our end users."

“Our mission is to fundamentally protect the customer as best as we can, as effectively as we can, and Snowpark for Python enables our team to build better detection models so we can do just this,” said Konstantin Berlin, Head of Artificial Intelligence, Sophos. “Everything in our industry revolves around Python, and Snowpark enables our data scientists with simple code that’s maintainable, and trackable, so we can significantly increase our pace of innovation.”