Google Cloud Blog
Follow
Boost BigQuery with Python: Managed Python UDFs now generally available
SQL is excellent for structured data analysis, but struggles with complex procedural logic, scientific computations, and machine learning. Python excels at these tasks. Previously, data practitioners faced infrastructure management overhead to run custom Python code. BigQuery Managed Python User-Defined Functions (UDFs) are now generally available, addressing this challenge. This feature allows users to execute custom Python code directly within BigQuery using SQL or BigQuery DataFrames. These UDFs leverage fully managed serverless resources that automatically scale. They offer flexibility by enabling access to the vast Python ecosystem, including libraries like NumPy and scikit-learn. Furthermore, they allow real-time integration with external APIs and Google Cloud services. Advanced users can optimize performance through vectorized processing with Pandas PyArrow, configurable container resources, and customizable concurrency. Streaming logs and real-time metrics facilitate debugging and monitoring. Python UDFs are billed under the BigQuery Services SKU and are eligible for cost commitments. Getting started involves exploring product documentation and public datasets.