Fabric Data Warehouse integrat... Note

Fabric Data Warehouse integration with Fabric User Data Functions (Preview)

Microsoft Fabric Data Warehouse now allows extending T-SQL with custom Python logic directly within the warehouse. This integration, currently in preview, eliminates the need for separate applications or pipelines to execute Python code. You can now create T-SQL external functions that reference Python functions defined in Fabric User Data Functions items. These functions can be invoked from familiar T-SQL statements like SELECT, INSERT, and UPDATE. The Data Warehouse then delegates the execution of these external functions to the Fabric User Data Functions runtime. This new capability unlocks powerful scenarios such as integrating with external APIs, sending notifications, and processing various file formats. It also facilitates accessing data across OneLake, Fabric databases, and Azure Cosmos DB. Sensitive data can be encrypted, decrypted, or masked using custom Python logic and organizational keys. Furthermore, data can be enriched and validated through services like geocoding or machine learning models. An example demonstrates using Python to classify IPv6 network events, which is complex in T-SQL but straightforward with Python's ipaddress library. This allows for persistent storage of network analysis results, simplifying downstream queries. Essentially, Fabric User Data Functions enable continuous work in T-SQL while leveraging Python for advanced functionalities previously requiring separate development efforts.