Simplify your streaming pipelines with new Pub/Sub Single Message Transforms

Pub/Sub Single Message Transforms (SMTs) are introduced to simplify data transformation directly within Pub/Sub. These transforms address the need to modify streaming data for various use cases, like data sharing and normalization. Previously, simple transformations required altering sources/destinations or using external components, leading to added overhead. Pub/Sub SMTs aim to streamline streaming architectures, following the model of Import Topics and Export Subscriptions. The first available SMT is JavaScript User-Defined Functions (UDFs), enabling lightweight modifications via JavaScript code. These UDFs allow for common transformations, enhanced filtering, and data masking within Pub/Sub. UDFs offer flexibility, simplifying pipelines and improving performance compared to alternative approaches. Transforms can be applied at the topic, subscription, or both levels to maintain decoupling between publishers and subscribers. Users can easily implement JavaScript UDFs through the Google Cloud console or the gcloud CLI. Future SMTs, including schema validation and AI inference, are planned for release.