Google Cloud Blog
Follow
Storage Insights datasets: How to optimize storage spend with deep visibility
Managing vast amounts of data in Google Cloud Storage presents challenges for storage administrators. Understanding storage spend, data distribution, and searching for specific metadata are common concerns. Storage Insights datasets, a feature of Storage Intelligence, address these by providing deep visibility into Cloud Storage data. These datasets transform raw storage metadata into structured, queryable data within BigQuery. This allows for familiar SQL analysis, offering insights into cost management and visibility. Key features include customizable scope, a queryable metadata dataset, and regular 24-hour updates. Use cases range from calculating showback and understanding storage class distribution to setting lifecycle and Autoclass policies. The datasets also aid in proactive cleanup by identifying duplicates and temporary objects. Further analysis can be done with tools like Looker Studio for visualization and dashboarding. By integrating with billing exports, a complete cost picture can be achieved.