Piqama: Pinterest Quota Manage... Note

Piqama: Pinterest Quota Management Ecosystem

Pinterest developed Piqama, a generic quota management platform, to oversee resource usage across various systems. Piqama manages the entire quota lifecycle, including schema management, validation, and update authorization through a centralized portal. The platform provides default enforcement and punishment strategies while allowing for customization via application-specific logic. Governance and optimization features include collecting usage statistics and enabling auto-rightsizing for efficient resource allocation. Budgets and quotas are interconnected, with chargeback systems influencing quota settings and resource allocation. Piqama is implemented in two areas: capacity-based quota management for the Big Data Processing Platform and rate-limiting quotas for online storage. In the Big Data Platform, Piqama manages memory, vcore, and concurrent applications, with automatic and manual adjustments to quotas. The Big Data Platform uses Yunikorn and a resource database for accurate quota calculation and enforcement. A new rate limiting framework for online storage is introduced to enhance system resource allocation and cost control. The rate-limiting framework aims to streamline lifecycle management, connect the rate limit to resource usage and utilize Piqama as its control plane. This approach offers a robust, flexible, and centralized solution for managing resources across Pinterest's diverse platforms to avoid any manual adjustments.
CdXz5zHNQW_acxjx5IRwX.png