Organizations often maintain two parallel identity systems: one from cloud providers like AWS and another for internal services. This post details how Apache Spark workloads on Amazon EMR bridge this gap, enabling them to gain a first-class internal identity from an initial cloud identity. Netflix utilizes a private PKI called Metatron for service-to-service authentication via mutual TLS, requiring an attestation process to verify workload identities. They also employ a "Data Project" concept, which owns data and has its own identity, ensuring consistency in access control and auditing.The core challenge is translating a cloud-based AWS execution role into the necessary internal identity for Spark jobs on managed compute. This solution establishes a direct, one-to-one mapping between each Data Project identity and a dedicated IAM role, with the Data Project service serving as the authority for this mapping. This mapping is crucial for translating statements from the cloud provider's vocabulary to the organization's internal one. To address the scalability issue of managing numerous IAM roles, these roles are deterministically sharded across a small pool of dedicated AWS accounts.Five components are involved: the control plane (launches jobs and signs metadata), the Data Project service (manages the identity-to-role mapping), the Identity service (verifies attestations and issues certificates), a Spark plugin (hooks into process bootstrapping), and AWS STS (acts as a notary). The process begins with the control plane signing a metadata payload containing workload details. Subsequently, the Spark driver plugin uses AWS credentials to create a pre-signed URL from AWS STS, proving possession of the cloud identity.The Identity service then corroborates these two independent claims: the role identified by AWS from the pre-signed URL and the signed metadata from the control plane. If they align, certificates are issued for the workload. This corroboration is vital as neither statement alone is sufficient; the provider's statement is unforgeable but underspecified, while the control plane's statement is well-specified but potentially replayable. The system addresses the fan-out problem for executors by having the driver attest once and distribute credentials securely. Certificates are short-lived and renewed through a repeatable attestation process within the driver JVM. The trust model ensures no single participant can issue an identity independently, and the workload itself is never required to vouch for its own legitimacy.