Cilium's ipcache scalability issue: Understanding identity distribution in Kubernetes clusters for optimized network policy.

Introduction: The Cilium ipcache Scalability Challenge Cilium’s ipcache, a critical component for enforcing identity-based network policies in Kubernetes, faces scalability limitations as clusters approach and exceed 1 million pods. Analogous to a centralized registry tracking unique resident IDs in a metropolis, the ipcache maps pod IP addresses to security identities, enabling fine-grained policy enforcement. However, its scalability bottleneck arises from the distribution of unique identities within the cluster. Each pod’s identity, derived from labels, annotations, and namespace, contributes to a mapping stored in the ipcache. As the number of distinct identities proliferates, the ipcache—a centralized, hash table-like structure—encounters increased collisions and operational overhead, directly degrading performance. The scalability challenge is rooted in the empirical distribution of pod identities. Real-world clusters exhibit bimodal patterns: a minority of large identity groups