K8s NetworkPolicy Default Deny: A 3-Step Guide to Safe Adoption Without Service Disruption

When a Working Service Stops: A Safe Adoption Guide for K8s NetworkPolicy Default Deny

"The service was communicating perfectly until yesterday, but this morning communication between certain microservices suddenly stopped."

If you have seen this in a Kubernetes environment, you have probably thought hard about NetworkPolicy at least once. NetworkPolicy is a powerful feature that can dramatically raise a cluster's security posture, but it is also a double-edged sword. Apply a single policy incorrectly, or fail to understand how your CNI behaves, and it can paralyze the entire service at the worst possible moment.

This guide goes beyond listing how to apply NetworkPolicy. It presents a systematic methodology for anticipating unintended service disruption (DoS) scenarios in production, understanding CNI-specific behavior, and introducing a Default Deny policy as safely as possible.

How Network Policies Work and the Pitfalls of Default Deny

What Is a NetworkPolicy?

A NetworkPolicy is a specification that explicitly defines who can send what traffic where, for particular Pods or Namespaces. By default, a Kubernetes cluster is close to a "permissive" mode that allows communication between all Pods. Applying a NetworkPolicy means breaking that default-allow rule and establishing the principle that only explicitly allowed traffic is permitted.

The Appeal and Risks of Default Deny

Default Deny is the strongest security model. It means "block all traffic that is not explicitly allowed," which aligns with a core principle of Zero Trust Architecture (ZTA).

⚠️ Warning: The Default Deny Trap Applying Default Deny blindly is very dangerous. If you omit even one seemingly minor dependency—a backend database, cache server, or monitoring agent—that service will silently enter a "no communication" state. This is not a simple outage; it can be a denial of service caused by your security policy.

Fundamental Differences in How CNI Plugins Enforce Policies

The entity that actually enforces NetworkPolicy in the cluster is the CNI (Container Network Interface) plugin. Calico and Cilium are the main players, and understanding how they process policies is essential for debugging.

Advice from a practitioner's perspective: If your cluster is very large and you need complex application-level control (for example, "allow only a specific API path"), eBPF-based Cilium can offer more flexibility and performance advantages in the long run. However, if you are already comfortable with Calico, or you only need IP/port-level control, it is safer to take a gradual approach that fits your current environment.

[Hands-on Guide] Introducing Default Deny Safely in 3 Steps

Introducing Default Deny is really the process of writing a complete allowlist. Follow these three steps in order.

Step 1: Establish a Baseline and Run in Logging Mode

First, do not deny traffic. Change your CNI settings so that you only log which traffic would violate the policy.

• Goal: Record 100% of the normal traffic flows between all services.
• Action: Apply a NetworkPolicy with policyTypes: [Ingress, Egress], but test in a mode where logging is enabled instead of blocking.

Step 2: Build and Apply the Allowlist (Whitelist)

Analyze the logs to identify every essential communication pair (Source Pod A $\rightarrow$ Destination Pod B: Port X). Based on that list, write a NetworkPolicy that contains only the minimum allow rules.

💡 Example Default Deny YAML (applied to Namespace secure-app)

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: secure-app
spec:
  podSelector: {} # 이 네임스페이스의 모든 파드에 적용
  policyTypes:
  - Ingress
  - Egress
  # 아래 규칙이 없으면, 모든 트래픽은 기본적으로 차단됨 (Default Deny)
  # 따라서, 여기에 '허용해야 할' 규칙만 명시적으로 추가해야 함.
  # 예시: Ingress로만 특정 네임스페이스의 Pod A로부터의 8080 포트만 허용
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: frontend
      namespaceSelector:
        matchLabels:
          name: ingress-ns
    ports:
    - protocol: TCP
      port: 8080

Step 3: Validate and Roll Out Gradually

Do not apply the finished policy to the entire namespace at once. Apply it sequentially, starting with the lowest-priority service groups, and observe that group's monitoring metrics (latency, error rate) for at least 24 hours.

🚨 5-Step Debugging Flow When You Suspect Traffic Is Being Blocked

If communication fails after applying a policy, work through these steps systematically.

1. Confirm scope: Check whether a NetworkPolicy is applied to the affected Pod/Namespace.
2. Review policies: Inspect all applied policies (Ingress/Egress) and verify that the required ports and protocols are explicitly allowed.
3. Network trace: Use tools such as tcpdump or netshoot to determine whether packets never arrive at all (L3/L4 issue) or only the response is missing (application issue).
4. Temporarily bypass the policy: Temporarily disable the suspected policy and confirm whether communication recovers, so you can isolate the offending policy.
5. Re-apply least privilege: Once you have the root cause, modify the policy to allow only the minimum permissions needed for that communication, then re-apply it.

This process lets you pinpoint exactly which rule blocked the traffic, rather than treating it as a vague failure.

References: Official Documentation

The primary source for the behavior, configuration, and errors discussed in this article is the following official documentation. Check it for version-specific options and exact behavior.

• Kubernetes official documentation