Out of the box, Kubernetes doesn’t really have particular capabilities when it comes to network observability. Sure, you can collect metrics, traces, and various other data points, but you still have to take those data points and send them to a particular tool to have them ingested.

The raw data exists, but nothing that truly allows you to observe it in a digestible fashion.

In this blog post, you’ll learn about a few different methods for implementing network observability on Kubernetes.

Kubeshark

Kubernetes is sending data and traffic back and forth between Kubernetes components constantly. Whether it’s you as the engineer sending requests to the API server, the Scheduler talking to the API Server, the Kubelet listening to requests from the API Server, and every other method of communication.

And that’s just the Kubernetes components. That’s not the traffic between the Pods and Services from a networking perspective.

Kubeshark gives you a lot of visibility into this.

When you use Kubeshark, you can download a Pcap and open it up in Wireshark.

By doing so, you can analyze the traffic from APIs to Pod calls to various components like the Kubelet.

eBPF and Security-Based CNI’s

When it comes to how a network is deployed, although it may not be specific to showing particular monitors or logs, a security-based CNI like Cilium or Calico is ideal. They both have the ability to implement eBPF, which is designed to give you more security and network observability out of the box.

You get certain engineering capabilities like:

• Insights on changes to network behavior.
• Service-to-service communication patterns.
• Microbursts of activity which could indicate a potential exploit.

A huge implementation from a Kubernetes perspective with eBPF is the ability to remove the need for kube-proxy and iptables, which gives you a better method of scaling and not having to worry about rules at the iptable level and instead manage those rules with eBPF.

Outside of eBPF, security-centric CNI’s like Cilium and Calico also give you the ability to use Network Policies to control Ingress and Egress traffic.

Service Mesh

When you're setting up a Service Mesh in Kubernetes, encryption is very much talked about. The whole idea is to implement mTLS, which provides secure connections between Services. The problem is that although mTLS and the overall encryption mechanisms are great, it doesn’t give you any observability. Yes, it gives you security, but not observability into the network.

The ability to see what's happening on your network and what services are talking to other services, including inbound and outbound requests to various entities, is crucial to understanding the networking layer in your cluster.

Combining something like Istio with Kiali (the Istio dashboard) helps a ton here.

Even though encryption is on top of everyone’s mind when it comes to Service Mesh, that’s 20-30% of what a Service Mesh does. The biggest part of a service mesh is its ability to implement network latency and network traffic observability.

Built-In Traffic Controllers

Last but certainly not least is built-in traffic observability that you’ll get from a Managed Kubernetes Service like AKS or EKS. Although these monitors won’t dive as deep to tell you about pod-to-pod or service-to-service communication as a Service Mesh will, they give you the ability to view network traffic at the cluster level. You can see traffic coming in and going out including bandwidth pipes. This is a good method to use if you or someone else notices a delay or slowness in getting to a particular Service or other component running in Kubernetes.