Life of a Packet in ISTIO — Envoy Proxy — Part 2
Recap: In Part 1 of the “Life of a Packet in ISTIO” series, we explored the concept of a service mesh and Istio, an open-source service mesh that provides features like traffic management, observability, policy enforcement, and service identity and security. We then delved into the traffic interception process in Istio’s data plane, which is composed of a collection of proxy services represented as sidecar containers in each Kubernetes pod using an extended Envoy proxy. We discussed sidecar injection and the roles of the istio-proxy and istio-init containers. We then covered traffic interception for inbound and outbound traffic, and analyzed the NAT table rules that handle traffic routing.
In this next part (2) of the “Life of a Packet in ISTIO” series, We will introduce Envoy proxy configuration, and discuss how it works for inbound and outbound traffic. By the end of this part, you will have a clear understanding of how Istio handles traffic between microservices in the service mesh.
In the world of the service mesh, Envoy has emerged as one of the most popular proxy servers used to handle the traffic between services. Developed by Lyft and now a graduated project of the Cloud Native Computing Foundation (CNCF), Envoy provides a scalable, high-performance, and extensible solution for handling network traffic between services.
Envoy is designed to work as a sidecar proxy, running alongside each service instance in a service mesh. It intercepts all traffic between services, providing features such as load balancing, traffic routing, traffic management, and security. Envoy is a powerful tool that can be configured in many different ways to meet the specific needs of your service mesh.
In this post, we will take a deep dive into the configuration of Envoy, exploring the different components that make up an Envoy configuration file, and how they work together to create a robust and flexible service mesh. We will also provide examples of how to configure Envoy for different use cases, such as service discovery, routing, and traffic shaping.
Whether you’re just getting started with Envoy or are a seasoned user looking to deepen your understanding of its configuration options, this blog post will provide the knowledge you need to take your service mesh to the next level.
If you’re interested in learning more about Kubernetes, check out my ‘Life of a Packet in Kubernetes’ blog for a deep dive into the Kubernetes network model and how packets flow in the cluster. You can find the blog post here: https://dramasamy.medium.com/life-of-a-packet-in-kubernetes-part-1-f9bc0909e051?source=friends_link&sk=9525ced03e4683afca531c21914625eb
Here are some of the specific ways that Envoy features help in Istio:
- Traffic management: Envoy’s dynamic routing capabilities allow Istio to manage traffic at the service mesh level. This enables Istio to perform traffic routing, load balancing, and service discovery without requiring changes to application code. Istio uses Envoy’s advanced routing features, including path-based routing, header-based routing, and fault injection, to implement advanced traffic management policies.
- Security: Envoy provides Istio with strong security features, including encryption and authentication. Istio uses Envoy’s TLS and mTLS support to secure communication between services in the service mesh. Envoy’s advanced access control features, including rate limiting and RBAC, allow Istio to enforce fine-grained security policies across the service mesh.
- Observability: Envoy’s observability features enable Istio to monitor and troubleshoot the service mesh. Envoy’s rich telemetry data, including detailed metrics and logs, can be used to diagnose issues and optimize performance. Istio uses Envoy’s distributed tracing capabilities to provide end-to-end tracing across the service mesh, allowing users to understand how requests flow through the mesh and identify performance bottlenecks.
- Extensibility: Envoy’s modular architecture allows Istio to easily add new functionality to the service mesh. Istio can add new Envoy filters or configure existing filters to implement custom logic, enabling users to extend Istio’s capabilities as needed.
Overall, Envoy’s rich feature set makes it an ideal choice for powering Istio’s service mesh platform. Envoy’s advanced traffic management, security, observability, and extensibility features enable Istio to provide a powerful and flexible platform for managing microservices at scale.
Envoy Proxy Processing Steps: How a Packet is Handled
When a packet arrives at an Envoy proxy, it goes through a series of steps before it reaches its final destination. These steps include the following:
At the ingress phase, the Envoy listener component accepts incoming connections from the downstream client on specific ports and protocols such as TCP, HTTP, or HTTPS. The listener filter chain provides SNI and other pre-TLS details and matches a network filter chain based on destination IP CIDR range, SNI, ALPN, source ports, etc. A transport socket, such as the TLS transport socket, is associated with the filter chain for secure communication.
The network filter chain decrypts the data from the TCP connection using the TLS transport socket on network reads, and the HTTP connection manager filters are the last in the chain. The HTTP/2 codec in the HTTP connection manager deframes and demultiplexes the decrypted data stream from the TLS connection into independent streams, handling each request and response.
For each HTTP stream, a corresponding HTTP filter chain is created and run. The request passes through custom filters that can read and modify the request. The router filter is the most important HTTP filter, sitting at the end of the chain, and selects a route and a cluster based on the headers of the incoming request. The headers are then forwarded to an upstream endpoint in that cluster. The router filter obtains an HTTP connection pool from the cluster manager for the matched cluster to handle this process.
Cluster-specific load balancing is performed to find an endpoint, and circuit breakers are checked to determine if a new stream is allowed. A new connection to the endpoint is created if the endpoint’s connection pool is empty or lacks capacity. The upstream endpoint connection’s HTTP/2 codec multiplexes and frames the request’s stream with any other streams going to that upstream over a single TCP connection. The upstream endpoint connection’s TLS transport socket encrypts these bytes and writes them to a TCP socket for the upstream connection.
The request, consisting of headers, and optional body and trailers, is proxied upstream, and the response is proxied downstream. The response passes through the HTTP filters in the opposite order from the request, starting at the router filter and passing through custom filters before being sent downstream. When the response is complete, the stream is destroyed. Post-request processing is done, including updating stats, writing to the access log, and finalizing trace spans.
Finally, the egress configuration specifies the endpoint information for the downstream client, including transport protocol, encryption, and other settings. Envoy can be configured to forward the response to the downstream client, using the appropriate protocol and transport mechanism, allowing you to configure filters to process the response from the upstream service, add response headers, set cookies, perform caching, and more, and also to configure the access log filters to log the response.
Overall, the Envoy proxy configuration is highly customizable and can be tailored to meet the specific needs of your application architecture. By carefully configuring each phase of the packet life cycle, you can achieve advanced traffic management, observability, and security features for your modern applications.
Hands-On Demo: Envoy front proxy
Let’s see a practical example of how to configure an Envoy proxy using the provided YAML configuration. In this example, we are configuring a listener to accept incoming traffic on port 8000 and forward it to a cluster named “service” using round-robin load balancing. The cluster is defined as a strict DNS type and has three endpoints, each with an address of service_blue, service_green, and service_red, all listening on port 3000.
The connection manager is configured to generate request IDs, use automatic codec type detection, and prefix statistics with “ingress_http.” It also specifies a virtual host named “service” that accepts traffic for any domain and a route matching any requests with a prefix of “/service.” The route is then forwarded to the “service” cluster.
To secure the communication between Envoy and its clients, we are using a self-signed certificate pair for Transport Layer Security (TLS) authentication. The certificate pair is fed to Envoy as an inline string, but it can also be provided as a file or fetched remotely via Secret Discovery Service (SDS) in a dynamic configuration scenario.
Overall, this configuration demonstrates how Envoy can be used as a powerful and flexible proxy to handle incoming traffic and distribute it to a cluster of endpoints with load balancing and security features.
To try out the example with the given configuration, please clone the GitHub repository at https://github.com/dramasamy/training.git and navigate to the
envoydirectory. From there, you can rundocker-compose upto start the Envoy proxy container and test the configuration.
Here are the steps to execute docker-compose and browse the page to see the Envoy load balancing:
Step 1: Clone the repository by running the following command in your terminal:
git clone https://github.com/dramasamy/training.gitStep 2: Navigate to the envoy directory by running the following command:
cd training/envoy/front-proxyStep 3: Execute docker-compose to start the containers by running the following command:
docker compose up -dStep 4: Once the containers are up and running, open your browser and navigate to https://localhost:8000/service. This will show you the response from one of the three services, with load balancing being performed by Envoy.
Step 5: Refresh the page a few times and observe that the response changes between the three services (blue, green, and red), demonstrating the round-robin load balancing performed by Envoy.
Tracing in Envoy
Tracing is an essential feature that helps developers in understanding how different services are communicating with each other. When applications are distributed, the request can span across multiple services, making it challenging to understand what’s happening across the network. Envoy provides an easy way to capture and visualize traces through its Zipkin tracing feature.
Hands-On Demo: Jaeger Tracing
The provided configuration file demonstrates how to enable the Zipkin tracer in Envoy. In this configuration, Envoy acts as a front proxy and listens on port 8000. The traffic_direction field indicates that it is an outbound listener. The tracing field is enabled, which enables the Zipkin tracer with the collector's configuration information. The collector cluster is specified as "jaeger," and the collector endpoint is specified as "/api/v2/spans." The shared_span_context field is set to false, indicating that the span context is not propagated between services.
The route configuration is specified under virtual_hosts. The decorator field is used to set the operation name in the span. In this example, the decorator is set to "checkAvailability." The use_remote_address field is set to true, indicating that Envoy should use the client's IP address to determine the source address of the request.
The clusters section defines the different backend services and their associated clusters. In this example, there is only one backend service called service1. The load_assignment field defines how the endpoints should be assigned to the service. In this case, the endpoint is defined as service1:8000.
Overall, this configuration file enables Envoy to capture trace information using the Zipkin tracer and propagate span context across services. Developers can use this information to gain insight into how different services are communicating and to debug issues in their distributed applications.
Step 1: Navigate to the ‘training/envoy/jaeger’ directory:
cd training/envoy/jaeger/Step 2: Start the docker containers by running:
docker-compose up -dStep 3: Open two browser tabs and navigate to http://localhost:8000/trace/1 and http://localhost:8000/trace/2
Step 4: Refresh the pages a few times to generate some traffic.
Step 5: Open another browser tab and navigate to http://localhost:10000/search to access Jaeger’s search page.
Step 6: In the Jaeger search page, select ‘front-proxy’ as the service from the dropdown menu and click the ‘Find Traces’ button.
Step 7: You should now see a list of traces generated by Envoy, including the traces generated by the traffic to http://localhost:8000/trace/1 and http://localhost:8000/trace/2. Click on a trace to view its details and explore the distributed tracing functionality of Envoy.
One of the most important benefits of tracing from Envoy is that it will take care of propagating the traces to the Jaeger service cluster. However, in order to fully take advantage of tracing, the application has to propagate trace headers that Envoy generates, while making calls to other services. In the flask app (see trace function in envoy/shared/flask/tracing/service.py) acting as service1 propagates the trace headers while making an outbound call to service2.
Envoy UDP Proxy
The User Datagram Protocol (UDP) example in Envoy provides a simple demonstration of how Envoy can be used to proxy UDP traffic. The example includes an upstream server that listens for UDP traffic on port 5005 and an Envoy proxy that listens for UDP traffic on port 10000 and proxies it to the upstream server. Additionally, Envoy provides an admin endpoint on port 10001 that provides statistics on UDP traffic. This example demonstrates how Envoy can be used to handle not only HTTP but also UDP traffic.
Hands-On Demo: UDP proxy
Step 1: Change the directory to envoy/udp in the Envoy repository.
cd training/envoy/udp/Step 2: Start the docker containers by running:
docker-compose up -dThis starts Envoy and an upstream server on port 5005.
Step 2: Send some UDP messages Send packets to the upstream server, proxied by Envoy using netcat. Run the following commands:
echo -n HELO | nc -4u -w1 127.0.0.1 10000
echo -n OLEH | nc -4u -w1 127.0.0.1 10000Step 3: Check the logs of the upstream UDP listener server To view the packets that were sent, check the logs of the upstream server by running the following command:
lab1 $ docker compose logs service-udp
service-udp_1 | Listening on UDP port 5005
service-udp_1 | HELO
service-udp_1 | OLEHStep 4: View the Envoy admin UDP stats View the stats provided by the Envoy admin endpoint by running the following command:
lab1 $ curl -s http://127.0.0.1:10001/stats | grep udp | grep -v "\: 0"
cluster.service_udp.default.total_match_count: 15
cluster.service_udp.max_host_weight: 1
cluster.service_udp.membership_change: 1
cluster.service_udp.membership_healthy: 1
cluster.service_udp.membership_total: 1
cluster.service_udp.udp.sess_tx_datagrams: 2
cluster.service_udp.update_attempt: 15
cluster.service_udp.update_no_rebuild: 14
cluster.service_udp.update_success: 15
cluster.service_udp.upstream_cx_tx_bytes_total: 8
udp.service.downstream_sess_rx_bytes: 8
udp.service.downstream_sess_rx_datagrams: 2
udp.service.downstream_sess_total: 2
udp.service.idle_timeout: 2
cluster.service_udp.upstream_cx_connect_ms: No recorded values
cluster.service_udp.upstream_cx_length_ms: No recorded valuesThis will display the non-zero stats, including upstream connection data and downstream session data.
Dynamic XDS Configuration — Updating Envoy’s Configuration on the Fly
Envoy XDS is a core component of Istio, which is a service mesh that provides features such as traffic management, security, and observability to microservices-based applications.
Istio uses the Envoy proxy as a sidecar container alongside each service instance to handle incoming and outgoing traffic for that instance. Envoy XDS is used by Istio to dynamically configure the Envoy sidecars with the service mesh configuration.
Here is a high-level overview of how Envoy XDS works.
- The configuration server serves as a centralized repository for all the configuration information that the Envoy proxy needs to route traffic in the service mesh.
- The configuration server provides dynamic configuration updates to the Envoy proxy via the XDS (Discovery Service) protocol. The XDS protocol uses gRPC to enable communication between the control plane and the data plane.
- When a new service is added to the service mesh or when an existing service is updated, the configuration server sends a new configuration update to the Envoy proxy.
- The Envoy proxy applies the new configuration update and begins routing traffic according to the new rules.
- The configuration server can also provide health check information to Envoy, which enables Envoy to route traffic only to healthy instances of a service.
- In this way, the control plane and the data plane work together to provide dynamic service discovery and traffic routing in the service mesh.
Hands-On Demo: Using Envoy XDS
Step 1: Change the directory to envoy/dynamic-config-cp in the Envoy repository.
cd training/envoy/dynamic-config-cp/Step 2: Start the proxy container along with two upstream HTTP echo servers, service1 and service2.
docker-compose up -d proxyStep 3: As the control plane has not yet been started, nothing should be responding on port 10000. To check this, dump the proxy’s static_clusters configuration by running the command curl -s http://localhost:19000/config_dump | jq '.configs[1].static_clusters'. The output should show the cluster named xds_cluster configured for the control plane.
lab1 $ curl -s http://localhost:19000/config_dump | jq '.configs[1].static_clusters'
[
{
"cluster": {
"@type": "type.googleapis.com/envoy.config.cluster.v3.Cluster",
"name": "xds_cluster",
"type": "STRICT_DNS",
"load_assignment": {
"cluster_name": "xds_cluster",
"endpoints": [
{
"lb_endpoints": [
{
"endpoint": {
"address": {
"socket_address": {
"address": "go-control-plane",
"port_value": 18000
}
}
}
}
]
}
]
},
"typed_extension_protocol_options": {
"envoy.extensions.upstreams.http.v3.HttpProtocolOptions": {
"@type": "type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions",
"explicit_http_config": {
"http2_protocol_options": {}
}
}
}
},
"last_updated": "2023-03-12T23:16:57.273Z"
}
]Step 4: No dynamic_active_clusters have been configured yet:
lab1 $ curl -s http://localhost:19000/config_dump | jq '.configs[1].dynamic_active_clusters'
nullStep 5: Start up the go-control-plane service by running the command docker compose up --build -d go-control-plane. You may need to wait a moment or two for it to become healthy.
Step 6: Once the control plane has started and is healthy, you should be able to make a request to port 10000, which will be served by service1.
lab1 $ curl http://localhost:10000
Request served by service1
GET / HTTP/1.1
Host: localhost:10000
Accept: */*
User-Agent: curl/7.85.0
X-Envoy-Expected-Rq-Timeout-Ms: 15000
X-Forwarded-Proto: http
X-Request-Id: 8df92d24-6656-44d2-90d9-53f00d3af091Step 7: To verify that the dynamic configuration is working, dump the proxy’s dynamic_active_clusters configuration by running the command curl -s http://localhost:19000/config_dump | jq '.configs[1].dynamic_active_clusters'. The output should show that the example_proxy_cluster is pointing to service1 and has a version of 1.
lab1 $ curl -s http://localhost:19000/config_dump | jq '.configs[1].dynamic_active_clusters'
[
{
"version_info": "1",
"cluster": {
"@type": "type.googleapis.com/envoy.config.cluster.v3.Cluster",
"name": "example_proxy_cluster",
"type": "LOGICAL_DNS",
"connect_timeout": "5s",
"dns_lookup_family": "V4_ONLY",
"load_assignment": {
"cluster_name": "example_proxy_cluster",
"endpoints": [
{
"lb_endpoints": [
{
"endpoint": {
"address": {
"socket_address": {
"address": "service1",
"port_value": 8080
}
}
}
}
]
}
]
}
},
"last_updated": "2023-03-12T23:26:24.879Z"
}
]Summary
In part two, we discussed load balancing and tracing in Envoy. We started by explaining the basic front proxy example and then added load balancing to it using the round-robin algorithm. Next, we explored how tracing works in Envoy and provided an example configuration for it. We also asked the user to execute Docker Compose and browse the page to see the load balancing and tracing in action.
Additionally, we introduced the concept of Envoy XDS and explained a sample configuration for it, including CDS and LDS. Finally, we briefly discussed how Envoy XDS works in Istio.
Stay tuned for part three, where we will dive into Istio’s control plane components.
References:
- Envoy documentation: https://www.envoyproxy.io/docs/envoy/latest/
- Envoy GitHub repository: https://github.com/envoyproxy/envoy
- Istio documentation: https://istio.io/latest/docs/
- Jaeger documentation: https://www.jaegertracing.io/docs/
- Zipkin documentation: https://zipkin.io/
Disclaimer
This article is for informational purposes only and does not provide any technical advice or recommendations. The views expressed in this article are solely those of the author and do not represent the views of any company or organization. Any action you take based on the information in this article is at your own risk.
