Introduction to Benchmarking

Overview of Benchmarking Objectives

The goal of this benchmark is to isolate the impact of sidecar proxy resource policy (CPU and memory limits) on three observable dataplane metrics:

1. Handshake latency – time from TCP SYN to TLS finished (or HTTP/2 settings) as seen by the client.
2. Queue growth – buildup of pending connections or requests in the proxy’s listen/accept queues and internal work queues.
3. Tail‑response behavior – high‑percentile latency (p99, p99.9) of application‑level responses after the proxy has processed the request.

By varying CPU and memory caps while keeping the application workload constant, we can observe when the proxy’s resource policy becomes the dominant failure mode, rather than the application code itself.

Importance of Understanding Performance Failure Modes

In service‑mesh architectures the sidecar proxy sits on the critical path for every request. When latency spikes or request loss occur, operators often first look at application logs or code paths. However, if the proxy is starved of CPU or memory, the observed symptoms (increased handshake time, queue buildup, tail latency) are proxy‑induced. Recognizing this shift prevents wasted effort on application‑level optimizations and directs attention to resource allocation, scheduling, or proxy configuration tuning.

Environment Setup and Configuration

Sidecar Proxy Configuration

We use Envoy proxy (v1.26) as the sidecar because it exposes fine‑grained stats, supports dynamic listener filters, and can be run as a standalone process. A minimal configuration that terminates TLS and forwards HTTP/1.1 to a local upstream (127.0.0.1:8080) is shown below:

static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address: { address: 0.0.0.0, port_value: 8443 }
    filter_chains:
    - filter_chain_match:
        transport_protocol: "tls"
      filters:
      - name: envoy.filters.network.tcp_proxy
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
          stat_prefix: tcp_proxy
          cluster: service_backend
          # Enable TLS termination
          tls_context:
            common_tls_context:
              tls_certificates:
              - certificate_chain:
                  filename: "/etc/envoy/certs/tls.crt"
                private_key:
                  filename: "/etc/envoy/certs/tls.key"
              validation_context:
                trusted_ca:
                  filename: "/etc/envoy/certs/ca.crt"
  clusters:
  - name: service_backend
    connect_timeout: 0.25s
    type: LOGICAL_DNS
    lb_policy: ROUND_ROBIN
    load_assignment:
      cluster_name: service_backend
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address: { address: 127.0.0.1, port_value: 8080 }
admin:
  access_log_path: /tmp/admin_access.log
  address:
    socket_address: { address: 127.0.0.1, port_value: 9901 }

The upstream (127.0.0.1:8080) is a simple Go HTTP server that returns a 200 OK with a 1‑KB payload and artificially adds a fixed 200 µs processing delay (time.Sleep(200*time.Microsecond)) to isolate proxy effects.

CPU and Memory Limit Constraints

Limits are enforced via Linux cgroups v2 (the default on modern kernels). In a Docker‑based test harness we launch Envoy with:

docker run --name envoy \
  --cpus="0.5" \               # 0.5 CPU core (50% of a single core)
  --memory="256m" \            # 256 MiB RAM
  --memory-swap="512m" \       # allow swap up to 512 MiB total
  -v $(pwd)/envoy.yaml:/etc/envoy/envoy.yaml:ro \
  -v $(pwd)/certs:/etc/envoy/certs:ro \
  envoyproxy/envoy:v1.26 \
  /usr/local/bin/envoy -c /etc/envoy/envoy.yaml --log-level warn

To sweep limits we create a matrix:

In Kubernetes the same limits are expressed via resources.requests and resources.limits on the sidecar container.

Benchmarking Tools and Methodologies

All tests run for a fixed duration (30 s) after a 5‑second warm‑up to let the proxy’s internal buffers stabilize. Each configuration is repeated three times; we report the median and the 95 % confidence interval.

Benchmarking Handshake Latency

Handshake Latency Measurement Techniques

Handshake latency comprises three phases:

1. TCP SYN‑SYN/ACK – measured via tcptrace on the SYN and SYN‑ACK timestamps.
2. TLS ClientHello‑ServerHello – captured via bpftrace probing SSL_read/SSL_write in Envoy’s BoringSSL layer.
3. TLS Finished – final flight; the moment the proxy sends the Finished message.

A combined bpftrace script:

#!/usr/bin/env bpftrace
#include <net/sock.h>
#include <linux/tcp.h>

tracepoint:tcp:tcp_retransmit_synack
{
    @syn[args->skaddr_v4->src_addr, args->skaddr_v4->dest_port] = nsecs;
}

tracepoint:tcp:tcp_retransmit_synack
/@syn[args->skaddr_v4->src_addr, args->skaddr_v4->dest_port]/
{
    $delta = nsecs - @syn[args->skaddr_v4->src_addr, args->skaddr_v4->dest_port];
    @tcp_handshake[args->skaddr_v4->src_addr, args->skaddr_v4->dest_port] = hist($delta);
    delete(@syn[args->skaddr_v4->src_addr, args->skaddr_v4->dest_port]);
}

/* TLS handshake – approximate via SSL state machine */
usdt:/usr/lib/x86_64-linux-gnu/libssl.so:ssl3_read_bytes
{
    @tls_start[tid] = nsecs;
}
usdt:/usr/lib/x86_64-linux-gnu/libssl.so:ssl3_write_bytes
/@tls_start[tid]/
{
    $delta = nsecs - @tls_start[tid];
    @tls_handshake[tid] = hist($delta);
    delete(@tls_start[tid]);
}

Running the script while wrk2 generates connections yields a histogram of TCP + TLS handshake latency.

Impact of CPU Limits on Handshake Latency

When Envoy’s CPU share falls below the rate at which it can process incoming packets, the NIC’s receive ring fills, the kernel backlog grows, and the proxy’s event loop is delayed. Observations from the matrix:

Interpretation: TCP handshake begins to rise noticeably at ≤0.5 core because the proxy cannot dequeue packets from the NIC fast enough, causing the kernel to stretch the SYN‑ACK RTT. TLS handshake follows the same trend, amplified by the extra CPU needed for BoringSSL operations.

Impact of Memory Limits on Handshake Latency

Memory pressure manifests differently. When the proxy’s RSS approaches the limit, the kernel begins reclaiming pages (increasing pgsteal and pgscan). If swap is enabled, latency spikes due to page‑fault stalls; if swap is disabled, the proxy may encounter ENOMEM when allocating buffers for TLS records, causing connection resets.

Observed effects (swap enabled, 256