Envoy Knowledge
Kip Landergren
(Updated: )
My Envoy knowledge base explaining an overview, the core idea, and key concepts.
Contents
Overview
Envoy was born out of a few intertwining needs:
- application developers should be able to abstract away the network. debugging network problems is difficult—even just reaching the point where you believe the problem is with the network and not with the application—and the more you can isolate application behavior from network / cluster behavior, the better operations will work
- discoverability (services, clusters, endpoints, etc) and observability (how is this service, from a network perspective, performing?) get reinvented and hand rolled everywhere. getting different implementations that solve different parts of these questions to talk to one another and be coordinated is difficult.
- being able to reuse the same network software at the edge as within your cluster pays dividends through simplification (cluster management involves n less pieces of software), reduced operational cognitive load, and increased ability for depth of expertise
- varying network requirements for services and clusters. some need TLS. some need TCP proxying. Others need HTTP routing. some need to ensure that an authentication check is run on every request.
- core count of available instances kept increasing, and the less parallelizable your proxy architecture was, the more non-trivial the amount of contention happened (e.g. 96 core w/ forking process model is much more likely to have contention than a 96 core w/ single process and 1 thread per core)
- async IO is now possible (note: I don’t really know what changed here; was it something at the OS level? was it library specific? e.g. I remember netty becoming popular. or is it simply a programming paradigm shift that has been proved out?)
And what was created boils down to:
- single process, multi-threaded, async IO architecture to scale out well to large number of cores
- proxying done at a base level as a set of byte filters
- connections get processed by a chain of additional filters, with those filters being user composable and user definable
- can be used as edge proxy or local proxy (“sidecar”) interchangeably (and without change in mode)
- make L7 routing use cases simple to configure / create
- prioritize observability by making it dead simple to send / consume stats as needed
- prioritize discoverability through xDS pattern (“x” as a placeholder meaning “anything” Discovery Service)
- allow static or dynamic configuration management (also through a service)
Core Idea
Make network transparent to applications.
Key Concepts
architecture-wise:
- async
- single threaded (no separate processes)
- byte filters
- L3/L4+ proxy
- http/2 native
- easy L7 routing
big ideas:
- compose filters
- centralize observability and discoverability (so you do not reinvent the wheel on every container)
here is how it works:
- single envoy process
- contains multiple listeners
- a listener:
- configured with filter chains
- a filter chain is:
- selected based on its match criteria
- composed of 1 or more L3/L4 filters
- a filter may be:
- read
- write
- read/write
- these filter chains allow management of handling the connection to do things like terminate TLS or enable grpc-web communication
- may also be configured with a listener filter, which are processed before network-level filters
- when a connection is made on a listener, the matching filter chain is selected and used to process subsequent events (e.g. data)
envoy can also be configured as a strict TCP proxy where all TCP traffic is passed through.
On configuration:
- static configuration is with a file
- envoy also supports dynamic configuration via xDS (x (placeholder) discovery service)
- static configuration is fine for my current local-proxy use case
- each resource has a specific type that must be referenced
- these types are checked against schema definitions to ensure correct behavior
Components
Filters
Network-Level Filters
HTTP Filters
- handled by the HTTP Connection Manager
Route Tables
A component of Filters that matches the route for a filter.
configuration as a service
evidently envoy can be configured to talk to a configuration service rather than rely on files?
Cluster Manager
HTTP Connection Manager
xDS (* Discovery Service)
Cluster Discovery Service (CDS)
Service Discovery
gRPC support
gRPC-web support
- use a specific filter
- look into grpc bridging
Envoy Terminology
- cluster
- A cluster is a group of logically similar upstream hosts that Envoy connects to. Envoy discovers the members of a cluster via service discovery. It optionally determines the health of cluster members via active health checking. The cluster member that Envoy routes a request to is determined by the load balancing policy.
- downstream
- A downstream host connects to Envoy, sends requests, and receives responses
- host
- An entity capable of network communication (application on a mobile phone, server, etc.). In this documentation a host is a logical network application. A physical piece of hardware could possibly have multiple hosts running on it as long as each of them can be independently addressed.
- listener
- A listener is a named network location (e.g., port, unix domain socket, etc.) that can be connected to by downstream clients. Envoy exposes one or more listeners that downstream hosts connect to.
- mesh
- A group of hosts that coordinate to provide a consistent network topology. In this documentation, an “Envoy mesh” is a group of Envoy proxies that form a message passing substrate for a distributed system comprised of many different services and application platforms.
- runtime configuration
- Out of band realtime configuration system deployed alongside Envoy. Configuration settings can be altered that will affect operation without needing to restart Envoy or change the primary configuration.
- upstream
- An upstream host receives connections and requests from Envoy and returns responses