Envoy Knowledge

Contents

• Overview
• Core Idea
• Key Concepts
• Components
  • Filters
    • Network-Level Filters
      • HTTP Filters
  • Route Tables
  • configuration as a service
  • Cluster Manager
  • HTTP Connection Manager
  • xDS (* Discovery Service)
    • Cluster Discovery Service (CDS)
    • Service Discovery
  • gRPC support
    • gRPC-web support
• Envoy Terminology

Overview

Envoy was born out of a few intertwining needs:

• application developers should be able to abstract away the network. debugging network problems is difficult—even just reaching the point where you believe the problem is with the network and not with the application—and the more you can isolate application behavior from network / cluster behavior, the better operations will work
• discoverability (services, clusters, endpoints, etc) and observability (how is this service, from a network perspective, performing?) get reinvented and hand rolled everywhere. getting different implementations that solve different parts of these questions to talk to one another and be coordinated is difficult.
• being able to reuse the same network software at the edge as within your cluster pays dividends through simplification (cluster management involves n less pieces of software), reduced operational cognitive load, and increased ability for depth of expertise
• varying network requirements for services and clusters. some need TLS. some need TCP proxying. Others need HTTP routing. some need to ensure that an authentication check is run on every request.
• core count of available instances kept increasing, and the less parallelizable your proxy architecture was, the more non-trivial the amount of contention happened (e.g. 96 core w/ forking process model is much more likely to have contention than a 96 core w/ single process and 1 thread per core)
• async IO is now possible (note: I don’t really know what changed here; was it something at the OS level? was it library specific? e.g. I remember netty becoming popular. or is it simply a programming paradigm shift that has been proved out?)

And what was created boils down to:

• single process, multi-threaded, async IO architecture to scale out well to large number of cores
• proxying done at a base level as a set of byte filters
• connections get processed by a chain of additional filters, with those filters being user composable and user definable
• can be used as edge proxy or local proxy (“sidecar”) interchangeably (and without change in mode)
• make L7 routing use cases simple to configure / create
• prioritize observability by making it dead simple to send / consume stats as needed
• prioritize discoverability through xDS pattern (“x” as a placeholder meaning “anything” Discovery Service)
• allow static or dynamic configuration management (also through a service)

Core Idea

Make network transparent to applications.

Key Concepts

architecture-wise:

• async
• single threaded (no separate processes)
• byte filters
• L3/L4+ proxy
• http/2 native
• easy L7 routing

big ideas:

• compose filters
• centralize observability and discoverability (so you do not reinvent the wheel on every container)

here is how it works:

• single envoy process
• contains multiple listeners
• a listener:
  • configured with filter chains
  • a filter chain is:
    • selected based on its match criteria
    • composed of 1 or more L3/L4 filters
    • a filter may be:
      • read
      • write
      • read/write
  • these filter chains allow management of handling the connection to do things like terminate TLS or enable grpc-web communication
  • may also be configured with a listener filter, which are processed before network-level filters
• when a connection is made on a listener, the matching filter chain is selected and used to process subsequent events (e.g. data)

envoy can also be configured as a strict TCP proxy where all TCP traffic is passed through.

On configuration:

• static configuration is with a file
• envoy also supports dynamic configuration via xDS (x (placeholder) discovery service)
• static configuration is fine for my current local-proxy use case
• each resource has a specific type that must be referenced
• these types are checked against schema definitions to ensure correct behavior

Components

Filters

Network-Level Filters

HTTP Filters

• handled by the HTTP Connection Manager

Route Tables

A component of Filters that matches the route for a filter.

configuration as a service

evidently envoy can be configured to talk to a configuration service rather than rely on files?

Cluster Manager

HTTP Connection Manager

xDS (* Discovery Service)

Cluster Discovery Service (CDS)

Service Discovery

gRPC support

gRPC-web support

• use a specific filter
• look into grpc bridging

Envoy Terminology

downstream

an entity that connects to Envoy, sends requests, and receives responses

upstream

an entity that receives connections and requests from Envoy and returns responses