Controllers

• System Controllers
  • ServiceRegistry
  • TrafficController
  • RawConfigTrafficController
    • HTTPServer
      • AccessLogVariable
    • GRPCServer
    • Pipeline
  • StatusSyncController
• Business Controllers
  • GlobalFilter
  • EaseMonitorMetrics
  • FaaSController
  • IngressController
  • ConsulServiceRegistry
  • EtcdServiceRegistry
  • EurekaServiceRegistry
  • ZookeeperServiceRegistry
  • NacosServiceRegistry
  • AutoCertManager
• Common Types
  • tracing.Spec
    • spanlimits.Spec
    • batchlimits.Spec
    • exporter.Spec
    • jaeger.Spec
    • zipkin.Spec
    • otlp.Spec
    • zipkin.DeprecatedSpec
  • ipfilter.Spec
  • httpserver.Rule
  • httpserver.Host
  • httpserver.Path
  • httpserver.Header
  • pipeline.Spec
  • pipeline.FlowNode
  • filters.Filter
  • grpcserver.Rule
  • grpcserver.Method
  • grpcserver.Header
  • easemonitormetrics.Kafka
  • nacos.ServerSpec
  • autocertmanager.DomainSpec
  • resilience.Policy
    • Retry Policy
    • CircuitBreaker Policy

As the architecture diagram shows, the controller is the core entity to control kinds of working. There are two kinds of controllers overall:

• System Controller: It is created one and only one instance in every Easegress node, which can’t be deleted. They mainly aim to control essential system-level stuff.
• Business Controller: It could be created, updated, deleted by admin operation. They control various resources such as mesh traffic, service discovery, faas, and so on.

In another view, Easegress as a traffic orchestration system, we could classify them into traffic controller and non-traffic controller:

• Traffic Controller: It invokes TrafficController to handle its specific traffic, such as MeshController.
• Non-Traffic Controller: It doesn’t handle business traffic, such as EurekaServiceRegistry, even though it has admin traffic with Eureka.

The two categories are conceptual, which means they are not strict distinctions. We just use them as terms to clarify controllers technically.

System Controllers

For now, all system controllers can not be configured. It may gain this capability if necessary in the future.

ServiceRegistry

We use the system controller ServiceRegistry as the service hub for all service registries. Current drivers are

• ConsulServiceRegistry
• EtcdServiceRegistry
• EurekaServiceRegistry
• ZookeeperServiceRegistry
• NacosServiceRegistry

The drivers need to offer notifying change periodically, and operations to the external service registry.

TrafficController

TrafficController handles the lifecycle of Traffic Gates (like HTTPServer) and Pipeline and their relationship. It manages the resource in a namespaced way. Traffic gates accepts incoming traffic and routes it to Pipelines in the same namespace. Most other controllers could handle traffic by leverage the ability of TrafficController..

RawConfigTrafficController

RawConfigTrafficController maps all traffic static configurations to TrafficController in the namespace default. We could use egctl to manage the configuration of servers and pipelines in the default namespace.

HTTPServer

HTTPServer is a server that listens on one port to route all traffic to available pipelines. Its simplest config looks like:

kind: HTTPServer
name: http-server-example
port: 80
rules:
  - paths:
    - pathPrefix: /pipeline
      backend: http-pipeline-example

AccessLogVariable

GRPCServer

The GRPCServer in Easegress provides robust functionality tailored to gRPC protocol interactions. With its IP filtering feature, traffic can be selectively allowed or blocked, ensuring that only desired clients can communicate with the services. Additionally, the server’s routing rules offer flexible methods to determine how each incoming request is processed and forwarded, based on host, method, headers, and other criteria.

name: server-grpc
kind: GRPCServer
port: 8080

---
title: The maximum number of connections allowed by gRPC Server.
linkTitle: The maximum number of connections allowed by gRPC Server.
weight: 1
---
---
title: Default value 10240
linkTitle: Default value 10240
weight: 1
---
maxConnections: 10240

---
title: IP Filter for all traffic under the server
linkTitle: IP Filter for all traffic under the server
weight: 1
---
ipFilter:
  blockIPs: []
  allowIPs: []
  blockByDefault: false

---
title: routing rules
linkTitle: routing rules
weight: 1
---
rules:
---
title: Rules for host matching.
linkTitle: Rules for host matching.
weight: 1
---
---
title: If not match, GRPCServer will check next rule.
linkTitle: If not match, GRPCServer will check next rule.
weight: 1
---
- host: <your-host>
  hostRegexp: <your-host-regexp>

  methods: 
  - method: /Sale/AddProduct # Exact method match
    backend: sale-pipeline

  - methodPrefix: /IT        # Matches method with the given prefix
    backend: it-pipeline

  - headers:                 # Matches by header
    - key: x-geo-country
      values: ["CN", "EU", "US"]
    - key: user-agent
      values: ["SaleClient/1.0.0"]
    matchAllHeader: false
    backend: header-pipeline

  - methodRegexp: .*         # Match by regexp
    backend: other-pipeline

---
title: more rules
linkTitle: more rules
weight: 1
---
- methods:
  ...

Example Requests

• A gRPC request with method /Sale/AddProduct will be routed to the sale-pipeline.
• Any request with a method starting with /IT (e.g., /IT/UpdateSoftware) will be directed to the it-pipeline.
• If a client sends a request with headers x-geo-country set to CN and user-agent set to SaleClient/1.0.0, it will be handled by the header-pipeline.
• All other requests (due to the wildcard methodRegexp) will be sent to the other-pipeline.

Configuration

The below parameters will help manage connections better

Pipeline

Pipeline is used to orchestrate filters. Its simplest config looks like:

name: http-pipeline-example1
kind: Pipeline
flow:
- filter: proxy

filters:
- name: proxy
  kind: Proxy
  pools:
  - servers:
    - url: http://127.0.0.1:9095

The flow defines the execution order of filters. You can use jumpIf to change the order.

For example, if a request’s header doesn’t have the key X-Id or its value is not user1 or user2, then the validator filter returns result invalid and the pipeline jumps to END.

name: http-pipeline-example2
kind: Pipeline
flow:
- filter: validator
  jumpIf:
    # END is a built-in filter, it stops the execution of the pipeline.
    invalid: END
- filter: proxy

filters:
- name: validator
  kind: Validator
  headers:
    X-Id:
      values: ["user1", "user2"]
- name: proxy
  kind: Proxy
  pools:
  - servers:
    - url: http://127.0.0.1:9095

jumpIf can only jump to filters behind the current filter.

The resilience field defines resilience policies, if a filter implements the filters.Resiliencer interface (for now, only the Proxy filter implements the interface), the pipeline injects the policies into the filter instance after creating it. A filter can implement the filters.Resiliencer interface to support resilience. There are two kinds of resilience, Retry and CircuitBreaker. Check resilience for more details. The following config adds a retry policy to the proxy filter:

name: http-pipeline-example3
kind: Pipeline
flow:
- filter: proxy

filters:
- name: proxy
  kind: Proxy
  pools:
  - servers:
    - url: http://127.0.0.1:9095
    retryPolicy: retry

resilience:
- name: retry
  kind: Retry
  maxAttempts: 3

In this case, if proxy returns non-empty results, then resilience retry reruns the proxy filter until proxy returns empty results or gets the max attempts.

The flow also supports namespace, so the pipeline can support workflows that contain multiple requests and responses.

name: http-pipeline-example4
kind: Pipeline
flow:
- filter: validator
  jumpIf:
    invalid: END
- filter: requestBuilderFoo
  namespace: foo
- filter: proxyFoo
  namespace: foo
- filter: requestBuilderBar
  namespace: bar
- filter: proxyBar
  namespace: bar
- filter: responseBuilder

filters:
- name: requestBuilder
  kind: RequestBuilder
  ...
...

In this case, requestBuilderFoo creates a request in namespace foo, and proxyFoo sends foo request and puts the response into namespace foo. requestBuilderBar creates a request in namespace bar and proxyBar sends bar request and puts the response into namespace bar. Finally, requestBuilder creates a response and puts it into the default namespace.

If not set, the filter works in the default namespace DEFAULT.

The alias in flow gives a filter an alias to help re-use the filter so that we can use the alias to distinguish each of its appearances in the flow.

name: http-pipeline-example5
kind: Pipeline
flow:
- filter: validator
  jumpIf:
    invalid: proxy2
- filter: proxy
---
title: when meeting filter END, the pipeline execution stops and returns.
linkTitle: when meeting filter END, the pipeline execution stops and returns.
weight: 1
---
- filter: END
- filter: proxy
  alias: proxy2
- filter: responseAdaptor

filters:
- name: proxy
  kind: Proxy
  ...

In this case, we give second proxy alias proxy2, so request is invalid, it jumps to second proxy.

The data field defines static user data for the pipeline, which can be accessed by filters. For example, in the below pipeline, the body of the result request of the RequestBuilder will be hello world, which is the value of data item foo.

name: http-pipeline-example6
kind: Pipeline
flow:
  ...

filters:
- name: requestBuilder
  kind: RequestBuilder
  template: |
    body: {{.data.PIPELINE.foo}}    

data:
  foo: "hello world"

StatusSyncController

No config.

Business Controllers

GlobalFilter

GlobalFilter is a special pipeline that can be executed before or/and after all pipelines in a server. For example:

name: globalFilter-example
kind: GlobalFilter
beforePipeline:
  flow:
  - filter: validator

  filters:
  - name: validator
    kind: Validator
    ...
---
name: server-example
kind: HTTPServer
globalFilter: globalFilter-example
...

In this case, all requests in HTTPServer server-example go through GlobalFilter globalFilter-example before executing any other pipelines.

EaseMonitorMetrics

EaseMonitorMetrics is adapted to monitor metrics of Easegress and send them to Kafka. The config looks like:

kind: EaseMonitorMetrics
name: easemonitor-metrics-example
kafka:
  brokers: ["127.0.0.1:9092"]
  topic: metrics

FaaSController

A FaaSController is a business controller for handling Easegress and FaaS products integration purposes. It abstracts FaasFunction, FaaSStore and, FaasProvider. Currently, we only support Knative type FaaSProvider.

For the full reference document please check - FaaS Controller

IngressController

The IngressController is an implementation of Kubernetes ingress controller, it watches Kubernetes Ingress, Service, Endpoints, and Secrets then translates them to Easegress HTTP server and pipelines. The config looks like:

kind: IngressController
name: ingress-controller-example
kubeConfig:
masterURL:
namespaces: ["default"]
ingressClass: easegress
httpServer:
  port: 8080
  https: false
  keepAlive: true
  keepAliveTimeout: 60s
  maxConnections: 10240

Note: IngressController uses kubeConfig and masterURL to connect to Kubernetes, at least one of them must be specified when deployed outside of a Kubernetes cluster, and both are optional when deployed inside a cluster.

ConsulServiceRegistry

ConsulServiceRegistry supports service discovery for Consul as backend. The config looks like:

kind: ConsulServiceRegistry
name: consul-service-registry-example
address: '127.0.0.1:8500'
scheme: http
syncInterval: 10s

EtcdServiceRegistry

EtcdServiceRegistry support service discovery for Etcd as backend. The config looks like:

kind: EtcdServiceRegistry
name: etcd-service-registry-example
endpoints: ['127.0.0.1:12379']
prefix: "/services/"
cacheTimeout: 10s

EurekaServiceRegistry

EurekaServiceRegistry supports service discovery for Eureka as backend. The config looks like:

kind: EurekaServiceRegistry
name: eureka-service-registry-example
endpoints: ['http://127.0.0.1:8761/eureka']
syncInterval: 10s

ZookeeperServiceRegistry

ZookeeperServiceRegistry supports service discovery for Zookeeper as backend. The config looks like:

kind: ZookeeperServiceRegistry
name: zookeeper-service-registry-example
zkservices: [127.0.0.1:2181]
prefix: /services
conntimeout: 6s
syncInterval: 10s

NacosServiceRegistry

NacosServiceRegistry supports service discovery for Nacos as backend. The config looks like:

kind: NacosServiceRegistry
name: nacos-service-registry-example
syncInterval: 10s
servers:
  - scheme: http
    port: 8848
    contextPath: /nacos
    ipAddr: 127.0.0.1

AutoCertManager

AutoCertManager automatically manage HTTPS certificates. The config looks like:

kind: AutoCertManager
name: autocert
email: [email protected]
directoryURL: https://acme-v02.api.letsencrypt.org/directory
renewBefore: 720h
enableHTTP01: true
enableTLSALPN01: true
enableDNS01: true
domains:
  - name: "*.megaease.com"
    dnsProvider:
      name: dnspod
      zone: megaease.com
      apiToken: <token value>

Common Types

tracing.Spec

spanlimits.Spec

batchlimits.Spec

exporter.Spec

jaeger.Spec

zipkin.Spec

otlp.Spec

zipkin.DeprecatedSpec

ipfilter.Spec

httpserver.Rule

Note: if host or hostRegexp is not empty, they will be added into hosts at runtime, and if the result hosts is empty, all hosts are matched.

httpserver.Host

httpserver.Path

httpserver.Header

There must be at least one of values and regexp.

pipeline.Spec

pipeline.FlowNode

filters.Filter

The self-defining specification of each filter references to filters.

grpcserver.Rule

grpcserver.Method

grpcserver.Header

easemonitormetrics.Kafka

nacos.ServerSpec

autocertmanager.DomainSpec

The fields in dnsProvider vary from DNS providers, but:

• name and zone are required for all DNS providers.
• nsAddress and nsNetwork are optional name server information for all DNS providers, if provided, AutoCertManager will leverage them to speed up the DNS record lookup. nsAddress is the address of the name server, must always include the port number, nsNetwork is the network protocol of name server, it should be udp in most cases.

Below table list other required fields for each supported DNS provider (Note: google is temporarily disabled due to dependency conflict):

resilience.Policy

Retry Policy

A retry policy configures how to retry a failed request.

CircuitBreaker Policy

CircuitBreaker leverges a finite state machine to implement the processing logic, the state machine has three states: CLOSED, OPEN, and HALF_OPEN. When the state is CLOSED, requests pass through normally, state transits to OPEN if request failure rate or slow request rate reach a configured threshold and requests will be shor-circuited in this state. After a configured duration, state transits from OPEN to HALF_OPEN, in which a limited number of requests are permitted to pass through while other requests are still short-circuited, and state transit to CLOSED or OPEN based on the results of the permitted requests.

When CLOSED, it uses a sliding window to store and aggregate the result of recent requests, the window can either be COUNT_BASED or TIME_BASED. The COUNT_BASED window aggregates the last N requests and the TIME_BASED window aggregates requests in the last N seconds, where N is the window size.

Below is an example configuration with both COUNT_BASED and TIME_BASED policies.

Policy circuit-breaker-example-count short-circuits requests if more than half of recent requests failed.

Policy circuit-breaker-example-time short-circuits requests if more than 60% of recent requests failed.

failed means that backend filter returns non-empty results.

kind: CircuitBreaker
name: circuit-breaker-example-count
slidingWindowType: COUNT_BASED
failureRateThreshold: 50
slidingWindowSize: 100

---
kind: CircuitBreaker
name: circuit-breaker-example-time
slidingWindowType: TIME_BASED
failureRateThreshold: 60
slidingWindowSize: 100

See more details about Retry, CircuitBreaker or other resilience polcies in here.