Skip to main content

Edge Delivery Platform

OpenResty acts as Cirrus CDN’s data plane, executing per-domain routing, caching, and TLS termination logic defined in LUA scripts. This chapter covers configuration templates, runtime behavior, purge processing, and telemetry emitted by the edge tier.

Nginx Configuration Template​

openresty/conf/nginx.conf.j2 is rendered at image build time with environment-controlled parameters (openresty/Dockerfile). Key directives include:

  • proxy_cache_path – Defines the shared cache (node_cache) stored on /data/cache.
  • map directives – Determine cache bypass behavior based on cookies ($has_cookie) and cache-control headers ($cc_nocache).
  • init_worker_by_lua_file – Executes init_worker.lua in each worker to start the purge subscriber and initialize metrics.
  • log_by_lua_file – Invokes log_metrics.lua after each request to update Prometheus counters.
  • listen blocks – Bind to HTTP (default 80) and HTTPS (default 443) ports, enable HTTP/2 and QUIC, and expose a separate listener on port 9145 for metrics and health checks.
  • ssl_certificate_by_lua_file – Loads per-domain certificates dynamically via ssl_loader.lua.

The template supports injection of resolver addresses, cache sizes, worker connections, and metrics access control at build time.

Access Router (Lua)​

openresty/conf/access_router.lua orchestrates request handling:

  1. Retrieves domain configuration from Redis using resty.redis. If Redis is unavailable, falls back to a 5-second shared dictionary cache (ngx.shared.dict_conf).
  2. Validates configuration; absence yields 404.
  3. Determines whether caching should be active (cache.enable flag).
  4. Parses origin definitions into a normalized list (scheme, host, port, sni, weight).
  5. Applies custom upstream headers from conf.upstream_headers, excluding Host unless explicitly set.
  6. Selects an origin using weighted round robin stored in a shared dictionary key (wrr:{domain}).
  7. Sets $origin_host, $origin_sni, and $backend Nginx variables to drive proxy_pass.
  8. Logs the selected backend via ngx.log(ngx.INFO, ...).

This logic delegates DNS resolution to Nginx itself (proxy_pass with hostname), simplifying Lua responsibilities.

Cache & Purge Mechanics​

  • Cache keys incorporate scheme, host, URI, and Range header to support byte-range caching.
  • proxy_cache_background_update enables background refreshes while serving stale content.
  • PURGE support uses nginx_cache_multipurge (compiled during the OpenResty image build):
    • API publishes commands to Redis channel cdn:purge.
    • redis_subscriber.lua runs in worker 0, consumes messages, and issues PURGE requests to /cache/purge with the target domain in the Host header.
    • cache_multipurge.lua leverages nginx_cache_keyfinder to calculate cache keys, supports wildcard purges, and integrates with configured cache zones.

TLS Handling​

ssl_loader.lua uses resty.lrucache to store up to 2000 SNI entries for 10 seconds. On handshake:

  • Fetches the server name (ngx.ssl.server_name()).
  • Attempts LRU lookup; on miss, pulls fullchain and key from cdn:cert:{domain} hash in Redis.
  • Parses PEM material via ngx.ssl.parse_pem_cert/parse_pem_priv_key.
  • Sets the cert and key for the current handshake.
  • Records Prometheus counters (via metric_ssl_handshake_errors) for parse or set failures.

Default dummy certificates (generated at image build time) ensure TLS handshakes succeed even before per-domain certs are available.

Metrics & Logging​

init_worker.lua:

  • Starts the Redis purge subscriber.
  • Initializes Prometheus metrics (requests, latency, cache status, upstream errors/timeouts, SSL errors, upstream RTT).

log_metrics.lua:

  • Increments counters per host/status.
  • Observes request latency and upstream response time histograms.
  • Counts cache statuses (HIT, MISS, STALE, etc.) and upstream errors.

Metrics are exposed via /metrics on port 9145, gated to loopback by default (configurable via NGX_METRICS_ALLOW). The same port serves /healthz, which returns 200 OK for external health probes (Celery).

Logs:

  • Access logs stream to /data/access-logs/access.log, managed by a dedicated logrotate container.
  • Error logs include Redis connectivity, configuration decoding, and backend selection messages helpful for debugging.

Security Controls​

  • Metrics endpoint is restricted to localhost by default; deployment engineers can whitelist additional CIDRs via build arguments.
  • Purge endpoint requires local access (127.0.0.1); external purge requests must flow through the API.
  • Redis connections default to 127.0.0.1:6379 for the host-networked OpenResty container. Production deployments should enforce TLS or restrict network boundaries accordingly.
  • Origin header injection prohibits overriding the Host header unless specified in upstream_headers["Host"].

Performance Considerations​

  • Weighted round robin is implemented in Lua; for large node counts, consider migrating to consistent hashing or using nginx upstream modules.
  • Cache zone size (NGX_CACHE_MAX_SIZE) is configurable; default is 1 GiB.
  • proxy_cache_lock prevents cache stampedes by serializing cache fill operations.
  • Client support includes HTTP/2 and QUIC, enhancing protocol coverage without additional configuration.

OpenResty faithfully executes control plane directives at request time. The next chapter highlights the frontend experience that surfaces these controls to operators through a Next.js interface.