Experiment Evaluation Proxy

The Evaluation Proxy is a Service to enable, enhance, and optimize local evaluation running within your infrastructure.

• Enable local evaluation on unsupported platforms: Use remote Evaluation APIs and SDKs to run local evaluation in your infrastructure.

• Automatically track exposure events for local evaluations: The proxy deduplicates identical exposure events for 24 hours.

• Enhance local evaluation with large cohort targeting: Targeted cohorts sync hourly to the Evaluation Proxy, and the proxy adds cohorts to the user before evaluation.

Configuration

The evaluation proxy is either configured using a yaml file (recommended, more configuration options), or using environment variables.

The default location for the configuration yaml file is /etc/evaluation-proxy-config.yaml. You may also configure the file location using the PROXY_CONFIG_FILE_PATH environment variable.

• projects (required)
• configuration (optional).

projects:
  - apiKey: "AMPLITUDE_API_KEY"
    secretKey: "YOUR SECRET KEY"
    managementKey: "YOUR MANAGEMENT API KEY"

configuration:
  redis:
    uri: "YOUR REDIS URI" # e.g. "redis://localhost:6379"

Environment configuration supports only a single project. The proxy uses environment variable configuration only if the configuration file isn't found.

• AMPLITUDE_API_KEY
  • Description: The project's API key.
• AMPLITUDE_SECRET_KEY
  • Description: The project's secret key.
• AMPLITUDE_EXPERIMENT_MANAGEMENT_API_KEY
  • Description: The Experiment management API key. Must be created for the same project as the configured API and secret key. Used to automatically access and update deployments used for the project.
• AMPLITUDE_REDIS_URI
  • Description: Optional. The entire URI to connect to Redis. Include the protocol, host, port, and optional username, password, and path (for example redis://localhost:6379).
• AMPLITUDE_REDIS_PREFIX
  • Description: Optional. A prefix for all keys the evaluation proxy saves in Redis. Defaults to amplitude.
• AMPLITUDE_REDIS_USE_CLUSTER
  • Description: Optional. If AMPLITUDE_REDIS_URI is a cluster URL, pass this as true. The setting defaults to false.
• AMPLITUDE_REDIS_READ_FROM
  • Description: Optional. Read routing strategy for Redis Cluster. Options: REPLICA_PREFERRED (default, prefer replicas) or ANY (read from any node in the cluster).
• AMPLITUDE_FLAG_CACHE_TTL_MILLIS
  • Description: Optional. How long, in milliseconds, each proxy instance serves flag configurations from memory before it reads them from Redis again. Defaults to 5000. Set to 0 to turn off the cache.
• AMPLITUDE_SERVER_ZONE
  • Description: Optional. The Amplitude data center, US or EU. Defaults to US. Sets the default for all four server URL variables below. Set it to EU for the EU data center.
• AMPLITUDE_SERVER_URL
  • Description: Optional. The server URL, including protocol and host, to fetch flags from. Overrides the AMPLITUDE_SERVER_ZONE default.
• AMPLITUDE_COHORT_SERVER_URL
  • Description: Optional. The server URL, including protocol and host, to download cohorts from. Overrides the AMPLITUDE_SERVER_ZONE default.
• AMPLITUDE_MANAGEMENT_SERVER_URL
  • Description: Optional. The server URL, including protocol and host, of the Experiment management API that the proxy uses to list deployments. Overrides the AMPLITUDE_SERVER_ZONE default.
• AMPLITUDE_ANALYTICS_SERVER_URL
  • Description: Optional. The full URL the proxy sends assignment and exposure events to. Overrides the AMPLITUDE_SERVER_ZONE default.

projects

A required array of objects with the following fields, all which are required.

• apiKey
  • Description: The project's API key.
• secretKey
  • Description: The project's secret key.
• managementKey
  • Description: The Experiment management API key. Must be created for the same project as the configured API and secret key. Used to automatically access and update deployments used for the project.

configuration

An optional object of extra configuration.

• redis
  • Description: Optional (Recommended). Refer to redis. Configure the proxy to use redis as persistent storage.
• flagSyncIntervalMillis
  • Description: Optional. The polling interval to update flag configurations (default 60000).
• flagCacheTtlMillis
  • Description: Optional. How long, in milliseconds, each proxy instance serves flag configurations from memory before it reads them from Redis again (default 5000). The cache reduces Redis load on evaluation requests. Flag changes written by another instance can take up to this long to appear. Set to 0 to turn off the cache.
• maxCohortSize
  • Description: Optional. The maximum size of targeted cohorts that the proxy can download (default 2147483647).
• serverZone
  • Description: Optional. The Amplitude data center, US or EU (default US). Sets the default for all four server URLs below.
• serverUrl
  • Description: Optional. The server URL, including protocol and host, to fetch flags from (default https://flag.lab.amplitude.com).
• cohortServerUrl
  • Description: Optional. The server URL, including protocol and host, to download cohorts from (default https://cohort-v2.lab.amplitude.com).
• managementServerUrl
  • Description: Optional. The server URL, including protocol and host, of the Experiment management API that the proxy uses to list deployments (default https://experiment.amplitude.com).
• analyticsServerUrl
  • Description: Optional. The full URL the proxy sends assignment and exposure events to (default https://api2.amplitude.com/2/httpapi).

EU data residency

To use the evaluation proxy with the EU data center, set serverZone to EU in the configuration file, or set the AMPLITUDE_SERVER_ZONE environment variable to EU. This sends flag, cohort, management API, and event traffic to the EU data center.

configuration:
  # Other configurations...
  serverZone: "EU"

redis

Configure the evaluation proxy to use Redis as a persistent storage. Highly recommended to enable the evaluation proxy to run efficiently.

• uri
  • Description: Required. The full URI to connect to Redis with. Include the protocol, host, port, and optional username, password, and path.
• readOnlyUri
  • Description: Optional. Optional URI to connect to read only replicas for high scaling high volume reads to Redis read replicas.
• useCluster
  • Description: Optional. If uri is a cluster URL, set this to true. Defaults to false.
• readFrom
  • Description: Optional. Read routing strategy for cluster mode only: REPLICA_PREFERRED (default, prefer replicas) or ANY (read from any node in the cluster).
• prefix
  • Description: Optional. A prefix for all keys saved by the evaluation proxy. Defaults to amplitude.

Deployment

The evaluation proxy is stateless. Deploy multiple instances behind a load balancer for high availability and scalability.

For example, a kubernetes deployment with greater than one replica.

Kubernetes

Use the evaluation proxy Helm chart to install the proxy service on kubernetes or generate the files needed to deploy the service manually. The repository also contains an example of running the evaluation proxy on kubernetes locally using minikube.

Helm

Add helm repo

helm repo add \
    evaluation-proxy-helm https://amplitude.github.io/evaluation-proxy-helm

Configure values.yaml

Configure the chart values. The recommended approach to configuring and installing the helm chart is using a values.yaml configuration file.

The chart's evaluationProxy value contents exactly match the evaluation proxy's configuration file fields.

evaluationProxy:
  # At least one project is required.
  projects:
    - apiKey: "AMPLITUDE_API_KEY"
      secretKey: "YOUR SECRET KEY"
      managementKey: "YOUR MANAGEMENT API KEY"
  configuration: {}
#    redis:
#      uri: "redis://redis-master.default.svc.cluster.local:6379"

Install helm chart

helm install -f values.yaml \
    evaluation-proxy evaluation-proxy-helm/evaluation-proxy

Docker

You may run the docker image directly. First, create a configuration file, then run the docker image mounting the file as a volume to the expected directory in the container.

docker run \
    -v CONFIG_FILE_PATH:/etc/evaluation-proxy-config.yaml \
    amplitudeinc/evaluation-proxy

Evaluation

The Evaluation Proxy exposes remote Evaluation API and SDK endpoints to run local evaluation within your cluster. This is useful to enable platforms and languages which aren't supported by local evaluation SDKs. As an added benefit, fetch requests made to the evaluation proxy can target cohorts of users, and have assignment events tracked automatically to Amplitude.

You must send requests to the service using http on port 3546.

Remote evaluation mode flags

The proxy runs local evaluation only. For flags set to remote evaluation mode, Amplitude sends the proxy a placeholder configuration without targeting rules, so the proxy can't evaluate them. As with local evaluation in the SDKs, the proxy returns the flag's default variant. It doesn't track assignment or exposure events for these flags. It logs a warning once per flag and increments the amplitude_proxy_remote_evaluation_flag_skipped_total metric.

To evaluate remote evaluation mode flags, send those requests to Amplitude's Evaluation API instead of the proxy. For example, use two remote evaluation clients: one with the proxy as its server URL for local evaluation mode flags, and one with the default server URL for remote evaluation mode flags. Request only the relevant flag keys from each client. If your SDK can't pass flag keys, put remote evaluation mode flags in a separate deployment.

Best practices

Production deployment

Resource requirements

Configure each pod with 4 CPU cores and 9 GiB RAM for a capacity of approximately 5,000 requests each second.

• Minimum replicas: Deploy at least two replicas for high availability.
• Horizontal scaling: Add pods to increase capacity. For example, four pods provide approximately 20,000 requests each second.

JVM heap for large cohorts

If you use local evaluation and download large cohorts greater than 5M users, set a 6 GiB max JVM heap size.

- name: JAVA_TOOL_OPTIONS
  value: "-Xms128m -Xmx6144m"

Redis configuration

Option 1: Standalone with replicas

Use for small deployments.

configuration:
  redis:
    uri: "rediss://primary:6379"
    readOnlyUri: "rediss://replica:6379" # Optional, for high-volume read scaling.

Recommended specs: 12+ GiB memory, cache.m7g.xlarge or equivalent.

Option 2: Redis Cluster (recommended for high scale)

Use Redis Cluster when you have 10M+ users or many large cohorts.

configuration:
  redis:
    uri: "rediss://cluster:6379"
    useCluster: true

Recommended specs: 2-3 shards, 1-2 replicas per shard, 12+ GiB per node.

Performance characteristics

Latency

• Normal: 1-5ms.
• During background cohort refresh: p95 latency can reach up to 50ms. This typically occurs when you add or remove a significant number of users from a cohort, or attach new large cohorts to a flag.

Cold start

Initial startup time scales with the number and size of targeted cohorts.

Set the readiness probe initialDelaySeconds: 600 to prevent pod restarts during startup.

Monitoring and alerts

The proxy exposes Prometheus metrics at http://proxy:9090/metrics. Proxy metrics use the amplitude_proxy_ prefix. The endpoint also includes standard JVM and process metrics (jvm_*, process_*, and system_*). Each counter appears after its first increment, so failure counters are absent until the first failure.

Configure these critical alerts:

# Error rate > 1%
sum(rate(amplitude_proxy_http_server_requests_seconds_count{status=~"5.."}[5m]))
  / sum(rate(amplitude_proxy_http_server_requests_seconds_count[5m])) > 0.01

# P95 latency > 100ms sustained on any pod (ignore during cohort refresh spikes)
max(amplitude_proxy_http_server_requests_seconds{quantile="0.95"}) > 0.1

# Memory > 85%
container_memory_usage_bytes / container_spec_memory_limit_bytes > 0.85

# Redis errors
rate(amplitude_proxy_redis_command_failure_total[5m]) > 0

# Flag or cohort sync failures
rate(amplitude_proxy_flags_fetch_failure_total[15m]) > 0
rate(amplitude_proxy_cohort_download_failure_total[15m]) > 0

Troubleshooting

Capacity planning