Skip to main content

Configure the AI Gateway

Enable the AI Gateway through the platform chart, then apply its custom resources in the order described below. See AI Gateway for ongoing configuration.

Prerequisites​

  • A corporate identity provider, configured once at global.stacklok.primaryIdp. The gateway ties every request to an identity from that provider. See Configure identity.
  • PostgreSQL, which the platform already requires. Budgets, pricing, and recorded spend live there.
  • Redis or Valkey, only if you intend to enable the detection result cache. It is optional and off by default. See PCI/PII controls.

Enable it​

Set the install toggle in your platform values and upgrade:

values.yaml
global:
  stacklok:
    aiGateway:
      enabled: true

This installs the AI Gateway operator and custom resource definitions. Apply an AIGateway resource to create a gateway instance.

Bring it up in this order​

Complete the following sequence before sending production traffic:

  1. Create budgets that cover every caller, before you enable the budget webhook target. Set an organization default for resolved directory users, or create budgets for individual users and groups. A caller with no applicable budget is refused. See Budgets and pricing.

  2. Enable gateway-level budget webhooks. Add the webhook target and its receiver configuration to your platform values, then upgrade the release:

    values.yaml
    global:
      webhooks:
        issuerRef:
          name: <WEBHOOK_CLUSTER_ISSUER>
          kind: ClusterIssuer
        caBundleSecret: <WEBHOOK_CA_BUNDLE_SECRET>

    enterprise-manager:
      webhookTLS:
        enabled: true
        port: 443
      webhookAuth:
        audience: <BUDGET_WEBHOOK_AUDIENCE>

    enterprise-ai-gateway-operator:
      upstream:
        budgetsWebhook:
          serviceName: <ENTERPRISE_MANAGER_SERVICE>
          port: 443
          audience: <BUDGET_WEBHOOK_AUDIENCE>

    Set serviceName to the Enterprise Manager Service in the same namespace as the gateway. The two audience values must match exactly. The operator adds admission and usage webhooks to every OIDC-enabled gateway it manages.

  3. Apply an AIGateway resource with at least one provider and one route. See Connect model providers.

  4. Verify. Confirm the gateway reports its providers ready and that budget enforcement probed successfully:

    kubectl get aigw -n <NAMESPACE>
    kubectl get aigw <NAME> -n <NAMESPACE> \
      -o jsonpath='{.status.webhooks}' | jq .

Content screening posture​

Detection failures deny requests by default. An experimental waiver can allow traffic during a rollout or incident, but it is unavailable on the stable release channel.

Next steps​