# Highflame Services

## Deploying Highflame on Kubernetes

A self-service guide to installing the Highflame AI security platform on your own Kubernetes cluster using the official Helm charts.

This guide is **platform-neutral** — the same workflow installs Highflame on any conformant Kubernetes 1.32+ cluster. Cloud native / managed K8s-specific notes are called out inline where they matter.

### What you are deploying

The Highflame platform is delivered as a collection of microservices that run together in a single Kubernetes namespace. The values files in this [directory](https://github.com/highflame-ai/highflame-iac/tree/main/helm-values) describe how each service should be configured for a self-hosted production install.

### Prerequisites

You will need the following before starting.

#### Tooling on your workstation

| Tool      | Minimum version | Notes                                 |
| --------- | --------------- | ------------------------------------- |
| `kubectl` | 1.32            | Configured against the target cluster |
| `helm`    | 3.x             | OCI support required                  |

#### Kubernetes cluster

| Requirement                      | Minimum                                                             |
| -------------------------------- | ------------------------------------------------------------------- |
| Kubernetes version               | 1.32                                                                |
| Core node pool                   | 3 nodes × 8 vCPU / 16 GB RAM                                        |
| GPU node pool (for Guard models) | 4 nodes × 4 vCPU / 16 GB RAM + GPU                                  |
| Ingress                          | ALB (EKS), or AGIC (AKS), or GCE (GKE), or Nginx Ingress Controller |
| Storage class                    | Block storage with high IOPS                                        |

#### External managed services

| Service        | Purpose                   | Sizing floor       |
| -------------- | ------------------------- | ------------------ |
| PostgreSQL 17+ | Primary data store        | 4 vCPU / 8 GB RAM  |
| Redis 7        | Caching                   | 4 GB RAM           |
| ClickHouse 24+ | Span and event analytics  | 4 vCPU / 16 GB RAM |
| Object storage | License + model artefacts | —                  |

#### Credentials and licenses

You will need the following from your Highflame representative:

* **Highflame license bundle** — `license.jwt` and `public.pem` (`license`)
* **Highflame Feature flag config** — `goff.yaml`
* **GitHub Container Registry access** — docker login password `HIGHFLAME_DOCKER_CRED`
* **Authz signing keys** — For microservice internal communications `authz-signing-keys`
* **Auth keys** — For microservice internal communications `auth-keys`
* **GCP service-account JSON** *(if using GCP-hosted detection models)* — `gcp-credential.json`
* **Identity provider config** — Clerk publishable + secret key (or your SAML/OIDC equivalent)

### Add the Highflame Helm repository

```bash
helm repo add highflame-charts https://highflame-ai.github.io/charts
helm repo update
helm search repo highflame-charts
```

You should see three charts:

| Chart                                | Used by                    |
| ------------------------------------ | -------------------------- |
| `highflame-charts/highflame-generic` | All core platform services |
| `highflame-charts/highflame-redteam` | RedTeam services           |
| `highflame-charts/highflame-ingress` | Optional ingress wiring    |

### Pick your chart and platform values

This repository ships two value-file variants:

| Directory                                                                         | Use when                                                                                                                 |
| --------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| [`EKS/`](https://github.com/highflame-ai/highflame-iac/tree/main/helm-values/EKS) | Deploying on Amazon EKS — uses ALB-friendly annotations, IRSA-compatible service accounts, and `gp3` block storage hints |
| [`AKS/`](https://github.com/highflame-ai/highflame-iac/tree/main/helm-values/AKS) | Deploying on Azure AKS — uses Azure-LB annotations and Workload Identity hints                                           |
| [`GKE/`](https://github.com/highflame-ai/highflame-iac/tree/main/helm-values/GKE) | Deploying on Google GKE — uses GCE-friendly annotations, GCP service accounts, and `pd-ssd` block storage hints          |

Pick the directory that matches your platform and use those files in every `helm upgrade --install` invocation below. The variable list ([`../docs/service-vars.md`](https://github.com/highflame-ai/highflame-iac/blob/main/docs/service-vars.md)) is the same for all.

### Configure environment variables

Export the chart versions and namespace you intend to deploy into. These are referenced by every subsequent command.

```bash
export HIGHFLAME_NAMESPACE="highflame-prod"
export HIGHFLAME_GENERIC_VER="1.0.6"
export HIGHFLAME_REDTEAM_VER="1.0.7"
export HIGHFLAME_INGRESS_VER="1.0.0"
```

Then, for each service, set the variables documented in [`../docs/service-vars.md`](https://github.com/highflame-ai/highflame-iac/blob/main/docs/service-vars.md). The values files use `${VAR_NAME}` placeholders that you should substitute (either with `envsubst`, your CI templating engine or by editing the file directly).

### Create the namespace

```bash
kubectl create namespace ${HIGHFLAME_NAMESPACE}
```

### Create the image pull secret

Highflame images are hosted on GitHub Container Registry. Create the pull secret in your namespace:

```bash
kubectl --namespace ${HIGHFLAME_NAMESPACE} create secret docker-registry highflame-registry-secret \
  --docker-server="ghcr.io" \
  --docker-username="highflame" \
  --docker-password="HIGHFLAME_DOCKER_CRED" \
  --docker-email="highflame@pat.com"
```

{% hint style="info" %}
`HIGHFLAME_DOCKER_CRED` will be shared by the Highflame representative
{% endhint %}

### Create platform secrets

#### Highflame license (`required`)

```bash
kubectl --namespace ${HIGHFLAME_NAMESPACE} create secret generic highflame-license \
  --from-file=./license \
  --dry-run=client -o yaml | kubectl apply -f -
```

{% hint style="info" %}
will be shared by the Highflame representative
{% endhint %}

#### Redis CA certificate (`only if your Redis enforces TLS`)

```bash
kubectl --namespace ${HIGHFLAME_NAMESPACE} create secret generic highflame-redis-cert \
  --from-file=redis-ca.pem \
  --dry-run=client -o yaml | kubectl apply -f -
```

#### GCP service-account credential (`required`)

```bash
kubectl --namespace ${HIGHFLAME_NAMESPACE} create secret generic highflame-gcp-cred \
  --from-file=gcp-credential.json \
  --dry-run=client -o yaml | kubectl apply -f -
```

{% hint style="info" %}
`gcp-credential.json` will be shared by the Highflame representative
{% endhint %}

#### Highflame signing keys - AuthZ signing keys (`required`)

```bash
kubectl --namespace ${HIGHFLAME_NAMESPACE} create secret generic highflame-signing-keys \
  --from-file=./authz-signing-keys \
  --dry-run=client -o yaml | kubectl apply -f -
```

{% hint style="info" %}
`authz-signing-keys` will be shared by the Highflame representative
{% endhint %}

#### Highflame auth keys - JWT keys (`required`)

```bash
kubectl --namespace ${HIGHFLAME_NAMESPACE} create secret generic highflame-auth-keys \
  --from-file=./auth-keys \
  --dry-run=client -o yaml | kubectl apply -f -
```

{% hint style="info" %}
`auth-keys` will be shared by the Highflame representative
{% endhint %}

#### Highflame Feature flag config (`required`)

```bash
kubectl --namespace ${HIGHFLAME_NAMESPACE} create secret generic highflame-goff \
    --from-file=goff.yaml \
    --dry-run=client -o yaml | kubectl apply -f -
```

{% hint style="info" %}
`goff.yaml` will be shared by the Highflame representative
{% endhint %}

### Install services in the correct order

The services have startup dependencies. Deploy them in the order below — each step should be reached `READY 1/1` before moving on.

* highflame-flag

```bash
export HIGHFLAME_SVC="highflame-flag"

helm upgrade --install ${HIGHFLAME_SVC} highflame-charts/highflame-generic \
  --namespace ${HIGHFLAME_NAMESPACE} \
  --version ${HIGHFLAME_GENERIC_VER} \
  -f ${HIGHFLAME_SVC}-helm-values-tmpl.yml \
  --timeout=15m

kubectl --namespace ${HIGHFLAME_NAMESPACE} get deployment ${HIGHFLAME_SVC}
```

* highflame-admin

```bash
export HIGHFLAME_SVC="highflame-admin"

helm upgrade --install ${HIGHFLAME_SVC} highflame-charts/highflame-generic \
  --namespace ${HIGHFLAME_NAMESPACE} \
  --version ${HIGHFLAME_GENERIC_VER} \
  -f ${HIGHFLAME_SVC}-helm-values-tmpl.yml \
  --timeout=15m

kubectl --namespace ${HIGHFLAME_NAMESPACE} get deployment ${HIGHFLAME_SVC}
```

{% hint style="info" %}
Admin runs its own database migrations on startup. The first install may take 2–5 minutes.
{% endhint %}

* highflame-authn

```bash
export HIGHFLAME_SVC="highflame-authn"

helm upgrade --install ${HIGHFLAME_SVC} highflame-charts/highflame-generic \
  --namespace ${HIGHFLAME_NAMESPACE} \
  --version ${HIGHFLAME_GENERIC_VER} \
  -f ${HIGHFLAME_SVC}-helm-values-tmpl.yml \
  --timeout=15m

kubectl --namespace ${HIGHFLAME_NAMESPACE} get deployment ${HIGHFLAME_SVC}
```

* highflame-authz

```bash
export HIGHFLAME_SVC="highflame-authz"

helm upgrade --install ${HIGHFLAME_SVC} highflame-charts/highflame-generic \
  --namespace ${HIGHFLAME_NAMESPACE} \
  --version ${HIGHFLAME_GENERIC_VER} \
  -f ${HIGHFLAME_SVC}-helm-values-tmpl.yml \
  --timeout=15m

kubectl --namespace ${HIGHFLAME_NAMESPACE} get deployment ${HIGHFLAME_SVC}
```

* highflame-cerberus

```bash
export HIGHFLAME_SVC="highflame-cerberus"

helm upgrade --install ${HIGHFLAME_SVC} highflame-charts/highflame-generic \
  --namespace ${HIGHFLAME_NAMESPACE} \
  --version ${HIGHFLAME_GENERIC_VER} \
  -f ${HIGHFLAME_SVC}-helm-values-tmpl.yml \
  --timeout=15m

kubectl --namespace ${HIGHFLAME_NAMESPACE} get deployment ${HIGHFLAME_SVC}
```

* highflame-collector

```bash
export HIGHFLAME_SVC="highflame-collector"

helm upgrade --install ${HIGHFLAME_SVC} highflame-charts/highflame-generic \
  --namespace ${HIGHFLAME_NAMESPACE} \
  --version ${HIGHFLAME_GENERIC_VER} \
  -f ${HIGHFLAME_SVC}-helm-values-tmpl.yml \
  --timeout=15m

kubectl --namespace ${HIGHFLAME_NAMESPACE} get deployment ${HIGHFLAME_SVC}
```

* highflame-dlp

```bash
export HIGHFLAME_SVC="highflame-dlp"

helm upgrade --install ${HIGHFLAME_SVC} highflame-charts/highflame-generic \
  --namespace ${HIGHFLAME_NAMESPACE} \
  --version ${HIGHFLAME_GENERIC_VER} \
  -f ${HIGHFLAME_SVC}-helm-values-tmpl.yml \
  --timeout=15m

kubectl --namespace ${HIGHFLAME_NAMESPACE} get deployment ${HIGHFLAME_SVC}
```

* highflame-firehog

```bash
export HIGHFLAME_SVC="highflame-firehog"

helm upgrade --install ${HIGHFLAME_SVC} highflame-charts/highflame-generic \
  --namespace ${HIGHFLAME_NAMESPACE} \
  --version ${HIGHFLAME_GENERIC_VER} \
  -f ${HIGHFLAME_SVC}-helm-values-tmpl.yml \
  --timeout=15m

kubectl --namespace ${HIGHFLAME_NAMESPACE} get deployment ${HIGHFLAME_SVC}
```

* highflame-guard

```bash
export HIGHFLAME_SVC="highflame-guard"

helm upgrade --install ${HIGHFLAME_SVC} highflame-charts/highflame-generic \
  --namespace ${HIGHFLAME_NAMESPACE} \
  --version ${HIGHFLAME_GENERIC_VER} \
  -f ${HIGHFLAME_SVC}-helm-values-tmpl.yml \
  --timeout=15m

kubectl --namespace ${HIGHFLAME_NAMESPACE} get deployment ${HIGHFLAME_SVC}
```

* highflame-guard-cm

```bash
export HIGHFLAME_SVC="highflame-guard-cm"

helm upgrade --install ${HIGHFLAME_SVC} highflame-charts/highflame-generic \
  --namespace ${HIGHFLAME_NAMESPACE} \
  --version ${HIGHFLAME_GENERIC_VER} \
  -f ${HIGHFLAME_SVC}-helm-values-tmpl.yml \
  --timeout=15m

kubectl --namespace ${HIGHFLAME_NAMESPACE} get deployment ${HIGHFLAME_SVC}
```

* highflame-guard-hallucination

```bash
export HIGHFLAME_SVC="highflame-guard-hallucination"

helm upgrade --install ${HIGHFLAME_SVC} highflame-charts/highflame-generic \
  --namespace ${HIGHFLAME_NAMESPACE} \
  --version ${HIGHFLAME_GENERIC_VER} \
  -f ${HIGHFLAME_SVC}-helm-values-tmpl.yml \
  --timeout=15m

kubectl --namespace ${HIGHFLAME_NAMESPACE} get deployment ${HIGHFLAME_SVC}
```

* highflame-guard-deepcontext

```bash
export HIGHFLAME_SVC="highflame-guard-deepcontext"

helm upgrade --install ${HIGHFLAME_SVC} highflame-charts/highflame-generic \
  --namespace ${HIGHFLAME_NAMESPACE} \
  --version ${HIGHFLAME_GENERIC_VER} \
  -f ${HIGHFLAME_SVC}-helm-values-tmpl.yml \
  --timeout=15m

kubectl --namespace ${HIGHFLAME_NAMESPACE} get deployment ${HIGHFLAME_SVC}
```

* highflame-observatory

```bash
export HIGHFLAME_SVC="highflame-observatory"

helm upgrade --install ${HIGHFLAME_SVC} highflame-charts/highflame-generic \
  --namespace ${HIGHFLAME_NAMESPACE} \
  --version ${HIGHFLAME_GENERIC_VER} \
  -f ${HIGHFLAME_SVC}-helm-values-tmpl.yml \
  --timeout=15m

kubectl --namespace ${HIGHFLAME_NAMESPACE} get deployment ${HIGHFLAME_SVC}
```

* highflame-redteam

```bash
export HIGHFLAME_SVC="highflame-redteam"

helm upgrade --install ${HIGHFLAME_SVC} highflame-charts/highflame-redteam \
  --namespace ${HIGHFLAME_NAMESPACE} \
  --version ${HIGHFLAME_REDTEAM_VER} \
  -f ${HIGHFLAME_SVC}-helm-values-tmpl.yml \
  --timeout=15m

kubectl --namespace ${HIGHFLAME_NAMESPACE} get deployment ${HIGHFLAME_SVC}
```

* highflame-redteam-lab1 (optional)

```bash
export HIGHFLAME_SVC="highflame-redteam-lab1"

helm upgrade --install ${HIGHFLAME_SVC} highflame-charts/highflame-redteam \
  --namespace ${HIGHFLAME_NAMESPACE} \
  --version ${HIGHFLAME_REDTEAM_VER} \
  -f ${HIGHFLAME_SVC}-helm-values-tmpl.yml \
  --timeout=15m

kubectl --namespace ${HIGHFLAME_NAMESPACE} get deployment ${HIGHFLAME_SVC}
```

* highflame-shield

```bash
export HIGHFLAME_SVC="highflame-shield"

helm upgrade --install ${HIGHFLAME_SVC} highflame-charts/highflame-generic \
  --namespace ${HIGHFLAME_NAMESPACE} \
  --version ${HIGHFLAME_GENERIC_VER} \
  -f ${HIGHFLAME_SVC}-helm-values-tmpl.yml \
  --timeout=15m

kubectl --namespace ${HIGHFLAME_NAMESPACE} get deployment ${HIGHFLAME_SVC}
```

* highflame-studio

```bash
export HIGHFLAME_SVC="highflame-studio"

helm upgrade --install ${HIGHFLAME_SVC} highflame-charts/highflame-generic \
  --namespace ${HIGHFLAME_NAMESPACE} \
  --version ${HIGHFLAME_GENERIC_VER} \
  -f ${HIGHFLAME_SVC}-helm-values-tmpl.yml \
  --timeout=15m

kubectl --namespace ${HIGHFLAME_NAMESPACE} get deployment ${HIGHFLAME_SVC}
```

### Verify the deployment

#### All pods running

```bash
kubectl --namespace ${HIGHFLAME_NAMESPACE} get pods
```

#### End-to-end smoke test (Optional)

A scripted smoke test is provided in [`../smoke_test/`](https://github.com/highflame-ai/highflame-iac/tree/main/smoke_test). Run it from your workstation against the deployed cluster to confirm the highflame components are responding correctly.

### Upgrading

To roll out a new chart or image version:

1. Update `HIGHFLAME_GENERIC_VER` / `HIGHFLAME_REDTEAM_VER` to the target.
2. Update `IMAGE_TAG` in each affected values file.
3. Re-run the relevant `helm upgrade --install` command.

`helm upgrade --install` is idempotent — running it again with the same values is a no-op.

> **Schema changes:** Admin migrations are run automatically on startup.

### Troubleshooting

| Symptom                                                       | Likely cause                                                        | Fix                                                                                                      |
| ------------------------------------------------------------- | ------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------- |
| `ImagePullBackOff` on every pod                               | Image pull secret missing or wrong                                  | Re-create `highflame-registry-secret`                                                                    |
| `CrashLoopBackOff` on Admin with `cannot connect to database` | Wrong `DB_HOST` / credentials, or network policy blocking egress    | `kubectl exec` into a pod and `psql` to the DB to confirm reachability                                   |
| Shield pods restart with `failed to verify license`           | License secret missing or `public.pem` does not match `license.jwt` | check the license secrets `highflame-license` and if it is missing, please create it                     |
| Guard pods stuck `Pending`                                    | No GPU node available or wrong `nodeSelector`                       | Confirm GPU node pool labels match the values file's `nodeSelector`                                      |
| Studio loads but shows 401                                    | Clerk config not set, or Admin can't issue tokens                   | Set Clerk env vars in `highflame-admin-helm-values-tmpl.yml` and `highflame-studio-helm-values-tmpl.yml` |
| Pods OOM-killed under load                                    | Default resource requests are conservative                          | Bump `resources.requests` / `resources.limits` in the values file                                        |

### Security checklist

Before exposing your install to production traffic, confirm:

* [ ] `HIGHFLAME_INTERNAL_SERVICE_SECRET` is identical across services but never exposed externally
* [ ] Postgres and Redis are reachable **only** from inside the cluster's VPC/VNet
* [ ] Ingress terminates TLS with a certificate from a trusted CA
* [ ] `IMAGE_TAG` is pinned to an immutable release tag (never `latest`)

{% content-ref url="/pages/KcYf2KneeVzvt8JemY13" %}
[Support](/deployment-guides/support.md)
{% endcontent-ref %}


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.highflame.ai/deployment-guides/highflame-services.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.