Azure

Highflame Deployment Docs for Azure cloud

Deployment on AKS

⚠️ Reference Only (If you are using Terraform)

The highflame-iac repository contains reference Terraform configurations for managing Highflame cloud resources. Do not copy these directly into your IaC code or run them against your cloud environment without first reviewing and patching them to match your account, networking, and security requirements.

Prerequisites

Ensure that the following tools and resources are installed and available:

  • Access to Azure Portal

  • Set of domain names for Highflame services (Highflame Team will share the list of services that require the ingress)

  • AKS cluster with at least 6 worker nodes for user mode nodepool (Best Practice: 3 - 4 nodes for System mode nodepool)

  • Create a log analytics workspace that can be used in AKS logging

  • Application gateway with SSL certs for AKS ingress integration

  • AKS addons to be installed

    • Storage profile for disk driver

    • OMS with the log analytics workspace created above

    • Ingress addon with AGIC

    • Autoscaler profile needs to be enabled

    • Enable Workload autoscaler

  • Create a default node pool for system nodes in the AKS cluster

  • Postgres Flexible Server with private endpoint

  • Azure Cache for Redis + private endpoint

  • Helm v3

  • Kubectl utility

  • All the cloud resources (managed services such as Postgres, Redis etc) should be in the same VNet, or those should be accessible from the Kubernetes Cluster

Cloud Resources and Sizing

Highflame components
Cloud Resources
Size

Highflame services

Deploying the services into your AKS cluster with help of the helm charts

  • CPU nodes: Setup atleast 3 nodes in the user mode nodepool and the worker node SKU can be Standard_F8s_v2 or Standard_D8as_v6

  • GPU nodes: Setup atleast 4 nodes in the user mode nodepool and the worker node SKU can be Standard_NC4as_T4_v3

Trascational Database

Postgres Flexible Server

The Postgres server can be single node or primary - secondary cluster and the SKU can be GP_Standard_D4ads_v5

Analytical Database

Clickhouse database server

Deploying into the AKS

Cache

Azure Cache for Redis

Redis SKU can be Standard

Object Store

Azure blob store

Highflame services required to store files and data in object store, such as Store the clickhouse db backup in the Azure blob store for DR as clickhouse is deploying into the AKS cluster

Logging

Log analytics workspace

The AKS service logs can be pushed to the Workspace with help of AKS addons

Authentication

Clerk

External - Managed by Highflame

Highflame Azure Identity

Prerequisites

Ensure that the following tools and resources are installed and available:

  • Access to the AKS cluster setup above

  • Ensure OIDC issuer is enabled on your AKS cluster

  • Enable Workload Identity on your AKS cluster

  • Azure CLI

User-Assigned Managed Identity (UAMI)

  1. Setting up the environment vars

  1. Create a User-Assigned Managed Identity

  1. Create a federated identity credentials

Resource Access - Azure Blob store

There will be a few Azure blob store requirements for HighFlame, and those were mentioned in the highflame service variables list.

For each Azure Blob store, this command needs to be run

  1. Setting up some additional environment vars

  1. Grant the managed identity a role that allows it to access Azure Blob Storage and its container

Logging Setup

Fluent Bit, a managed solution from Azure, is deployed as a DaemonSet in the AKS cluster to collect logs from each node. It is configured to:

  • Fetch the logs from the /var/log/containers/*.log for each microservice deployed in the AKS

  • Filter the logs from the deployed namespace only

  • Parse Kubernetes logs using the Kubernetes filter and create an individual log group with the highflame service name

  • Forward logs to Azure log analytics workspace

Analytical Database setup - Clickhouse on AKS

This document is the long-form runbook for installing ClickHouse on an AKS cluster using the Altinity ClickHouse Operator, and an Azure blob store-backed clickhouse-backup sidecar.

What gets deployed

The full stack lives entirely inside a dedicated clickhouse namespace and consists of five logical components:

Component
Kind
Source
Notes

Altinity operator

Helm release ch-operator

Upstream chart

Watches ClickHouseInstallation (CHI) and ClickHouseKeeperInstallation (CHK) CRDs

Service account, ConfigMap, Secret

k8s resources

highflame-clickhouse-deps.yml

IRSA-annotated clickhouse-sa, clickhouse-cm, and clickhouse-secrets

ClickHouse Keeper

ClickHouseKeeperInstallation/ch

highflame-clickhouse-values.yml

1 replica, 20Gi PVC. Replaces ZooKeeper

ClickHouse cluster

ClickHouseInstallation/ch

highflame-clickhouse-values.yml

1 shard × 1 replica, 4 CPU / 12 Gi RAM, 512 Gi PVC, clickhouse-backup sidecar

Backup config

k8s resources

highflame-clickhouse-backup-config.yml

Backup config for Azure blob store integration

CronJob + Job

k8s resources

highflame-clickhouse-addons-values.yml

12-hourly remote backups, 60-backup retention; setup Job applies 3-day TTL on system.*_log tables

Pinned versions (verify before installing):

  • Operator chart: 0.25.5

  • clickhouse/clickhouse-server: 25.10.2

  • clickhouse/clickhouse-keeper: 25.10.2

  • altinity/clickhouse-backup: 2.6.39

After installation, the in-cluster service endpoints are:

  • HTTP: clickhouse-ch.clickhouse.svc.cluster.local:8123

  • Native TCP: clickhouse-ch.clickhouse.svc.cluster.local:9000

  • Keeper: keeper-ch.clickhouse.svc.cluster.local:2181

These match the CLICKHOUSE_HOST ConfigMap value and the zookeeper.nodes block in the CHI spec

Prerequisites

Before you start, you need:

  1. An AKS cluster with kubectl context pointing at it (kubectl config current-context).

  2. A storage class that supports ReadWriteOnce PVCs — Premium_LRS is recommended. Both the Keeper and the ClickHouse server use the cluster default unless you uncomment storageClassName in the CHI/CHK templates.

  3. An Azure blob store and container for remote backups, in the same Azure subscription as the cluster. Versioning + lifecycle rules are recommended but not required.

  4. An User-Assigned Managed Identity (UAMI) for the backup Service Account (IRSA) with permission to the Azure blob store and the container Storage Blob Data Contributor on that blob store. The UAMI must allow the OIDC provider of the cluster to assume it for system:serviceaccount:clickhouse:clickhouse-sa.

  5. helm ≥ 3.10

Config files

highflame-clickhouse-deps.yml

Defines:

  • ServiceAccount/clickhouse-sa — annotation azure.workload.identity/client-id must be set to the UAMI created in the prerequisites.

  • ConfigMap/clickhouse-cmCLICKHOUSE_HOST, BACKUP_KEEP=60, CREATE_DB_LIST (comma-separated DBs to create on first boot), BACKUP_DB_LIST=highflame (comma-separated DBs to back up).

  • Secret/clickhouse-secrets — admin / readonly / backup usernames + passwords.

You must edit:

Field
Required value

azure.workload.identity/client-id

IRSA UAMI

CH_ADMIN_PASSWORD

strong password for admin user highflame_admin

CH_READONLY_PASSWORD

strong password for readonly user highflame_readonly

CH_BACKUP_PASSWORD

strong password for backup user highflame_backup

highflame-clickhouse-values.yml

Defines the ClickHouseKeeperInstallation/ch and ClickHouseInstallation/ch custom resources.

Other useful knobs in this file (edit if your environment differs):

  • templates.podTemplates[clickhouse].spec.containers[clickhouse].resources — default request and limit are both cpu: 4, memory: 12Gi.

  • templates.volumeClaimTemplates[clickhouse-data].resources.requests.storage — default 512Gi.

  • templates.podTemplates[clickhouse].spec.nodeSelector — uncomment to pin to a specific nodegroup.

highflame-clickhouse-backup-config.yml

Renders into Secret/clickhouse-backup-config (mounted into the sidecar at /opt/backup-config.yml). Update:

  • BLOB_STORE — Azure blob store name

  • BLOB_CONTAINER — Azure blob container name

  • clickhouse.username / clickhouse.password to match CH_BACKUP_USERNAME / CH_BACKUP_PASSWORD from the previous step

  • The default path: highflame-clickhouse is a key prefix inside the azure blob container — change it per environment if you share a azure blob store between environments.

highflame-clickhouse-addons-values.yml

Defines:

  • CronJob/clickhouse-backup — runs every 12h (0 */12 * * *), iterates BACKUP_DB_LIST from the ConfigMap, calls clickhouse-backup create_remote per DB, and prunes old remote backups beyond BACKUP_KEEP.

  • Job/clickhouse-setup — one-shot job that waits for ClickHouse to answer /ping, creates databases listed in CREATE_DB_LIST, and applies MODIFY TTL event_date + INTERVAL 3 DAY to every system.*_log table.

Install — step by step

Run from the repo root unless noted otherwise.

Namespace

Operator

Verify the operator pod is running, and the CRDs are installed:

You should see

  • clickhouseinstallations.clickhouse.altinity.com

  • clickhouseinstallationtemplates.clickhouse.altinity.com

  • clickhousekeeperinstallations.clickhouse-keeper.altinity.com

Dependencies (SA + ConfigMap + Secret)

Keeper + ClickHouse cluster

Verify the deployment:

Backup secret

The setup Job runs once and exits Completed. The CronJob fires every 12h

Clickhouse Azure Blob Store backup management (From the backup containers)

List down the remote backup

Restore from remote backup (For DR)

Highflame service deployment

Follow this documentation to deploy Highflame services to your Kubernetes cluster using Helm charts.

Whether you're standing up a fresh environment or upgrading an existing one, this guide has you covered end to end.

Highflame Services

Last updated