diff --git a/documentation/Production Setup/cluster-install.md b/documentation/Production Setup/cluster-install.md index 3fec1f0..ead182d 100644 --- a/documentation/Production Setup/cluster-install.md +++ b/documentation/Production Setup/cluster-install.md @@ -725,6 +725,90 @@ Let op: apply alleen triggert niet altijd een rollout als er geen inhoudelijke s - Kubernetes cluster overview (metrics) - Kubernetes cluster logs (controlplane logs) +### Phase 11: Flower Setup + +#### Overzicht +Flower is de Celery monitoring UI. We deployen Flower in de namespace `monitoring` via de bjw-s/app-template Helm chart. Er is geen Ingress; toegang gebeurt enkel lokaal via `kubectl port-forward`. Verbinding naar Redis gebruikt TLS met je private CA; hostname‑verificatie staat uit omdat je via IP verbindt. + +#### Helm repository toevoegen +```bash +helm repo add bjw-s https://bjw-s-labs.github.io/helm-charts +helm repo update +helm search repo bjw-s/app-template +``` + +#### Deploy (aanbevolen: alleen Flower via Helm CLI) +Gebruik gerichte commando’s zodat enkel Flower wordt beheerd door Helm en de rest van de monitoring stack ongemoeid blijft. +```bash +# 1) ExternalSecrets en NetworkPolicy aanmaken +kubectl apply -f scaleway/manifests/base/monitoring/flower/externalsecrets.yaml +kubectl apply -f scaleway/manifests/base/monitoring/flower/networkpolicy.yaml + +# 2) Flower installeren via Helm (alleen deze release) +helm upgrade --install flower bjw-s/app-template \ + -n monitoring --create-namespace \ + -f scaleway/manifests/base/monitoring/flower/values.yaml +``` +Wat dit deployt: +- ExternalSecrets: `flower-redis` (REDIS_USER/PASS/URL/PORT) en `flower-ca` (REDIS_CERT) uit `scaleway-cluster-secret-store` +- Flower via Helm (bjw-s/app-template): + - Image: `mher/flower:2.0.1` (gepind) + - Start: `/usr/local/bin/celery --broker=$(BROKER) flower --address=0.0.0.0 --port=5555` + - TLS naar Redis met CA-mount op `/etc/ssl/redis/ca.pem` en `ssl_check_hostname=false` + - Hardened securityContext (non-root, read-only rootfs, capabilities drop) + - Probes en resource requests/limits +- Service: ClusterIP `flower` op poort 5555 +- NetworkPolicy: ingress default-deny; egress enkel naar Redis (172.16.16.2:6379/TCP) en CoreDNS (53 TCP/UDP) + +#### Verifiëren +```bash +# Helm release en resources +helm list -n monitoring +kubectl -n monitoring get externalsecret +kubectl -n monitoring get secret | grep flower +kubectl -n monitoring get deploy,po,svc | grep flower +kubectl -n monitoring logs deploy/flower --tail=200 || true +``` + +#### Toegang (port-forward) +```bash +kubectl -n monitoring port-forward svc/flower 5555:5555 +# Browser: http://localhost:5555 +``` + +#### Security & TLS +- Geen Ingress/extern verkeer; enkel port-forward. +- TLS naar Redis met CA-mount op `/etc/ssl/redis/ca.pem`. +- Omdat je Redis via IP aanspreekt, staat `ssl_check_hostname=false`. +- Strikte egress NetworkPolicy: update het IP indien je Redis IP verandert. + +#### Troubleshooting +```bash +# Secrets en ExternalSecrets +kubectl -n monitoring describe externalsecret flower-redis +kubectl -n monitoring describe externalsecret flower-ca + +# Pods & logs +kubectl -n monitoring get pods -l app=flower -w +kubectl -n monitoring logs deploy/flower --tail=200 + +# NetworkPolicy +kubectl -n monitoring describe networkpolicy flower-policy +``` + +#### Alternatief: Kustomize rendering (let op!) +Je kunt Flower ook via Kustomize renderen samen met de monitoring chart: +```bash +kubectl kustomize --enable-helm scaleway/manifests/base/monitoring | kubectl apply -f - +``` +Let op: dit rendert en apply’t álle resources in de monitoring Kustomization, inclusief de kube-prometheus-stack chart. Gebruik dit alleen als je bewust de volledige monitoring stack declaratief wil bijwerken. + +#### Migratie & Opschonen +Als je eerder de losse Deployment/Service hebt gebruikt: +```bash +kubectl -n monitoring delete deploy flower --ignore-not-found +kubectl -n monitoring delete svc flower --ignore-not-found +``` ## Verification and Testing diff --git a/eveai_api/__init__.py b/eveai_api/__init__.py index 75bc08e..5d5bf55 100644 --- a/eveai_api/__init__.py +++ b/eveai_api/__init__.py @@ -30,6 +30,8 @@ def create_app(config_file=None): match environment: case 'development': app.config.from_object(get_config('dev')) + case 'staging': + app.config.from_object(get_config('staging')) case 'production': app.config.from_object(get_config('prod')) case _: diff --git a/scaleway/manifests/base/applications/backend/eveai-chat-workers/deployment.yaml b/scaleway/manifests/base/applications/backend/eveai-chat-workers/deployment.yaml index c4ec657..cf00069 100644 --- a/scaleway/manifests/base/applications/backend/eveai-chat-workers/deployment.yaml +++ b/scaleway/manifests/base/applications/backend/eveai-chat-workers/deployment.yaml @@ -26,6 +26,8 @@ spec: - secretRef: name: eveai-secrets env: + - name: FLASK_ENV + value: "staging" - name: COMPONENT_NAME value: "eveai_chat_workers" - name: ROLE diff --git a/scaleway/manifests/base/applications/backend/eveai-entitlements/deployment.yaml b/scaleway/manifests/base/applications/backend/eveai-entitlements/deployment.yaml index fcc42ab..1b6a6cc 100644 --- a/scaleway/manifests/base/applications/backend/eveai-entitlements/deployment.yaml +++ b/scaleway/manifests/base/applications/backend/eveai-entitlements/deployment.yaml @@ -26,6 +26,8 @@ spec: - secretRef: name: eveai-secrets env: + - name: FLASK_ENV + value: "staging" - name: COMPONENT_NAME value: "eveai_entitlements" - name: ROLE diff --git a/scaleway/manifests/base/applications/backend/eveai-workers/deployment.yaml b/scaleway/manifests/base/applications/backend/eveai-workers/deployment.yaml index 0cfcc23..7c220c1 100644 --- a/scaleway/manifests/base/applications/backend/eveai-workers/deployment.yaml +++ b/scaleway/manifests/base/applications/backend/eveai-workers/deployment.yaml @@ -26,6 +26,8 @@ spec: - secretRef: name: eveai-secrets env: + - name: FLASK_ENV + value: "staging" - name: COMPONENT_NAME value: "eveai_workers" - name: ROLE diff --git a/scaleway/manifests/base/applications/frontend/eveai-api/deployment.yaml b/scaleway/manifests/base/applications/frontend/eveai-api/deployment.yaml index 00fa373..2b785ad 100644 --- a/scaleway/manifests/base/applications/frontend/eveai-api/deployment.yaml +++ b/scaleway/manifests/base/applications/frontend/eveai-api/deployment.yaml @@ -29,6 +29,8 @@ spec: - secretRef: name: eveai-secrets env: + - name: FLASK_ENV + value: "staging" - name: COMPONENT_NAME value: "eveai_api" - name: ROLE diff --git a/scaleway/manifests/base/applications/frontend/eveai-app/deployment.yaml b/scaleway/manifests/base/applications/frontend/eveai-app/deployment.yaml index 24c2149..7c0e414 100644 --- a/scaleway/manifests/base/applications/frontend/eveai-app/deployment.yaml +++ b/scaleway/manifests/base/applications/frontend/eveai-app/deployment.yaml @@ -29,6 +29,8 @@ spec: - secretRef: name: eveai-secrets env: + - name: FLASK_ENV + value: "staging" - name: COMPONENT_NAME value: "eveai_app" - name: ROLE diff --git a/scaleway/manifests/base/applications/frontend/eveai-chat-client/deployment.yaml b/scaleway/manifests/base/applications/frontend/eveai-chat-client/deployment.yaml index 27c0082..7103b92 100644 --- a/scaleway/manifests/base/applications/frontend/eveai-chat-client/deployment.yaml +++ b/scaleway/manifests/base/applications/frontend/eveai-chat-client/deployment.yaml @@ -29,6 +29,8 @@ spec: - secretRef: name: eveai-secrets env: + - name: FLASK_ENV + value: "staging" - name: COMPONENT_NAME value: "eveai_chat_client" - name: ROLE diff --git a/scaleway/manifests/base/applications/ops/jobs/00-env-check-job.yaml b/scaleway/manifests/base/applications/ops/jobs/00-env-check-job.yaml index 10d04d1..b7ae163 100644 --- a/scaleway/manifests/base/applications/ops/jobs/00-env-check-job.yaml +++ b/scaleway/manifests/base/applications/ops/jobs/00-env-check-job.yaml @@ -28,6 +28,8 @@ spec: - secretRef: name: eveai-secrets env: + - name: FLASK_ENV + value: "staging" - name: FLASK_APP value: "/app/scripts/run.py" - name: COMPONENT_NAME diff --git a/scaleway/manifests/base/applications/ops/jobs/02-db-bootstrap-ext-job.yaml b/scaleway/manifests/base/applications/ops/jobs/02-db-bootstrap-ext-job.yaml index 5f5bad8..f5a7af1 100644 --- a/scaleway/manifests/base/applications/ops/jobs/02-db-bootstrap-ext-job.yaml +++ b/scaleway/manifests/base/applications/ops/jobs/02-db-bootstrap-ext-job.yaml @@ -29,6 +29,8 @@ spec: - secretRef: name: eveai-secrets env: + - name: FLASK_ENV + value: "staging" - name: FLASK_APP value: "/app/scripts/run.py" - name: COMPONENT_NAME diff --git a/scaleway/manifests/base/applications/ops/jobs/03-db-migrate-public-job.yaml b/scaleway/manifests/base/applications/ops/jobs/03-db-migrate-public-job.yaml index e0ec750..774c9ad 100644 --- a/scaleway/manifests/base/applications/ops/jobs/03-db-migrate-public-job.yaml +++ b/scaleway/manifests/base/applications/ops/jobs/03-db-migrate-public-job.yaml @@ -29,6 +29,8 @@ spec: - secretRef: name: eveai-secrets env: + - name: FLASK_ENV + value: "staging" - name: FLASK_APP value: "/app/scripts/run.py" - name: COMPONENT_NAME diff --git a/scaleway/manifests/base/applications/ops/jobs/04-db-migrate-tenant-job.yaml b/scaleway/manifests/base/applications/ops/jobs/04-db-migrate-tenant-job.yaml index 8147385..a42f418 100644 --- a/scaleway/manifests/base/applications/ops/jobs/04-db-migrate-tenant-job.yaml +++ b/scaleway/manifests/base/applications/ops/jobs/04-db-migrate-tenant-job.yaml @@ -29,6 +29,8 @@ spec: - secretRef: name: eveai-secrets env: + - name: FLASK_ENV + value: "staging" - name: FLASK_APP value: "/app/scripts/run.py" - name: COMPONENT_NAME diff --git a/scaleway/manifests/base/applications/ops/jobs/05-seed-or-init-data-job.yaml b/scaleway/manifests/base/applications/ops/jobs/05-seed-or-init-data-job.yaml index 97465e3..0754d3e 100644 --- a/scaleway/manifests/base/applications/ops/jobs/05-seed-or-init-data-job.yaml +++ b/scaleway/manifests/base/applications/ops/jobs/05-seed-or-init-data-job.yaml @@ -29,6 +29,8 @@ spec: - secretRef: name: eveai-secrets env: + - name: FLASK_ENV + value: "staging" - name: FLASK_APP value: "/app/scripts/run.py" - name: COMPONENT_NAME diff --git a/scaleway/manifests/base/applications/ops/jobs/06-verify-minimal-job.yaml b/scaleway/manifests/base/applications/ops/jobs/06-verify-minimal-job.yaml index 1726cb7..7cc8693 100644 --- a/scaleway/manifests/base/applications/ops/jobs/06-verify-minimal-job.yaml +++ b/scaleway/manifests/base/applications/ops/jobs/06-verify-minimal-job.yaml @@ -29,6 +29,8 @@ spec: - secretRef: name: eveai-secrets env: + - name: FLASK_ENV + value: "staging" - name: FLASK_APP value: "/app/scripts/run.py" - name: COMPONENT_NAME diff --git a/scaleway/manifests/base/monitoring/flower/externalsecrets.yaml b/scaleway/manifests/base/monitoring/flower/externalsecrets.yaml new file mode 100644 index 0000000..8270276 --- /dev/null +++ b/scaleway/manifests/base/monitoring/flower/externalsecrets.yaml @@ -0,0 +1,48 @@ +apiVersion: external-secrets.io/v1 +kind: ExternalSecret +metadata: + name: flower-redis + namespace: monitoring +spec: + refreshInterval: 300s + secretStoreRef: + name: scaleway-cluster-secret-store + kind: ClusterSecretStore + target: + name: flower-redis + creationPolicy: Owner + data: + - secretKey: REDIS_USER + remoteRef: + key: name:eveai-redis + property: REDIS_USER + - secretKey: REDIS_PASS + remoteRef: + key: name:eveai-redis + property: REDIS_PASS + - secretKey: REDIS_URL + remoteRef: + key: name:eveai-redis + property: REDIS_URL + - secretKey: REDIS_PORT + remoteRef: + key: name:eveai-redis + property: REDIS_PORT +--- +apiVersion: external-secrets.io/v1 +kind: ExternalSecret +metadata: + name: flower-ca + namespace: monitoring +spec: + refreshInterval: 300s + secretStoreRef: + name: scaleway-cluster-secret-store + kind: ClusterSecretStore + target: + name: flower-ca + creationPolicy: Owner + data: + - secretKey: REDIS_CERT + remoteRef: + key: name:eveai-redis-certificate # note: no property \ No newline at end of file diff --git a/scaleway/manifests/base/monitoring/flower/networkpolicy.yaml b/scaleway/manifests/base/monitoring/flower/networkpolicy.yaml new file mode 100644 index 0000000..2538a64 --- /dev/null +++ b/scaleway/manifests/base/monitoring/flower/networkpolicy.yaml @@ -0,0 +1,44 @@ +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: flower-policy + namespace: monitoring +spec: + podSelector: + matchLabels: + app: flower + policyTypes: + - Ingress + - Egress + # Default deny all ingress by providing empty ingress rules + ingress: [] + egress: + # Allow egress to Redis (TLS) on specific IP and port + - to: + - ipBlock: + cidr: 172.16.16.2/32 + ports: + - protocol: TCP + port: 6379 + # Allow DNS queries to CoreDNS in kube-system (UDP 53) + - to: + - namespaceSelector: + matchLabels: + kubernetes.io/metadata.name: kube-system + podSelector: + matchLabels: + k8s-app: kube-dns + ports: + - protocol: UDP + port: 53 + # Also allow TCP 53 (some resolvers use TCP for large responses) + - to: + - namespaceSelector: + matchLabels: + kubernetes.io/metadata.name: kube-system + podSelector: + matchLabels: + k8s-app: kube-dns + ports: + - protocol: TCP + port: 53 diff --git a/scaleway/manifests/base/monitoring/kustomization.yaml b/scaleway/manifests/base/monitoring/kustomization.yaml index 2cbb296..3c706ec 100644 --- a/scaleway/manifests/base/monitoring/kustomization.yaml +++ b/scaleway/manifests/base/monitoring/kustomization.yaml @@ -3,6 +3,10 @@ kind: Kustomization namespace: monitoring +resources: + - flower/externalsecrets.yaml + - flower/networkpolicy.yaml + helmCharts: - name: kube-prometheus-stack repo: https://prometheus-community.github.io/helm-charts