- Addition of FLASK_ENV setting for all eveai services

- Addition of flower to the monitoring stack
2025-09-09 21:07:10 +02:00
parent 10ca344c84
commit 6fbaff45a8
17 changed files with 206 additions and 0 deletions
--- a/Setup/cluster-install.md
+++ b/Setup/cluster-install.md
@@ -725,6 +725,90 @@ Let op: apply alleen triggert niet altijd een rollout als er geen inhoudelijke s
  - Kubernetes cluster overview (metrics)
  - Kubernetes cluster logs (controlplane logs)

+### Phase 11: Flower Setup
+
+#### Overzicht
+Flower is de Celery monitoring UI. We deployen Flower in de namespace `monitoring` via de bjw-s/app-template Helm chart. Er is geen Ingress; toegang gebeurt enkel lokaal via `kubectl port-forward`. Verbinding naar Redis gebruikt TLS met je private CA; hostname‑verificatie staat uit omdat je via IP verbindt.
+
+#### Helm repository toevoegen
+```bash
+helm repo add bjw-s https://bjw-s-labs.github.io/helm-charts
+helm repo update
+helm search repo bjw-s/app-template
+```
+
+#### Deploy (aanbevolen: alleen Flower via Helm CLI)
+Gebruik gerichte commando’s zodat enkel Flower wordt beheerd door Helm en de rest van de monitoring stack ongemoeid blijft.
+```bash
+# 1) ExternalSecrets en NetworkPolicy aanmaken
+kubectl apply -f scaleway/manifests/base/monitoring/flower/externalsecrets.yaml
+kubectl apply -f scaleway/manifests/base/monitoring/flower/networkpolicy.yaml
+
+# 2) Flower installeren via Helm (alleen deze release)
+helm upgrade --install flower bjw-s/app-template \
+  -n monitoring --create-namespace \
+  -f scaleway/manifests/base/monitoring/flower/values.yaml
+```
+Wat dit deployt:
+- ExternalSecrets: `flower-redis` (REDIS_USER/PASS/URL/PORT) en `flower-ca` (REDIS_CERT) uit `scaleway-cluster-secret-store`
+- Flower via Helm (bjw-s/app-template):
+  - Image: `mher/flower:2.0.1` (gepind)
+  - Start: `/usr/local/bin/celery --broker=$(BROKER) flower --address=0.0.0.0 --port=5555`
+  - TLS naar Redis met CA-mount op `/etc/ssl/redis/ca.pem` en `ssl_check_hostname=false`
+  - Hardened securityContext (non-root, read-only rootfs, capabilities drop)
+  - Probes en resource requests/limits
+- Service: ClusterIP `flower` op poort 5555
+- NetworkPolicy: ingress default-deny; egress enkel naar Redis (172.16.16.2:6379/TCP) en CoreDNS (53 TCP/UDP)
+
+#### Verifiëren
+```bash
+# Helm release en resources
+helm list -n monitoring
+kubectl -n monitoring get externalsecret
+kubectl -n monitoring get secret | grep flower
+kubectl -n monitoring get deploy,po,svc | grep flower
+kubectl -n monitoring logs deploy/flower --tail=200 || true
+```
+
+#### Toegang (port-forward)
+```bash
+kubectl -n monitoring port-forward svc/flower 5555:5555
+# Browser: http://localhost:5555
+```
+
+#### Security & TLS
+- Geen Ingress/extern verkeer; enkel port-forward.
+- TLS naar Redis met CA-mount op `/etc/ssl/redis/ca.pem`.
+- Omdat je Redis via IP aanspreekt, staat `ssl_check_hostname=false`.
+- Strikte egress NetworkPolicy: update het IP indien je Redis IP verandert.
+
+#### Troubleshooting
+```bash
+# Secrets en ExternalSecrets
+kubectl -n monitoring describe externalsecret flower-redis
+kubectl -n monitoring describe externalsecret flower-ca
+
+# Pods & logs
+kubectl -n monitoring get pods -l app=flower -w
+kubectl -n monitoring logs deploy/flower --tail=200
+
+# NetworkPolicy
+kubectl -n monitoring describe networkpolicy flower-policy
+```
+
+#### Alternatief: Kustomize rendering (let op!)
+Je kunt Flower ook via Kustomize renderen samen met de monitoring chart:
+```bash
+kubectl kustomize --enable-helm scaleway/manifests/base/monitoring | kubectl apply -f -
+```
+Let op: dit rendert en apply’t álle resources in de monitoring Kustomization, inclusief de kube-prometheus-stack chart. Gebruik dit alleen als je bewust de volledige monitoring stack declaratief wil bijwerken.
+
+#### Migratie & Opschonen
+Als je eerder de losse Deployment/Service hebt gebruikt:
+```bash
+kubectl -n monitoring delete deploy flower --ignore-not-found
+kubectl -n monitoring delete svc flower --ignore-not-found
+```

 ## Verification and Testing

--- a/eveai_api/init.py
+++ b/eveai_api/init.py
@@ -30,6 +30,8 @@ def create_app(config_file=None):
    match environment:
        case 'development':
            app.config.from_object(get_config('dev'))
+        case 'staging':
+            app.config.from_object(get_config('staging'))
        case 'production':
            app.config.from_object(get_config('prod'))
        case _:
--- a/scaleway/manifests/base/applications/backend/eveai-chat-workers/deployment.yaml
+++ b/scaleway/manifests/base/applications/backend/eveai-chat-workers/deployment.yaml
@@ -26,6 +26,8 @@ spec:
        - secretRef:
            name: eveai-secrets
        env:
+        - name: FLASK_ENV
+          value: "staging"
        - name: COMPONENT_NAME
          value: "eveai_chat_workers"
        - name: ROLE
--- a/scaleway/manifests/base/applications/backend/eveai-entitlements/deployment.yaml
+++ b/scaleway/manifests/base/applications/backend/eveai-entitlements/deployment.yaml
@@ -26,6 +26,8 @@ spec:
        - secretRef:
            name: eveai-secrets
        env:
+        - name: FLASK_ENV
+          value: "staging"
        - name: COMPONENT_NAME
          value: "eveai_entitlements"
        - name: ROLE
--- a/scaleway/manifests/base/applications/backend/eveai-workers/deployment.yaml
+++ b/scaleway/manifests/base/applications/backend/eveai-workers/deployment.yaml
@@ -26,6 +26,8 @@ spec:
        - secretRef:
            name: eveai-secrets
        env:
+        - name: FLASK_ENV
+          value: "staging"
        - name: COMPONENT_NAME
          value: "eveai_workers"
        - name: ROLE
--- a/scaleway/manifests/base/applications/frontend/eveai-api/deployment.yaml
+++ b/scaleway/manifests/base/applications/frontend/eveai-api/deployment.yaml
@@ -29,6 +29,8 @@ spec:
        - secretRef:
            name: eveai-secrets
        env:
+        - name: FLASK_ENV
+          value: "staging"
        - name: COMPONENT_NAME
          value: "eveai_api"
        - name: ROLE
--- a/scaleway/manifests/base/applications/frontend/eveai-app/deployment.yaml
+++ b/scaleway/manifests/base/applications/frontend/eveai-app/deployment.yaml
@@ -29,6 +29,8 @@ spec:
        - secretRef:
            name: eveai-secrets
        env:
+        - name: FLASK_ENV
+          value: "staging"
        - name: COMPONENT_NAME
          value: "eveai_app"
        - name: ROLE
--- a/scaleway/manifests/base/applications/frontend/eveai-chat-client/deployment.yaml
+++ b/scaleway/manifests/base/applications/frontend/eveai-chat-client/deployment.yaml
@@ -29,6 +29,8 @@ spec:
        - secretRef:
            name: eveai-secrets
        env:
+        - name: FLASK_ENV
+          value: "staging"
        - name: COMPONENT_NAME
          value: "eveai_chat_client"
        - name: ROLE
--- a/scaleway/manifests/base/applications/ops/jobs/00-env-check-job.yaml
+++ b/scaleway/manifests/base/applications/ops/jobs/00-env-check-job.yaml
@@ -28,6 +28,8 @@ spec:
        - secretRef:
            name: eveai-secrets
        env:
+        - name: FLASK_ENV
+          value: "staging"
        - name: FLASK_APP
          value: "/app/scripts/run.py"
        - name: COMPONENT_NAME
--- a/scaleway/manifests/base/applications/ops/jobs/02-db-bootstrap-ext-job.yaml
+++ b/scaleway/manifests/base/applications/ops/jobs/02-db-bootstrap-ext-job.yaml
@@ -29,6 +29,8 @@ spec:
        - secretRef:
            name: eveai-secrets
        env:
+        - name: FLASK_ENV
+          value: "staging"
        - name: FLASK_APP
          value: "/app/scripts/run.py"
        - name: COMPONENT_NAME
--- a/scaleway/manifests/base/applications/ops/jobs/03-db-migrate-public-job.yaml
+++ b/scaleway/manifests/base/applications/ops/jobs/03-db-migrate-public-job.yaml
@@ -29,6 +29,8 @@ spec:
        - secretRef:
            name: eveai-secrets
        env:
+        - name: FLASK_ENV
+          value: "staging"
        - name: FLASK_APP
          value: "/app/scripts/run.py"
        - name: COMPONENT_NAME
--- a/scaleway/manifests/base/applications/ops/jobs/04-db-migrate-tenant-job.yaml
+++ b/scaleway/manifests/base/applications/ops/jobs/04-db-migrate-tenant-job.yaml
@@ -29,6 +29,8 @@ spec:
        - secretRef:
            name: eveai-secrets
        env:
+        - name: FLASK_ENV
+          value: "staging"
        - name: FLASK_APP
          value: "/app/scripts/run.py"
        - name: COMPONENT_NAME
--- a/scaleway/manifests/base/applications/ops/jobs/05-seed-or-init-data-job.yaml
+++ b/scaleway/manifests/base/applications/ops/jobs/05-seed-or-init-data-job.yaml
@@ -29,6 +29,8 @@ spec:
        - secretRef:
            name: eveai-secrets
        env:
+        - name: FLASK_ENV
+          value: "staging"
        - name: FLASK_APP
          value: "/app/scripts/run.py"
        - name: COMPONENT_NAME
--- a/scaleway/manifests/base/applications/ops/jobs/06-verify-minimal-job.yaml
+++ b/scaleway/manifests/base/applications/ops/jobs/06-verify-minimal-job.yaml
@@ -29,6 +29,8 @@ spec:
        - secretRef:
            name: eveai-secrets
        env:
+        - name: FLASK_ENV
+          value: "staging"
        - name: FLASK_APP
          value: "/app/scripts/run.py"
        - name: COMPONENT_NAME
--- a/scaleway/manifests/base/monitoring/flower/externalsecrets.yaml
+++ b/scaleway/manifests/base/monitoring/flower/externalsecrets.yaml
@@ -0,0 +1,48 @@
+apiVersion: external-secrets.io/v1
+kind: ExternalSecret
+metadata:
+  name: flower-redis
+  namespace: monitoring
+spec:
+  refreshInterval: 300s
+  secretStoreRef:
+    name: scaleway-cluster-secret-store
+    kind: ClusterSecretStore
+  target:
+    name: flower-redis
+    creationPolicy: Owner
+  data:
+    - secretKey: REDIS_USER
+      remoteRef:
+        key: name:eveai-redis
+        property: REDIS_USER
+    - secretKey: REDIS_PASS
+      remoteRef:
+        key: name:eveai-redis
+        property: REDIS_PASS
+    - secretKey: REDIS_URL
+      remoteRef:
+        key: name:eveai-redis
+        property: REDIS_URL
+    - secretKey: REDIS_PORT
+      remoteRef:
+        key: name:eveai-redis
+        property: REDIS_PORT
+---
+apiVersion: external-secrets.io/v1
+kind: ExternalSecret
+metadata:
+  name: flower-ca
+  namespace: monitoring
+spec:
+  refreshInterval: 300s
+  secretStoreRef:
+    name: scaleway-cluster-secret-store
+    kind: ClusterSecretStore
+  target:
+    name: flower-ca
+    creationPolicy: Owner
+  data:
+    - secretKey: REDIS_CERT
+      remoteRef:
+        key: name:eveai-redis-certificate   # note: no property
--- a/scaleway/manifests/base/monitoring/flower/networkpolicy.yaml
+++ b/scaleway/manifests/base/monitoring/flower/networkpolicy.yaml
@@ -0,0 +1,44 @@
+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
+metadata:
+  name: flower-policy
+  namespace: monitoring
+spec:
+  podSelector:
+    matchLabels:
+      app: flower
+  policyTypes:
+    - Ingress
+    - Egress
+  # Default deny all ingress by providing empty ingress rules
+  ingress: []
+  egress:
+    # Allow egress to Redis (TLS) on specific IP and port
+    - to:
+        - ipBlock:
+            cidr: 172.16.16.2/32
+      ports:
+        - protocol: TCP
+          port: 6379
+    # Allow DNS queries to CoreDNS in kube-system (UDP 53)
+    - to:
+        - namespaceSelector:
+            matchLabels:
+              kubernetes.io/metadata.name: kube-system
+          podSelector:
+            matchLabels:
+              k8s-app: kube-dns
+      ports:
+        - protocol: UDP
+          port: 53
+    # Also allow TCP 53 (some resolvers use TCP for large responses)
+    - to:
+        - namespaceSelector:
+            matchLabels:
+              kubernetes.io/metadata.name: kube-system
+          podSelector:
+            matchLabels:
+              k8s-app: kube-dns
+      ports:
+        - protocol: TCP
+          port: 53
--- a/scaleway/manifests/base/monitoring/kustomization.yaml
+++ b/scaleway/manifests/base/monitoring/kustomization.yaml
@@ -3,6 +3,10 @@ kind: Kustomization

 namespace: monitoring

+resources:
+  - flower/externalsecrets.yaml
+  - flower/networkpolicy.yaml
+
 helmCharts:
 - name: kube-prometheus-stack
  repo: https://prometheus-community.github.io/helm-charts