- Check-in voordat we aan bugfix beginnen te werken.

- Introductie van static-files serving met standaard nginx (niet ons docker nginx image), en een rsync service om static files te synchroniseren. Nog niet volledig afgewerkt!
2025-08-21 05:48:03 +02:00
parent 9c63ecb17f
commit 4c00d33bc3
10 changed files with 467 additions and 194 deletions
--- a/k8s/dev/INGRESS_MIGRATION_SUMMARY.md
+++ b/k8s/dev/INGRESS_MIGRATION_SUMMARY.md
@@ -1,157 +0,0 @@
-# EveAI Kubernetes Ingress Migration - Complete Implementation
-
-## Migration Summary
-
-The migration from nginx reverse proxy to Kubernetes Ingress has been successfully implemented. This migration provides a production-ready, native Kubernetes solution for HTTP routing.
-
-## Changes Made
-
-### 1. Setup Script Updates
-**File: `setup-dev-cluster.sh`**
- ✅ Added `install_ingress_controller()` function
- ✅ Automatically installs NGINX Ingress Controller for Kind
- ✅ Updated main() function to include Ingress Controller installation
- ✅ Updated final output to show Ingress-based access URLs
-
-### 2. New Configuration Files
-
-**File: `static-files-service.yaml`** ✅
- ConfigMap with nginx configuration for static file serving
- Deployment with initContainer to copy static files from existing nginx image
- Service (ClusterIP) for internal access
- Optimized for production with proper caching headers
-
-**File: `eveai-ingress.yaml`** ✅
- Ingress resource with path-based routing
- Routes: `/static/`, `/admin/`, `/api/`, `/chat-client/`, `/`
- Proper annotations for proxy settings and URL rewriting
- Host-based routing for `minty.ask-eve-ai-local.com`
-
-**File: `monitoring-services.yaml`** ✅
- Extracted monitoring services from nginx-monitoring-services.yaml
- Contains: Flower, Prometheus, Grafana deployments and services
- No nginx components included
-
-### 3. Deployment Script Updates
-**File: `deploy-all-services.sh`**
- ✅ Replaced `deploy_nginx_monitoring()` with `deploy_static_ingress()` and `deploy_monitoring_only()`
- ✅ Added `test_connectivity_ingress()` function for Ingress endpoint testing
- ✅ Added `show_connection_info_ingress()` function with updated URLs
- ✅ Updated main() function to use new deployment functions
-
-## Architecture Changes
-
-### Before (nginx reverse proxy):
-```
-Client → nginx:3080 → {eveai_app:5001, eveai_api:5003, eveai_chat_client:5004}
-```
-
-### After (Kubernetes Ingress):
-```
-Client → Ingress Controller:3080 → {
-  /static/* → static-files-service:80
-  /admin/* → eveai-app-service:5001  
-  /api/* → eveai-api-service:5003
-  /chat-client/* → eveai-chat-client-service:5004
-}
-```
-
-## Benefits Achieved
-
-1. **Native Kubernetes**: Using standard Ingress resources instead of custom nginx
-2. **Production Ready**: Separate static files service with optimized caching
-3. **Scalable**: Static files service can be scaled independently
-4. **Maintainable**: Declarative YAML configuration instead of nginx.conf
-5. **No CORS Issues**: All traffic goes through same host (as correctly identified)
-6. **URL Rewriting**: Handled by existing `nginx_utils.py` via Ingress headers
-
-## Usage Instructions
-
-### 1. Complete Cluster Setup (One Command)
-```bash
-cd k8s/dev
-./setup-dev-cluster.sh
-```
-This now automatically:
- Creates Kind cluster
- Installs NGINX Ingress Controller
- Applies base manifests
-
-### 2. Deploy All Services
-```bash
-./deploy-all-services.sh
-```
-This now:
- Deploys application services
- Deploys static files service
- Deploys Ingress configuration
- Deploys monitoring services separately
-
-### 3. Access Services (via Ingress)
- **Main App**: http://minty.ask-eve-ai-local.com:3080/admin/
- **API**: http://minty.ask-eve-ai-local.com:3080/api/
- **Chat Client**: http://minty.ask-eve-ai-local.com:3080/chat-client/
- **Static Files**: http://minty.ask-eve-ai-local.com:3080/static/
-
-### 4. Monitoring (Direct Access)
- **Flower**: http://minty.ask-eve-ai-local.com:3007
- **Prometheus**: http://minty.ask-eve-ai-local.com:3010
- **Grafana**: http://minty.ask-eve-ai-local.com:3012
-
-## Validation Status
-
-✅ All YAML files validated for syntax correctness
-✅ Setup script updated and tested
-✅ Deployment script updated and tested
-✅ Ingress configuration created with proper routing
-✅ Static files service configured with production optimizations
-
-## Files Modified/Created
-
-### Modified Files:
- `setup-dev-cluster.sh` - Added Ingress Controller installation
- `deploy-all-services.sh` - Updated for Ingress deployment
-
-### New Files:
- `static-files-service.yaml` - Dedicated static files service
- `eveai-ingress.yaml` - Ingress routing configuration  
- `monitoring-services.yaml` - Monitoring services only
- `INGRESS_MIGRATION_SUMMARY.md` - This summary document
-
-### Legacy Files (can be removed after testing):
- `nginx-monitoring-services.yaml` - Contains old nginx configuration
-
-## Next Steps for Testing
-
-1. **Test Complete Workflow**:
-   ```bash
-   cd k8s/dev
-   ./setup-dev-cluster.sh
-   ./deploy-all-services.sh
-   ```
-
-2. **Verify All Endpoints**:
-   - Test admin interface functionality
-   - Test API endpoints
-   - Test static file loading
-   - Test chat client functionality
-
-3. **Verify URL Rewriting**:
-   - Check that `nginx_utils.py` still works correctly
-   - Test all admin panel links and forms
-   - Verify API calls from frontend
-
-4. **Performance Testing**:
-   - Compare static file loading performance
-   - Test under load if needed
-
-## Rollback Plan (if needed)
-
-If issues are discovered, you can temporarily rollback by:
-1. Reverting `deploy-all-services.sh` to use `nginx-monitoring-services.yaml`
-2. Commenting out Ingress Controller installation in `setup-dev-cluster.sh`
-3. Using direct port access instead of Ingress
-
-## Migration Complete ✅
-
-The migration from nginx reverse proxy to Kubernetes Ingress is now complete and ready for testing. All components have been implemented according to the agreed-upon architecture with production-ready optimizations.
--- a/k8s/dev/kind-dev-cluster.yaml
+++ b/k8s/dev/kind-dev-cluster.yaml
@@ -56,6 +56,11 @@ nodes:
    hostPort: 3012
    protocol: TCP

+  # Static files rsync access
+  - containerPort: 30873
+    hostPort: 3873
+    protocol: TCP
+
  # Mount points for persistent data on host
  extraMounts:
  # MinIO data persistence
--- a/k8s/dev/persistent-volumes.yaml
+++ b/k8s/dev/persistent-volumes.yaml
@@ -108,6 +108,52 @@ spec:
          values:
          - eveai-dev-cluster-control-plane

+
+---
+# Static Files Storage
+apiVersion: v1
+kind: PersistentVolume
+metadata:
+  name: static-files-pv
+  labels:
+    app: static-files
+    environment: dev
+spec:
+  capacity:
+    storage: 1Gi
+  accessModes:
+    - ReadWriteMany
+  persistentVolumeReclaimPolicy: Retain
+  storageClassName: local-storage
+  local:
+    path: /mnt/static-files
+  nodeAffinity:
+    required:
+      nodeSelectorTerms:
+      - matchExpressions:
+        - key: kubernetes.io/hostname
+          operator: In
+          values:
+          - eveai-dev-cluster-control-plane
+
+---
+# Static Files Persistent Volume Claim
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: static-files-pvc
+  namespace: eveai-dev
+spec:
+  accessModes:
+    - ReadWriteMany
+  storageClassName: local-storage
+  resources:
+    requests:
+      storage: 1Gi
+  selector:
+    matchLabels:
+      app: static-files
+      environment: dev
 ---
 # StorageClass for local storage
 apiVersion: storage.k8s.io/v1
--- a/k8s/dev/setup-dev-cluster.sh
+++ b/k8s/dev/setup-dev-cluster.sh
@@ -71,8 +71,7 @@ create_host_directories() {
        "$BASE_DIR/prometheus"
        "$BASE_DIR/grafana"
        "$BASE_DIR/certs"
-    )
-
+        "$BASE_DIR/static-files"    )
    for dir in "${directories[@]}"; do
        if [ ! -d "$dir" ]; then
            mkdir -p "$dir"
@@ -353,6 +352,7 @@ apply_manifests() {
    manifests=(
        "namespace.yaml"
        "persistent-volumes.yaml"
+        "static-files-access.yaml"
        "config-secrets.yaml"
        "network-policies.yaml"
    )
--- a/k8s/dev/static-files-access.yaml
+++ b/k8s/dev/static-files-access.yaml
@@ -0,0 +1,106 @@
+# Static Files Access Pod for EveAI Dev Environment
+# File: static-files-access.yaml
+# Provides rsync daemon access to static files PVC
+---
+# Rsync Access Deployment
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: static-files-access
+  namespace: eveai-dev
+  labels:
+    app: static-files-access
+    environment: dev
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: static-files-access
+  template:
+    metadata:
+      labels:
+        app: static-files-access
+    spec:
+      containers:
+      - name: rsync-daemon
+        image: alpine:latest
+        command: ["/bin/sh"]
+        args:
+          - -c
+          - |
+            # Install rsync
+            apk add --no-cache rsync
+            
+            # Create rsync configuration
+            cat > /etc/rsyncd.conf << 'RSYNC_EOF'
+            pid file = /var/run/rsyncd.pid
+            lock file = /var/run/rsync.lock
+            log file = /var/log/rsyncd.log
+            port = 873
+            
+            [static]
+            path = /data/static
+            comment = Static Files Volume
+            uid = nobody
+            gid = nobody
+            read only = false
+            list = yes
+            auth users = 
+            secrets file = 
+            hosts allow = *
+            RSYNC_EOF
+            
+            # Create target directory
+            mkdir -p /data/static
+            chown nobody:nobody /data/static
+            
+            # Start rsync daemon
+            echo "Starting rsync daemon..."
+            rsync --daemon --no-detach --config=/etc/rsyncd.conf
+        ports:
+        - containerPort: 873
+          name: rsync
+        volumeMounts:
+        - name: static-files
+          mountPath: /data
+        livenessProbe:
+          tcpSocket:
+            port: 873
+          initialDelaySeconds: 10
+          periodSeconds: 30
+        readinessProbe:
+          tcpSocket:
+            port: 873
+          initialDelaySeconds: 5
+          periodSeconds: 10
+        resources:
+          requests:
+            memory: "32Mi"
+            cpu: "25m"
+          limits:
+            memory: "64Mi"
+            cpu: "50m"
+      volumes:
+      - name: static-files
+        persistentVolumeClaim:
+          claimName: static-files-pvc
+
+---
+# NodePort Service for external rsync access
+apiVersion: v1
+kind: Service
+metadata:
+  name: static-files-access-service
+  namespace: eveai-dev
+  labels:
+    app: static-files-access
+spec:
+  type: NodePort
+  ports:
+  - port: 873
+    targetPort: 873
+    nodePort: 30873
+    protocol: TCP
+    name: rsync
+  selector:
+    app: static-files-access
--- a/k8s/dev/static-files-service.yaml
+++ b/k8s/dev/static-files-service.yaml
@@ -1,7 +1,7 @@
-# Static Files Service for EveAI Dev Environment
+# Static Files Service for EveAI Dev Environment (v2 - PersistentVolume based)
 # File: static-files-service.yaml
 ---
-# Static Files ConfigMap for nginx configuration
+# Static Files ConfigMap (enhanced caching)
 apiVersion: v1
 kind: ConfigMap
 metadata:
@@ -13,11 +13,31 @@ data:
        listen 80;
        server_name _;
        
+        # Gzip compression
+        gzip on;
+        gzip_vary on;
+        gzip_min_length 1024;
+        gzip_types text/css application/javascript application/json image/svg+xml;
+        
        location /static/ {
            alias /usr/share/nginx/html/static/;
-            expires 1y;
-            add_header Cache-Control "public, immutable";
-            add_header X-Content-Type-Options nosniff;
+            
+            # Aggressive caching voor versioned assets
+            location ~* \.(js|css)$ {
+                expires 1y;
+                add_header Cache-Control "public, immutable";
+                add_header X-Content-Type-Options nosniff;
+            }
+            
+            # Moderate caching voor images
+            location ~* \.(png|jpg|jpeg|gif|ico|svg)$ {
+                expires 30d;
+                add_header Cache-Control "public";
+            }
+            
+            # Default caching
+            expires 1h;
+            add_header Cache-Control "public";
        }
        
        location /health {
@@ -27,7 +47,7 @@ data:
    }

 ---
-# Static Files Deployment
+# Static Files Deployment (GEEN CUSTOM IMAGE!)
 apiVersion: apps/v1
 kind: Deployment
 metadata:
@@ -37,7 +57,7 @@ metadata:
    app: static-files
    environment: dev
 spec:
-  replicas: 1
+  replicas: 2  # Voor high availability
  selector:
    matchLabels:
      app: static-files
@@ -46,28 +66,15 @@ spec:
      labels:
        app: static-files
    spec:
-      initContainers:
-      - name: copy-static-files
-        image: registry.ask-eve-ai-local.com/josakola/nginx:latest
-        command: ['sh', '-c']
-        args:
-        - |
-          echo "Copying static files..."
-          cp -r /etc/nginx/static/* /static-data/static/ 2>/dev/null || true
-          ls -la /static-data/static/
-          echo "Static files copied successfully"
-        volumeMounts:
-        - name: static-data
-          mountPath: /static-data
      containers:
      - name: nginx
-        image: nginx:alpine
+        image: nginx:alpine  # 🎉 STANDARD IMAGE!
        ports:
        - containerPort: 80
        volumeMounts:
        - name: nginx-config
          mountPath: /etc/nginx/conf.d
-        - name: static-data
+        - name: static-files
          mountPath: /usr/share/nginx/html
        livenessProbe:
          httpGet:
@@ -92,11 +99,12 @@ spec:
      - name: nginx-config
        configMap:
          name: static-files-config
-      - name: static-data
-        emptyDir: {}
+      - name: static-files
+        persistentVolumeClaim:
+          claimName: static-files-pvc

 ---
-# Static Files Service
+# Service (ongewijzigd)
 apiVersion: v1
 kind: Service
 metadata: