pieter/eveAI
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

- Definition of extra eveai_ops service to run (db) jobs

Browse Source 
- Definition of manifests for all jobs
- Definition of manifests for all eveai services
This commit is contained in:
Josako
2025-09-03 15:20:54 +02:00
parent 898bb32318
commit 2a0c92b064
34 changed files with 1345 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
docker/Dockerfile.base
View File

@@ -24,6 +24,7 @@ RUN pip install --no-cache-dir -r requirements.txt
COPY ../common /app/common
COPY ../config /app/config
COPY ../scripts /app/scripts
COPY ../patched_packages /app/patched_packages
RUN chown -R appuser:appuser /app && chmod +x /app/scripts/start.sh

45
docker/compose_dev.yaml
View File

@@ -57,6 +57,51 @@ services:
networks:
- eveai-dev-network
eveai_ops:
image: ${REGISTRY_PREFIX:-}josakola/eveai_ops:latest
build:
context: ..
dockerfile: ./docker/eveai_ops/Dockerfile
ports:
- 3002:8080 # Dev app volgens port schema
expose:
- 8000
environment:
<<: *common-variables
COMPONENT_NAME: eveai_ops
ROLE: web
PORT: 8080
WORKERS: 1 # Dev: lagere concurrency
WORKER_CLASS: gevent
WORKER_CONN: 100
LOGLEVEL: debug # Lowercase voor gunicorn
MAX_REQUESTS: 1000
MAX_REQUESTS_JITTER: 100
volumes:
- ../eveai_ops:/app/eveai_ops
- ../common:/app/common
- ../content:/app/content
- ../config:/app/config
- ../migrations:/app/migrations
- ../scripts:/app/scripts
- ../patched_packages:/app/patched_packages
- ./eveai_logs:/app/logs
depends_on:
db:
condition: service_healthy
redis:
condition: service_healthy
minio:
condition: service_healthy
healthcheck:
test: [ "CMD", "curl", "-f", "http://localhost:8080/healthz/ready" ]
interval: 30s
timeout: 10s
retries: 3
start_period: 60s
networks:
- eveai-dev-network
eveai_app:
image: ${REGISTRY_PREFIX:-}josakola/eveai_app:latest
build:

4
docker/eveai_ops/Dockerfile Normal file
View File

@@ -0,0 +1,4 @@
FROM registry.ask-eve-ai-local.com/josakola/eveai-base:latest
# Copy the source code into the container.
COPY eveai_ops /app/eveai_ops
COPY migrations /app/migrations

2
docker/eveai_workers/Dockerfile
View File

@@ -1,4 +1,4 @@
FROM registry.ask-eve-ai-local.com/josakola/eveai-base:latest
pcupFROM registry.ask-eve-ai-local.com/josakola/eveai-base:latest
# Service-specific packages (ffmpeg only needed for this service)
USER root

32
documentation/Production Setup/cluster-install.md
View File

@@ -462,6 +462,38 @@ kubectl -n tools port-forward svc/pgadmin-pgadmin4 8080:80
# Login with PGADMIN_DEFAULT_EMAIL / PGADMIN_DEFAULT_PASSWORD (from eveai-pgadmin-admin)
```
### Phase 8: RedisInsight Tool Deployment
### Phase 9: Ops Jobs Invocation (if required)
Run the DB ops scripts manually in order. Each manifest uses generateName; use kubectl create.
```bash
kubectl create -f scaleway/manifests/base/applications/ops/jobs/00-env-check-job.yaml
kubectl wait --for=condition=complete job -n eveai-staging -l job-type=env-check --timeout=600s
kubectl create -f scaleway/manifests/base/applications/ops/jobs/02-db-bootstrap-ext-job.yaml
kubectl wait --for=condition=complete job -n eveai-staging -l job-type=db-bootstrap-ext --timeout=1800s
kubectl create -f scaleway/manifests/base/applications/ops/jobs/03-db-migrate-public-job.yaml
kubectl wait --for=condition=complete job -n eveai-staging -l job-type=db-migrate-public --timeout=1800s
kubectl create -f scaleway/manifests/base/applications/ops/jobs/04-db-migrate-tenant-job.yaml
kubectl wait --for=condition=complete job -n eveai-staging -l job-type=db-migrate-tenant --timeout=3600s
kubectl create -f scaleway/manifests/base/applications/ops/jobs/05-seed-or-init-data-job.yaml
kubectl wait --for=condition=complete job -n eveai-staging -l job-type=db-seed-or-init --timeout=1800s
kubectl create -f scaleway/manifests/base/applications/ops/jobs/06-verify-minimal-job.yaml
kubectl wait --for=condition=complete job -n eveai-staging -l job-type=db-verify-minimal --timeout=900s
```
### Phase 10: Application Services Deployment
## Verification and Testing

133
documentation/Production Setup/phase-8-application-services.md Normal file
View File

@@ -0,0 +1,133 @@
# Phase 8: Application Services (Staging)
This guide describes how to deploy EveAI application services to the Scaleway Kubernetes cluster, building on Phases 1–7 in cluster-install.md.
## Prerequisites
- Ingress-NGINX running with external IP
- cert-manager installed and Certificate evie-staging-tls is READY (via HTTP ACME first, then HTTPS-only)
- External Secrets Operator installed; Kubernetes Secret eveai-secrets exists in namespace eveai-staging
- Verification service deployed and reachable via /verify
- Optional: Monitoring stack running, Pushgateway deployed or reachable; PUSH_GATEWAY_HOST/PORT available to apps (via eveai-secrets)
## What we deploy (structure)
- Frontend (web) services
- eveai-app → exposed at /admin
- eveai-api → exposed at /api
- eveai-chat-client → exposed at /client
- Backend worker services (internal)
- eveai-workers (queue: embeddings)
- eveai-chat-workers (queue: llm_interactions)
- eveai-entitlements (queue: entitlements)
- Ops Jobs (manual DB ops)
- 00-env-check
- 02-db-bootstrap-ext
- 03-db-migrate-public
- 04-db-migrate-tenant
- 05-seed-or-init-data
- 06-verify-minimal
Manifests are under:
- scaleway/manifests/base/applications/frontend/
- scaleway/manifests/base/applications/backend/
- scaleway/manifests/base/applications/ops/jobs/
- Aggregate kustomization: scaleway/manifests/base/applications/kustomization.yaml
## Step 1: Validate secrets
```bash
kubectl get secret eveai-secrets -n eveai-staging
kubectl get secret eveai-secrets -n eveai-staging -o jsonpath='{.data}' | jq 'keys'
```
Confirm presence of DB_*, REDIS_*, OPENAI_API_KEY, MISTRAL_API_KEY, JWT_SECRET_KEY, API_ENCRYPTION_KEY, MINIO_*, PUSH_GATEWAY_HOST, PUSH_GATEWAY_PORT.
## Step 2: Deploy Ops Jobs (manual pre-deploy)
Run the DB ops scripts manually in order. Each manifest uses generateName; use kubectl create.
```bash
kubectl create -f scaleway/manifests/base/applications/ops/jobs/00-env-check-job.yaml
kubectl wait --for=condition=complete job -n eveai-staging -l job-type=env-check --timeout=600s
kubectl create -f scaleway/manifests/base/applications/ops/jobs/02-db-bootstrap-ext-job.yaml
kubectl wait --for=condition=complete job -n eveai-staging -l job-type=db-bootstrap-ext --timeout=1800s
kubectl create -f scaleway/manifests/base/applications/ops/jobs/03-db-migrate-public-job.yaml
kubectl wait --for=condition=complete job -n eveai-staging -l job-type=db-migrate-public --timeout=1800s
kubectl create -f scaleway/manifests/base/applications/ops/jobs/04-db-migrate-tenant-job.yaml
kubectl wait --for=condition=complete job -n eveai-staging -l job-type=db-migrate-tenant --timeout=3600s
kubectl create -f scaleway/manifests/base/applications/ops/jobs/05-seed-or-init-data-job.yaml
kubectl wait --for=condition=complete job -n eveai-staging -l job-type=db-seed-or-init --timeout=1800s
kubectl create -f scaleway/manifests/base/applications/ops/jobs/06-verify-minimal-job.yaml
kubectl wait --for=condition=complete job -n eveai-staging -l job-type=db-verify-minimal --timeout=900s
```
View logs:
```bash
kubectl -n eveai-staging get jobs
kubectl -n eveai-staging logs job/<created-job-name>
```
## Step 3: Deploy backend workers
```bash
kubectl apply -k scaleway/manifests/base/applications/backend/
kubectl -n eveai-staging get deploy | egrep 'eveai-(workers|chat-workers|entitlements)'
# Optional: quick logs
kubectl -n eveai-staging logs deploy/eveai-workers --tail=100 || true
kubectl -n eveai-staging logs deploy/eveai-chat-workers --tail=100 || true
kubectl -n eveai-staging logs deploy/eveai-entitlements --tail=100 || true
```
## Step 4: Deploy frontend services
```bash
kubectl apply -k scaleway/manifests/base/applications/frontend/
kubectl -n eveai-staging get deploy,svc | egrep 'eveai-(app|api|chat-client)'
```
## Step 5: Verify Ingress routes
The HTTPS ingress has paths enabled for /admin, /api, /client. Verify:
```bash
kubectl -n eveai-staging describe ingress eveai-staging-ingress
curl -k https://evie-staging.askeveai.com/verify/health
curl -k https://evie-staging.askeveai.com/admin/healthz/ready
curl -k https://evie-staging.askeveai.com/api/healthz/ready
curl -k https://evie-staging.askeveai.com/client/healthz/ready
```
## Resources and probes (staging defaults)
- Web (app, api, chat-client):
- requests: 150m CPU, 256Mi RAM; limits: 500m CPU, 512Mi RAM; replicas: 1
- readiness/liveness: GET /healthz/ready
- Workers:
- eveai-workers: req 200m/512Mi, lim 1CPU/1Gi
- eveai-chat-workers: req 500m/1Gi, lim 2CPU/3Gi
- eveai-entitlements: req 100m/256Mi, lim 500m/512Mi
## Pushgateway usage
- Ensure PUSH_GATEWAY_HOST and PUSH_GATEWAY_PORT are provided (e.g., pushgateway.monitoring.svc.cluster.local:9091), typically via eveai-secrets or a ConfigMap.
- Apps will continue to push business metrics; Prometheus scrapes the Pushgateway.
## Bunny.net WAF (TODO)
- Configure Pull Zone for evie-staging.askeveai.com
- Set Origin to the LoadBalancer IP with HTTPS and Host header evie-staging.askeveai.com
- Define rate limits primarily on /api, looser on /client; enable bot filtering
- Only switch DNS (CNAME) to Bunny after TLS issuance completed directly against LoadBalancer
## Troubleshooting
```bash
kubectl get all -n eveai-staging
kubectl get events -n eveai-staging --sort-by=.lastTimestamp
kubectl describe ingress eveai-staging-ingress -n eveai-staging
kubectl logs -n eveai-staging deploy/eveai-api --tail=200
```
## Rollback / Cleanup
```bash
# Remove frontend/backend (keeps verification and other base resources)
kubectl delete -k scaleway/manifests/base/applications/frontend/
kubectl delete -k scaleway/manifests/base/applications/backend/
# Jobs are kept for history due to ttlSecondsAfterFinished; to delete immediately:
kubectl -n eveai-staging delete jobs --all
```

86
eveai_ops/__init__.py Normal file
View File

@@ -0,0 +1,86 @@
import logging
import os
from flask import Flask
from werkzeug.middleware.proxy_fix import ProxyFix
import logging.config
from common.extensions import db, migrate
from config.logging_config import configure_logging
from config.config import get_config
def create_app(config_file=None):
app = Flask(__name__, static_url_path='/static')
# Ensure all necessary headers are handled
app.wsgi_app = ProxyFix(app.wsgi_app, x_for=1, x_proto=1, x_host=1, x_port=1)
environment = os.getenv('FLASK_ENV', 'development')
match environment:
case 'development':
app.config.from_object(get_config('dev'))
case 'staging':
app.config.from_object(get_config('staging'))
case 'production':
app.config.from_object(get_config('prod'))
case _:
app.config.from_object(get_config('dev'))
app.config['SESSION_KEY_PREFIX'] = 'eveai_ops_'
try:
os.makedirs(app.instance_path)
except OSError:
pass
# Configureer logging op basis van de omgeving (K8s of traditioneel)
try:
configure_logging()
logger = logging.getLogger(__name__)
# Test dat logging werkt
logger.debug("Logging test in eveai_ops")
except Exception as e:
print(f"Critical Error Initialising Error: {str(e)}")
import traceback
traceback.print_exc()
logger.info("eveai_ops starting up")
# Register extensions
register_extensions(app)
# Register Blueprints
register_blueprints(app)
# Register Cache Handlers
register_cache_handlers(app)
# Debugging settings
if app.config['DEBUG'] is True:
app.logger.setLevel(logging.DEBUG)
security_logger = logging.getLogger('flask_security')
security_logger.setLevel(logging.DEBUG)
sqlalchemy_logger = logging.getLogger('sqlalchemy.engine')
sqlalchemy_logger.setLevel(logging.DEBUG)
# log_request_middleware(app) # Add this when debugging nginx or another proxy
app.logger.info(f"EveAI Ops Server Started Successfully (PID: {os.getpid()})")
app.logger.info("-------------------------------------------------------------------------------------------------")
return app
def register_extensions(app):
db.init_app(app)
migrate.init_app(app, db)
def register_blueprints(app):
# minimal health blueprint
from .healthz import healthz_bp
app.register_blueprint(healthz_bp)
def register_cache_handlers(app):
pass

61
scaleway/manifests/base/applications/backend/eveai-chat-workers/deployment.yaml Normal file
View File

@@ -0,0 +1,61 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: eveai-chat-workers
namespace: eveai-staging
labels:
app: eveai-chat-workers
component: backend
role: worker
spec:
replicas: 1
selector:
matchLabels:
app: eveai-chat-workers
template:
metadata:
labels:
app: eveai-chat-workers
component: backend
role: worker
spec:
containers:
- name: eveai-chat-workers
image: josakola/eveai_chat_workers:latest
envFrom:
- secretRef:
name: eveai-secrets
env:
- name: COMPONENT_NAME
value: "eveai_chat_workers"
- name: ROLE
value: "worker"
- name: CELERY_CONCURRENCY
value: "1"
- name: CELERY_LOGLEVEL
value: "DEBUG"
- name: CELERY_MAX_TASKS_PER_CHILD
value: "1000"
- name: CELERY_PREFETCH
value: "1"
- name: CELERY_QUEUE_NAME
value: "llm_interactions"
- name: PUSH_GATEWAY_HOST
valueFrom:
secretKeyRef:
name: eveai-secrets
key: PUSH_GATEWAY_HOST
optional: true
- name: PUSH_GATEWAY_PORT
valueFrom:
secretKeyRef:
name: eveai-secrets
key: PUSH_GATEWAY_PORT
optional: true
resources:
requests:
cpu: "500m"
memory: "1Gi"
limits:
cpu: "2"
memory: "3Gi"

61
scaleway/manifests/base/applications/backend/eveai-entitlements/deployment.yaml Normal file
View File

@@ -0,0 +1,61 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: eveai-entitlements
namespace: eveai-staging
labels:
app: eveai-entitlements
component: backend
role: worker
spec:
replicas: 1
selector:
matchLabels:
app: eveai-entitlements
template:
metadata:
labels:
app: eveai-entitlements
component: backend
role: worker
spec:
containers:
- name: eveai-entitlements
image: josakola/eveai_entitlements:latest
envFrom:
- secretRef:
name: eveai-secrets
env:
- name: COMPONENT_NAME
value: "eveai_entitlements"
- name: ROLE
value: "worker"
- name: CELERY_CONCURRENCY
value: "1"
- name: CELERY_LOGLEVEL
value: "DEBUG"
- name: CELERY_MAX_TASKS_PER_CHILD
value: "1000"
- name: CELERY_PREFETCH
value: "1"
- name: CELERY_QUEUE_NAME
value: "entitlements"
- name: PUSH_GATEWAY_HOST
valueFrom:
secretKeyRef:
name: eveai-secrets
key: PUSH_GATEWAY_HOST
optional: true
- name: PUSH_GATEWAY_PORT
valueFrom:
secretKeyRef:
name: eveai-secrets
key: PUSH_GATEWAY_PORT
optional: true
resources:
requests:
cpu: "100m"
memory: "256Mi"
limits:
cpu: "500m"
memory: "512Mi"

61
scaleway/manifests/base/applications/backend/eveai-workers/deployment.yaml Normal file
View File

@@ -0,0 +1,61 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: eveai-workers
namespace: eveai-staging
labels:
app: eveai-workers
component: backend
role: worker
spec:
replicas: 1
selector:
matchLabels:
app: eveai-workers
template:
metadata:
labels:
app: eveai-workers
component: backend
role: worker
spec:
containers:
- name: eveai-workers
image: josakola/eveai_workers:latest
envFrom:
- secretRef:
name: eveai-secrets
env:
- name: COMPONENT_NAME
value: "eveai_workers"
- name: ROLE
value: "worker"
- name: CELERY_CONCURRENCY
value: "1"
- name: CELERY_LOGLEVEL
value: "DEBUG"
- name: CELERY_MAX_TASKS_PER_CHILD
value: "1000"
- name: CELERY_PREFETCH
value: "1"
- name: CELERY_QUEUE_NAME
value: "embeddings"
- name: PUSH_GATEWAY_HOST
valueFrom:
secretKeyRef:
name: eveai-secrets
key: PUSH_GATEWAY_HOST
optional: true
- name: PUSH_GATEWAY_PORT
valueFrom:
secretKeyRef:
name: eveai-secrets
key: PUSH_GATEWAY_PORT
optional: true
resources:
requests:
cpu: "200m"
memory: "512Mi"
limits:
cpu: "1"
memory: "1Gi"

7
scaleway/manifests/base/applications/backend/kustomization.yaml Normal file
View File

@@ -0,0 +1,7 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: eveai-staging
resources:
- eveai-workers/deployment.yaml
- eveai-chat-workers/deployment.yaml
- eveai-entitlements/deployment.yaml

84
scaleway/manifests/base/applications/frontend/eveai-api/deployment.yaml Normal file
View File

@@ -0,0 +1,84 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: eveai-api
namespace: eveai-staging
labels:
app: eveai-api
component: frontend
role: web
spec:
replicas: 1
selector:
matchLabels:
app: eveai-api
template:
metadata:
labels:
app: eveai-api
component: frontend
role: web
spec:
containers:
- name: eveai-api
image: josakola/eveai_api:latest
ports:
- containerPort: 8080
name: http
envFrom:
- secretRef:
name: eveai-secrets
env:
- name: COMPONENT_NAME
value: "eveai_api"
- name: ROLE
value: "web"
- name: PORT
value: "8080"
- name: PUSH_GATEWAY_HOST
valueFrom:
secretKeyRef:
name: eveai-secrets
key: PUSH_GATEWAY_HOST
optional: true
- name: PUSH_GATEWAY_PORT
valueFrom:
secretKeyRef:
name: eveai-secrets
key: PUSH_GATEWAY_PORT
optional: true
resources:
requests:
cpu: "150m"
memory: "256Mi"
limits:
cpu: "500m"
memory: "512Mi"
readinessProbe:
httpGet:
path: /healthz/ready
port: http
initialDelaySeconds: 10
periodSeconds: 10
livenessProbe:
httpGet:
path: /healthz/ready
port: http
initialDelaySeconds: 20
periodSeconds: 20
---
apiVersion: v1
kind: Service
metadata:
name: eveai-api-service
namespace: eveai-staging
labels:
app: eveai-api
spec:
selector:
app: eveai-api
ports:
- name: http
port: 80
targetPort: 8080
type: ClusterIP

84
scaleway/manifests/base/applications/frontend/eveai-app/deployment.yaml Normal file
View File

@@ -0,0 +1,84 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: eveai-app
namespace: eveai-staging
labels:
app: eveai-app
component: frontend
role: web
spec:
replicas: 1
selector:
matchLabels:
app: eveai-app
template:
metadata:
labels:
app: eveai-app
component: frontend
role: web
spec:
containers:
- name: eveai-app
image: josakola/eveai_app:latest
ports:
- containerPort: 8080
name: http
envFrom:
- secretRef:
name: eveai-secrets
env:
- name: COMPONENT_NAME
value: "eveai_app"
- name: ROLE
value: "web"
- name: PORT
value: "8080"
- name: PUSH_GATEWAY_HOST
valueFrom:
secretKeyRef:
name: eveai-secrets
key: PUSH_GATEWAY_HOST
optional: true
- name: PUSH_GATEWAY_PORT
valueFrom:
secretKeyRef:
name: eveai-secrets
key: PUSH_GATEWAY_PORT
optional: true
resources:
requests:
cpu: "150m"
memory: "256Mi"
limits:
cpu: "500m"
memory: "512Mi"
readinessProbe:
httpGet:
path: /healthz/ready
port: http
initialDelaySeconds: 10
periodSeconds: 10
livenessProbe:
httpGet:
path: /healthz/ready
port: http
initialDelaySeconds: 20
periodSeconds: 20
---
apiVersion: v1
kind: Service
metadata:
name: eveai-app-service
namespace: eveai-staging
labels:
app: eveai-app
spec:
selector:
app: eveai-app
ports:
- name: http
port: 80
targetPort: 8080
type: ClusterIP

84
scaleway/manifests/base/applications/frontend/eveai-chat-client/deployment.yaml Normal file
View File

@@ -0,0 +1,84 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: eveai-chat-client
namespace: eveai-staging
labels:
app: eveai-chat-client
component: frontend
role: web
spec:
replicas: 1
selector:
matchLabels:
app: eveai-chat-client
template:
metadata:
labels:
app: eveai-chat-client
component: frontend
role: web
spec:
containers:
- name: eveai-chat-client
image: josakola/eveai_chat_client:latest
ports:
- containerPort: 8080
name: http
envFrom:
- secretRef:
name: eveai-secrets
env:
- name: COMPONENT_NAME
value: "eveai_chat_client"
- name: ROLE
value: "web"
- name: PORT
value: "8080"
- name: PUSH_GATEWAY_HOST
valueFrom:
secretKeyRef:
name: eveai-secrets
key: PUSH_GATEWAY_HOST
optional: true
- name: PUSH_GATEWAY_PORT
valueFrom:
secretKeyRef:
name: eveai-secrets
key: PUSH_GATEWAY_PORT
optional: true
resources:
requests:
cpu: "150m"
memory: "256Mi"
limits:
cpu: "500m"
memory: "512Mi"
readinessProbe:
httpGet:
path: /healthz/ready
port: http
initialDelaySeconds: 10
periodSeconds: 10
livenessProbe:
httpGet:
path: /healthz/ready
port: http
initialDelaySeconds: 20
periodSeconds: 20
---
apiVersion: v1
kind: Service
metadata:
name: eveai-chat-client-service
namespace: eveai-staging
labels:
app: eveai-chat-client
spec:
selector:
app: eveai-chat-client
ports:
- name: http
port: 80
targetPort: 8080
type: ClusterIP

7
scaleway/manifests/base/applications/frontend/kustomization.yaml Normal file
View File

@@ -0,0 +1,7 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: eveai-staging
resources:
- eveai-app/deployment.yaml
- eveai-api/deployment.yaml
- eveai-chat-client/deployment.yaml

8
scaleway/manifests/base/applications/kustomization.yaml Normal file
View File

@@ -0,0 +1,8 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: eveai-staging
resources:
- verification/
- frontend/
- backend/
- ops/jobs/

34
scaleway/manifests/base/applications/ops/jobs/00-env-check-job.yaml Normal file
View File

@@ -0,0 +1,34 @@
apiVersion: batch/v1
kind: Job
metadata:
generateName: dbops-env-check-
namespace: eveai-staging
labels:
app: eveai
component: ops
job-type: env-check
spec:
ttlSecondsAfterFinished: 1800
backoffLimit: 2
template:
metadata:
labels:
app: eveai
component: ops
job-type: env-check
spec:
restartPolicy: Never
containers:
- name: dbops
image: josakola/eveai_ops:latest
envFrom:
- secretRef:
name: eveai-secrets
command: ["/bin/bash","-lc","/app/scripts/dbops/00-env-check.sh"]
resources:
requests:
cpu: "100m"
memory: "128Mi"
limits:
cpu: "500m"
memory: "512Mi"

35
scaleway/manifests/base/applications/ops/jobs/02-db-bootstrap-ext-job.yaml Normal file
View File

@@ -0,0 +1,35 @@
apiVersion: batch/v1
kind: Job
metadata:
generateName: dbops-bootstrap-ext-
namespace: eveai-staging
labels:
app: eveai
component: ops
job-type: db-bootstrap-ext
spec:
ttlSecondsAfterFinished: 1800
backoffLimit: 2
activeDeadlineSeconds: 1800
template:
metadata:
labels:
app: eveai
component: ops
job-type: db-bootstrap-ext
spec:
restartPolicy: Never
containers:
- name: dbops
image: josakola/eveai_ops:latest
envFrom:
- secretRef:
name: eveai-secrets
command: ["/bin/bash","-lc","/app/scripts/dbops/02-db-bootstrap-ext.sh"]
resources:
requests:
cpu: "100m"
memory: "128Mi"
limits:
cpu: "500m"
memory: "512Mi"

35
scaleway/manifests/base/applications/ops/jobs/03-db-migrate-public-job.yaml Normal file
View File

@@ -0,0 +1,35 @@
apiVersion: batch/v1
kind: Job
metadata:
generateName: dbops-migrate-public-
namespace: eveai-staging
labels:
app: eveai
component: ops
job-type: db-migrate-public
spec:
ttlSecondsAfterFinished: 1800
backoffLimit: 2
activeDeadlineSeconds: 1800
template:
metadata:
labels:
app: eveai
component: ops
job-type: db-migrate-public
spec:
restartPolicy: Never
containers:
- name: dbops
image: josakola/eveai_ops:latest
envFrom:
- secretRef:
name: eveai-secrets
command: ["/bin/bash","-lc","/app/scripts/dbops/03-db-migrate-public.sh"]
resources:
requests:
cpu: "100m"
memory: "128Mi"
limits:
cpu: "500m"
memory: "512Mi"

35
scaleway/manifests/base/applications/ops/jobs/04-db-migrate-tenant-job.yaml Normal file
View File

@@ -0,0 +1,35 @@
apiVersion: batch/v1
kind: Job
metadata:
generateName: dbops-migrate-tenant-
namespace: eveai-staging
labels:
app: eveai
component: ops
job-type: db-migrate-tenant
spec:
ttlSecondsAfterFinished: 1800
backoffLimit: 2
activeDeadlineSeconds: 3600
template:
metadata:
labels:
app: eveai
component: ops
job-type: db-migrate-tenant
spec:
restartPolicy: Never
containers:
- name: dbops
image: josakola/eveai_ops:latest
envFrom:
- secretRef:
name: eveai-secrets
command: ["/bin/bash","-lc","/app/scripts/dbops/04-db-migrate-tenant.sh"]
resources:
requests:
cpu: "200m"
memory: "256Mi"
limits:
cpu: "1"
memory: "1Gi"

35
scaleway/manifests/base/applications/ops/jobs/05-seed-or-init-data-job.yaml Normal file
View File

@@ -0,0 +1,35 @@
apiVersion: batch/v1
kind: Job
metadata:
generateName: dbops-seed-or-init-
namespace: eveai-staging
labels:
app: eveai
component: ops
job-type: db-seed-or-init
spec:
ttlSecondsAfterFinished: 1800
backoffLimit: 2
activeDeadlineSeconds: 1800
template:
metadata:
labels:
app: eveai
component: ops
job-type: db-seed-or-init
spec:
restartPolicy: Never
containers:
- name: dbops
image: josakola/eveai_ops:latest
envFrom:
- secretRef:
name: eveai-secrets
command: ["/bin/bash","-lc","/app/scripts/dbops/05-seed-or-init-data.sh"]
resources:
requests:
cpu: "100m"
memory: "128Mi"
limits:
cpu: "500m"
memory: "512Mi"

35
scaleway/manifests/base/applications/ops/jobs/06-verify-minimal-job.yaml Normal file
View File

@@ -0,0 +1,35 @@
apiVersion: batch/v1
kind: Job
metadata:
generateName: dbops-verify-minimal-
namespace: eveai-staging
labels:
app: eveai
component: ops
job-type: db-verify-minimal
spec:
ttlSecondsAfterFinished: 1800
backoffLimit: 2
activeDeadlineSeconds: 900
template:
metadata:
labels:
app: eveai
component: ops
job-type: db-verify-minimal
spec:
restartPolicy: Never
containers:
- name: dbops
image: josakola/eveai_ops:latest
envFrom:
- secretRef:
name: eveai-secrets
command: ["/bin/bash","-lc","/app/scripts/dbops/06-verify-minimal.sh"]
resources:
requests:
cpu: "100m"
memory: "128Mi"
limits:
cpu: "500m"
memory: "512Mi"

10
scaleway/manifests/base/applications/ops/jobs/kustomization.yaml Normal file
View File

@@ -0,0 +1,10 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: eveai-staging
resources:
- 00-env-check-job.yaml
- 02-db-bootstrap-ext-job.yaml
- 03-db-migrate-public-job.yaml
- 04-db-migrate-tenant-job.yaml
- 05-seed-or-init-data-job.yaml
- 06-verify-minimal-job.yaml

47
scaleway/manifests/base/networking/ingress-https.yaml
View File

@@ -33,33 +33,30 @@ spec:
port:
number: 80
# Future services (ready for deployment)
# Admin service
# - path: /admin
# pathType: Prefix
# backend:
# service:
# name: eveai-app-service
# port:
# number: 80
# Application services
- path: /admin
pathType: Prefix
backend:
service:
name: eveai-app-service
port:
number: 80
# API service
# - path: /api
# pathType: Prefix
# backend:
# service:
# name: eveai-api-service
# port:
# number: 80
- path: /api
pathType: Prefix
backend:
service:
name: eveai-api-service
port:
number: 80
# Client/Frontend service
# - path: /client
# pathType: Prefix
# backend:
# service:
# name: eveai-chat-client-service
# port:
# number: 80
- path: /client
pathType: Prefix
backend:
service:
name: eveai-chat-client-service
port:
number: 80
# Monitoring (when deployed)
# - path: /monitoring

40
scripts/dbops/00-env-check.sh Executable file
View File

@@ -0,0 +1,40 @@
#!/usr/bin/env bash
set -Eeuo pipefail
log() { echo "[$(date -u +'%Y-%m-%dT%H:%M:%SZ')] $*"; }
fail() { echo "ERROR: $*" >&2; exit 1; }
# Required env vars
REQUIRED_VARS=(DB_HOST DB_PORT DB_NAME DB_USER DB_PASS)
MISSING=()
for v in "${REQUIRED_VARS[@]}"; do
if [[ -z "${!v-}" ]]; then MISSING+=("$v"); fi
done
# Defaults
export PROJECT_DIR="${PROJECT_DIR:-/app}"
export FLASK_APP="${FLASK_APP:-${PROJECT_DIR}/scripts/run.py}"
export COMPONENT_NAME="${COMPONENT_NAME:-eveai_app}"
export PYTHONPATH="${PYTHONPATH:-${PROJECT_DIR}:${PYTHONPATH-}}"
if ((${#MISSING[@]})); then
fail "Missing required env vars: ${MISSING[*]}"
fi
# Tools check
need() { command -v "$1" >/dev/null 2>&1 || fail "Required tool not found in PATH: $1"; }
need psql
need pg_isready
need flask
log "Environment OK"
log "DB_HOST=$DB_HOST DB_PORT=$DB_PORT DB_NAME=$DB_NAME"
log "PROJECT_DIR=$PROJECT_DIR"
log "FLASK_APP=$FLASK_APP COMPONENT_NAME=$COMPONENT_NAME"
# Versions (do not leak secrets)
psql --version || true
flask --version || true
log "00-env-check completed successfully."

28
scripts/dbops/01-wait-for-db.sh Executable file
View File

@@ -0,0 +1,28 @@
#!/usr/bin/env bash
set -Eeuo pipefail
log() { echo "[$(date -u +'%Y-%m-%dT%H:%M:%SZ')] $*"; }
fail() { echo "ERROR: $*" >&2; exit 1; }
: "${DB_HOST:?DB_HOST required}"
: "${DB_PORT:?DB_PORT required}"
: "${DB_NAME:?DB_NAME required}"
INTERVAL=${WAIT_FOR_DB_INTERVAL:-2}
TIMEOUT=${WAIT_FOR_DB_TIMEOUT:-30}
need() { command -v "$1" >/dev/null 2>&1 || fail "Required tool not found in PATH: $1"; }
need pg_isready
log "Waiting for database to be ready (timeout=${TIMEOUT}s, interval=${INTERVAL}s)"
end=$((SECONDS + TIMEOUT))
while (( SECONDS < end )); do
if pg_isready -h "$DB_HOST" -p "$DB_PORT" -d "$DB_NAME" >/dev/null 2>&1; then
log "Postgres is ready."
exit 0
fi
sleep "$INTERVAL"
done
fail "Database not ready within ${TIMEOUT}s."

24
scripts/dbops/02-db-bootstrap-ext.sh Executable file
View File

@@ -0,0 +1,24 @@
#!/usr/bin/env bash
set -Eeuo pipefail
log() { echo "[$(date -u +'%Y-%m-%dT%H:%M:%SZ')] $*"; }
fail() { echo "ERROR: $*" >&2; exit 1; }
: "${DB_HOST:?DB_HOST required}"
: "${DB_PORT:?DB_PORT required}"
: "${DB_NAME:?DB_NAME required}"
: "${DB_USER:?DB_USER required}"
: "${DB_PASS:?DB_PASS required}"
export PGPASSWORD="$DB_PASS"
log "Ensuring pgvector extension exists in database '$DB_NAME'"
# Verify DB exists and is reachable
if ! psql -U "$DB_USER" -h "$DB_HOST" -p "$DB_PORT" -d "$DB_NAME" -tAc "SELECT 1" >/dev/null 2>&1; then
fail "Unable to connect to database '$DB_NAME'. Ensure it exists and credentials are valid."
fi
psql -U "$DB_USER" -h "$DB_HOST" -p "$DB_PORT" -d "$DB_NAME" -v ON_ERROR_STOP=1 -c "CREATE EXTENSION IF NOT EXISTS vector;"
log "pgvector extension ensured (CREATE EXTENSION IF NOT EXISTS vector)."

20
scripts/dbops/03-db-migrate-public.sh Executable file
View File

@@ -0,0 +1,20 @@
#!/usr/bin/env bash
set -Eeuo pipefail
log() { echo "[$(date -u +'%Y-%m-%dT%H:%M:%SZ')] $*"; }
fail() { echo "ERROR: $*" >&2; exit 1; }
# Env requirements
REQUIRED_VARS=(DB_HOST DB_PORT DB_NAME DB_USER DB_PASS)
for v in "${REQUIRED_VARS[@]}"; do : "${!v:?$v required}"; done
export PROJECT_DIR="${PROJECT_DIR:-/app}"
export FLASK_APP="${FLASK_APP:-${PROJECT_DIR}/scripts/run.py}"
export COMPONENT_NAME="${COMPONENT_NAME:-eveai_app}"
export PYTHONPATH="${PYTHONPATH:-${PROJECT_DIR}:${PYTHONPATH-}}"
export PGPASSWORD="$DB_PASS"
log "Applying Alembic migrations to the public schema..."
flask db upgrade -d "${PROJECT_DIR}/migrations/public"
log "Finished applying migrations to the public schema."

20
scripts/dbops/04-db-migrate-tenant.sh Executable file
View File

@@ -0,0 +1,20 @@
#!/usr/bin/env bash
set -Eeuo pipefail
log() { echo "[$(date -u +'%Y-%m-%dT%H:%M:%SZ')] $*"; }
fail() { echo "ERROR: $*" >&2; exit 1; }
# Env requirements
REQUIRED_VARS=(DB_HOST DB_PORT DB_NAME DB_USER DB_PASS)
for v in "${REQUIRED_VARS[@]}"; do : "${!v:?$v required}"; done
export PROJECT_DIR="${PROJECT_DIR:-/app}"
export FLASK_APP="${FLASK_APP:-${PROJECT_DIR}/scripts/run.py}"
export COMPONENT_NAME="${COMPONENT_NAME:-eveai_app}"
export PYTHONPATH="${PYTHONPATH:-${PROJECT_DIR}:${PYTHONPATH-}}"
export PGPASSWORD="$DB_PASS"
log "Applying Alembic migrations to the tenant schemas (single run handling all tenants)..."
flask db upgrade -d "${PROJECT_DIR}/migrations/tenant"
log "Finished applying migrations to the tenant schemas."

18
scripts/dbops/05-seed-or-init-data.sh Executable file
View File

@@ -0,0 +1,18 @@
#!/usr/bin/env bash
set -Eeuo pipefail
log() { echo "[$(date -u +'%Y-%m-%dT%H:%M:%SZ')] $*"; }
fail() { echo "ERROR: $*" >&2; exit 1; }
export PROJECT_DIR="${PROJECT_DIR:-/app}"
SCRIPT_PATH="${PROJECT_DIR}/scripts/initialize_data.py"
[[ -f "$SCRIPT_PATH" ]] || fail "Seed/init script not found: $SCRIPT_PATH"
export FLASK_APP="${FLASK_APP:-${PROJECT_DIR}/scripts/run.py}"
export COMPONENT_NAME="${COMPONENT_NAME:-eveai_app}"
export PYTHONPATH="${PYTHONPATH:-${PROJECT_DIR}:${PYTHONPATH-}}"
log "Running initialize_data.py (idempotent one-off per environment)..."
python "$SCRIPT_PATH"
log "initialize_data.py completed."

26
scripts/dbops/06-verify-minimal.sh Executable file
View File

@@ -0,0 +1,26 @@
#!/usr/bin/env bash
set -Eeuo pipefail
log() { echo "[$(date -u +'%Y-%m-%dT%H:%M:%SZ')] $*"; }
fail() { echo "ERROR: $*" >&2; exit 1; }
: "${DB_HOST:?DB_HOST required}"
: "${DB_PORT:?DB_PORT required}"
: "${DB_NAME:?DB_NAME required}"
: "${DB_USER:?DB_USER required}"
: "${DB_PASS:?DB_PASS required}"
export PGPASSWORD="$DB_PASS"
log "Verifying DB connectivity and pgvector extension in '$DB_NAME'"
# Connectivity check
psql -U "$DB_USER" -h "$DB_HOST" -p "$DB_PORT" -d "$DB_NAME" -tAc "SELECT 1" >/dev/null 2>&1 || fail "Cannot connect to database '$DB_NAME'"
# pgvector check
HAS_VECTOR=$(psql -U "$DB_USER" -h "$DB_HOST" -p "$DB_PORT" -d "$DB_NAME" -tAc "SELECT 1 FROM pg_extension WHERE extname='vector'" | tr -d '[:space:]')
if [[ "$HAS_VECTOR" != "1" ]]; then
fail "pgvector extension not found in '$DB_NAME'"
fi
log "Verification OK: DB reachable and pgvector extension present."

25
scripts/dbops/90-initial-cluster-db-create.sh Executable file
View File

@@ -0,0 +1,25 @@
#!/usr/bin/env bash
set -Eeuo pipefail
log() { echo "[$(date -u +'%Y-%m-%dT%H:%M:%SZ')] $*"; }
fail() { echo "ERROR: $*" >&2; exit 1; }
: "${DB_HOST:?DB_HOST required}"
: "${DB_PORT:?DB_PORT required}"
: "${DB_NAME:?DB_NAME required}"
: "${DB_USER:?DB_USER required}"
: "${DB_PASS:?DB_PASS required}"
export PGPASSWORD="$DB_PASS"
log "Checking if database '$DB_NAME' exists..."
EXISTS=$(psql -U "$DB_USER" -h "$DB_HOST" -p "$DB_PORT" -d postgres -tAc "SELECT 1 FROM pg_database WHERE datname='${DB_NAME}'" | tr -d '[:space:]') || true
if [[ "$EXISTS" == "1" ]]; then
log "Database '$DB_NAME' already exists. Nothing to do."
exit 0
fi
log "Creating database '$DB_NAME'..."
psql -U "$DB_USER" -h "$DB_HOST" -p "$DB_PORT" -d postgres -v ON_ERROR_STOP=1 -c "CREATE DATABASE \"$DB_NAME\";"
log "Database '$DB_NAME' created successfully."

4
scripts/dbops/99-cleanup.sh Executable file
View File

@@ -0,0 +1,4 @@
#!/usr/bin/env bash
set -Eeuo pipefail
echo "[$(date -u +'%Y-%m-%dT%H:%M:%SZ')] Cleanup complete (no-op)."

140
scripts/dbops/README.md Executable file
View File

@@ -0,0 +1,140 @@
# DB Operations (Manual)
Scripts to run database initialization and migrations manually during your release process. Default wait timeout is 30s with 2s interval.
Required env (typical):
- DB_HOST, DB_PORT, DB_NAME, DB_USER, DB_PASS
- PROJECT_DIR (defaults /app)
- FLASK_APP (defaults to ${PROJECT_DIR}/scripts/run.py)
- COMPONENT_NAME (defaults to eveai_app)
- PYTHONPATH (defaults to ${PROJECT_DIR})
Order (standard: DB already exists):
1. 00-env-check.sh
2. 01-wait-for-db.sh (WAIT_FOR_DB_TIMEOUT=30 WAIT_FOR_DB_INTERVAL=2)
3. 02-db-bootstrap-ext.sh (creates pgvector extension if missing)
4. 03-db-migrate-public.sh
5. 04-db-migrate-tenant.sh (single run for all tenants)
6. 06-verify-minimal.sh
Optional, one-off for a new environment where DB does not exist yet:
- 90-initial-cluster-db-create.sh (run before step 3)
Optional seed/init data (one-off per env, idempotent):
- 05-seed-or-init-data.sh
Notes:
- Scripts are idempotent where applicable.
- Do not echo secrets. Password is read from DB_PASS via PGPASSWORD env.
- Ensure you run these inside the app image/container or any environment with psql, pg_isready, flask and the project code present.
## Podman quickstart (manual testing)
Below are two simple ways to run these scripts in a Podman-based setup. Choose the approach that best matches your workflow.
Prerequisites:
- A reachable PostgreSQL 16 instance and credentials (DB_HOST, DB_PORT, DB_NAME, DB_USER, DB_PASS)
- Your application code available inside the container at /app (default PROJECT_DIR)
- Tools available in the container: psql, pg_isready, flask, python
- docker/Dockerfile.base already includes psql; use it to build a tools image if needed
Environment variables (typical):
- DB_HOST, DB_PORT, DB_NAME, DB_USER, DB_PASS
- PROJECT_DIR=/app (default)
- FLASK_APP=/app/scripts/run.py (default)
- COMPONENT_NAME=eveai_app (default)
- PYTHONPATH="/app" (default)
Order to run (standard):
1) scripts/dbops/00-env-check.sh
2) scripts/dbops/01-wait-for-db.sh (WAIT_FOR_DB_TIMEOUT=30 WAIT_FOR_DB_INTERVAL=2)
3) scripts/dbops/02-db-bootstrap-ext.sh
4) scripts/dbops/03-db-migrate-public.sh
5) scripts/dbops/04-db-migrate-tenant.sh
6) scripts/dbops/06-verify-minimal.sh
Optional (one-time for a new environment where the DB does not yet exist):
- scripts/dbops/90-initial-cluster-db-create.sh (run before step 3)
Optional seed/init (one-time per environment):
- scripts/dbops/05-seed-or-init-data.sh
---
Approach A — Exec into an existing app container
1. Start your stack (e.g., via Podman Compose) so the app container and DB are running.
2. Find the app container name:
podman ps
3. Exec into the container shell:
podman exec -it <app-container-name> bash
4. Export DB variables inside the container (adjust values):
export DB_HOST=127.0.0.1
export DB_PORT=5432
export DB_NAME=eveai
export DB_USER=postgres
export DB_PASS=yourpassword
# Optional (defaults exist)
export PROJECT_DIR=/app
export FLASK_APP=/app/scripts/run.py
export COMPONENT_NAME=eveai_app
export PYTHONPATH="/app"
5. Run the scripts in order, for example:
scripts/dbops/00-env-check.sh && \
scripts/dbops/01-wait-for-db.sh && \
scripts/dbops/02-db-bootstrap-ext.sh && \
scripts/dbops/03-db-migrate-public.sh && \
scripts/dbops/04-db-migrate-tenant.sh && \
scripts/dbops/06-verify-minimal.sh
6. Optionally seed/init once per environment:
scripts/dbops/05-seed-or-init-data.sh
Notes:
- If your DB runs in a separate container, DB_HOST should be that container's network hostname (often the service name from compose). If using host networking, 127.0.0.1 may work.
---
Approach B — Use a one-off tools container based on Dockerfile.base
This runs the scripts without depending on a running app container. It mounts the repo at /app and uses the base image with psql installed.
1. Build the tools image (from repo root):
podman build -f docker/Dockerfile.base -t eveai/tools:local .
2. Run a one-off container with your env vars and the repo mounted:
podman run --rm -it \
--name eveai-dbops \
--network host \
-e DB_HOST=127.0.0.1 \
-e DB_PORT=5432 \
-e DB_NAME=eveai \
-e DB_USER=postgres \
-e DB_PASS=yourpassword \
-e PROJECT_DIR=/app \
-e FLASK_APP=/app/scripts/run.py \
-e COMPONENT_NAME=eveai_app \
-e PYTHONPATH="/app" \
-e WAIT_FOR_DB_TIMEOUT=30 \
-e WAIT_FOR_DB_INTERVAL=2 \
-v "$(pwd)":/app \
-w /app \
eveai/tools:local \
bash -lc 'scripts/dbops/00-env-check.sh && scripts/dbops/01-wait-for-db.sh && scripts/dbops/02-db-bootstrap-ext.sh && scripts/dbops/03-db-migrate-public.sh && scripts/dbops/04-db-migrate-tenant.sh && scripts/dbops/06-verify-minimal.sh'
3. Optional seed/init once per environment:
podman run --rm -it --network host \
-e DB_HOST -e DB_PORT -e DB_NAME -e DB_USER -e DB_PASS \
-e PROJECT_DIR -e FLASK_APP -e COMPONENT_NAME -e PYTHONPATH \
-v "$(pwd)":/app -w /app \
eveai/tools:local \
bash -lc 'scripts/dbops/05-seed-or-init-data.sh'
Notes:
- --network host simplifies connectivity on Linux. If you cannot use it, use a user-defined Podman network and set DB_HOST to the DB container hostname.
- Do not echo secrets; scripts read DB_PASS via the PGPASSWORD env variable automatically.
- If flask is missing at runtime, ensure your tools image has Python deps installed or run inside your actual app image/container that already has them.
Troubleshooting
- Connection refused: verify DB_HOST/PORT and network mode; try --network host.
- Authentication failed: check DB_USER/DB_PASS and that the user has rights on DB_NAME.
- pgvector missing even after step 3: ensure your DB user has rights to CREATE EXTENSION in DB_NAME.
- Flask command not found: run within the app image or install project dependencies in the tools image before running.