- Ensure prefix is passed for all services
- Add eveai-tem secret (Scaleway Transactional Email) to allow sending emails - Adapted security URLs - Certification problem in regions solved - Redis insight added to tools in k8s - Introduced new way of connection pooling for Redis - TRA-79 - intrernal server error bij registreren catalog
This commit is contained in:
Josako
@@ -18,13 +18,29 @@ kubectl cluster-info
|
||||
- Kubernetes cluster running
|
||||
- Managed services configured (PostgreSQL, Redis, MinIO)
|
||||
- Secrets stored in Scaleway Secret Manager:
|
||||
- `eveai-app-keys`, `eveai-mistral`, `eveai-object-storage`
|
||||
- `eveai-app-keys`, `eveai-mistral`, `eveai-object-storage`, `eveai-tem`
|
||||
- `eveai-openai`, `eveai-postgresql`, `eveai-redis`, `eveai-redis-certificate`
|
||||
- Flexible IP address (LoadBalancer)
|
||||
- Eerst een loadbalancer aanmaken met publiek IP
|
||||
- Daarna de loadbalancer verwijderen maar flexible IPs behouden
|
||||
- Dit externe IP is het IP adres dat moet worden verwerkt in ingress-values.yaml!
|
||||
|
||||
## CDN Setup (Bunny.net - Optional)
|
||||
|
||||
### Configure Pull Zone
|
||||
- Create Pull zone: evie-staging
|
||||
- Origin: https://[LoadBalancer-IP] (note HTTPS!) -> pas later in het proces gekend
|
||||
- Host header: evie-staging.askeveai.com
|
||||
- Force SSL: Enabled
|
||||
- In the pull zone's Caching - General settings, ensure to disable 'Strip Response Cookies'
|
||||
- Define edge rules for
|
||||
- Redirecting the root
|
||||
- Redirecting security urls
|
||||
|
||||
### Update DNS (eurodns) for CDN
|
||||
- Change A-record to CNAME pointing to CDN endpoint
|
||||
- Or update A-record to CDN IP
|
||||
|
||||
## New Modular Deployment Process
|
||||
|
||||
### Phase 1: Infrastructure Foundation
|
||||
@@ -230,7 +246,7 @@ kubectl create secret generic scaleway-credentials \
|
||||
|
||||
#### Stap 3: Verifieer SecretStore configuratie
|
||||
|
||||
Verifieer bestand: `scaleway/manifests/base/secrets/scaleway-secret-store.yaml`. Daar moet de juiste project ID worden ingevoerd.
|
||||
Verifieer bestand: `scaleway/manifests/base/secrets/clustersecretstore-scaleway.yaml`. Daar moet de juiste project ID worden ingevoerd.
|
||||
|
||||
#### Stap 4: Verifieer ExternalSecret resource
|
||||
|
||||
@@ -245,7 +261,7 @@ Verifieer bestand: `scaleway/manifests/base/secrets/eveai-external-secrets.yaml`
|
||||
|
||||
```bash
|
||||
# Deploy SecretStore
|
||||
kubectl apply -f scaleway/manifests/base/secrets/scaleway-secret-store.yaml
|
||||
kubectl apply -f scaleway/manifests/base/secrets/clustersecretstore-scaleway.yaml
|
||||
|
||||
# Deploy ExternalSecret
|
||||
kubectl apply -f scaleway/manifests/base/secrets/eveai-external-secrets.yaml
|
||||
@@ -281,7 +297,13 @@ metadata:
|
||||
name: eveai-app
|
||||
namespace: eveai-staging
|
||||
spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
app: eveai-app
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: eveai-app
|
||||
spec:
|
||||
containers:
|
||||
- name: eveai-app
|
||||
@@ -296,15 +318,17 @@ spec:
|
||||
Voor SSL Redis connecties met het certificaat:
|
||||
|
||||
```python
|
||||
# In je config.py
|
||||
# Voorbeeld in je config.py
|
||||
import tempfile
|
||||
import ssl
|
||||
import redis
|
||||
from os import environ
|
||||
|
||||
class StagingConfig(Config):
|
||||
REDIS_CERT_DATA = environ.get('REDIS_CERT')
|
||||
|
||||
class StagingConfig:
|
||||
def __init__(self):
|
||||
self.REDIS_CERT_DATA = environ.get('REDIS_CERT')
|
||||
self.REDIS_BASE_URI = environ.get('REDIS_BASE_URI', 'redis://localhost:6379/0')
|
||||
|
||||
def create_redis_connection(self):
|
||||
if self.REDIS_CERT_DATA:
|
||||
# Schrijf certificaat naar tijdelijk bestand
|
||||
@@ -320,9 +344,11 @@ class StagingConfig(Config):
|
||||
)
|
||||
else:
|
||||
return redis.from_url(self.REDIS_BASE_URI)
|
||||
|
||||
|
||||
# Gebruik voor session Redis
|
||||
SESSION_REDIS = property(lambda self: self.create_redis_connection())
|
||||
@property
|
||||
def SESSION_REDIS(self):
|
||||
return self.create_redis_connection()
|
||||
```
|
||||
|
||||
#### Scaleway Secret Manager Vereisten
|
||||
@@ -389,8 +415,20 @@ kubectl -n eveai-staging describe certificate evie-staging-tls
|
||||
|
||||
Dit kan even duren. Maar zodra het certificaat is aangemaakt, kan je de de https-only ingress opzetten:
|
||||
|
||||
#### Apply per-prefix headers (moet bestaan vóór de Ingress die ernaar verwijst)
|
||||
```bash
|
||||
kubectl apply -f scaleway/manifests/base/networking/headers-configmaps.yaml
|
||||
```
|
||||
kubectl apply -f scaleway/manifests/base/networking/ingress-https.yaml
|
||||
|
||||
#### Apply ingresses
|
||||
```bash
|
||||
kubectl apply -f scaleway/manifests/base/networking/ingress-https.yaml # alleen /verify
|
||||
kubectl apply -f scaleway/manifests/base/networking/ingress-admin.yaml # /admin → eveai-app-service
|
||||
kubectl apply -f scaleway/manifests/base/networking/ingress-api.yaml # /api → eveai-api-service
|
||||
kubectl apply -f scaleway/manifests/base/networking/ingress-chat-client.yaml # /chat-client → eveai-chat-client-service
|
||||
|
||||
# Alternatief: via overlay (mits kustomization.yaml is bijgewerkt)
|
||||
kubectl apply -k scaleway/manifests/overlays/staging/
|
||||
```
|
||||
|
||||
Om bunny.net te gebruiken:
|
||||
@@ -418,6 +456,30 @@ kubectl get ingress -n eveai-staging
|
||||
kubectl get certificates -n eveai-staging
|
||||
```
|
||||
|
||||
### Verificatie commando's
|
||||
|
||||
Controleer ingresses en headers:
|
||||
|
||||
```bash
|
||||
kubectl -n eveai-staging get ing
|
||||
kubectl -n eveai-staging describe ing eveai-admin-ingress
|
||||
kubectl -n eveai-staging describe ing eveai-api-ingress
|
||||
kubectl -n eveai-staging describe ing eveai-chat-client-ingress
|
||||
kubectl -n eveai-staging describe ing eveai-staging-ingress # bevat /verify
|
||||
kubectl -n eveai-staging get cm eveai-admin-headers eveai-api-headers eveai-chat-headers -o yaml
|
||||
```
|
||||
|
||||
- In elke prefix-Ingress moeten de annotations zichtbaar zijn: use-regex: true, rewrite-target: /$2, proxy-set-headers: eveai-staging/eveai--headers.
|
||||
- In de ConfigMaps moet de key X-Forwarded-Prefix de juiste waarde hebben (/admin, /api, /chat-client).
|
||||
End-to-end testen:
|
||||
|
||||
- https://evie-staging.askeveai.com/admin/login → loginpagina. In app-logs zie je PATH zonder /admin (door rewrite) maar URL met /admin.
|
||||
- Na login: 302 Location: /admin/user/tenant_overview.
|
||||
- API: https://evie-staging.askeveai.com/api/… → backend ontvangt pad zonder /api.
|
||||
- Chat client: https://evie-staging.askeveai.com/chat-client/… → juiste service.
|
||||
- Verify: https://evie-staging.askeveai.com/verify → ongewijzigd via ingress-https.yaml.
|
||||
- Root: zolang Bunny rule niet actief is, geen automatische redirect op / (verwacht gedrag).
|
||||
|
||||
### Phase 7: Install PgAdmin Tool
|
||||
|
||||
#### Secret eveai-pgadmin-admin in Scaleway Secret Manager aanmaken (indien niet bestaat)
|
||||
@@ -464,6 +526,52 @@ kubectl -n tools port-forward svc/pgadmin-pgadmin4 8080:80
|
||||
|
||||
### Phase 8: RedisInsight Tool Deployment
|
||||
|
||||
#### Installatie via kubectl (zonder Helm)
|
||||
Gebruik een eenvoudig manifest met Deployment + Service + PVC in de `tools` namespace. Dit vermijdt externe chart repositories en extra authenticatie.
|
||||
```bash
|
||||
# Apply manifest (maakt namespace tools aan indien nodig)
|
||||
kubectl apply -f scaleway/manifests/base/tools/redisinsight/redisinsight.yaml
|
||||
|
||||
# Controleer resources
|
||||
kubectl -n tools get pods,svc,pvc
|
||||
```
|
||||
|
||||
#### (Optioneel) ExternalSecrets voor gemak (eigenlijk niet nodig)
|
||||
Indien je de Redis-credentials en CA-cert in namespace `tools` wil spiegelen (handig om het CA-bestand eenvoudig te exporteren en/of later provisioning te doen):
|
||||
```bash
|
||||
kubectl apply -f scaleway/manifests/base/tools/redisinsight/externalsecrets.yaml
|
||||
kubectl -n tools get externalsecret
|
||||
kubectl -n tools get secret | grep redisinsight
|
||||
```
|
||||
|
||||
CA-bestand lokaal opslaan voor UI-upload (alleen nodig als je ExternalSecrets gebruikte):
|
||||
```bash
|
||||
kubectl -n tools get secret redisinsight-ca -o jsonpath='{.data.REDIS_CERT}' | base64 -d > /tmp/redis-ca.pem
|
||||
```
|
||||
|
||||
#### Port Forward, Local Access
|
||||
```bash
|
||||
# RedisInsight v2 luistert op poort 5540
|
||||
kubectl -n tools port-forward svc/redisinsight 5540:5540
|
||||
# Browser: http://localhost:5540
|
||||
```
|
||||
|
||||
#### UI: Redis verbinden
|
||||
- Host: `172.16.16.2`
|
||||
- Port: `6379`
|
||||
- Auth: username `luke`, password uit secret (eveai-redis of redisinsight-redis)
|
||||
- TLS: zet TLS aan en upload het CA-certificaat (PEM)
|
||||
- Certificaatverificatie: omdat je via IP verbindt en geen hostname in het certificaat staat, kan strict verify falen. Zet dan "Verify server certificate"/"Check server identity" uit in de UI. Dit is normaal bij private networking via IP.
|
||||
|
||||
#### Troubleshooting
|
||||
- Controleer pods, service en PVC in `tools`:
|
||||
```bash
|
||||
kubectl -n tools get pods,svc,pvc
|
||||
```
|
||||
- NetworkPolicies: indien actief, laat egress toe van `tools` → `172.16.16.2:6379`.
|
||||
- TLS-issues via IP: zet verify uit of gebruik een DNS-hostnaam die met het cert overeenkomt (indien beschikbaar).
|
||||
- PVC niet bound: specificeer een geldige `storageClassName` in het manifest.
|
||||
|
||||
|
||||
### Phase 9: Application Services Deployment
|
||||
|
||||
@@ -663,21 +771,6 @@ nslookup evie-staging.askeveai.com
|
||||
curl https://evie-staging.askeveai.com/verify/
|
||||
```
|
||||
|
||||
## CDN Setup (Bunny.net - Optional)
|
||||
|
||||
### Configure Pull Zone
|
||||
- Create Pull zone: evie-staging
|
||||
- Origin: https://[LoadBalancer-IP] (note HTTPS!)
|
||||
- Host header: evie-staging.askeveai.com
|
||||
- Force SSL: Enabled
|
||||
|
||||
### Update DNS for CDN
|
||||
- Change A-record to CNAME pointing to CDN endpoint
|
||||
- Or update A-record to CDN IP
|
||||
|
||||
## Bunny.net notes
|
||||
|
||||
- In the pull zone's Caching - General settings, ensure to disable 'Strip Response Cookies'
|
||||
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user