eveAI/scripts/dbops/README.md

# DB Operations (Manual)

Scripts to run database initialization and migrations manually during your release process. Default wait timeout is 30s with 2s interval.

Required env (typical):
- DB_HOST, DB_PORT, DB_NAME, DB_USER, DB_PASS
- PROJECT_DIR (defaults /app)
- FLASK_APP (defaults to ${PROJECT_DIR}/scripts/run.py)
- COMPONENT_NAME (defaults to eveai_app)
- PYTHONPATH (defaults to ${PROJECT_DIR})

Order (standard: DB already exists):
1. 00-env-check.sh
2. 01-wait-for-db.sh  (WAIT_FOR_DB_TIMEOUT=30 WAIT_FOR_DB_INTERVAL=2)
3. 02-db-bootstrap-ext.sh  (creates pgvector extension if missing)
4. 03-db-migrate-public.sh
5. 04-db-migrate-tenant.sh  (single run for all tenants)
6. 06-verify-minimal.sh

Optional, one-off for a new environment where DB does not exist yet:
- 90-initial-cluster-db-create.sh (run before step 3)

Optional seed/init data (one-off per env, idempotent):
- 05-seed-or-init-data.sh

Notes:
- Scripts are idempotent where applicable.
- Do not echo secrets. Password is read from DB_PASS via PGPASSWORD env.
- Ensure you run these inside the app image/container or any environment with psql, pg_isready, flask and the project code present.


## Podman quickstart (manual testing)

Below are two simple ways to run these scripts in a Podman-based setup. Choose the approach that best matches your workflow.

Prerequisites:
- A reachable PostgreSQL 16 instance and credentials (DB_HOST, DB_PORT, DB_NAME, DB_USER, DB_PASS)
- Your application code available inside the container at /app (default PROJECT_DIR)
- Tools available in the container: psql, pg_isready, flask, python
  - docker/Dockerfile.base already includes psql; use it to build a tools image if needed

Environment variables (typical):
- DB_HOST, DB_PORT, DB_NAME, DB_USER, DB_PASS
- PROJECT_DIR=/app (default)
- FLASK_APP=/app/scripts/run.py (default)
- COMPONENT_NAME=eveai_app (default)
- PYTHONPATH="/app" (default)

Order to run (standard):
1) scripts/dbops/00-env-check.sh
2) scripts/dbops/01-wait-for-db.sh  (WAIT_FOR_DB_TIMEOUT=30 WAIT_FOR_DB_INTERVAL=2)
3) scripts/dbops/02-db-bootstrap-ext.sh
4) scripts/dbops/03-db-migrate-public.sh
5) scripts/dbops/04-db-migrate-tenant.sh
6) scripts/dbops/06-verify-minimal.sh

Optional (one-time for a new environment where the DB does not yet exist):
- scripts/dbops/90-initial-cluster-db-create.sh (run before step 3)

Optional seed/init (one-time per environment):
- scripts/dbops/05-seed-or-init-data.sh

---

Approach A — Exec into an existing app container
1. Start your stack (e.g., via Podman Compose) so the app container and DB are running.
2. Find the app container name:
   podman ps
3. Exec into the container shell:
   podman exec -it <app-container-name> bash
4. Export DB variables inside the container (adjust values):
   export DB_HOST=127.0.0.1
   export DB_PORT=5432
   export DB_NAME=eveai
   export DB_USER=postgres
   export DB_PASS=yourpassword
   # Optional (defaults exist)
   export PROJECT_DIR=/app
   export FLASK_APP=/app/scripts/run.py
   export COMPONENT_NAME=eveai_app
   export PYTHONPATH="/app"
5. Run the scripts in order, for example:
   scripts/dbops/00-env-check.sh && \
   scripts/dbops/01-wait-for-db.sh && \
   scripts/dbops/02-db-bootstrap-ext.sh && \
   scripts/dbops/03-db-migrate-public.sh && \
   scripts/dbops/04-db-migrate-tenant.sh && \
   scripts/dbops/06-verify-minimal.sh
6. Optionally seed/init once per environment:
   scripts/dbops/05-seed-or-init-data.sh

Notes:
- If your DB runs in a separate container, DB_HOST should be that container's network hostname (often the service name from compose). If using host networking, 127.0.0.1 may work.

---

Approach B — Use a one-off tools container based on Dockerfile.base
This runs the scripts without depending on a running app container. It mounts the repo at /app and uses the base image with psql installed.

1. Build the tools image (from repo root):
   podman build -f docker/Dockerfile.base -t eveai/tools:local .

2. Run a one-off container with your env vars and the repo mounted:
   podman run --rm -it \
     --name eveai-dbops \
     --network host \
     -e DB_HOST=127.0.0.1 \
     -e DB_PORT=5432 \
     -e DB_NAME=eveai \
     -e DB_USER=postgres \
     -e DB_PASS=yourpassword \
     -e PROJECT_DIR=/app \
     -e FLASK_APP=/app/scripts/run.py \
     -e COMPONENT_NAME=eveai_app \
     -e PYTHONPATH="/app" \
     -e WAIT_FOR_DB_TIMEOUT=30 \
     -e WAIT_FOR_DB_INTERVAL=2 \
     -v "$(pwd)":/app \
     -w /app \
     eveai/tools:local \
     bash -lc 'scripts/dbops/00-env-check.sh && scripts/dbops/01-wait-for-db.sh && scripts/dbops/02-db-bootstrap-ext.sh && scripts/dbops/03-db-migrate-public.sh && scripts/dbops/04-db-migrate-tenant.sh && scripts/dbops/06-verify-minimal.sh'

3. Optional seed/init once per environment:
   podman run --rm -it --network host \
     -e DB_HOST -e DB_PORT -e DB_NAME -e DB_USER -e DB_PASS \
     -e PROJECT_DIR -e FLASK_APP -e COMPONENT_NAME -e PYTHONPATH \
     -v "$(pwd)":/app -w /app \
     eveai/tools:local \
     bash -lc 'scripts/dbops/05-seed-or-init-data.sh'

Notes:
- --network host simplifies connectivity on Linux. If you cannot use it, use a user-defined Podman network and set DB_HOST to the DB container hostname.
- Do not echo secrets; scripts read DB_PASS via the PGPASSWORD env variable automatically.
- If flask is missing at runtime, ensure your tools image has Python deps installed or run inside your actual app image/container that already has them.

Troubleshooting
- Connection refused: verify DB_HOST/PORT and network mode; try --network host.
- Authentication failed: check DB_USER/DB_PASS and that the user has rights on DB_NAME.
- pgvector missing even after step 3: ensure your DB user has rights to CREATE EXTENSION in DB_NAME.
- Flask command not found: run within the app image or install project dependencies in the tools image before running.