v2

sidebar_position: 6

Deployment Guide

Purpose: Complete deployment instructions for Loreax across all environments (local, hyena, production).
Status: Complete
Last Updated: April 2026

Loreax supports multiple deployment targets. This guide covers all deployment strategies from local development to production.


📐 Architecture Overview

Loreax exposes three logical surfaces from a single Laravel codebase. Routing between them is done by host (subdomain), not by path:

Surface Purpose Production Example
API Versioned JSON API (/v1/*) api.loreax.bervant.co.ke
Admin Filament back-office panel admin.loreax.bervant.co.ke
Dev Laravel-rendered documentation (wiki, API reference, playground) dev.loreax.bervant.co.ke

The application reads domain configuration from environment variables:

  • LOREAX_API_DOMAIN — API host (e.g., api.loreax.bervant.co.ke)
  • LOREAX_ADMIN_DOMAIN — Admin panel host (e.g., admin.loreax.bervant.co.ke)
  • LOREAX_DEV_DOMAIN — Developer portal host (e.g., dev.loreax.bervant.co.ke)
  • LOREAX_API_PREFIX — API path prefix (api/v1 for local/test, v1 for hyena/prod)

Environments

Name Target Details
local Developer machine Single host, all surfaces under /
test CI/CD pipeline Single host, all surfaces under /
hyena EC2 35.178.52.158 Three subdomains (staging environment)
prod Kubernetes (TBD) Three subdomains (future production)


🐆 Hyena Environment (EC2 Staging)

Status: Active staging environment
Host: 35.178.52.158 (shared EC2 instance)
Repository: deployment/ec2/

The hyena environment runs on a shared EC2 host that powers multiple Bervant applications. All Loreax-specific components are namespaced (loreax-*) to avoid conflicts.

Infrastructure Details

Component Value
App root /var/www/loreax/
App user www-data
PHP version 8.4 (FPM socket: /run/php/php8.4-fpm.sock)
Web server nginx 1.24+
Database PostgreSQL 16 (shared, database: loreax_hyena)
Cache/Queue Redis 7 (shared, DB indexes: 10-13)
Logs MongoDB (shared, database: loreax_hyena_logs)
Storage AWS S3 (loreax-hyena bucket)

Subdomain Configuration

Each subdomain points to the same Laravel codebase but exposes different functionality:

Host Document Root Exposes Blocks
api.loreax.bervant.co.ke /var/www/loreax/public /v1/* API, /health, /ready, / welcome /admin, /dev
admin.loreax.bervant.co.ke /var/www/loreax/public /admin Filament panel API endpoints
dev.loreax.bervant.co.ke /var/www/loreax/public /wiki, /api-reference, /api-playground (Laravel-rendered) /admin, /v1/*

DNS Configuration

Set these A-records to point at the EC2 instance:

api.loreax.bervant.co.ke   A 35.178.52.158
admin.loreax.bervant.co.ke A 35.178.52.158
dev.loreax.bervant.co.ke   A 35.178.52.158

First-Time Provisioning

Run once on the host as a sudoer to set up the environment:

# Download and run the provisioning script
sudo bash <(curl -fsSL https://raw.githubusercontent.com/bervant/loreax-core/master/deployment/ec2/scripts/provision.sh) \
  --repo git@github.com:bervant/loreax-core.git \
  --branch master

The provision.sh script performs the following:

  1. ✓ Verifies/installs baseline tooling (composer, bun, git, PHP 8.4)
  2. Auto-installs any missing PHP 8.4 extensions (pdo_pgsql, mbstring, xml, curl, zip, gd, redis, pcntl, mongodb) — no manual apt-get required
  3. ✓ Clones repository to /var/www/loreax/ (owned by www-data)
  4. ✓ Seeds /var/www/loreax/.env from deployment/ec2/env/hyena.env.template
  5. ✓ Installs nginx configs for all three subdomains
  6. ✓ Installs and enables systemd units (Horizon, Scheduler)
  7. ✓ Reloads nginx

Environment Configuration

After provisioning, edit /var/www/loreax/.env and fill in the required secrets:

# Generate APP_KEY (one time only)
KEY="base64:$(php8.4 -r 'echo base64_encode(random_bytes(32));')"
sudo sed -i "s|^APP_KEY=.*|APP_KEY=${KEY}|" /var/www/loreax/.env

# Edit .env and update these critical values:
sudo nano /var/www/loreax/.env

Critical configuration values:

# Application
APP_KEY=base64:xxxxx                    # Generated above
APP_ENV=hyena
APP_DEBUG=false
APP_URL=https://api.loreax.bervant.co.ke
APP_FRONTEND_URL=https://admin.loreax.bervant.co.ke

# Subdomain routing (REQUIRED for multi-surface hosting)
LOREAX_API_DOMAIN=api.loreax.bervant.co.ke
LOREAX_ADMIN_DOMAIN=admin.loreax.bervant.co.ke
LOREAX_DEV_DOMAIN=dev.loreax.bervant.co.ke
LOREAX_API_PREFIX=v1                    # No /api/ prefix on dedicated subdomain

# Database
DB_PASSWORD=__CHANGE_ME__

# MongoDB (for request logs)
MONGODB_DSN="mongodb://loreax_hyena:__CHANGE_ME__@127.0.0.1:27017/loreax_hyena_logs?authSource=admin"

# AWS S3 (media storage)
AWS_ACCESS_KEY_ID=__CHANGE_ME__
AWS_SECRET_ACCESS_KEY=__CHANGE_ME__
AWS_BUCKET=loreax-hyena

# Payment Gateway
KASHIER_MERCHANT_KEY=__CHANGE_ME__
KASHIER_MERCHANT_SECRET=__CHANGE_ME__

# Email Service
MAIL_HOST=__CHANGE_ME__
MAIL_USERNAME=__CHANGE_ME__
MAIL_PASSWORD=__CHANGE_ME__

# OAuth (optional)
GOOGLE_CLIENT_ID=__CHANGE_ME__
GOOGLE_CLIENT_SECRET=__CHANGE_ME__
FACEBOOK_CLIENT_ID=__CHANGE_ME__
FACEBOOK_CLIENT_SECRET=__CHANGE_ME__

Template reference: deployment/ec2/env/hyena.env.template

Developer Documentation Configuration

The developer documentation (wiki, API reference, API playground) is served by Laravel:

  1. Routes: All documentation is served through Laravel at /wiki, /api-reference, and /api-playground paths on the dev subdomain
  2. No Build Step: These pages are dynamically rendered by Laravel — no static build or separate build step required
  3. OpenAPI Spec: Generated via php artisan l5-swagger:generate during deployment to storage/api-docs/api-docs.json
  4. Downloads: OpenAPI JSON, Postman, and Bruno collections are available at /api-reference/download/* endpoints

If the developer documentation is not working:

# Check if OpenAPI spec exists
ls -la /var/www/loreax/storage/api-docs/api-docs.json

# Regenerate OpenAPI spec
cd /var/www/loreax
sudo -u www-data php8.4 artisan l5-swagger:generate

# Check nginx is proxying to Laravel correctly
sudo nginx -T | grep -A 10 "dev.loreax.bervant.co.ke"

# Check build output logs
# Check Laravel logs
sudo -u www-data cat /var/www/loreax/storage/logs/laravel.log | grep -i "api-reference\|wiki\|playground"

### First Deployment

After configuring `.env`, run the deployment script:

```bash
# Run first deployment
sudo /var/www/loreax/deployment/ec2/scripts/deploy.sh

# Enable and start Horizon (queue workers)
sudo systemctl enable --now loreax-horizon.service

# Verify services are running
sudo systemctl status loreax-horizon.service
sudo systemctl status loreax-scheduler.timer

TLS/SSL Configuration

The shared host uses certbot for TLS certificates. After DNS propagates:

# Request certificates for all three subdomains
sudo certbot --nginx \
  -d api.loreax.bervant.co.ke \
  -d admin.loreax.bervant.co.ke \
  -d dev.loreax.bervant.co.ke

# Verify automatic renewal is enabled
sudo systemctl status certbot.timer

After certificates are issued, uncomment the SSL blocks in the nginx configs:

# Edit each nginx config and uncomment the SSL lines
sudo nano /etc/nginx/sites-available/api.loreax.bervant.co.ke.conf
sudo nano /etc/nginx/sites-available/admin.loreax.bervant.co.ke.conf
sudo nano /etc/nginx/sites-available/dev.loreax.bervant.co.ke.conf

# Test and reload
sudo nginx -t
sudo systemctl reload nginx

Continuous Deployment (GitHub Actions)

Automated deployments are triggered by pushes to the master branch via GitHub Actions.

Workflow: .github/workflows/deploy-hyena.yml

Required repository secrets (Settings → Secrets → Actions):

Secret Value
HYENA_SSH_HOST 35.178.52.158
HYENA_SSH_USER Sudoer username (with NOPASSWD)
HYENA_SSH_PRIVATE_KEY OpenSSH private key
HYENA_SSH_KNOWN_HOSTS Output of ssh-keyscan -H 35.178.52.158

The deploy SSH user must have passwordless sudo access. Create a sudoers entry:

# /etc/sudoers.d/loreax-deploy
deploy_user ALL=(ALL) NOPASSWD: /var/www/loreax/deployment/ec2/scripts/deploy.sh, \
                                 /usr/bin/install, /bin/ln, /usr/sbin/nginx, \
                                 /bin/systemctl, /usr/bin/sudo

Deployment Script Workflow

When deployment/ec2/scripts/deploy.sh runs, it performs these steps:

  1. ✓ Pulls latest code from configured branch
  2. ✓ Syncs nginx and systemd configs from the working tree (unless --skip-infra)
  3. Fixes storage/ and bootstrap/cache/ ownership — ensures www-data can write after any git operations that touched file ownership
  4. ✓ Installs Composer dependencies (--no-dev --optimize-autoloader)
  5. ✓ Installs Node dependencies and builds root JS assets (bun install --frozen-lockfile)
  6. ✓ Generates OpenAPI spec (l5-swagger:generate)
  7. ✓ Enters maintenance mode, runs migrations, exits maintenance mode
  8. ✓ Rebuilds caches (config, route, event, view, Filament components)
  9. ✓ Reloads php-fpm and restarts Horizon workers

Operational Commands

Log Monitoring

# Horizon worker logs
sudo journalctl -u loreax-horizon.service -f

# Nginx access logs
sudo tail -f /var/log/nginx/loreax-api.access.log
sudo tail -f /var/log/nginx/loreax-admin.access.log
sudo tail -f /var/log/nginx/loreax-dev.access.log

# Nginx error logs
sudo tail -f /var/log/nginx/loreax-api.error.log
sudo tail -f /var/log/nginx/loreax-admin.error.log
sudo tail -f /var/log/nginx/loreax-dev.error.log

# Laravel application logs
sudo -u www-data tail -f /var/www/loreax/storage/logs/laravel.log

Rollback to Previous Version

# Rollback to a specific commit or tag
sudo /var/www/loreax/deployment/ec2/scripts/rollback.sh --to <sha-or-tag>

# Example: Rollback to previous commit
sudo /var/www/loreax/deployment/ec2/scripts/rollback.sh --to HEAD~1

# Example: Rollback to tag
sudo /var/www/loreax/deployment/ec2/scripts/rollback.sh --to v1.2.3

Note: Schema rollbacks are NOT automatic. Run migrate:rollback if needed.

Maintenance Mode

# Enable maintenance mode with bypass secret
sudo /var/www/loreax/deployment/ec2/scripts/maintenance.sh on --secret <bypass-token>

# Disable maintenance mode
sudo /var/www/loreax/deployment/ec2/scripts/maintenance.sh off

# Check status
php artisan down --show-status

During maintenance, users see a 503 page. Bypass with ?secret=<bypass-token> in URL.

Service Management

# Restart Horizon workers
sudo systemctl restart loreax-horizon.service

# Stop Horizon (for maintenance)
sudo systemctl stop loreax-horizon.service

# Check Horizon status
sudo systemctl status loreax-horizon.service

# Manually run scheduler
sudo systemctl start loreax-scheduler.service

# Check scheduler timer
sudo systemctl status loreax-scheduler.timer

Artisan Commands

# Run artisan commands as www-data
sudo -u www-data php /var/www/loreax/artisan <command>

# Examples:
sudo -u www-data php /var/www/loreax/artisan cache:clear
sudo -u www-data php /var/www/loreax/artisan migrate:status
sudo -u www-data php /var/www/loreax/artisan queue:work --once
sudo -u www-data php /var/www/loreax/artisan horizon:status

Systemd Units

Loreax uses two systemd units for background processing:

Horizon Service (loreax-horizon.service)

Processes queued jobs (payments, notifications, media processing, etc.).

[Unit]
Description=Loreax Horizon Worker (queues + scheduler dispatchers)
After=network.target redis-server.service postgresql.service

[Service]
Type=simple
User=www-data
Group=www-data
Restart=always
RestartSec=3
WorkingDirectory=/var/www/loreax
ExecStart=/usr/bin/php8.4 artisan horizon
ExecStop=/usr/bin/php8.4 artisan horizon:terminate
KillSignal=SIGTERM
TimeoutStopSec=120

[Install]
WantedBy=multi-user.target

Location: /etc/systemd/system/loreax-horizon.service

Scheduler Timer (loreax-scheduler.timer + loreax-scheduler.service)

Runs Laravel's scheduled tasks every minute.

# loreax-scheduler.timer
[Unit]
Description=Loreax Scheduler Timer

[Timer]
OnBootSec=1min
OnUnitActiveSec=1min
Unit=loreax-scheduler.service

[Install]
WantedBy=timers.target

# loreax-scheduler.service
[Unit]
Description=Loreax Scheduler

[Service]
Type=oneshot
User=www-data
Group=www-data
WorkingDirectory=/var/www/loreax
ExecStart=/usr/bin/php8.4 artisan schedule:run

Location: /etc/systemd/system/loreax-scheduler.{service,timer}

Nginx Configuration

Each subdomain has its own nginx config with surface-specific routing:

API Host (api.loreax.bervant.co.ke.conf):

  • Serves /v1/* API endpoints
  • Blocks /admin and /dev paths
  • Sets fastcgi_param LOREAX_SURFACE api

Admin Host (admin.loreax.bervant.co.ke.conf):

  • Serves /admin Filament panel
  • Blocks API endpoints (/v1/*)
  • Sets fastcgi_param LOREAX_SURFACE admin

Dev Host (dev.loreax.bervant.co.ke.conf):

  • Document root is /var/www/loreax/public — All requests are proxied through Laravel/php-fpm
  • /health, /ready, and all documentation paths (/wiki, /api-reference, /api-playground) are served by Laravel
  • Static assets (CSS, JS, images) are served directly with 1-year cache headers
  • Blocks /admin and /v1/* paths
  • Sets fastcgi_param LOREAX_SURFACE dev

Location: /etc/nginx/sites-available/*.loreax.bervant.co.ke.conf

Health Checks

Verify deployment success:

# API welcome — confirms the API host is routed correctly
curl https://api.loreax.bervant.co.ke/
# Expected: {"message":"Loreax (hyena) API V1"}

# Liveness probe
curl https://api.loreax.bervant.co.ke/health
# Expected: {"status":"ok"}

# Readiness probe (checks DB, cache, storage, MPESA reachability)
curl https://api.loreax.bervant.co.ke/ready
# Expected: {"status":"ready","services":{...}}

# Admin panel (should redirect to login)
curl -I https://admin.loreax.bervant.co.ke/admin
# Expected: 302 redirect

# Developer documentation (rendered by Laravel at subdomain root)
curl -I https://dev.loreax.bervant.co.ke/wiki
# Expected: 200 OK (Laravel-rendered wiki page)

# API Reference
curl -I https://dev.loreax.bervant.co.ke/api-reference
# Expected: 200 OK (Laravel-rendered API reference)

# API Playground
curl -I https://dev.loreax.bervant.co.ke/api-playground
# Expected: 200 OK (Laravel-rendered playground)

# OpenAPI spec download
curl https://dev.loreax.bervant.co.ke/api-reference/download/openapi
# Expected: Valid JSON OpenAPI 3.0 document

Operational Scripts

All scripts live in deployment/ec2/scripts/ and must be run from the repo root or the EC2 host.

install.sh — Full infra install / refresh

Idempotent. Installs (or refreshes) nginx configs, symlinks, and worker units from the working tree. Run after provision.sh or whenever you want to push infra config changes without a full deploy.

# Install everything (nginx + systemd workers)
sudo /var/www/loreax/deployment/ec2/scripts/install.sh

# Also request TLS certs via certbot
sudo /var/www/loreax/deployment/ec2/scripts/install.sh --ssl --email admin@loreax.bervant.co.ke

# Dry-run to preview what would change
sudo /var/www/loreax/deployment/ec2/scripts/install.sh --dry-run

ops.sh — Targeted day-to-day operations

Lighter-weight than install.sh. Use for targeted ops tasks like resyncing a single nginx config with an automatic backup.

# Resync all nginx configs (backs up existing before overwriting)
bash deployment/ec2/scripts/ops.sh resync nginx --all

# Resync a specific surface
bash deployment/ec2/scripts/ops.sh resync nginx --app=api
bash deployment/ec2/scripts/ops.sh resync nginx --app=dev,admin

# See help
bash deployment/ec2/scripts/ops.sh --help

resync nginx does the following for each targeted conf:

  1. Backs up the current /etc/nginx/sites-available/<conf> as <conf>.bak.<timestamp>
  2. Copies the new conf from deployment/ec2/nginx/<conf>
  3. Ensures the symlink in sites-enabled/ is present
  4. Runs nginx -t — if this fails, nginx is not reloaded and backups are left for manual recovery
  5. Reloads nginx on success

deploy.sh — Standard deployment

# Deploy current HEAD of the tracked branch
sudo /var/www/loreax/deployment/ec2/scripts/deploy.sh

# Deploy a specific ref (branch, tag, or SHA)
sudo /var/www/loreax/deployment/ec2/scripts/deploy.sh --ref v1.2.3

# Skip rebuilding assets (useful for config-only changes)
sudo /var/www/loreax/deployment/ec2/scripts/deploy.sh --skip-build

# Skip nginx/systemd sync (useful when infra hasn't changed)
sudo /var/www/loreax/deployment/ec2/scripts/deploy.sh --skip-infra

Troubleshooting

Issue: 502 Bad Gateway

# Check PHP-FPM is running
sudo systemctl status php8.4-fpm

# Check PHP-FPM socket exists
ls -la /run/php/php8.4-fpm.sock

# Check nginx error logs
sudo tail -50 /var/log/nginx/loreax-*.error.log

Issue: 500 Internal Server Error

# Check Laravel logs
sudo -u www-data tail -50 /var/www/loreax/storage/logs/laravel.log

# Fix storage and bootstrap/cache permissions (most common cause)
sudo chown -R www-data:www-data /var/www/loreax/storage /var/www/loreax/bootstrap/cache
sudo chmod -R ug+rwX /var/www/loreax/storage /var/www/loreax/bootstrap/cache

Issue: Horizon Not Processing Jobs

# Check Horizon is running
sudo systemctl status loreax-horizon.service

# Check Horizon logs
sudo journalctl -u loreax-horizon.service -n 100

# Check Redis connection
redis-cli -n 12 LLEN loreax_hyena:default

# Restart Horizon
sudo systemctl restart loreax-horizon.service

Issue: Database Connection Failed

# Check PostgreSQL is running
sudo systemctl status postgresql

# Test connection with credentials from .env
psql -h localhost -U loreax_hyena -d loreax_hyena

# Check .env database credentials
sudo -u www-data grep -E "^DB_" /var/www/loreax/.env

Issue: Developer Portal (dev.loreax.bervant.co.ke) Not Working

Symptom: Accessing https://dev.loreax.bervant.co.ke shows 404 or shows Laravel error page.

Root Cause: The developer documentation pages are not loading correctly from Laravel.

Solution:

# 1. Verify routes are configured correctly
curl -I https://dev.loreax.bervant.co.ke/wiki
# Should return: 200 OK (not 404)

# 2. Check if OpenAPI spec exists
ls -la /var/www/loreax/storage/api-docs/api-docs.json
# Should exist and be recent (generated during deployment)

# 3. If spec is missing, regenerate it:
cd /var/www/loreax
sudo -u www-data php8.4 artisan l5-swagger:generate

# 4. Check nginx configuration
sudo nginx -T | grep -A 20 "server_name dev.loreax.bervant.co.ke"
# Root should be: /var/www/loreax/public
# Should proxy all requests through @laravel location

# 5. Reload nginx to pick up any config changes
sudo nginx -t && sudo systemctl reload nginx

# 6. Test directly
curl -I https://dev.loreax.bervant.co.ke/wiki
curl -I https://dev.loreax.bervant.co.ke/api-reference
curl -I https://dev.loreax.bervant.co.ke/api-playground
# All should return: 200 OK

# 7. Check nginx access logs for errors
sudo tail -f /var/log/nginx/loreax-dev.access.log
sudo tail -f /var/log/nginx/loreax-dev.error.log

# 8. Check Laravel logs
sudo -u www-data tail -f /var/www/loreax/storage/logs/laravel.log

Common mistakes:

  • ❌ Nginx root points to wrong directory (should be /var/www/loreax/public)
  • ❌ OpenAPI spec not generated (php artisan l5-swagger:generate)
  • ❌ Routes not updated after deployment (clear route cache: php artisan route:cache)
  • ❌ TLS certificate not installed (HTTP works but HTTPS doesn't)
  • ❌ php-fpm not running or socket permissions incorrect

Fix route cache issue:

# Clear and rebuild route cache
cd /var/www/loreax
sudo -u www-data php8.4 artisan route:clear
sudo -u www-data php8.4 artisan route:cache
sudo -u www-data php8.4 artisan config:cache

Security Considerations

  • ✓ All nginx configs enforce HTTPS (HTTP redirects to HTTPS)
  • ✓ Security headers added: X-Frame-Options, X-Content-Type-Options, Referrer-Policy
  • .env file permissions: 600 (owner read/write only)
  • ✓ Storage and cache directories: 775 (group writable)
  • ✓ Hidden directories blocked: location ~ /\.(?!well-known).* { deny all; }
  • ✓ Admin panel requires authentication (Filament + MFA)
  • ✓ API requires Bearer token authentication
  • ✓ Rate limiting enforced (config/rate_limits.php)

What This Setup Does NOT Do

  • Does not touch other apps on the shared host (all resources are loreax- prefixed)
  • Does not manage Postgres/Redis/MongoDB (shared services, dedicated databases/indexes per app)
  • Does not provision wildcard TLS (individual certificates per subdomain)
  • Does not provide atomic releases (clone-in-place; ~30s downtime during deploy)
  • Does not auto-rollback migrations (must be done manually if needed)

☸️ Kubernetes Deployment (Future Production)

Status: Placeholder (hyena is current staging)

When scaling beyond EC2, production will use a managed Kubernetes cluster (likely AWS EKS) with the same three-surface architecture.

Planned Architecture

Ingress Controller (NGINX/ALB)
├── api.loreax.<domain> → Deployment: loreax-api (3+ replicas)
├── admin.loreax.<domain> → Deployment: loreax-admin (2+ replicas)
└── dev.loreax.<domain> → Deployment: loreax-dev (static site)

Background Services:
├── Deployment: loreax-horizon (4+ workers)
└── CronJob: loreax-scheduler (runs every minute)

Managed Services:
├── RDS PostgreSQL (Multi-AZ, automated backups)
├── ElastiCache Redis (replication enabled)
├── DocumentDB MongoDB (optional, for logs)
└── S3 (media storage with CloudFront CDN)

Deployment Structure (Planned)

deployment/k8s/
├── base/                # Kustomize base manifests
│   ├── api/            # API deployment + service
│   ├── admin/          # Admin deployment + service
│   ├── dev/            # Dev portal deployment + service
│   ├── horizon/        # Queue worker deployment
│   ├── scheduler/      # CronJob for schedule:run
│   └── ingress/        # Ingress controller config
└── overlays/
    ├── hyena/          # Hyena-specific config (if migrated)
    └── prod/           # Production-specific config

Key Features (When Implemented)

  • Zero-downtime deployments (rolling updates)
  • Horizontal auto-scaling (HPA based on CPU/memory)
  • Health checks (liveness + readiness probes)
  • Secret management (Kubernetes Secrets or AWS Secrets Manager)
  • Persistent volumes (for cache warming if needed)
  • Multi-region (future: DR in eu-west-1)

For now, see deployment/k8s/README.md for planning notes.


� Deployment Scripts Reference

Loreax includes four automation scripts in deployment/ec2/scripts/ for managing the EC2 environment.

provision.sh — First-Time Setup

Purpose: Bootstrap a fresh EC2 host with Loreax infrastructure.
Run: Once per environment (new host setup).
Requires: Root/sudo access.

sudo bash <(curl -fsSL https://raw.githubusercontent.com/bervant/loreax-core/master/deployment/ec2/scripts/provision.sh) \
  --repo git@github.com:bervant/loreax-core.git \
  --branch master

What it does:

  1. Verifies/installs system dependencies (git, composer, bun, PHP 8.4)
  2. Clones repository to /var/www/loreax/
  3. Sets ownership to www-data:www-data
  4. Seeds .env from deployment/ec2/env/hyena.env.template
  5. Installs nginx site configs for all three subdomains
  6. Installs systemd units (Horizon + Scheduler)
  7. Enables and starts the scheduler timer
  8. Reloads nginx

Options:

  • --repo <url> — Git repository URL (required)
  • --branch <name> — Branch to checkout (default: master)
  • --app-dir <path> — Installation directory (default: /var/www/loreax)
  • --app-user <name> — App ownership user (default: www-data)

After provisioning:

  1. Edit /var/www/loreax/.env and fill in secrets
  2. Generate APP_KEY: php8.4 artisan key:generate
  3. Run first deployment: sudo /var/www/loreax/deployment/ec2/scripts/deploy.sh

deploy.sh — Deploy New Version

Purpose: Pull latest code, build assets, migrate database, reload services.
Run: Every deployment (manual or automated via CI/CD).
Requires: Sudo access.

sudo /var/www/loreax/deployment/ec2/scripts/deploy.sh [options]

What it does:

  1. Fetches latest commits from origin
  2. Checks out specified ref (or fast-forwards current branch)
  3. Reinstalls nginx configs and systemd units from working tree
  4. Installs Composer dependencies (--no-dev --optimize-autoloader)
  5. Builds frontend assets (bun run build)
  6. Generates OpenAPI spec (l5-swagger:generate)
  7. Enters maintenance mode with bypass secret
  8. Runs database migrations (--force)
  9. Warms caches (config, routes, events, views, Filament components)
  10. Links storage directory
  11. Exits maintenance mode
  12. Reloads PHP-FPM and restarts Horizon

Options:

  • --ref <git-ref> — Deploy specific commit/tag/branch (default: fast-forward current branch)
  • --skip-build — Skip Composer/npm installation and asset building
  • --skip-infra — Skip nginx/systemd reinstallation (config-only updates)

Examples:

# Deploy latest code from current branch
sudo /var/www/loreax/deployment/ec2/scripts/deploy.sh

# Deploy specific tag
sudo /var/www/loreax/deployment/ec2/scripts/deploy.sh --ref v1.5.0

# Deploy specific commit
sudo /var/www/loreax/deployment/ec2/scripts/deploy.sh --ref a1b2c3d

# Deploy without rebuilding (config-only changes)
sudo /var/www/loreax/deployment/ec2/scripts/deploy.sh --skip-build --skip-infra

Deployment downtime: ~30-60 seconds (maintenance mode window).

Bypass maintenance mode: During deployment, access the site with ?secret=<DEPLOY_SECRET> (generated randomly each deploy, printed to console).


rollback.sh — Revert to Previous Version

Purpose: Roll back to a previous commit or tag when deployment goes wrong.
Run: On-demand when issues are detected post-deployment.
Requires: Sudo access.

sudo /var/www/loreax/deployment/ec2/scripts/rollback.sh --to <sha-or-tag>

What it does:

  1. Checks out specified git ref (commit SHA or tag)
  2. Runs deploy.sh workflow (build + migrate + cache + reload)

Important notes:

  • Schema rollbacks are NOT automatic — run migrate:rollback manually if needed
  • Does NOT restore data — only code version
  • If migrations were added in the bad deploy, manually roll them back first

Examples:

# Rollback to previous commit
sudo /var/www/loreax/deployment/ec2/scripts/rollback.sh --to HEAD~1

# Rollback to specific tag
sudo /var/www/loreax/deployment/ec2/scripts/rollback.sh --to v1.4.2

# Rollback to commit SHA
sudo /var/www/loreax/deployment/ec2/scripts/rollback.sh --to a1b2c3d4e5f

# Check current deployed version
cd /var/www/loreax && git log --oneline | head -5

Rollback with migration revert:

# 1. Check migration status
sudo -u www-data php /var/www/loreax/artisan migrate:status

# 2. Rollback migrations added in bad deploy
sudo -u www-data php /var/www/loreax/artisan migrate:rollback --step=2

# 3. Rollback code
sudo /var/www/loreax/deployment/ec2/scripts/rollback.sh --to v1.4.2

maintenance.sh — Maintenance Mode Control

Purpose: Enable/disable maintenance mode with optional bypass secret.
Run: Before planned maintenance windows or during incident response.
Requires: Sudo access.

sudo /var/www/loreax/deployment/ec2/scripts/maintenance.sh on --secret <bypass-token>
sudo /var/www/loreax/deployment/ec2/scripts/maintenance.sh off

What it does:

  • ON: Puts application in maintenance mode (returns 503 to all requests)
  • OFF: Exits maintenance mode (resumes normal operation)

With bypass secret: Admins can access the site during maintenance with ?secret=<bypass-token> in URL.

Examples:

# Enable maintenance mode with bypass
sudo /var/www/loreax/deployment/ec2/scripts/maintenance.sh on --secret admin123

# Access site during maintenance
# https://api.loreax.bervant.co.ke/v1/me?secret=admin123

# Disable maintenance mode
sudo /var/www/loreax/deployment/ec2/scripts/maintenance.sh off

# Check current status
sudo -u www-data php /var/www/loreax/artisan down --show-status

Use cases:

  • Planned database maintenance
  • Long-running migrations (>1 minute)
  • Infrastructure updates (Redis/Postgres restarts)
  • Emergency incident response

Note: deploy.sh automatically enters/exits maintenance mode, so manual use is only needed for extended maintenance windows.


🔄 CI/CD Automation (GitHub Actions)

Loreax includes automated deployment to Hyena via GitHub Actions. Deployments are triggered automatically on pushes to master branch or can be triggered manually.

Workflow Configuration

File: .github/workflows/deploy-hyena.yml

Triggers:

  • Automatic: Push to master branch
  • Manual: Workflow dispatch (Actions tab in GitHub)

Environment: hyena
Target: EC2 35.178.52.158
Timeout: 20 minutes
Concurrency: Only one deployment at a time (cancel-in-progress: false)

Required GitHub Secrets

Configure these in Repository Settings → Secrets and variables → Actions:

Secret Name Description Example
CORE_SSH_KEY OpenSSH private key for deployment user -----BEGIN OPENSSH PRIVATE KEY-----...
CORE_IP EC2 host IP address 35.178.52.158

The deployment SSH user must have passwordless sudo access via /etc/sudoers.d/loreax-deploy:

# /etc/sudoers.d/loreax-deploy
deploy_user ALL=(ALL) NOPASSWD: /var/www/loreax/deployment/ec2/scripts/deploy.sh, \
                                 /usr/bin/install, /bin/ln, /usr/sbin/nginx, \
                                 /bin/systemctl, /usr/bin/sudo

Workflow Steps

When triggered, the GitHub Actions workflow:

  1. Resolves deploy ref — Uses manual input ref or current commit SHA
  2. Configures SSH — Loads private key from secrets
  3. Connects to EC2 — SSH as deployment user to 35.178.52.158
  4. Updates deploy script — Pulls latest master to get current scripts
  5. Runs deployment — Executes deploy.sh --ref <sha>
  6. Smoke tests — Verifies /health endpoint responds successfully

Deployment flow:

GitHub Push (master)
    ↓
GitHub Actions triggered
    ↓
SSH to EC2 (ubuntu@35.178.52.158)
    ↓
Pull latest code as www-data
    ↓
Run deploy.sh --ref <sha>
    ↓
Build assets → Migrate → Cache → Reload
    ↓
Smoke test /health endpoint
    ↓
✓ Deployment complete

Manual Deployment via GitHub UI

To manually deploy a specific version:

  1. Go to Actions tab in GitHub repository
  2. Select Deploy (hyena) workflow
  3. Click Run workflow button
  4. Enter optional git ref (branch, tag, or SHA)
  5. Click Run workflow

Examples:

  • Leave blank to deploy latest master
  • Enter v1.5.0 to deploy specific tag
  • Enter feature/new-feature to deploy from branch
  • Enter a1b2c3d4 to deploy specific commit

Monitoring Deployments

In GitHub Actions:

  • View workflow runs in Actions tab
  • Each step shows detailed logs
  • Failed deployments marked with ❌
  • Successful deployments marked with ✓

On the server:

# View deployment history
cd /var/www/loreax && git log --oneline | head -10

# Check currently deployed version
cd /var/www/loreax && git log --oneline -1

# View recent deployments in system logs
sudo journalctl -u loreax-horizon.service --since "1 hour ago" | grep -i "restart"

Troubleshooting CI/CD Deployments

Issue: SSH connection failed

# Check GitHub secrets are set correctly
- CORE_SSH_KEY must contain private key (not public key)
- CORE_IP must be 35.178.52.158

# Verify SSH key is added to server
cat ~/.ssh/authorized_keys  # on EC2 host

Issue: Permission denied (sudo)

# Verify sudoers entry exists
sudo cat /etc/sudoers.d/loreax-deploy

# Test sudo access
sudo -u deploy_user sudo /var/www/loreax/deployment/ec2/scripts/deploy.sh --help

Issue: Deployment times out

# Increase timeout in workflow (default 20 minutes)
timeout-minutes: 30

# Or optimize deployment:
# - Reduce asset build time
# - Skip unnecessary steps with --skip-build

Issue: Smoke test fails

# Check health endpoint manually
curl https://api.loreax.bervant.co.ke/health

# Check nginx is running
sudo systemctl status nginx

# Check PHP-FPM is running
sudo systemctl status php8.4-fpm

# Check application logs
sudo -u www-data tail -50 /var/www/loreax/storage/logs/laravel.log

Deployment Notifications

To add Slack/email notifications on deployment success/failure, update the workflow:

# Add at end of workflow
- name: Notify deployment status
  if: always()
  uses: 8398a7/action-slack@v3
  with:
    status: ${{ job.status }}
    webhook_url: ${{ secrets.SLACK_WEBHOOK_URL }}
    text: |
      Deployment to hyena: ${{ job.status }}
      Ref: ${{ steps.ref.outputs.value }}
      URL: https://api.loreax.bervant.co.ke

🚀 Alternative: Laravel Cloud (Managed Platform)

Laravel Cloud is the easiest way to deploy and scale Loreax in production without managing infrastructure.

Prerequisites

  • GitHub account with access to bervant/loreax-core
  • Laravel Cloud account (free trial available)
  • Valid credit card for production environment

Deploy in 5 Minutes

  1. Connect GitHub Repository

    # Visit: https://laravel.cloud
    # Click "New Project"
    # Select: bervant/loreax-core
    # Authorize GitHub access
    
  2. Configure Environment

    # Laravel Cloud dashboard → Environment Variables
    # Copy from your local .env:
    - APP_ENV=production
    - APP_DEBUG=false
    - APP_KEY=base64:xxxxx (must be set)
    - DB_HOST, DB_NAME, DB_USERNAME, DB_PASSWORD
    - REDIS_HOST, REDIS_PASSWORD
    - AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
    - MPESA_API_KEY, MPESA_CONSUMER_KEY
    # And all other services (SQS, SendGrid, etc.)
    
  3. Deploy

    # Push to main branch (or configured branch)
    git push origin main
    
    # Laravel Cloud automatically:
    # ✓ Runs migrations
    # ✓ Builds Docker image
    # ✓ Scales workers
    # ✓ Configures SSL/TLS
    # ✓ Sets up backups
    
  4. Verify

    # Check deployment status
    curl https://api.loreax.cloud/health
    # Expected: {"status":"operational","timestamp":"..."}
    
    # Access admin panel
    https://loreax.cloud/admin
    

Cost: Starting $99/month (includes PostgreSQL, Redis, Horizon workers, SSL, backups).


📋 Pre-Deployment Checklist

Before deploying to any environment, verify:

Code Quality

  • All tests pass: ./vendor/bin/phpunit
  • No linting issues: ./vendor/bin/pint --test
  • Static analysis passes: ./vendor/bin/phpstan analyse
  • Git history clean: git log --oneline | head -10

Secrets & Configuration

  • APP_KEY generated: php artisan key:generate
  • All env vars set (use .env.example as template)
  • No secrets in code or git history
  • Database migrations reviewed and tested
  • Feature flags configured for rollout strategy
  • Rate limits appropriate for production

Database & Infrastructure

  • PostgreSQL 16+ with automated backups
  • Redis 7+ for cache/session/queues with replication
  • MongoDB 7+ for request logs (optional but recommended)
  • AWS S3 buckets created for media with versioning
  • Email service configured (SendGrid recommended)
  • MPESA sandbox credentials tested

Security

  • HTTPS/TLS enforced (automatic with Laravel Cloud)
  • CORS properly configured for your domain
  • Rate limiting configured in config/rate_limits.php
  • Admin panel behind strong authentication
  • Database encryption at rest enabled
  • Log retention policy set (sensitive logs redacted)

Monitoring & Logging

  • Application error logging configured
  • Request logger pointing to MongoDB
  • Uptime monitoring set up (UptimeRobot, etc.)
  • Alerts configured for error rates >1%
  • Dashboard access for on-call team

🔧 Self-Hosted Deployment

For deployment outside Laravel Cloud (AWS EC2, DigitalOcean, etc.):

System Requirements

Server:

  • Ubuntu 22.04 LTS (or equivalent)
  • 2+ CPU cores
  • 4+ GB RAM (8+ for production)
  • 40+ GB SSD storage
  • Public IP with fixed domain

Services:

  • PostgreSQL 16+ (apt install postgresql-16)
  • Redis 7+ (apt install redis-server)
  • PHP 8.4 with extensions (apt install php8.4-{fpm,mysql,redis,bcmath,curl,gd,intl,mbstring,zip})
  • Nginx 1.24+ (apt install nginx)
  • Supervisor for queue workers (apt install supervisor)

Installation Steps

  1. Clone Repository

    sudo mkdir -p /srv/loreax
    cd /srv/loreax
    sudo git clone https://github.com/bervant/loreax-core.git .
    sudo chown -R www-data:www-data /srv/loreax
    
  2. Install Dependencies

    cd /srv/loreax
    sudo -u www-data composer install --no-dev --optimize-autoloader
    sudo -u www-data npm install
    npm run build
    
  3. Environment Setup

    sudo -u www-data cp .env.example .env
    sudo -u www-data php artisan key:generate
    # Edit .env with your production values
    sudo -u www-data chmod 600 .env
    
  4. Database Setup

    # Create PostgreSQL user and database
    sudo -u postgres createuser loreax_user -P
    sudo -u postgres createdb loreax_db -O loreax_user
    
    # Run migrations
    sudo -u www-data php artisan migrate --force
    sudo -u www-data php artisan db:seed --class=DatabaseSeeder
    
  5. Nginx Configuration

    # /etc/nginx/sites-available/loreax.conf
    server {
        listen 80;
        server_name api.loreax.cloud;
    
        root /srv/loreax/public;
        index index.php;
    
        # Redirect HTTP to HTTPS
        return 301 https://$server_name$request_uri;
    }
    
    server {
        listen 443 ssl http2;
        server_name api.loreax.cloud;
    
        root /srv/loreax/public;
        index index.php;
    
        ssl_certificate /etc/letsencrypt/live/api.loreax.cloud/fullchain.pem;
        ssl_certificate_key /etc/letsencrypt/live/api.loreax.cloud/privkey.pem;
    
        location / {
            try_files $uri $uri/ /index.php?$query_string;
        }
    
        location ~ \.php$ {
            fastcgi_pass unix:/run/php/php8.4-fpm.sock;
            fastcgi_index index.php;
            fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
            include fastcgi_params;
        }
    
        location ~ /\.ht {
            deny all;
        }
    }
    
  6. Enable Nginx & Reload

    sudo ln -s /etc/nginx/sites-available/loreax.conf /etc/nginx/sites-enabled/
    sudo nginx -t
    sudo systemctl reload nginx
    
  7. SSL Certificate (Let's Encrypt)

    sudo apt install certbot python3-certbot-nginx
    sudo certbot certonly --nginx -d api.loreax.cloud
    # Certificates auto-renew via cron
    
  8. Queue Workers (Supervisor)

    ; /etc/supervisor/conf.d/loreax.conf
    [program:loreax-worker]
    process_name=%(program_name)s_%(process_num)02d
    command=php /srv/loreax/artisan queue:work redis --sleep=3 --tries=3
    autostart=true
    autorestart=true
    numprocs=4
    user=www-data
    
    sudo supervisorctl reread
    sudo supervisorctl update
    sudo supervisorctl start loreax-worker:*
    
  9. Cron Jobs

    # /etc/cron.d/loreax
    * * * * * www-data cd /srv/loreax && php artisan schedule:run >> /dev/null 2>&1
    
  10. SSL Auto-Renewal (Certbot)

    sudo systemctl enable certbot.timer
    sudo systemctl start certbot.timer
    

🔄 Continuous Deployment Pipeline

GitHub Actions (Automated Testing Before Merge)

# .github/workflows/tests.yml
name: Tests & Quality
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    
    services:
      postgres:
        image: postgres:16
        env:
          POSTGRES_PASSWORD: password
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
    
    steps:
      - uses: actions/checkout@v3
      
      - name: Setup PHP
        uses: shivammathur/setup-php@v2
        with:
          php-version: '8.4'
          extensions: curl,gd,intl,mbstring,zip,redis
          coverage: xdebug
      
      - name: Install Dependencies
        run: composer install --no-interaction
      
      - name: Run Tests
        env:
          DB_CONNECTION: pgsql
          DB_HOST: localhost
          DB_DATABASE: loreax_test
          DB_USERNAME: postgres
          DB_PASSWORD: password
        run: ./vendor/bin/phpunit
      
      - name: Code Style (Pint)
        run: ./vendor/bin/pint --test
      
      - name: Static Analysis (Larastan)
        run: ./vendor/bin/phpstan analyse
      
      - name: Upload Coverage
        uses: codecov/codecov-action@v3
        if: always()

Staging Deployment

# After PR merge to `develop` branch
# Automatically deploys to: https://staging-api.loreax.cloud

# Trigger with:
git push origin develop

Production Deployment

# After PR merge to `main` branch (requires approval)
# Automatically deploys to: https://api.loreax.cloud

# Trigger with:
git push origin main

# To rollback:
git revert HEAD
git push origin main

🔍 Post-Deployment Verification

After deploying, verify everything is working:

Health Checks

# Application health
curl https://api.loreax.cloud/health
# Expected: {"status":"operational"}

# Database connectivity
curl https://api.loreax.cloud/ready
# Expected: {"database":"ready","cache":"ready",...}

# API endpoint (requires auth)
curl -X GET https://api.loreax.cloud/api/v1/me \
  -H "Authorization: Bearer <token>"
# Expected: 200 with user data

# Admin panel
curl -I https://api.loreax.cloud/admin
# Expected: 302 redirect (not authenticated yet)

Monitoring

  1. Error Rates

    # Check Laravel logs for errors
    tail -f storage/logs/laravel.log
    # Should be <1% of total requests
    
  2. Response Times

    # Check request logger (MongoDB)
    # Expected median: <200ms
    # Expected p99: <1000ms
    
  3. Queue Depth

    # Check pending jobs
    php artisan queue:pending
    # Should be near 0 (workers keeping up)
    
  4. Database Connections

    # Monitor PostgreSQL connections
    SELECT count(*) FROM pg_stat_activity;
    # Should be <20 (connection pooling working)
    

📊 Scaling Strategy

Horizontal Scaling (Multiple Servers)

Load Balancer (nginx or AWS ALB)

                    ┌─────────────────┐
                    │  Load Balancer  │
                    │   (sticky       │
                    │   sessions)     │
                    └────────┬────────┘
                  ┌─────────────────────┐
        ┌─────────┴──────────┬──────────┴──────────┐
        │                    │                     │
    ┌───▼─────┐          ┌───▼─────┐         ┌────▼────┐
    │ Server1 │          │ Server2 │         │ Server3 │
    │  (App)  │          │  (App)  │         │  (App)  │
    └───┬─────┘          └───┬─────┘         └────┬────┘
        │                    │                     │
        └────────┬───────────┴──────────┬──────────┘
                 │ Database Connection  │
            ┌────▼─────────────────────▼────┐
            │     PostgreSQL (Primary)       │
            │     Backup: Replication        │
            └────────────────────────────────┘

Queue Workers Distribution

# Spread workers across servers
# Server 1: 4 workers on redis queue
# Server 2: 4 workers on redis queue  
# Server 3: 4 workers on redis queue
# Total: 12 concurrent job processors

# Configure in /etc/supervisor/conf.d/loreax.conf
numprocs=4  # per server

Session Storage

// config/session.php
'driver' => 'redis',  // Use Redis, not files
'connection' => 'default',

Vertical Scaling (Bigger Servers)

Incremental resource increases:

  1. RAM: 4GB → 8GB → 16GB → 32GB
  2. CPU: 2 cores → 4 cores → 8 cores → 16 cores
  3. Storage: Monitor with df -h and expand as needed

🚨 Emergency Procedures

Rollback to Previous Version

# If deployment causes issues:
git revert HEAD
git push origin main

# Laravel Cloud auto-detects and deploys previous version
# (Or manually via dashboard)

# Monitor logs:
tail -f storage/logs/laravel.log

Database Disaster Recovery

# If database is corrupted:
# 1. Restore from automated backup (daily snapshots)
# 2. Run migrations fresh
php artisan migrate:fresh --seed

# 3. Alert team to data loss window
# 4. Monitor ledger integrity
php artisan ledger:audit

Queue Backlog

# If jobs are piling up:
# 1. Add more workers
supervisorctl add loreax-worker:*

# 2. Or restart queue (resets partial jobs)
php artisan queue:flush

# 3. Monitor with:
php artisan queue:pending

📚 Additional Resources


✅ Deployment Checklist

Use this checklist for each Hyena (staging) or production deployment:

PRE-DEPLOYMENT (Hyena/EC2)
━━━━━━━━━━━━━━━━━━━━━━━━
☐ Code review approved by 2+ engineers
☐ All tests passing (./vendor/bin/phpunit)
☐ Code style verified (./vendor/bin/pint --test)
☐ Static analysis passing (./vendor/bin/phpstan analyse)
☐ Migrations tested locally/staging
☐ Feature flags configured for rollout
☐ .env secrets verified (APP_KEY, DB_PASSWORD, AWS keys, etc.)
☐ Rate limits appropriate (config/rate_limits.php)
☐ Database backup confirmed (pg_dump scheduled)
☐ Monitoring alerts configured

HYENA DEPLOYMENT
━━━━━━━━━━━━━━━━
☐ Branch pushed to GitHub (triggers CI if configured)
☐ SSH access to 35.178.52.158 verified
☐ Notify team via Slack/email
☐ Check current version: cd /var/www/loreax && git log --oneline | head -1
☐ Run deployment: sudo /var/www/loreax/deployment/ec2/scripts/deploy.sh
☐ Monitor deployment output for errors
☐ Deployment completes successfully (exit code 0)

VERIFICATION (First 5 Minutes)
━━━━━━━━━━━━━━━━━━━━━━━━━━━
☐ Health check: curl https://api.loreax.bervant.co.ke/health
☐ Readiness check: curl https://api.loreax.bervant.co.ke/ready
☐ Admin panel accessible: https://admin.loreax.bervant.co.ke/admin
☐ Developer portal: https://dev.loreax.bervant.co.ke/wiki
☐ OpenAPI spec: https://dev.loreax.bervant.co.ke/api-reference
☐ Monitor error logs: sudo tail -f /var/log/nginx/loreax-*.error.log
☐ Monitor Laravel logs: sudo -u www-data tail -f /var/www/loreax/storage/logs/laravel.log
☐ Check Horizon: sudo systemctl status loreax-horizon.service
☐ Check queue depth: sudo -u www-data php /var/www/loreax/artisan horizon:status
☐ Check database connections: no errors in Laravel logs

POST-DEPLOYMENT (30 Minutes)
━━━━━━━━━━━━━━━━━━━━━━━━━━━
☐ All health checks passing consistently
☐ No error spikes in logs
☐ Response times normal (<500ms p99)
☐ Feature flags working as expected
☐ Database integrity verified: php artisan ledger:audit (if financial changes)
☐ Queue backlog cleared (workers processing normally)
☐ TLS certificates valid: curl -vI https://api.loreax.bervant.co.ke 2>&1 | grep -i "expire"
☐ Update deployment log with SHA and timestamp
☐ Notify stakeholders of successful deployment

ROLLBACK (If Issues Detected)
━━━━━━━━━━━━━━━━━━━━━━━━━━━
☐ Identify last known good version (git SHA or tag)
☐ Check if migrations need reverting: php artisan migrate:status
☐ Run rollback: sudo /var/www/loreax/deployment/ec2/scripts/rollback.sh --to <sha>
☐ Verify rollback successful
☐ Monitor logs to confirm issue resolved
☐ Document incident and root cause
☐ Plan fix for next deployment

📚 Quick Reference

Common deployment scenarios:

# Standard deployment (current branch)
sudo /var/www/loreax/deployment/ec2/scripts/deploy.sh

# Deploy specific version
sudo /var/www/loreax/deployment/ec2/scripts/deploy.sh --ref v1.5.0

# Emergency rollback
sudo /var/www/loreax/deployment/ec2/scripts/rollback.sh --to HEAD~1

# Planned maintenance window
sudo /var/www/loreax/deployment/ec2/scripts/maintenance.sh on --secret admin123
# ... perform maintenance ...
sudo /var/www/loreax/deployment/ec2/scripts/maintenance.sh off

Health check URLs:

  • API: https://api.loreax.bervant.co.ke/health
  • Readiness: https://api.loreax.bervant.co.ke/ready
  • Admin: https://admin.loreax.bervant.co.ke/admin
  • Dev Portal: https://dev.loreax.bervant.co.ke/wiki
  • OpenAPI: https://dev.loreax.bervant.co.ke/api-reference

Log locations:

  • Nginx: /var/log/nginx/loreax-{api,admin,dev}.{access,error}.log
  • Laravel: /var/www/loreax/storage/logs/laravel.log
  • Horizon: sudo journalctl -u loreax-horizon.service -f

Service management:

  • Horizon: sudo systemctl {status|restart|stop} loreax-horizon.service
  • Scheduler: sudo systemctl {status|start} loreax-scheduler.service
  • PHP-FPM: sudo systemctl {status|reload} php8.4-fpm
  • Nginx: sudo systemctl {status|reload} nginx

Ready to deploy? For Hyena (staging), follow the Hyena Environment guide above. For future production, see Kubernetes Deployment. 🚀