diff --git a/cline_docs/activeContext.md b/cline_docs/activeContext.md index 2d8fc3f8..828ca8c0 100644 --- a/cline_docs/activeContext.md +++ b/cline_docs/activeContext.md @@ -1,11 +1,245 @@ # Active Context - Current Work Status -## Current Task: PDF DOWNLOAD ISSUE FULLY RESOLVED ✅ +## Current Task: PRODUCTION SERVER MIGRATION COMPLETED ✅ +**MAD Lawsuit Website Successfully Migrated to New Infrastructure**: +- **Old Server**: chrishaulmark.com (DigitalOcean VPS - being decommissioned) +- **New Server**: 10.4.0.205 (Dedicated server behind NAT) +- **Status**: FULLY OPERATIONAL WITH ALL DATA MIGRATED ✅ + +### Migration Implementation Details +**Infrastructure Changes**: +1. **Package Upgrades** ✅ COMPLETED + - **Frontend**: React 18→19, Next.js 15→16, Tailwind 3→4 + - **Backend**: Prisma 5→7, Express 4→5 + - **All Dependencies**: Updated to latest stable versions + - **Docker Images**: Rebuilt with new package versions + +2. **Caddy Reverse Proxy Fix** ✅ COMPLETED + - **Problem**: Docker DNS resolving container hostnames to wrong network IPs + - **Impact**: Caddy timing out trying to reach containers + - **Solution**: Updated Caddyfile to use direct IP addresses on caddy_network + - **Frontend IP**: 172.18.0.6:806 (instead of hostname) + - **Backend IP**: 172.18.0.5:901 (instead of hostname) + - **File**: `/home/chaulmark/docker/caddy/config/Caddyfile` on server + +3. **Data Migration** ✅ COMPLETED + - **Database**: PostgreSQL data (47MB) migrated from old server + - **PDF Files**: 158MB of court documents (69 files) migrated + - **Redis Cache**: Cache data migrated + - **Method**: Created tar archive, transferred via rsync, extracted on new server + - **Location**: `/docker/websites/mad-lawsuit/` on 10.4.0.205 + +4. **SSL Certificates** ✅ COMPLETED + - **Provider**: Let's Encrypt via Caddy + - **Domains**: mad-lawsuit.org, files.mad-lawsuit.org + - **Status**: Active and auto-renewing + +### Final Production Architecture +``` +┌─────────────────────────────────────────────────────────┐ +│ Caddy Reverse Proxy (10.4.0.205) │ +│ │ +│ mad-lawsuit.org → 172.18.0.6:806 (Frontend) │ +│ files.mad-lawsuit.org → 172.18.0.5:901 (Backend) │ +└─────────────────────────────────────────────────────────┘ + │ + ├─────────────────────────────┐ + │ │ + ┌──────▼──────┐ ┌──────▼──────┐ + │ Frontend │ │ Backend │ + │ Port 806 │──────────────│ Port 901 │ + │ Next.js 16 │ API Calls │ Express 5 │ + │ React 19 │ │ Prisma 7 │ + └─────────────┘ └─────────────┘ + │ + ┌──────▼──────┐ + │ PostgreSQL │ + │ + Redis │ + └─────────────┘ +``` + +### Deployment Server Information +**IMPORTANT**: All deployments now go to **10.4.0.205** (NOT chrishaulmark.com) + +**Server Details**: +- **Hostname**: public-websites (internal: 10.4.0.205) +- **External Access**: Via NAT through dedicated server +- **SSH Access**: `ssh chaulmark@10.4.0.205` +- **Project Location**: `~/websites/mad-lawsuit.org/` +- **Docker Volumes**: `/docker/websites/mad-lawsuit/` + +**Deployment Commands**: +```bash +# SSH to production server +ssh chaulmark@10.4.0.205 + +# Navigate to project +cd ~/websites/mad-lawsuit.org + +# Pull latest code +git pull + +# Rebuild and restart containers +docker compose down +docker compose build +docker compose up -d + +# Check container status +docker compose ps +docker compose logs -f +``` + +### Verification ✅ +- **Website**: https://mad-lawsuit.org (HTTP/2 200 ✓) +- **Backend API**: https://files.mad-lawsuit.org (HTTP/2 200 ✓) +- **Database**: 63 docket entries with all historical data ✓ +- **PDF Files**: 69 court documents accessible ✓ +- **All Containers**: Running and healthy ✓ + +### Git Commits Made +- Migration completed with all package upgrades +- Caddyfile updated with direct IP addresses +- All changes committed to repository + +## Previous Task: CACHE CONTROL HEADERS ADDED ✅ +**MAD Lawsuit Website Cache Prevention Implemented**: +- **Problem**: HTTP response caching causing stale 404 responses for PDF downloads +- **Root Cause**: Caddy/CDN caching API responses, persisting even after fixes deployed +- **Solution**: Added Cache-Control headers to prevent future caching +- **Status**: CACHE HEADERS ACTIVE, LEGACY CACHE EXPIRING ⏳ + +### Cache Control Implementation Details +**Problem Identified**: +- After implementing dedicated backend subdomain, some PDF downloads still returned 404 +- Testing revealed cached responses from before the fix was deployed +- Cache persisted through multiple Caddy restarts and data directory clearing +- Indicates upstream CDN or aggressive HTTP caching layer + +**Solution Implemented** ✅: +1. **Added Cache-Control Headers to Caddyfile** + - **Location**: `mad-lawsuit.org` block in Caddyfile + - **Headers**: `Cache-Control: no-store, no-cache, must-revalidate` + - **Scope**: All `/api/*` routes + - **Purpose**: Prevent new caches from forming + +2. **Caddyfile Update**: +```caddy +mad-lawsuit.org { + # Disable caching for API routes to prevent stale responses + header /api/* { + Cache-Control "no-store, no-cache, must-revalidate" + } + reverse_proxy 172.18.0.1:806 +} +``` + +3. **Deployment Process**: + - Updated local Caddyfile + - SCP'd to server: `~/docker/caddy/config/Caddyfile` + - Restarted Caddy: `docker compose -f caddy-compose.yml down && up -d` + - Verified headers: `cache-control: no-store, no-cache, must-revalidate` ✅ + +**Current Status**: +- ✅ Cache-Control headers active and being sent +- ✅ API endpoint working: `https://mad-lawsuit.org/api/docket-entries` returns 63 entries +- ✅ Direct backend working: All PDFs accessible via `https://files.mad-lawsuit.org` +- ⏳ Legacy cached 404s: Documents 4, 11, 13 still have cached responses (will expire naturally) +- ✅ New requests: Will NOT be cached due to Cache-Control headers + +**Workaround for Users**: +- Use direct backend URL for immediate access: `https://files.mad-lawsuit.org/api/documents/{id}/download` +- Frontend proxy URLs will work correctly once legacy cache expires (typically 24 hours) + +## Previous Task: DEDICATED BACKEND SUBDOMAIN IMPLEMENTED ✅ +**MAD Lawsuit Website Backend Moved to files.mad-lawsuit.org**: +- **Problem**: PDF downloads unreliable through frontend proxy tunnel +- **Root Cause**: Frontend proxy (Next.js) creating instability for file downloads +- **Solution**: Created dedicated backend subdomain with direct Caddy routing +- **Status**: BACKEND SUBDOMAIN FULLY OPERATIONAL ✅ + +### Dedicated Backend Subdomain Implementation Details +**Architecture Changes**: +1. **Backend Port Change** ✅ COMPLETED + - **Changed**: Backend port from 809 → 901 (9xx range for backends) + - **Reason**: Follow port structure convention (8xx for frontends, 9xx for backends) + - **File**: `docker-compose.yml` + +2. **New Subdomain Configuration** ✅ COMPLETED + - **Created**: `files.mad-lawsuit.org` subdomain for backend API + - **DNS**: Already resolving to same IP as mad-lawsuit.org + - **Caddy**: Direct reverse proxy to backend on port 901 + - **File**: `Caddyfile` + +3. **Simplified Frontend Routing** ✅ COMPLETED + - **Changed**: `mad-lawsuit.org` now only proxies to frontend (port 806) + - **Removed**: Complex path-based routing (/api/* handling) + - **Result**: Cleaner, more reliable frontend routing + +4. **Frontend API Proxy Update** ✅ COMPLETED + - **Changed**: Frontend now calls `https://files.mad-lawsuit.org/api/*` + - **Previous**: Called internal Docker container `http://mad-lawsuit-backend-1:3001` + - **Benefit**: Direct backend access, no proxy tunnel issues + - **File**: `frontend/src/app/api/[...path]/route.ts` + +5. **Backend CORS Update** ✅ COMPLETED + - **Added**: Both `mad-lawsuit.org` and `files.mad-lawsuit.org` to allowed origins + - **Reason**: Backend needs to accept requests from both domains + - **File**: `backend/src/index.ts` + +### Final Architecture +``` +┌─────────────────────────────────────────────────────────┐ +│ Caddy Reverse Proxy │ +│ │ +│ mad-lawsuit.org → 172.18.0.1:806 (Frontend) │ +│ files.mad-lawsuit.org → 172.18.0.1:901 (Backend) │ +└─────────────────────────────────────────────────────────┘ + │ + ├─────────────────────────────┐ + │ │ + ┌──────▼──────┐ ┌──────▼──────┐ + │ Frontend │ │ Backend │ + │ Port 806 │──────────────│ Port 901 │ + │ Next.js │ API Calls │ Express.js │ + └─────────────┘ └─────────────┘ +``` + +### Benefits of New Architecture +1. **Reliability**: Direct backend access eliminates proxy tunnel issues +2. **Stability**: Backend isolated from frontend routing problems +3. **Performance**: Reduced latency without frontend proxy overhead +4. **Maintainability**: Cleaner separation of concerns +5. **Scalability**: Can scale frontend/backend independently +6. **Debugging**: Easier to troubleshoot backend-specific issues + +### Deployment Status ✅ +- **Code Changes**: Committed to Git (commit a2595246) +- **Caddyfile**: Uploaded and reloaded on server +- **Containers**: Rebuilt and restarted with new configuration +- **Verification**: Both domains responding correctly + - `https://mad-lawsuit.org` → Frontend (HTTP 200) + - `https://files.mad-lawsuit.org/api/health` → Backend (HTTP 200) + +### Git Commit +``` +commit a2595246 +Implement dedicated backend subdomain files.mad-lawsuit.org + +- Changed backend port from 809 to 901 (9xx range for backends) +- Updated docker-compose.yml to expose backend on port 901 +- Created new Caddyfile with files.mad-lawsuit.org subdomain +- Simplified mad-lawsuit.org to only proxy to frontend (port 806) +- Updated frontend API proxy to use https://files.mad-lawsuit.org +- Updated backend CORS to allow both mad-lawsuit.org and files.mad-lawsuit.org +- This fixes PDF download reliability issues by providing direct backend access +``` + +## Previous Task: PDF DOWNLOAD ISSUE FULLY RESOLVED ✅ **MAD Lawsuit Website PDF Download Fixed via Direct Backend Routing**: - **Problem**: PDF downloads failing with "Endpoint not found" error - **Root Cause**: Caddy was routing ALL requests (including /api/*) to frontend, frontend's Next.js proxy couldn't properly handle the requests - **Solution**: Exposed backend on port 809 and configured Caddy to route /api/* directly to backend -- **Status**: PDF DOWNLOADS WORKING ✅ +- **Status**: PDF DOWNLOADS WORKING ✅ (Now superseded by dedicated subdomain) ### PDF Download Fix Implementation Details **Issues Resolved**: diff --git a/cline_docs/techContext.md b/cline_docs/techContext.md index e8107488..c257a669 100644 --- a/cline_docs/techContext.md +++ b/cline_docs/techContext.md @@ -3,17 +3,17 @@ ## Technology Stack - PRODUCTION READY ### Frontend Stack -- **Framework**: Next.js 15.3.4 (React 18.3.1) +- **Framework**: Next.js 16.0.1 (React 19.0.0) - **Language**: TypeScript 5.8.3 -- **Styling**: Tailwind CSS 3.4.17 with custom DeafGain theme +- **Styling**: Tailwind CSS 4.0.0 with custom DeafGain theme - **Icons**: Lucide React for modern iconography - **Package Manager**: pnpm for efficient dependency management - **Build**: Next.js optimized production builds ### Backend Stack - **Runtime**: Node.js with TypeScript 5.8.3 -- **Framework**: Express.js 4.21.2 -- **Database ORM**: Prisma 5.22.0 +- **Framework**: Express.js 5.0.1 +- **Database ORM**: Prisma 7.0.0 - **Authentication**: JWT with bcrypt password hashing - **File Upload**: Multer with UUID naming strategy - **Email**: Nodemailer with Gmail SMTP integration @@ -271,11 +271,52 @@ GOOGLE_APP_PASSWORD=ojvlysraxwjriwzy - **Database**: PostgreSQL container on port 5432 - **Cache**: Redis container on port 6379 -### Production Readiness +### Production Environment +**IMPORTANT**: Production server is **10.4.0.205** (NOT chrishaulmark.com) + +- **Server**: 10.4.0.205 (public-websites VM behind NAT) +- **SSH Access**: `ssh chaulmark@10.4.0.205` +- **Project Location**: `~/websites/mad-lawsuit.org/` +- **Docker Volumes**: `/docker/websites/mad-lawsuit/` +- **Frontend Port**: 806 (internal Docker network) +- **Backend Port**: 901 (internal Docker network) +- **Reverse Proxy**: Caddy on caddy_network + - Frontend IP: 172.18.0.6:806 + - Backend IP: 172.18.0.5:901 + +### Production URLs +- **Public Website**: https://mad-lawsuit.org +- **Backend API**: https://files.mad-lawsuit.org +- **SSL Certificates**: Let's Encrypt via Caddy (auto-renewing) + +### Production Deployment Process +```bash +# SSH to production server +ssh chaulmark@10.4.0.205 + +# Navigate to project +cd ~/websites/mad-lawsuit.org + +# Pull latest code +git pull + +# Rebuild and restart containers +docker compose down +docker compose build +docker compose up -d + +# Check container status +docker compose ps +docker compose logs -f +``` + +### Production Infrastructure - **Docker Compose**: Multi-service container setup - **Environment Variables**: Secure configuration management - **Database Migrations**: Automated schema updates -- **File Storage**: Organized upload directory structure +- **File Storage**: `/docker/websites/mad-lawsuit/uploads/` (158MB of PDFs) +- **Database**: `/docker/websites/mad-lawsuit/postgres_data/` (47MB) +- **Redis Cache**: `/docker/websites/mad-lawsuit/redis_data/` - **Logging**: Comprehensive request and error logging ## Current System Status