# Active Context - Current Work Status

## Current Task: PRODUCTION SERVER MIGRATION COMPLETED ✅
**MAD Lawsuit Website Successfully Migrated to New Infrastructure**:
- **Old Server**: chrishaulmark.com (DigitalOcean VPS - being decommissioned)
- **New Server**: 10.4.0.205 (Dedicated server behind NAT)
- **Status**: FULLY OPERATIONAL WITH ALL DATA MIGRATED ✅

### Migration Implementation Details
**Infrastructure Changes**:
1. **Package Upgrades** ✅ COMPLETED
   - **Frontend**: React 18→19, Next.js 15→16, Tailwind 3→4
   - **Backend**: Prisma 5→7, Express 4→5
   - **All Dependencies**: Updated to latest stable versions
   - **Docker Images**: Rebuilt with new package versions

2. **Caddy Reverse Proxy Fix** ✅ COMPLETED
   - **Problem**: Docker DNS resolving container hostnames to wrong network IPs
   - **Impact**: Caddy timing out trying to reach containers
   - **Solution**: Updated Caddyfile to use direct IP addresses on caddy_network
   - **Frontend IP**: 172.18.0.6:806 (instead of hostname)
   - **Backend IP**: 172.18.0.5:901 (instead of hostname)
   - **File**: `/home/chaulmark/docker/caddy/config/Caddyfile` on server

3. **Data Migration** ✅ COMPLETED
   - **Database**: PostgreSQL data (47MB) migrated from old server
   - **PDF Files**: 158MB of court documents (69 files) migrated
   - **Redis Cache**: Cache data migrated
   - **Method**: Created tar archive, transferred via rsync, extracted on new server
   - **Location**: `/docker/websites/mad-lawsuit/` on 10.4.0.205

4. **SSL Certificates** ✅ COMPLETED
   - **Provider**: Let's Encrypt via Caddy
   - **Domains**: mad-lawsuit.org, files.mad-lawsuit.org
   - **Status**: Active and auto-renewing

### Final Production Architecture
```
┌─────────────────────────────────────────────────────────┐
│              Caddy Reverse Proxy (10.4.0.205)          │
│                                                         │
│  mad-lawsuit.org → 172.18.0.6:806 (Frontend)          │
│  files.mad-lawsuit.org → 172.18.0.5:901 (Backend)     │
└─────────────────────────────────────────────────────────┘
                           │
                           ├─────────────────────────────┐
                           │                             │
                    ┌──────▼──────┐              ┌──────▼──────┐
                    │  Frontend   │              │   Backend   │
                    │  Port 806   │──────────────│   Port 901  │
                    │  Next.js 16 │   API Calls  │  Express 5  │
                    │  React 19   │              │  Prisma 7   │
                    └─────────────┘              └─────────────┘
                                                        │
                                                 ┌──────▼──────┐
                                                 │  PostgreSQL │
                                                 │  + Redis    │
                                                 └─────────────┘
```

### Deployment Server Information
**IMPORTANT**: All deployments now go to **10.4.0.205** (NOT chrishaulmark.com)

**Server Details**:
- **Hostname**: public-websites (internal: 10.4.0.205)
- **External Access**: Via NAT through dedicated server
- **SSH Access**: `ssh chaulmark@10.4.0.205`
- **Project Location**: `~/websites/mad-lawsuit.org/`
- **Docker Volumes**: `/docker/websites/mad-lawsuit/`

**Deployment Commands**:
```bash
# SSH to production server
ssh chaulmark@10.4.0.205

# Navigate to project
cd ~/websites/mad-lawsuit.org

# Pull latest code
git pull

# Rebuild and restart containers
docker compose down
docker compose build
docker compose up -d

# Check container status
docker compose ps
docker compose logs -f
```

### Verification ✅
- **Website**: https://mad-lawsuit.org (HTTP/2 200 ✓)
- **Backend API**: https://files.mad-lawsuit.org (HTTP/2 200 ✓)
- **Database**: 63 docket entries with all historical data ✓
- **PDF Files**: 69 court documents accessible ✓
- **All Containers**: Running and healthy ✓

### Git Commits Made
- Migration completed with all package upgrades
- Caddyfile updated with direct IP addresses
- All changes committed to repository

## Previous Task: CACHE CONTROL HEADERS ADDED ✅
**MAD Lawsuit Website Cache Prevention Implemented**:
- **Problem**: HTTP response caching causing stale 404 responses for PDF downloads
- **Root Cause**: Caddy/CDN caching API responses, persisting even after fixes deployed
- **Solution**: Added Cache-Control headers to prevent future caching
- **Status**: CACHE HEADERS ACTIVE, LEGACY CACHE EXPIRING ⏳

### Cache Control Implementation Details
**Problem Identified**:
- After implementing dedicated backend subdomain, some PDF downloads still returned 404
- Testing revealed cached responses from before the fix was deployed
- Cache persisted through multiple Caddy restarts and data directory clearing
- Indicates upstream CDN or aggressive HTTP caching layer

**Solution Implemented** ✅:
1. **Added Cache-Control Headers to Caddyfile**
   - **Location**: `mad-lawsuit.org` block in Caddyfile
   - **Headers**: `Cache-Control: no-store, no-cache, must-revalidate`
   - **Scope**: All `/api/*` routes
   - **Purpose**: Prevent new caches from forming

2. **Caddyfile Update**:
```caddy
mad-lawsuit.org {
    # Disable caching for API routes to prevent stale responses
    header /api/* {
        Cache-Control "no-store, no-cache, must-revalidate"
    }
    reverse_proxy 172.18.0.1:806
}
```

3. **Deployment Process**:
   - Updated local Caddyfile
   - SCP'd to server: `~/docker/caddy/config/Caddyfile`
   - Restarted Caddy: `docker compose -f caddy-compose.yml down && up -d`
   - Verified headers: `cache-control: no-store, no-cache, must-revalidate` ✅

**Current Status**:
- ✅ Cache-Control headers active and being sent
- ✅ API endpoint working: `https://mad-lawsuit.org/api/docket-entries` returns 63 entries
- ✅ Direct backend working: All PDFs accessible via `https://files.mad-lawsuit.org`
- ⏳ Legacy cached 404s: Documents 4, 11, 13 still have cached responses (will expire naturally)
- ✅ New requests: Will NOT be cached due to Cache-Control headers

**Workaround for Users**:
- Use direct backend URL for immediate access: `https://files.mad-lawsuit.org/api/documents/{id}/download`
- Frontend proxy URLs will work correctly once legacy cache expires (typically 24 hours)

## Previous Task: DEDICATED BACKEND SUBDOMAIN IMPLEMENTED ✅
**MAD Lawsuit Website Backend Moved to files.mad-lawsuit.org**:
- **Problem**: PDF downloads unreliable through frontend proxy tunnel
- **Root Cause**: Frontend proxy (Next.js) creating instability for file downloads
- **Solution**: Created dedicated backend subdomain with direct Caddy routing
- **Status**: BACKEND SUBDOMAIN FULLY OPERATIONAL ✅

### Dedicated Backend Subdomain Implementation Details
**Architecture Changes**:
1. **Backend Port Change** ✅ COMPLETED
   - **Changed**: Backend port from 809 → 901 (9xx range for backends)
   - **Reason**: Follow port structure convention (8xx for frontends, 9xx for backends)
   - **File**: `docker-compose.yml`

2. **New Subdomain Configuration** ✅ COMPLETED
   - **Created**: `files.mad-lawsuit.org` subdomain for backend API
   - **DNS**: Already resolving to same IP as mad-lawsuit.org
   - **Caddy**: Direct reverse proxy to backend on port 901
   - **File**: `Caddyfile`

3. **Simplified Frontend Routing** ✅ COMPLETED
   - **Changed**: `mad-lawsuit.org` now only proxies to frontend (port 806)
   - **Removed**: Complex path-based routing (/api/* handling)
   - **Result**: Cleaner, more reliable frontend routing

4. **Frontend API Proxy Update** ✅ COMPLETED
   - **Changed**: Frontend now calls `https://files.mad-lawsuit.org/api/*`
   - **Previous**: Called internal Docker container `http://mad-lawsuit-backend-1:3001`
   - **Benefit**: Direct backend access, no proxy tunnel issues
   - **File**: `frontend/src/app/api/[...path]/route.ts`

5. **Backend CORS Update** ✅ COMPLETED
   - **Added**: Both `mad-lawsuit.org` and `files.mad-lawsuit.org` to allowed origins
   - **Reason**: Backend needs to accept requests from both domains
   - **File**: `backend/src/index.ts`

### Final Architecture
```
┌─────────────────────────────────────────────────────────┐
│                    Caddy Reverse Proxy                  │
│                                                         │
│  mad-lawsuit.org → 172.18.0.1:806 (Frontend)          │
│  files.mad-lawsuit.org → 172.18.0.1:901 (Backend)     │
└─────────────────────────────────────────────────────────┘
                           │
                           ├─────────────────────────────┐
                           │                             │
                    ┌──────▼──────┐              ┌──────▼──────┐
                    │  Frontend   │              │   Backend   │
                    │  Port 806   │──────────────│   Port 901  │
                    │  Next.js    │   API Calls  │  Express.js │
                    └─────────────┘              └─────────────┘
```

### Benefits of New Architecture
1. **Reliability**: Direct backend access eliminates proxy tunnel issues
2. **Stability**: Backend isolated from frontend routing problems
3. **Performance**: Reduced latency without frontend proxy overhead
4. **Maintainability**: Cleaner separation of concerns
5. **Scalability**: Can scale frontend/backend independently
6. **Debugging**: Easier to troubleshoot backend-specific issues

### Deployment Status ✅
- **Code Changes**: Committed to Git (commit a2595246)
- **Caddyfile**: Uploaded and reloaded on server
- **Containers**: Rebuilt and restarted with new configuration
- **Verification**: Both domains responding correctly
  - `https://mad-lawsuit.org` → Frontend (HTTP 200)
  - `https://files.mad-lawsuit.org/api/health` → Backend (HTTP 200)

### Git Commit
```
commit a2595246
Implement dedicated backend subdomain files.mad-lawsuit.org

- Changed backend port from 809 to 901 (9xx range for backends)
- Updated docker-compose.yml to expose backend on port 901
- Created new Caddyfile with files.mad-lawsuit.org subdomain
- Simplified mad-lawsuit.org to only proxy to frontend (port 806)
- Updated frontend API proxy to use https://files.mad-lawsuit.org
- Updated backend CORS to allow both mad-lawsuit.org and files.mad-lawsuit.org
- This fixes PDF download reliability issues by providing direct backend access
```

## Previous Task: PDF DOWNLOAD ISSUE FULLY RESOLVED ✅
**MAD Lawsuit Website PDF Download Fixed via Direct Backend Routing**:
- **Problem**: PDF downloads failing with "Endpoint not found" error
- **Root Cause**: Caddy was routing ALL requests (including /api/*) to frontend, frontend's Next.js proxy couldn't properly handle the requests
- **Solution**: Exposed backend on port 809 and configured Caddy to route /api/* directly to backend
- **Status**: PDF DOWNLOADS WORKING ✅ (Now superseded by dedicated subdomain)

### PDF Download Fix Implementation Details
**Issues Resolved**:
1. **Caddy Routing Issue** ✅ FIXED
   - **Problem**: Caddy configuration only proxied to frontend (port 806), no separate API routing
   - **Impact**: All /api/* requests went through frontend's Next.js proxy, causing routing issues
   - **Solution**: Exposed backend on port 809 and configured Caddy to route /api/* directly to backend
   - **Files**: `docker-compose.yml`, Caddy configuration

2. **Backend Port Exposure** ✅ FIXED
   - **Problem**: Backend not exposed to host, only accessible within Docker network
   - **Solution**: Added `ports: - "809:3001"` to backend service in docker-compose.yml
   - **File**: `docker-compose.yml`

3. **Caddy Configuration Update** ✅ FIXED
   - **Problem**: Simple reverse proxy to frontend didn't handle API routes separately
   - **Solution**: Added handle blocks to route /api/* to backend (809) and everything else to frontend (806)
   - **File**: `/home/chaulmark/docker/caddy/config/Caddyfile`

### Final Caddy Configuration
```
mad-lawsuit.org {
    # Route API requests directly to backend
    handle /api/* {
        reverse_proxy 172.18.0.1:809
    }

    # Route everything else to frontend
    handle {
        reverse_proxy 172.18.0.1:806
    }
}
```

### Verification ✅
```bash
# Backend accessible on port 809
curl http://localhost:809/api/documents/9/download
# HTTP 200 - PDF content returned

# Public domain working
curl https://mad-lawsuit.org/api/documents/9/download
# HTTP 200 - PDF content returned
```

### Git Commits Made
1. `13d219db` - Expose backend on port 809 for direct Caddy routing to fix PDF download

## Previous Task: DOCKER BUILD ISSUES FULLY RESOLVED ✅
**MAD Lawsuit Website Docker Deployment Successfully Fixed**:
- **Problem**: Docker containers failing to build and run due to pnpm and Prisma issues
- **Root Cause**: Prisma CLI in devDependencies but needed in production for client generation
- **Solution**: Dependency restructuring + Docker build process optimization
- **Status**: ALL CONTAINERS RUNNING AND HEALTHY ✅

### Docker Fix Implementation Details
**Issues Resolved**:
1. **pnpm TTY Error** ✅ FIXED
   - **Problem**: `pnpm prune --prod` failing with TTY error in Docker build
   - **Solution**: Added `ENV CI=true` to Dockerfile before prune command
   - **File**: `backend/Dockerfile`

2. **Prisma Client Missing After Prune** ✅ FIXED
   - **Problem**: `prisma` CLI in devDependencies, removed during production prune
   - **Impact**: Prisma client generation failing, backend crashing with MODULE_NOT_FOUND
   - **Solution**: Moved `prisma` from devDependencies to dependencies in package.json
   - **Files**: `backend/package.json`, `backend/pnpm-lock.yaml`

3. **Outdated Lockfile** ✅ FIXED
   - **Problem**: pnpm-lock.yaml outdated after dependency changes
   - **Impact**: `pnpm install --frozen-lockfile` failing in Docker build
   - **Solution**: Regenerated lockfile locally and committed to repository

### Final Docker Build Process
```dockerfile
# Install all dependencies
RUN pnpm install --frozen-lockfile

# Generate Prisma client (first time)
RUN pnpm prisma generate

# Build application
RUN pnpm build

# Remove dev dependencies (but keep prisma CLI)
ENV CI=true
RUN pnpm prune --prod

# Regenerate Prisma client (after prune, using production deps)
RUN pnpm prisma generate
```

### Current Container Status - ALL HEALTHY ✅
- **mad-lawsuit-frontend-1**: Running and healthy on port 806
- **mad-lawsuit-backend-1**: Running and healthy with working database connections
- **mad-lawsuit-postgres-1**: Running on port 5432
- **mad-lawsuit-redis-1**: Running on port 6379

### Backend Verification ✅
```
> node dist/index.js
prisma:info Starting a postgresql pool with 3 connections.
prisma:query SELECT 1
prisma:query SELECT "public"."docket_entries"...
```
- Prisma client working correctly
- Database connections established
- Queries executing successfully

### Git Commits Made
1. `b291ee9b` - Fix Docker build: Add CI=true env var for pnpm prune command
2. `d087ca9c` - Fix Prisma client: Regenerate after pnpm prune to restore missing binaries
3. `0035b7cc` - Fix Prisma generate: Use npx instead of pnpm after prune removes CLI
4. `cf071cc4` - Fix Prisma prune: Use --config.ignore-scripts=false to preserve Prisma client
5. `b59ae90c` - Fix Prisma dependencies: Move prisma CLI to production deps and regenerate after prune
6. `f962327a` - Update pnpm-lock.yaml after moving prisma to production dependencies

## Previous Task: GMAIL SMUGGLER FULLY OPERATIONAL ✅
**SMTP Relay Solution Successfully Implemented**:
- **DigitalOcean Server (chrishaulmark.com)**: SMTP ports 25, 465, 587 BLOCKED by DigitalOcean
- **Dedicated Server (74.80.182.50)**: All SMTP ports WORKING ✅
- **Gmail Smuggler Container**: DEPLOYED and RUNNING ✅ on port 2525 (unblocked)

**Solution**: DNS override + port 2525 SMTP relay bypasses DigitalOcean SMTP blocking completely.

### Gmail Smuggler Final Configuration
- **Container Name**: `gmail-smuggler`
- **Location**: `10.4.0.206:~/gmail-smuggler/` (internal VM)
- **Status**: Running with Docker Compose auto-restart
- **Port**: 2525 (external) → 587 (internal, forwarding to smtp.gmail.com:587)
- **Technology**: Alpine Linux + socat TCP relay
- **OPNsense**: Port forwarding 2525 → 10.4.0.206:2525

### Implementation Details
1. **DNS Override**: `74.80.182.50    smtp.gmail.com` in `/etc/hosts` ✅
2. **Backend Configuration**: `SMTP_PORT=2525` in `backend/.env` ✅
3. **Port Forwarding**: OPNsense forwards port 2525 to internal VM ✅
4. **Connection Verified**: DigitalOcean VPS → 74.80.182.50:2525 → Gmail ✅

### Email Flow
```
Court Docket Website → smtp.gmail.com:2525 → 74.80.182.50:2525 → 10.4.0.206:2525 → Gmail SMTP
```

## Previous Task: COMPLETED ✅
**DeafGain LLC Footer Addition** - Added "Designed by DeafGain LLC" footer to the court docket website.

## Previous Task: COMPLETED ✅
**Email System URL Fix** - Updated email notifications to use production domain and added unsubscribe instructions.

## Previous Task: COMPLETED ✅
**PDF Upload Issue Resolution** - Successfully identified and fixed all issues preventing PDF document uploads in the admin dashboard.

## Recent Work Completed

### DeafGain LLC Footer Addition - COMPLETED ✅
**Problem**: Website was missing professional branding footer like other DeafGain websites.

**Solution Implemented**:

1. **Footer Design** ✅ ADDED
   - **Implementation**: Added footer section at bottom of home page
   - **Styling**: Matches website theme with dark background and yellow accent text
   - **Content**: "Designed by DeafGain LLC" with link to http://deafgain.org
   - **File**: `frontend/src/app/page.tsx`

2. **Visual Consistency** ✅ ACHIEVED
   - **Reference**: Based on Chris Haulmark website footer implementation
   - **Colors**: Gray text with yellow (#fbbf24) DeafGain LLC link
   - **Hover Effect**: Color changes to darker yellow (#f59e0b) on hover
   - **Layout**: Centered text in footer section

### Footer Implementation:
```jsx
<footer style={{ 
  width: '100%',
  padding: '2rem 0',
  backgroundColor: 'rgba(57, 64, 83, 0.95)',
  borderTop: '1px solid #4E4A59',
  textAlign: 'center',
  display: 'block'
}}>
  <div style={{ textAlign: 'center', display: 'block', width: '100%' }}>
    <span style={{ color: '#9ca3af' }}>Designed by </span>
    <a href="http://deafgain.org" target="_blank" rel="noopener noreferrer" 
       style={{ color: '#fbbf24', textDecoration: 'none' }}>
      DeafGain LLC
    </a>
  </div>
</footer>
```

### Email System URL Fix - COMPLETED ✅
**Problem**: Email notifications were linking to localhost instead of production domain, and missing unsubscribe instructions.

**Issues Fixed**:

1. **Production URL Issue** ✅ FIXED
   - **Problem**: Email links pointed to `localhost:3000` instead of production domain
   - **Impact**: Subscribers couldn't access website from email notifications
   - **Solution**: Updated all email links to use `https://mad-lawsuit.org/`
   - **File**: `backend/src/services/emailService.ts`

2. **Email Text Improvement** ✅ FIXED
   - **Problem**: Generic "court docket website" text in emails
   - **Impact**: Less professional email presentation
   - **Solution**: Changed to "You can view the complete docket and any associated documents by visiting the website:" with website as clickable link
   - **File**: `backend/src/services/emailService.ts`

3. **Missing Unsubscribe Instructions** ✅ FIXED
   - **Problem**: No clear unsubscribe instructions for email recipients
   - **Impact**: Users couldn't easily unsubscribe from notifications
   - **Solution**: Added "To unsubscribe, please send an email to eliza.kragh@gmail.com" in footer
   - **File**: `backend/src/services/emailService.ts`

### Email Changes Made:
```html
<!-- Before -->
<p>You can view the complete docket and any associated documents by visiting the court docket website:</p>
<a href="${process.env['FRONTEND_URL'] || 'http://localhost:3000'}" class="button">View Court Docket</a>

<!-- After -->
<p>You can view the complete docket and any associated documents by visiting the <a href="https://mad-lawsuit.org/">website</a>:</p>
<a href="https://mad-lawsuit.org/" class="button">View Court Docket</a>
<p>To unsubscribe, please send an email to eliza.kragh@gmail.com</p>
```

### PDF Upload Issue - FULLY RESOLVED ✅
**Problem**: Users could not upload PDF documents through the admin dashboard, receiving various errors.

**Root Causes Identified and Fixed**:

1. **Network Connectivity Issue** ✅ FIXED
   - **Problem**: Frontend container only on `caddy_network`, backend on `app-network` 
   - **Impact**: Frontend couldn't reach backend for API calls
   - **Solution**: Added frontend to both networks in `docker-compose.yml`
   - **Commit**: 910bd525

2. **MIME Type Validation Issue** ✅ FIXED
   - **Problem**: Backend fileFilter only accepted `application/pdf` MIME type
   - **Impact**: Valid PDFs rejected due to browser MIME detection variations
   - **Solution**: Enhanced fileFilter to accept multiple PDF MIME types + file extension fallback
   - **File**: `backend/src/routes/documents.ts`
   - **Commit**: 9ab09196

3. **Missing Form Data Issue** ✅ FIXED
   - **Problem**: Frontend only sending `file` and `docketEntryId`, missing required `title` field
   - **Impact**: Backend validation failing with "title is required" error
   - **Solution**: Added all required fields to FormData in dashboard upload
   - **File**: `frontend/src/app/admin/dashboard/page.tsx`
   - **Commit**: 8027ef0a

### Technical Details

**Network Architecture Fixed**:
```yaml
frontend:
  networks:
    - app-network      # Added for backend communication
    - caddy_network    # Existing for reverse proxy
```

**Enhanced PDF Detection**:
```javascript
const allowedMimeTypes = [
  'application/pdf',
  'application/x-pdf', 
  'application/acrobat',
  'applications/vnd.pdf',
  'text/pdf',
  'text/x-pdf'
];
```

**Complete Form Data**:
```javascript
uploadFormData.append('file', selectedFile);
uploadFormData.append('docketEntryId', data.entry.id.toString());
uploadFormData.append('title', selectedFile.name.replace('.pdf', ''));
uploadFormData.append('summary', '');
uploadFormData.append('notes', '');
```

## Current Status

### What's Working ✅
- **Docker containers all running and healthy**
- **Backend with working Prisma database connections**
- **Frontend accessible on port 806**
- Network connectivity between frontend and backend
- PDF MIME type detection (multiple formats)
- Form validation with all required fields
- Error logging and debugging
- **Email system with production URLs** (https://mad-lawsuit.org/)
- **Professional unsubscribe instructions** (eliza.kragh@gmail.com)
- **DeafGain LLC footer branding** with professional styling
- All code committed and pushed to Git

### Next Steps for User
1. **Website is fully operational** - no further action needed
2. **All containers healthy and running** on remote server
3. **Database connections working** - Prisma queries executing successfully
4. **Email notifications functional** with proper URLs
5. **Professional branding** with DeafGain LLC footer

### Files Modified (Docker Fix)
- `backend/Dockerfile` - Added CI=true, optimized Prisma generation
- `backend/package.json` - Moved prisma from devDependencies to dependencies
- `backend/pnpm-lock.yaml` - Updated lockfile for new dependency structure

### Files Modified (Previous Tasks)
- `docker-compose.yml` - Network configuration
- `backend/src/routes/documents.ts` - Enhanced PDF validation
- `frontend/src/app/admin/dashboard/page.tsx` - Fixed form data
- **`backend/src/services/emailService.ts` - Updated URLs and unsubscribe info**
- **`frontend/src/app/page.tsx` - Added DeafGain LLC footer**

### Deployment Notes
- **Docker deployment successful** - all containers running on remote server
- **No further deployment needed** - website fully operational
- All fixes are committed to Git repository
- Backend and frontend both successfully deployed with latest fixes

## Investigation Process
1. **Docker Build Analysis**: Identified pnpm TTY and Prisma dependency issues
2. **Dependency Management**: Restructured package.json for proper production builds
3. **Build Process Optimization**: Enhanced Dockerfile for reliable Prisma client generation
4. **Systematic Testing**: Verified all containers healthy and database connections working

The MAD lawsuit website Docker deployment is now fully operational with all containers running successfully.
