# Phase 1: GCS Migration - Implementation Guide

## Overview
This guide documents the implementation of Google Cloud Storage (GCS) migration for the CBT system.

## What Was Implemented

### 1. Package Installation
- ✅ Installed `league/flysystem-google-cloud-storage:^3.0`
- ✅ Dependencies: Google Cloud Storage SDK, JWT auth, GRPC

### 2. Configuration Files

#### `config/filesystems.php`
Added new GCS disk configuration:
```php
'gcs' => [
    'driver' => 'gcs',
    'project_id' => env('GCS_PROJECT_ID'),
    'key_file' => env('GCS_KEY_FILE'),
    'bucket' => env('GCS_BUCKET'),
    'path_prefix' => env('GCS_PATH_PREFIX', ''),
    'visibility' => 'private',
    'throw' => false,
    'report' => false,
],
```

#### `.env.example`
Added GCS environment variables:
```env
# Google Cloud Storage Configuration
GCS_PROJECT_ID=
GCS_KEY_FILE=storage/app/gcs_credentials.json
GCS_BUCKET=
GCS_PATH_PREFIX=
```

### 3. Artisan Commands

#### `storage:test-gcs`
Tests GCS connection with comprehensive checks:
- Validates configuration
- Tests write operation
- Tests read operation
- Tests existence check
- Tests file size retrieval
- Tests delete operation
- Tests file listing

Usage:
```bash
php artisan storage:test-gcs
```

#### `storage:migrate-to-gcs`
Migrates files from local storage to GCS with:
- Batch processing (default: 50 records per batch)
- Dry-run mode for testing
- Progress tracking
- Error reporting
- Automatic path transformation

New path formats:
- **Question images:** `processed/{lbb_id}/images/{question_id}.webp`
- **Question audio:** `tenants/{lbb_id}/audio/{question_id}.{ext}`
- **Option images:** `processed/{lbb_id}/images/option_{option_id}.webp`
- **Token proofs:** `tenants/{lbb_id}/proofs/token_{order_id}.{ext}`
- **Withdraw proofs:** `tenants/{lbb_id}/proofs/withdraw_{withdraw_id}.{ext}`
- **LBB logos:** `tenants/{lbb_id}/logos/{lbb_id}.{ext}`

Usage:
```bash
# Dry run (test without making changes)
php artisan storage:migrate-to-gcs --dry-run

# Full migration with confirmation
php artisan storage:migrate-to-gcs

# Force migration without confirmation
php artisan storage:migrate-to-gcs --force

# Custom batch size
php artisan storage:migrate-to-gcs --batch=100
```

## Setup Instructions

### Step 1: Create GCS Bucket
1. Go to Google Cloud Console
2. Create a new bucket (or use existing)
3. **Important:** Set bucket access to **Private** (not public)
4. Note the bucket name

### Step 2: Create Service Account
1. Go to IAM & Admin > Service Accounts
2. Create a new service account
3. Grant these roles to the service account:
   - `Storage Object Admin` (or `Storage Object Creator` + `Storage Object Viewer`)
4. Create and download a JSON key file

### Step 3: Configure Credentials
1. Place the JSON key file in your project:
   ```bash
   mv ~/Downloads/service-account-key.json storage/app/gcs_credentials.json
   ```

2. Add to `.env` file:
   ```env
   GCS_PROJECT_ID=your-project-id
   GCS_KEY_FILE=storage/app/gcs_credentials.json
   GCS_BUCKET=your-bucket-name
   GCS_PATH_PREFIX=  # Optional: add prefix if needed
   ```

### Step 4: Test Connection
```bash
php artisan storage:test-gcs
```

If successful, you should see:
```
✓ All GCS tests passed!
```

If failed, check the error messages and:
- Verify credentials file path
- Check bucket name spelling
- Ensure service account has proper permissions
- Verify network connectivity

### Step 5: Run Migration

#### First: Dry Run
```bash
php artisan storage:migrate-to-gcs --dry-run
```

This will show what will happen without actually uploading files.

#### Second: Full Migration
```bash
php artisan storage:migrate-to-gcs
```

The script will:
1. Process all files in batches
2. Upload to GCS with new path format
3. Update database records
4. Show progress every 10 files
5. Report any errors

### Step 6: Verify Migration
After migration completes:

1. Check GCS bucket for files
2. Run dry run again to see if all files are skipped (already migrated)
3. Test application with new storage

### Step 7: Switch to GCS (Optional)
Once verified, update `.env`:
```env
FILESYSTEM_DISK=gcs
```

**Important:** Do NOT delete local files until you're absolutely sure everything works with GCS.

## Migration Features

### Safe Migration
- ✅ Local files are NOT deleted automatically
- ✅ Database updates happen only after successful upload
- ✅ Already-migrated files are skipped
- ✅ Comprehensive error reporting

### Batch Processing
- Processes records in configurable batches (default: 50)
- Prevents memory issues with large datasets
- Shows progress updates

### Error Handling
- Individual file failures don't stop entire migration
- All errors are logged and reported at the end
- Failed files can be retried by running migration again

### Rollback Support
Since local files are preserved, you can easily rollback by:
1. Updating database paths back to original format
2. Keeping FILESYSTEM_DISK=local

## Models Migrated

The migration script handles these models:

| Model | Fields | New Path Format |
|-------|--------|-----------------|
| Question | `image_path`, `audio_path` | `processed/{lbb_id}/images/{id}.webp`<br>`tenants/{lbb_id}/audio/{id}.{ext}` |
| QuestionOption | `image_path` | `processed/{lbb_id}/images/option_{id}.webp` |
| TokenOrder | `proof` | `tenants/{lbb_id}/proofs/token_{id}.{ext}` |
| Withdraw | `proof` | `tenants/{lbb_id}/proofs/withdraw_{id}.{ext}` |
| LbbSetting | `logo_path` | `tenants/{lbb_id}/logos/{lbb_id}.{ext}` |

## Important Notes

### Database Columns
- No schema changes required
- Columns already exist and store relative paths
- Migration only updates path values

### File Visibility
- All files stored as **private** in GCS
- Access will be through signed URLs (Phase 4)
- No public URLs exposed

### Tenant Isolation
- Files are organized by `lbb_id` (tenant ID)
- Each tenant's files are in separate directory structures
- Path prefix ensures proper isolation

### Image Format
- All images are stored as `.webp` format
- Maintains compression and optimization
- Consistent file extensions

## Troubleshooting

### Connection Issues
```bash
# Check if credentials file exists
ls -la storage/app/gcs_credentials.json

# Verify .env values
php artisan tinker
>>> env('GCS_PROJECT_ID')
>>> env('GCS_BUCKET')
```

### Permission Issues
Ensure service account has these IAM roles:
- `Storage Object Admin` (recommended)
- OR: `Storage Object Creator` + `Storage Object Viewer` + `Storage Object Deleter`

### Bucket Access
- Verify bucket exists in correct project
- Check bucket region/locations
- Ensure bucket is accessible from your environment

## Next Steps

After completing Phase 1:
1. ✅ GCS is configured and tested
2. ✅ Files are migrated to GCS
3. ✅ Database paths are updated
4. ✅ System can use GCS storage

Proceed to **Phase 2: Signed URL Direct Upload**