|
| 1 | +# postgres-backup |
| 2 | + |
| 3 | +A lightweight, automated PostgreSQL backup service that creates compressed backups and uploads them to S3-compatible storage. |
| 4 | + |
| 5 | +## Features |
| 6 | + |
| 7 | +- **Scheduled Backups**: Configurable cron-style scheduling using `robfig/cron` |
| 8 | +- **Compression**: Automatic gzip compression to save storage space |
| 9 | +- **S3 Integration**: Upload backups to AWS S3 or S3-compatible storage (MinIO, etc.) |
| 10 | +- **Retry Logic**: Automatic retry on S3 upload failures |
| 11 | +- **Local Cleanup**: Removes local backup files after successful S3 upload |
| 12 | +- **Flexible Configuration**: Configure via YAML file or environment variables |
| 13 | +- **Docker Support**: Ready-to-use Docker container |
| 14 | + |
| 15 | +## Quick Start |
| 16 | + |
| 17 | +### Using Docker |
| 18 | + |
| 19 | +```bash |
| 20 | +docker run -v $(pwd)/config.yaml:/app/config.yaml \ |
| 21 | + ghcr.io/010codingcollective/postgres-backup:latest |
| 22 | +``` |
| 23 | + |
| 24 | +### Using Docker Compose |
| 25 | + |
| 26 | +```yaml |
| 27 | +version: '3.8' |
| 28 | +services: |
| 29 | + postgres-backup: |
| 30 | + image: ghcr.io/010codingcollective/postgres-backup:latest |
| 31 | + volumes: |
| 32 | + - ./config.yaml:/app/config.yaml |
| 33 | + environment: |
| 34 | + - POSTGRES_PASSWORD=your_password |
| 35 | + - S3_SECRET_ACCESS_KEY=your_secret_key |
| 36 | +``` |
| 37 | +
|
| 38 | +## Configuration |
| 39 | +
|
| 40 | +Configuration can be provided via: |
| 41 | +1. YAML file (`config.yaml`) |
| 42 | +2. Environment variables (highest priority) |
| 43 | + |
| 44 | +### Example Configuration |
| 45 | + |
| 46 | +```yaml |
| 47 | +schedule: "@daily" # Cron format: @daily, @hourly, or "0 2 * * *" |
| 48 | +run_at_startup: false # Run backup immediately on startup |
| 49 | +postgres_database: mydb |
| 50 | +postgres_user: postgres |
| 51 | +postgres_password: secret |
| 52 | +postgres_host: localhost |
| 53 | +postgres_port: "5432" |
| 54 | +postgres_extra_opts: "--schema=public --blobs" |
| 55 | +
|
| 56 | +s3: |
| 57 | + endpoint: "" # Leave empty for AWS S3, or set for MinIO/S3-compatible |
| 58 | + region: us-east-1 |
| 59 | + bucket: my-backup-bucket |
| 60 | + access_key_id: AKIAIOSFODNN7EXAMPLE |
| 61 | + secret_access_key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY |
| 62 | + use_path_style: false # Set to true for MinIO and most S3-compatible services |
| 63 | +``` |
| 64 | + |
| 65 | +### Environment Variables |
| 66 | + |
| 67 | +All configuration can be overridden with environment variables: |
| 68 | + |
| 69 | +```bash |
| 70 | +SCHEDULE="@daily" |
| 71 | +RUN_AT_STARTUP="false" |
| 72 | +POSTGRES_DATABASE="mydb" |
| 73 | +POSTGRES_USER="postgres" |
| 74 | +POSTGRES_PASSWORD="secret" |
| 75 | +POSTGRES_HOST="localhost" |
| 76 | +POSTGRES_PORT="5432" |
| 77 | +POSTGRES_EXTRA_OPTS="--schema=public --blobs" |
| 78 | +S3_ENDPOINT="" |
| 79 | +S3_REGION="us-east-1" |
| 80 | +S3_BUCKET="my-backup-bucket" |
| 81 | +S3_ACCESS_KEY_ID="AKIAIOSFODNN7EXAMPLE" |
| 82 | +S3_SECRET_ACCESS_KEY="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" |
| 83 | +S3_USE_PATH_STYLE="false" |
| 84 | +``` |
| 85 | + |
| 86 | +## Schedule Format |
| 87 | + |
| 88 | +The `schedule` field uses cron format. Common examples: |
| 89 | + |
| 90 | +- `@daily` - Run once per day at midnight |
| 91 | +- `@hourly` - Run once per hour |
| 92 | +- `@every 6h` - Run every 6 hours |
| 93 | +- `0 2 * * *` - Run at 2:00 AM daily |
| 94 | +- `0 */6 * * *` - Run every 6 hours |
| 95 | +- `0 2 * * 0` - Run at 2:00 AM every Sunday |
| 96 | + |
| 97 | +## Backup Storage |
| 98 | + |
| 99 | +Backups are stored in S3 with the following structure: |
| 100 | + |
| 101 | +``` |
| 102 | +s3://your-bucket/backups/{database}-backup-{timestamp}.sql.gz |
| 103 | +``` |
| 104 | + |
| 105 | +Example: `backups/mydb-backup-20250128-020000.sql.gz` |
| 106 | + |
| 107 | +## Building from Source |
| 108 | + |
| 109 | +### Prerequisites |
| 110 | + |
| 111 | +- Go 1.23+ |
| 112 | +- PostgreSQL client tools (`pg_dump`) |
| 113 | + |
| 114 | +### Build |
| 115 | + |
| 116 | +```bash |
| 117 | +go build -o pg-backup . |
| 118 | +``` |
| 119 | + |
| 120 | +### Run |
| 121 | + |
| 122 | +```bash |
| 123 | +./pg-backup |
| 124 | +``` |
| 125 | + |
| 126 | +## Building Docker Image |
| 127 | + |
| 128 | +```bash |
| 129 | +docker build -t postgres-backup . |
| 130 | +``` |
| 131 | + |
| 132 | +Or using buildah: |
| 133 | + |
| 134 | +```bash |
| 135 | +buildah bud -f Dockerfile -t postgres-backup . |
| 136 | +``` |
| 137 | + |
| 138 | +## How It Works |
| 139 | + |
| 140 | +1. **Scheduler** starts based on configured cron schedule |
| 141 | +2. **pg_dump** creates a PostgreSQL backup |
| 142 | +3. **Compression** compresses the backup with gzip |
| 143 | +4. **S3 Upload** uploads the compressed backup to S3 (with retry) |
| 144 | +5. **Cleanup** removes the local backup file after successful upload |
| 145 | + |
| 146 | +## Error Handling |
| 147 | + |
| 148 | +- If `pg_dump` fails, the backup job is aborted |
| 149 | +- If S3 upload fails, it retries once |
| 150 | +- If both S3 upload attempts fail, the local backup file is retained and logged |
| 151 | +- If S3 is misconfigured, the application fails at startup |
| 152 | + |
| 153 | +## License |
| 154 | + |
| 155 | +MIT License - see [LICENSE](LICENSE) file for details |
| 156 | + |
| 157 | +## Contributing |
| 158 | + |
| 159 | +Contributions are welcome! Please open an issue or submit a pull request. |
0 commit comments