SemScore Logo
New App

SemScore App is Live on Google Play!

Free S3 Storage Guide: How to Host MinIO on Docker with External Drives and Cloudflare Tunnels

H
Harish Kumar R
14 min read (2698 words)13 reads
Free S3 Storage Guide: How to Host MinIO on Docker with External Drives and Cloudflare Tunnels

Stop paying monthly cloud storage fees. Turn an external hard drive attached to a Mac Mini or Windows PC into a 100% free, enterprise-grade S3-compatible object storage server using Docker and Cloudflare Zero Trust Tunnels.

As software engineers and DevOps practitioners, we rely on S3-compatible object storage to back up application databases, store user file uploads, and retain historical site snapshots. However, managed cloud object storage providers (such as AWS S3DigitalOcean SpacesGoogle Cloud Storage, or Wasabi) charge monthly storage fees per gigabyte alongside bandwidth egress charges that escalate as your application footprint grows.

If you have a spare computer (such as a Mac Mini, Intel NUC, or Windows PC) and an external hard drive, you can build your own 100% free S3 storage server. Using open-source MinIODocker, and Cloudflare Zero Trust Tunnels, you get a production-ready, encrypted, self-hosted S3 object storage server with unlimited API operations and zero monthly cloud invoices.

⚡ OD2 Free Developer Tool

Testing Email & SMTP Integrations?

Launch a private SMTP sandbox to capture, inspect, and debug mock emails without sending to real users.

Launch SMTP Sandbox →

Here is a complete guide covering real-world developer use cases, an architectural decision matrix, full setup steps, and automated sample data generation scripts to test your free S3 storage setup.

Real-World Developer Use Cases

Self-hosting an S3 object storage server on bare-metal hardware backed by Cloudflare Tunnels solves several practical engineering problems:

Use Case 1: Automated Offsite Cloud Server Backups

When running cloud application orchestrators like CoolifyCapRover, or Docker Swarm on VPS instances (Hetzner, DigitalOcean, AWS EC2), storing database dumps on the same VPS creates a single point of failure. Setting up your self-hosted MinIO instance gives your cloud servers a remote, offsite S3 destination that dumps nightly PostgreSQLMySQL, or MongoDB backups straight onto a physical drive on your desk.

Use Case 2: Local & Staging Environment Object Storage

Developers often test file upload logic in staging or local environments. Using live AWS S3 buckets for staging runs the risk of leaving orphan test files behind or incurring unintended storage charges. A self-hosted MinIO bucket provides a zero-cost, 100% API-compatible local staging target that behaves identically to AWS S3.

Use Case 3: Synthetic Data Generation & Load Testing

When benchmarking file upload pipelines or testing bulk data ingest services, uploading thousands of synthetic test files to public cloud storage generates unexpected API request charges. A local free S3 server allows software teams to generate, write, and delete gigabytes of sample test data without paying per-request fees.

Use Case 4: Free Media Archive & Side Project Storage

For side projects, indie applications, or self-hosted media libraries, recurring cloud storage bills eat into project margins. Mapping an external 1TB or 2TB hard drive to MinIO provides virtually unlimited storage space without monthly invoice anxiety.

Decision Framework: Self-Hosted Free S3 vs. AWS S3 & Cloud Storage

Before setting up your local infrastructure, evaluate how self-hosted MinIO compares to public cloud storage providers across cost, bandwidth, latency, and maintenance:

Feature / MetricAWS S3 StandardDigitalOcean SpacesBackblaze B2Self-Hosted Free S3 (MinIO + Drive)
Monthly Storage Cost$0.023 / GB / month$5 / month (first 250GB)$0.006 / GB / month$0 / month (uses local physical drive)
Egress Bandwidth Cost$0.09 / GB transferred$0.01 / GB over limit$0.01 / GB transferred$0 / month (routed via Cloudflare Tunnel)
API Request Fees$0.005 per 1,000 PUTsIncluded in base tier$0.004 per 10,000 PUTs$0 / month (Unlimited API calls)
Network ConfigurationPublic Internet / IAMPublic Internet / TokenPublic Internet / KeyOutbound Tunnel (No static IP or open ports)
Physical Storage LimitVirtually UnlimitedTiered / ScalableScalableDependent on external drive capacity
Maintenance OverheadZero (Managed AWS)Zero (Managed DO)Zero (Managed B2)Low (Docker container + OS auto-updates)

When to Choose Self-Hosted Free S3

Self-hosting your own free S3 storage cluster is the ideal choice when:

  • Automating Offsite Backups: You need a secure offsite backup target for cloud servers managed by orchestrators like Coolify.
  • Eliminating Storage & Egress Invoices: You want to avoid monthly cloud storage bills for storing multi-gigabyte database dumps, video archives, or system snapshots.
  • Bypassing CGNAT & Home Network Constraints: You operate on a residential internet connection without a static IP address or open router ports.
  • Retaining Physical Data Ownership: You want complete physical custody of sensitive database backups stored on local hardware on your desk.

When to Choose AWS S3 or Public Cloud Storage

Public cloud providers remain necessary when:

  • Multi-Region Global Availability: Your application requires global edge replication across multiple AWS availability zones with 99.999999999% (11 nines) of durability SLA.
  • Petabyte-Scale Elastic Storage: You store petabytes of user uploads that burst dynamically beyond local physical drive capacities.
  • Strict Corporate Compliance: Your enterprise requires SOC2 Type II, HIPAA, or PCI-DSS cloud provider compliance attestations.

The Core Challenge: Self-Hosting Behind Home Networks

Running a self-hosted S3 storage server on a residential network typically encounters three networking obstacles:

  1. Dynamic Public IPs: Residential ISPs change your home IP address without notice, breaking static DNS endpoints.
  2. Carrier-Grade NAT (CGNAT): Modern ISPs place residential subscribers behind shared CGNAT gateways, blocking traditional inbound port forwarding on home routers.
  3. Network Security Hazards: Exposing raw storage ports (port 9000 for S3 API or port 9001 for console UI) invites automated brute-force attacks and port scans.

We solve all three challenges using Cloudflare Zero Trust Tunnels (cloudflared). The local daemon initiates an outbound encrypted gRPC connection over TLS to Cloudflare edge nodes. Incoming requests to your domain (e.g., https://s3.example.com) are authenticated by Cloudflare and proxied through the outbound tunnel directly to your local MinIO container.

Comprehensive System Architecture Summary

The complete architecture operates across four distinct layers:

  • Source Layer (Cloud Production): Production servers, application databases (PostgreSQL, MySQL, MongoDB), and volume backups managed by Coolify or automated cron tasks.
  • Storage Layer (Bare-Metal Local Appliance): A dedicated local machine (Apple Mac Mini running macOS or a Windows PC) running MinIO in Docker, mapped to an external physical drive partition.
  • Transport & Perimeter Security Layer (Zero Trust Tunnel): An outbound encrypted tunnel (cloudflare/cloudflared) providing an exclusive gateway into your local S3 engine.
  • Data & Control Planes:
    • S3 Data API (https://s3.example.com): Securely mapped through the tunnel to host.docker.internal:9000 (MinIO S3 Engine).
    • Control UI (https://s3-console.example.com): Securely mapped through the tunnel to host.docker.internal:9001 (MinIO Web Console).

Step 1: Prepping the Physical External Hard Drive

Because your local machine runs as a 24/7 storage node, partitioning and formatting your external media correctly is critical for long-term data safety.

Format Recommendation: ExFAT

Format the dedicated backup partition using the ExFAT file system. ExFAT provides cross-platform compatibility across macOS, Windows, and Linux. If your Mac Mini or Windows host suffers a hardware failure, you can unplug the external drive and connect it directly to any workstation to retrieve raw object files immediately.

Mount Paths Across Operating Systems

Ensure your external drive mounts to a fixed directory path:

  • macOS Mount Location/Volumes/sitebackups (or /Volumes/backup_drive)
  • Windows Mount LocationD:\MinIOData (assign a fixed drive letter via Disk Management)

Create a dedicated data folder on the root of the partition:

# macOS
mkdir -p /Volumes/sitebackups/minio_data

# Windows PowerShell
New-Item -ItemType Directory -Path "D:\MinIOData\minio_data" -Force

Step 2: Deploying MinIO S3 Storage via Docker

MinIO is a high-performance, S3-compatible object storage server. Running MinIO inside Docker isolates its runtime environment and binds external host drive volumes directly to the storage container.

Launch Command for macOS

Run the following command in Terminal on macOS:

docker run -d \
  --name minio-s3 \
  --restart unless-stopped \
  -v /Volumes/sitebackups/minio_data:/data \
  -e "MINIO_ROOT_USER=admin_user" \
  -e "MINIO_ROOT_PASSWORD=YourSecureMasterPassword" \
  minio/minio server /data --console-address ":9001"

Launch Command for Windows PowerShell

Run the following command in PowerShell on Windows:

docker run -d `
  --name minio-s3 `
  --restart unless-stopped `
  -v D:\MinIOData\minio_data:/data `
  -e "MINIO_ROOT_USER=admin_user" `
  -e "MINIO_ROOT_PASSWORD=YourSecureMasterPassword" `
  minio/minio server /data --console-address ":9001"

Command Parameter Breakdown

  • -d: Runs the container in detached background mode.
  • --restart unless-stopped: Ensures MinIO automatically restarts if the container crashes or when Docker starts on system reboot.
  • -v [HOST_PATH]:/data: Maps your local external drive folder directly into the container’s internal /data directory.
  • MINIO_ROOT_USER & MINIO_ROOT_PASSWORD: Sets the master administrative credentials for the object storage cluster.
  • server /data --console-address ":9001": Instructs MinIO to serve S3 object data on port 9000 while launching the Web Console interface on port 9001.

Step 3: Creating Buckets and Access Keys in MinIO Console

Once the container is running, open your web browser locally and navigate to:

http://localhost:9001

Log in using your administrative credentials (MINIO_ROOT_USER and MINIO_ROOT_PASSWORD).

1. Create the Backup Bucket

  1. In the left navigation panel, click Buckets.
  2. Click Create Bucket.
  3. Set the Bucket Name to sitebackups (or your preferred identifier).
  4. Click Create Bucket.

2. Generate Dedicated Service Credentials

To establish a secure connection between Coolify and your Mac Mini, you need a matching Access Key and Secret Key. In the S3 ecosystem, these act as your dedicated cryptographic username and password.

Depending on your security workflow, you can authenticate using one of two methods:

Option A: Use Your Master Root Credentials (Fastest)

Because MinIO runs inside a secure, private Docker container on your Mac Mini, your master root login details act as your global S3 API keys by default. You can bypass the web dashboard entirely and plug these straight into Coolify:

  • Access Key ID: (The custom Username you assigned to your Docker container)
  • Secret Access Key: (The custom master password you assigned to your Docker container)

Option B: Use Service Accounts (For Granular Control)

If you prefer to keep your master root password hidden and create a unique credential pair strictly for Coolify, you can generate an isolated API key pair:

  • Log in to your secure admin interface at https://s3-console.example.com.
  • Click on your user profile icon or navigate to the user management section.
  • Select the Service Accounts or Access Keys tab.
  • Click Create Access Key to generate a randomized pairing.
  • Important: Copy the keys or click Download Credentials immediately. MinIO hides the Secret Key permanently once you close this window.

Step 4: Bridging the WAN via Cloudflare Zero Trust Tunnels

To connect remote cloud servers to your local MinIO instance without static public IPs or router port forwarding, deploy a Cloudflare Tunnel daemon container (cloudflared).

1. Retrieve Your Cloudflare Tunnel Token

  1. Log into your Cloudflare Dashboard and navigate to Zero Trust > Networks > Tunnels.
  2. Click Create a Tunnel.
  3. Choose Cloudflared as the connector type.
  4. Name your tunnel (e.g., macmini-s3-tunnel) and click Save Tunnel.
  5. Copy the unique tunnel token string following --token.

2. Run the Cloudflared Docker Container

Execute the daemon container on your local storage host:

docker run -d \
  --name cloudflare-tunnel \
  --restart unless-stopped \
  cloudflare/cloudflared:latest tunnel --no-autoupdate run --token YOUR_CLOUDFLARE_ZERO_TRUST_TOKEN

3. Configure Public Hostname Routing Rules

Inside the Cloudflare Zero Trust Tunnel setup page, add two Public Hostnames pointing back to your local container network using host.docker.internal:

  • Public Hostname 1 (S3 API Endpoint):
    • Subdomains3
    • Domainexample.com (your custom domain)
    • Service TypeHTTP
    • URLhost.docker.internal:9000
  • Public Hostname 2 (S3 Console UI):
    • Subdomains3-console
    • Domainexample.com
    • Service TypeHTTP
    • URLhost.docker.internal:9001

Cloudflare automatically handles wildcard SSL certificate provisioning. Traffic sent to https://s3.example.com is encrypted over HTTPS through Cloudflare’s edge network and decrypted cleanly at host.docker.internal:9000 inside your local host machine.

Step 5: Testing with Automated Sample Data Generators

To verify that your self-hosted S3 storage server is accepting uploads properly, you can use these copy-pasteable sample data generator scripts in Node.js and Python.

Automated Test Script in Node.js (AWS SDK v3)

First, install the AWS S3 client package:

npm install @aws-sdk/client-s3

Then create test-s3-upload.js and run it:

import { S3Client, PutObjectCommand, ListObjectsV2Command } from '@aws-sdk/client-s3';

// Initialize S3 client pointing to your custom Cloudflare Tunnel endpoint
const s3Client = new S3Client({
  endpoint: 'https://s3.example.com',
  region: 'us-east-1',
  credentials: {
    accessKeyId: 'YOUR_ACCESS_KEY',
    secretAccessKey: 'YOUR_SECRET_KEY',
  },
  forcePathStyle: true, // Required for MinIO path-style bucket access
});

async function runSampleDataTest() {
  const bucketName = 'sitebackups';
  
  // Generate sample JSON database dump payload
  const sampleDatabaseDump = JSON.stringify({
    timestamp: new Date().toISOString(),
    status: 'HEALTHY',
    tables: [
      { name: 'users', recordsCount: 1420 },
      { name: 'orders', recordsCount: 9850 },
      { name: 'logs', recordsCount: 45200 }
    ],
    metadata: { environment: 'staging', testRunId: 'sample-run-101' }
  }, null, 2);

  const fileKey = `synthetic-backups/sample-db-dump-${Date.now()}.json`;

  try {
    console.log(`[+] Uploading sample test file: ${fileKey}...`);
    
    await s3Client.send(new PutObjectCommand({
      Bucket: bucketName,
      Key: fileKey,
      Body: Buffer.from(sampleDatabaseDump),
      ContentType: 'application/json',
    }));

    console.log('[✓] Upload successful!');

    // List objects in the bucket to verify persistence
    const listResult = await s3Client.send(new ListObjectsV2Command({ Bucket: bucketName }));
    console.log(`[✓] Bucket Contents (${listResult.Contents?.length || 0} items):`);
    listResult.Contents?.forEach(item => {
      console.log(`  - ${item.Key} (${item.Size} bytes)`);
    });

  } catch (err) {
    console.error('[!] S3 Upload Error:', err.message);
  }
}

runSampleDataTest();

Automated Test Script in Python (Boto3)

Install boto3:

pip install boto3

Create test_s3_upload.py and run it:

import boto3
import json
import time

# Configure Boto3 client targeting your self-hosted S3 endpoint
s3 = boto3.client(
    's3',
    endpoint_url='https://s3.example.com',
    aws_access_key_id='YOUR_ACCESS_KEY',
    aws_secret_access_key='YOUR_SECRET_KEY',
    region_name='us-east-1'
)

bucket_name = 'sitebackups'
file_key = f"python-tests/mock-telemetry-{int(time.time())}.json"

# Create synthetic sample telemetry data
sample_payload = json.dumps({
    "app": "Telemetry Service",
    "event": "SYNTHETIC_DATA_GEN_TEST",
    "metrics": { "cpu_usage": 14.2, "memory_mb": 512, "active_connections": 8 },
    "status": "PASS"
})

print(f"[+] Uploading sample payload to {file_key}...")
s3.put_object(
    Bucket=bucket_name,
    Key=file_key,
    Body=sample_payload.encode('utf-8'),
    ContentType='application/json'
)

print("[✓] Upload complete! Listing bucket objects:")
response = s3.list_objects_v2(Bucket=bucket_name)
for obj in response.get('Contents', []):
    print(f"  - {obj['Key']} ({obj['Size']} bytes)")

When you run either test script, you can verify that the sample files appear instantly inside your MinIO Web Console (https://s3-console.example.com) and on your physical external hard drive.

Step 6: Connecting Coolify for Autonomous Nightly Backups

With your storage server fully validated by sample data scripts, you can configure your remote application orchestrator (Coolify) to send nightly production backups.

1. Register the S3 Destination in Coolify

  1. Log into your Coolify dashboard (e.g., https://coolify.example.com).
  2. Navigate to Keys & Destinations > Destinations > Add New Destination.
  3. Select S3 Storage.

2. Configure S3 Parameters

Fill in your custom domain details:

  • NameLocal Mac Mini Free S3
  • Endpointhttps://s3.example.com (Do not add port :9000; Cloudflare routes traffic automatically).
  • Access Key ID[YOUR_GENERATED_ACCESS_KEY]
  • Secret Access Key[YOUR_GENERATED_SECRET_KEY]
  • Bucketsitebackups
  • Regionus-east-1 (MinIO accepts standard region headers, guaranteeing compatibility with AWS S3 SDK calls).

3. Validate Connection

Click Validate Connection. Coolify will execute an S3 HeadBucket / ListObjectsV2 API call against your custom endpoint. A green validation banner confirms that your remote cloud server can write directly onto your desk’s external drive.

Advanced Reliability & Disaster Recovery Strategies

Handling Host Reboots and External Drive Mount Delays

When your local Mac Mini or Windows PC reboots, Docker Desktop or the Docker Engine daemon starts automatically. If the external drive takes a few seconds to spin up, Docker might create an empty local folder at the mount path before the drive mounts.

To prevent this:

  • On macOS, configure Docker Desktop in system settings to launch automatically on user login after drive mounting.
  • On Windows, ensure your external disk is assigned a persistent drive letter (D:\) in diskmgmt.msc and set the Docker Desktop service startup to Automatic (Delayed Start).

Disaster Recovery File Inspection

Because MinIO stores data objects directly on the ExFAT file system inside the /data directory, your backup data remains human-readable. If you disconnect your external drive and connect it directly to another computer, you will find your uploaded backup archives organized in folders corresponding to your MinIO bucket names.

FAQ & Common Troubleshooting

Error: 502 Bad Gateway when accessing s3.example.com

  • Cause: Cloudflare Tunnel cannot communicate with MinIO over host.docker.internal.
  • Fix: Ensure cloudflared and minio-s3 are running on the same host. On Linux hosts where host.docker.internal is disabled by default, add --add-host=host.docker.internal:host-gateway to your cloudflared Docker container run command, or join both containers to a custom Docker bridge network.

Error: SignatureDoesNotMatch or InvalidAccessKeyId

  • Cause: The S3 credentials entered in your script or Coolify do not match the Access Key generated in the MinIO Console, or the system time on your storage host is out of sync.
  • Fix: Verify clock synchronization on your storage host using NTP. AWS S3 API signatures reject requests if system clocks drift by more than 15 minutes.

Error: Connection Refused on S3 API Operations

  • Cause: Client code is trying to connect via plain HTTP or using port :9000 in the endpoint URL while Cloudflare expects HTTPS on standard port 443.
  • Fix: Use https://s3.example.com without appending :9000 in the endpoint configuration.

How does this setup handle ISP disconnects or dynamic IP shifts?

  • Answer: The cloudflared daemon maintains long-lived outbound connection pools to Cloudflare edge routers. If your ISP drops your connection or assigns a new public IP address, cloudflared automatically reconnects to Cloudflare edge nodes within milliseconds without requiring DNS record updates.

Conclusion: Achieving Total Cloud Data Sovereignty

By combining open-source object storage (MinIO), containerization (Docker), Zero Trust network tunnels (Cloudflare Tunnels), and automated cloud backup scheduling (Coolify), you eliminate monthly S3 storage bills while maintaining full physical custody of your production data archives.

Whether hosting on an old Mac Mini, a spare Windows PC, or an Intel NUC, this architecture delivers a resilient, enterprise-grade cloud backup target operating entirely on your bare-metal storage media.

Recommended Next Reads

Developer Publishing Portal

Have an Engineering Tutorial or Tool to Share?

Publish your software engineering guides, API walkthroughs, and technical insights on the OD2 Blog. Review our publishing conditions, formatting standards, and guidelines.

Loading Sponsor...