All posts in Web Technology

S3 and S3-com­pat­i­ble stor­age is an indis­pens­able com­po­nent of mod­ern web appli­ca­tions, espe­cial­ly when deploy­ing with con­tain­ers like Dock­er and Kubernetes.

When devel­op­ing an appli­ca­tion local­ly, although Lar­avel and oth­er frame­works have uni­fied stor­age helpers allow­ing switch­ing between file sys­tems (usu­al­ly local and S3), there are many quirks of object stor­age that will be over­looked if using a local filesystem.

Rather than use a host­ed S3 sys­tem dur­ing devel­op­ment, we can save costs, laten­cy (and time) and band­width using a local S3-com­pat­i­ble serv­er. MinIO was the go-to until recent­ly with con­tro­ver­sial license changes. RustFS is a good choice for local devel­op­ment. Here is how to set it up with Dock­er and Lar­avel for local development:

RustFS Dock­er:

1
2
3
4
5
6
7
8
9
10
11
docker run \

  -p 9000:9000 \

  -p 9001:9001 \

  -v rust-data:/data \

  -v rust-logs:/logs \

  -e RUSTFS_ACCESS_KEY=rustfsadmin \

  -e RUSTFS_SECRET_KEY=rustfsadmin \

  -e RUSTFS_CONSOLE_ENABLE=true \

  -e RUSTFS_SERVER_DOMAINS=storage.mydevdomain.test \

  -e RUSTFS_SIG_HEADER_WHITELIST="if-modified-since,range" \

  rustfs/rustfs:latest

To con­fig­ure this as a stor­age disk in Lar­avel, we can either cre­ate a new disk, or alter the S3 disk. You’ll have to decide which approach to take, which may depend on if you have mul­ti­ple disks and providers.

The admin inter­face of RustFS is very easy to use. If you are intend­ing files to be pub­lic and using Lar­avel, it is essen­tial to set your file sys­tem as pub­lic, and also set your RustFS buck­et as pub­lic. From expe­ri­ence you will just get silent errors with no error logs to help you.

config/filesystems.php

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
        's3' => [

            'driver' => 's3',

            'key' => env('AWS_ACCESS_KEY_ID'),

            'secret' => env('AWS_SECRET_ACCESS_KEY'),

            'region' => env('AWS_DEFAULT_REGION'),

            'bucket' => env('AWS_BUCKET'),

            'url' => env('AWS_URL'),

            'endpoint' => env('AWS_ENDPOINT'),

            'visibility' => 'public',

            'use_path_style_endpoint' => env('AWS_USE_PATH_STYLE_ENDPOINT', false),

            'throw' => false,

            'report' => false,

            'http' => [

                'verify' => env('AWS_SSL_VERIFY', true),

            ],

        ],

From expe­ri­ence I’ve found that RustFS requires use_path_style_endpoint, as the URL struc­ture has the buck­et name as the first path in the URL, con­trary to S3. This is the same in many com­pat­i­ble servers.

Dur­ing devel­op­ment, the option to ignore self-signed cer­tifi­cate errors is important.

The final step is to set up a reverse proxy on your favorite web serv­er to serve the files. Here’s an exam­ple con­fig in Nginx (don’t for­get to update your hosts file):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
server {

    listen 80;

    listen 443 ssl;

    server_name storage.mydevdomain.test bucketname.storage.mydevdomain.test;



    location / {

        proxy_pass http://127.0.0.1:9900;



        # Preserve client + host info (important for S3-style services)

        proxy_set_header Host $host;

        proxy_set_header X-Real-IP $remote_addr;

        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        proxy_set_header X-Forwarded-Proto $scheme;



        # RustFS / S3 compatibility tweaks

        proxy_http_version 1.1;

        proxy_set_header Connection "";



        # Large uploads support

        client_max_body_size 0;

        proxy_request_buffering off;

    }



    # SSL (Laragon)

    ssl_certificate "/etc/ssl/mycert.crt";

    ssl_certificate_key "/etc/ssl/mykey.key";

    ssl_session_timeout 5m;

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

    ssl_ciphers ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv3:+EXP;

    ssl_prefer_server_ciphers on;



    charset utf-8;

}

That’s it! Now you have local S3-com­pat­i­ble stor­age for local devel­op­ment, that won’t cost you in stor­age and ingress/egress fees, has low laten­cy and won’t use up your bandwidth.