Deploying an S3 Compatible Block Storage Server with MinIO

MinIO [Link] is an open source [Link] high performance object storage API written in GO compatible with Amazon S3 cloud storage. It can handle unstructured data (photos, videos, log files, backups, container images, etc) with size up to 5 TB per object.

Its enterprise class features are capable of strip files in multiple disks and reliably recover the data even under the failure of multiple drives. And based on the implemented parity/hash logic it heals corrupted files on the fly ensuring data integrity.

Moreover, MinIO features server-site encryption, continuous site replication, bucket versioning, deletion protection, event notification, lifecycle management, access rules, work as a cluster, integration to public cloud services…

DEPLOYMENT METHODS

From a binary:
- You literally download the binary version to your CPU architecture and run it with no installation required or install it from the package manager.
From a container:
- With a single line command a new instance of the application can be executed. Great for evaluation and development.
From the source code:
- With the usage of the Golang native features, you can easily pulls and compiles the binary that can be executed. The most secure way.

BINARY EXECUTION

Linux

export MINIO_ROOT_USER=admin
export MINIO_ROOT_PASSWORD=strongpassword
wget https://dl.min.io/server/minio/release/linux-amd64/minio
chmod +x minio
./minio server /PATH/data --console-address ":9001"

ARM (Raspberry PI)

export MINIO_ROOT_USER=admin
export MINIO_ROOT_PASSWORD=strongpassword
wget https://dl.min.io/server/minio/release/linux-arm64/minio
chmod +x minio
./minio server /PATH/data --console-address ":9001"

MacOS

export MINIO_ROOT_USER=admin
export MINIO_ROOT_PASSWORD=strongpassword
wget https://dl.min.io/server/minio/release/darwin-amd64/minio
chmod +x minio
./minio server /PATH/data --console-address ":9001"

Windows

PS> Invoke-WebRequest -Uri "https://dl.min.io/server/minio/release/windows-amd64/minio.exe" -OutFile "C:\minio.exe"
PS> setx MINIO_ROOT_USER admin
PS> setx MINIO_ROOT_PASSWORD strongpassword
PS> C:\minio.exe server D:\Data --console-address ":9001"

PACKAGE MANAGER INSTALLATION

Linux

Or for Debian-based distributions:

wget https://dl.min.io/server/minio/release/linux-amd64/minio_20220620231345.0.0_amd64.deb
sudo dpkg -i minio_20220620231345.0.0_amd64.deb

Or for RHEL-based distributions:

wget https://dl.min.io/server/minio/release/linux-amd64/minio-20220620231345.0.0.x86_64.rpm
sudo rpm -i minio-20220620231345.0.0.x86_64.rpm

ARM (Raspberry PI)

Or for Debian-based distributions:

wget https://dl.min.io/server/minio/release/linux-arm64/minio_20220620231345.0.0_arm64.deb
sudo dpkg -i minio_20220620231345.0.0_amd64.deb

Or for RHEL-based distributions:

wget https://dl.min.io/server/minio/release/linux-arm64/minio-20220620231345.0.0.aarch64.rpm
sudo rpm -i minio-20220620231345.0.0.x86_64.rpm

MacOS

brew install minio/stable/minio
minio server start

DOCKER DEPLOYMENT

Using Docker

sudo docker run -p 9000:9000 -p 9001:9001 -v /PATH/data:/data quay.io/minio/minio server /PATH/data --console-address ":9001"

Using Podman:

podman run -p 9000:9000 -p 9001:9001 -v /PATH/data:/data quay.io/minio/minio server /PATH/data --console-address ":9001"

COMPILING THE SOURCE CODE

Using GO

GO111MODULE=on go install github.com/minio/minio@latest

Manually

git clone https://github.com/minio/minio.git
cd minio/
make
make install

FIREWALL SETTINGS

UFW
- ```
sudo ufw allow 9000:9001/tcp
```

Firewall-CMD

sudo firewall-cmd --zone=public --add-port=9000-9001/tcp --permanent
sudo firewall-cmd --reload

BONUS

Securing

sudo useradd -r minioUser -s /sbin/nologin
sudo mkdir -p /PATH/data
sudo chown minioUser:minioUser /PATH/data
sudo chmod 770 /PATH/data

Configuration File

```
sudo nano /etc/default/minio
```

MINIO_ROOT_USER=admin
MINIO_ROOT_PASSWORD=strongpassword 
MINIO_VOLUMES="/PATH/data"
MINIO_OPTS="--certs-dir /home/minioUser/.minio/certs --console-address :9001"

sudo chown minioUser:minioUser /etc/default/minio
sudo chmod 770 /etc/default/minio

Running as a Service

sudo nano /etc/systemd/system/minio.service

[Unit]
Description=MinIO
Wants=network-online.target
After=network-online.target

[Service]
WorkingDirectory=/usr/local/
User=minioUser
Group=minioUser
ProtectProc=invisible

EnvironmentFile=/etc/default/minio
ExecStart=/usr/local/bin/minio server $MINIO_OPTS $MINIO_VOLUMES
Restart=always
LimitNOFILE=1048576
TasksMax=infinity
TimeoutStopSec=infinity
SendSIGKILL=no

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable minio
sudo systemctl start minio
sudo systemctl status minio

MinIO Client
- Download
  - ```
  wget https://dl.min.io/client/mc/release/linux-amd64/mc
  chmod +x mc
  ./mc --help
  sudo mv mc /usr/local/bin
  mc --help
```
- Configuration
  - ```
  ./mc alias set <ALIAS> http://<YOUR-S3-ENDPOINT>:9000 <YOUR-ACCESS-KEY> <YOUR-SECRET-KEY>
  ./mc ls <ALIAS>
```
- Basic commands (<ALIAS> = s3)
  - mc ls s3
    - List buckets.
  - mc mb s3/new-bucket
    - Make/create a new bucket.
  - mc rm s3/new-bucket
    - Remove/delete a bucket.
  - mc ls s3/bucket
    - List content of the bucket.
  - mc cp localFile s3/bucket/remoteFile
    - Copy file from Local to Remote.
  - mc cp s3/bucket/remoteFile localFile
    - Copy file from Remote to Local.
  - mc cp s3/backup/fileName s3/backup/copiedFile
    - Copy file from and to the same bucket (same for between buckets).
  - mc cp s3/backup/fileName s3/backup/newFIle
    - Copy file in the same bucket (same for between buckets).
  - mc mv s3/backup/fileName s3/backup/renamedFile
    - Rename (or move) file in the same bucket (same for between buckets).
  - mc rm s3/backup/fileToRemove
    - Remove/delete a file (object).
  - mc find s3 –name “*.zip”
    - Find/search in all buckets for a file by name (also by regex, max depth, age…).
  - mc find s3/bucket –name “*.zip” –exec “mc cp {} .”
    - Spawn an external process for each matching object to copy it from to local (or any other action).
  - mc mirror backup/ s3/backup
    - Synchronize a local directory recursively with a remote location (also between remote locations).
  - mc update
    - Update the client to the latest
- Additional commands
  - cat
  - head
  - pipe
  - sql
  - stat
  - tree
  - du
  - retention
  - legalhold
  - share
  - version
  - ilm
  - encrypt
  - event
  - watch
  - undo
  - anonymous
  - tag
  - diff
  - replicate
- Administrative commands
  - mc admin config get s3 site
    - Get config information.
  - mc admin config set s3 site name=mys3
    - Set config information.
  - mc admin service restart s3
    - Restart the service.
- Upgrading
  - ```
  mc admin update s3
```
- Backups
  - ```
  tar zcf - /PATH | mc pipe -q <ALIAS>/<BUCKET>/fileName.tar.gz
  mysqldump --no-tablespaces --lock-tables=false dbName | gzip -c | mc pipe -q <ALIAS>/<BUCKET>/fileName.tar.gz
```
- Versioning and Life Cycle
  - ```
  mc mb --with-lock <ALIAS>/<BUCKET>
  mc version enable <ALIAS>/<BUCKET>
  mc ilm rule add --noncurrent-expire-days 30 <ALIAS>/<BUCKET>
  mc ilm rule add --expire-days 90 --noncurrent-expire-days 30 <ALIAS>/<BUCKET>
  mc ls --versions <ALIAS>/<BUCKET>
```

Encrypting

I recommend using a web server between (such as Apache or NGINX – or event Varnish) the client and the MinIO server to facilitate the management of certificates, virtual hosts, load balance traffic, cache…

But MinIO has its own resources for encrypted communication with self-signed certificate or public certificate (from Let’s Encrypt) using CertBot, for example:

sudo apt install snapd
sudo snap install core 
sudo snap refresh core
sudo snap install --classic certbot
sudo ln -s /snap/bin/certbot /usr/bin/certbot
sudo certbot certonly --standalone --agree-tos --no-eff-email --staple-ocsp --preferred-challenges http -m [email protected] -d minio.domain.com
sudo cp /etc/letsencrypt/live/minio.domain.com/privkey.pem /home/minioUser/.minio/certs/private.key
sudo cp /etc/letsencrypt/live/minio.domain.com/fullchain.pem /home/minioUser/.minio/certs/public.crt
sudo ufw allow 80
sudo ufw allow 443

```
sudo nano /etc/default/minio
```

MINIO_SERVER_URL="https://minio.domain.com:9000"

```
sudo systemctl restart minio
```

Optional

export MINIO_DOMAIN=domain.com
export MINIO_SITE_NAME=mysite
export MINIO_SITE_REGION=global
export MINIO_SITE_COMMENT="My Block Storage"
export MINIO_API_REQUESTS_MAX=1600
export MINIO_API_REQUESTS_DEADLINE=1m
export MINIO_API_CORS_ALLOW_ORIGIN="https://domain-a.com,https://domain-b.com"
export MINIO_API_REMOTE_TRANSPORT_DEADLINE=2h
export MINIO_BROWSER=off

Multiple resources

Maps the resource based on the host address.

minio server https://minio-{1...4}.example.net/mnt/disk-{1...4} --console-address ":9001"

Directory Tree

While the the certificates are stored inside the home by default inside the home directory of the user that runs the binary:
- /home/minioUser/.minio/
  - ```
  └── certs
      └── CAs
```

All the configuration for the instance is stored in another hidden directory inside the data directory (volume directory) as follows:

/PATH/data/.minio.sys/

├── buckets
├── config
│   ├── config.json
│   │   └── xl.meta
│   └── iam
│       ├── format.json
│       │   └── xl.meta
│       ├── policydb
│       │   └── users
│       │       └── username.json
│       │           └── xl.meta
│       ├── sts
│       │   ├── 2G67ZRC44SCKLVLLZVGT
│       │   │   └── identity.json
│       │   │       └── xl.meta
│       │   ├── 3KEUK011NRQIOOS0AK0R
│       │   │   └── identity.json
│       │   │       └── xl.meta
│       │   └── AAQ83Z1Z279SWO7VMH5T
│       │       └── identity.json
│       │           └── xl.meta
│       └── users
│           └── username
│               └── identity.json
│                   └── xl.meta
├── format.json
├── ilm
│   └── deletion-journal.bin
├── multipart
├── tmp
└── tmp-old

READ MORE

Configure a Reverse Proxy with HTTP for the MinIO [Link].

Configure a Reverse Proxy with Cache for the MinIO [Link].

Check the awesome backup tools Duplicati (open source) [Link], Duplicity [Link] and Duplicacy [Link] that are S3 compatible, among many other features.