Introduction and what ObjectiveFS is:
This document explains how to setup the ObjectiveFS software to connect to StorageGRID. ObjectiveFS is a scalable S3 filesystem which gives the ability to mount a Vault as a filesystem on a Linux or MacOS.
The support for this is limited at present, but it does work, but no guarantees on performance can be stated at this time, and technically this is an unsupported configuration, but it does work, and may be useful to ingest data onto StorageGRID
At the time of writing this document, ObjectiveFS has not been through official testing and is therefore not subject to official support. As with all untested applications, mileage may vary and any customers considering this product for production use should do so at their own risk and only after robust testing within their environment.
Test system environment
The system has been tested against a lab StorageGRID with no certificates installed, with 6 storage nodes, 2 gateway nodes and a single primary admin Node.
DNS configuration
Very important and will not work without it – in this case we are using s3.yourdomain.com defined to point at the High availability gateway nodes. This need to be resolvable via DNS, and not cheating using hosts files, as it will not work.
ObjectiveFS Considerations
ObjectiveFS is a distributed shared POSIX filesystem that connects direct to any S3 compliant object store. Once installed you can mount the filesystem like any other disk.
It was initially designed just for AWS S3, but will minor changes it can work with any S3 compliant object store. The tweaks here were tested on a lab system with proper certificates and domains.
ObjectiveFS Install Steps
Initial setup – refer to documents on ObjectiveFS web site:
https://objectivefs.com/user/downloads
Available for MacOS and Linux.
Run the command (after installation):
Mount.objectivefs config:
Enter License Key
Access Key
Secret Key
Default Region – this would be the aws region, but in this case we will use the dns name of the load balancer endpoint previous defined in SG, this needs to be resolvable by DNS, and you need to. Include the appropriate port number.
In the example below, we have DNS configured for s3.yourdomain.com to point to the IP address of the VIP for HA Gateway, and have specified the port as the port defined in the Load Balancing Group of 10433, which is enabled for http, as ObjectiveFS provides client side end to end encryption.
If you want to use HTTPS, then use Stunnel – covered later and also on their website:
These settings will be stored in files in the directory /etc/objectivefs.env on the machine you are running these steps on.
Create New bucket to install files to
This step creates a new bucket where the files will be installed. An existing bucket can be used but the bucket needs to dedicated to the ObjectiveFS filesystem, or thing will go badly wrong.
Mount.objectivefs create ‘bucketname’
If this fails – see sample output below, then you will need to create a new file ‘PATHSYTLE’ in the install directory of /etc/objectivefs.env with the contents of ‘1’. This error is caused by invalid domain names and certificates. (more to follow on this one)
Mount the filesystem
Finally mount the S3 filesystem into an empty directory.
Mount.objectivefs bucketname directoryname
Only thing to be aware of here is the directory name needs to be absolute.
All done, write away.
Example Install steps and output
root@adminnode:/etc# mount.objectivefs config
Creating config in /etc/objectivefs.env
Enter ObjectiveFS license: abcdefghijklmnop
Enter Access Key Id: xxxxxxxxxxxxxxxxxx
Enter Secret Access Key: yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
Enter Default Region (optional): http://s3.yourdomain.com:10443
Creating:
/etc/objectivefs.env
/etc/objectivefs.env/OBJECTIVEFS_LICENSE
/etc/objectivefs.env/AWS_ACCESS_KEY_ID
/etc/objectivefs.env/AWS_SECRET_ACCESS_KEY
/etc/objectivefs.env/AWS_DEFAULT_REGION
ObjectiveFS was successfully configured.
root@adminnode:/etc# mount.objectivefs create bucket10
Passphrase (for http://s3yourdomain.com:10443/bucket10):
Verify passphrase (for http://s3.yourdomain.com:10443/bucket10):
ERROR: s3.yourdomain.com:10443 response: 405 Method Not Allowed, MethodNotAllowed, server: StorageGRID/11.4.0, x-amz-request-id: 1600360187614609, x-amz-id-2: 12684097
mount.objectivefs: unable to create filesystem: MethodNotAllowed
ADDING PATHSTYLE:
root@adminnode:/etc# mount.objectivefs create -vv bucket10
Passphrase (for http://s3.yourdomain.com:10443/bucket10):
Verify passphrase (for http://s3yourdomain.com:10443/bucket10):
NOTE: 3 PUT, 2 LIST, 0 GET, 0 DELETE, 1.604 kB IN, 2.686 kB OUT, CACHE [0.0% HIT, 0.0% META, 0.0% DATA, 2.544 kB OS], DISK [0.0% HIT]
NOTE: Successfully created filesystem: http://s3.yourdomain.com:10443/bucket10 in the region
Mounting:
labuser@adminnode:~$ sudo mount.objectivefs bucket10 mountobject
Passphrase (for http://s3.yourdomain.com:10443/bucket10):
mount.objectivefs: please use absolute path to mount directory
labuser@adminnode:~$ sudo mount.objectivefs bucket10 /home/labuser/mountobject
Passphrase (for http://s3.yourdomain.com:10443/bucket10):
labuser@adminnode:~$ cd mountobject/
labuser@adminnode:~/mountobject$ touch test.txt
labuser@adminnode:~/mountobject$ sudo vim test.txt
labuser@adminnode:~/mountobject$ ls -ll
total 2
-rw-rw-rw- 1 root root 605 Sep 17 16:55 README
-rw-rw-r– 1 labuser labuser 22 Sep 17 17:02 test.txt
labuser@adminnode:~/mountobject$
If you want to use https, you can use stunnel – although ObjectiveFS says it uses built in client side encryption, so the traffic is encrypted anyway.