Alluxio Mount Commands Options Needed for IBM Cleversafe

Description

Alluxio mount commands for IBM cleversafe required additional parameters to be supplied during mount. This was discovered when running runTests against the cleversafe bucket which showed the following: Caused by: alluxio.exception.status.UnknownException: com.amazonaws.services.s3.model.AmazonS3Exception: Forbidden (Service: Amazon S3; Status Code: 403; Error Code: 403 Forbidden; Request ID: 963AD12B616EC0AC; S3 Extended Request ID: 71ocKUynTwhhXeOXzkykK7FwROul3TORgsuFhBIZYZpjkZ8XMLjWKa/qLndzn2Ukh0bsx9VAMMk=), S3 Extended Request ID: 71ocKUynTwhhXeOXzkykK7FwROul3TORgsuFhBIZYZpjkZ8XMLjWKa/qLndzn2Ukh0bsx9VAMMk=

Then also added the endpoint property fs mount --option alluxio.underfs.s3.endpoint=https://lb1.ait.cleversafelabs.com as well which then resulted in the following error: Caused by: alluxio.exception.status.UnknownException: com.amazonaws.SdkClientException: Unable to execute HTTP request: handshake alert: unrecognized_name

The above error was resolved by also including the --option alluxio.underfs.s3.disable.dns.buckets=true option when mounting the bucket. However once resolved it reverted to an 404 error:Caused by: alluxio.exception.status.UnknownException: com.amazonaws.services.s3.model.AmazonS3Exception: The specified bucket does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchBucket; Request ID: fb3af696-ba89-46f8-90d1-20941362645a; S3 Extended Request ID: null), S3 Extended Request ID: null

Following that realized that the `s3a://1ade4379-f1a2-43cb-a50f-6ffd86266068.lb1.ait.cleversafelabs.com` path in the mount command should be 's3a://1ade4379-f1a2-43cb-a50f-6ffd86266068' as 1ade4379-f1a2-43cb-a50f-6ffd86266068 is the Vault bucket name and the endpoint(https://lb1.ait.cleversafelabs.com) is to be specified in the endpoint option only

Environment

None

Status

Assignee

Bin Fan

Reporter

Madan Kumar

Labels

None

Components

Affects versions

1.8.1

Priority

Minor
Configure