loadufs for S3 ufs does not work

Description

the loadufs script does not work with an S3 UFS.

example 1:

./alluxio loadufs /<path> s3n://<bucket_name>/src

We get the following exception (the /tou/ folder is S3 does not have a corresponding tou_$folder$ object)
alluxio.exception.FileDoesNotExistException: Path /<path>/uiq/extract/02-03-16/tou does not exist
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at alluxio.exception.AlluxioException.from(AlluxioException.java:72)
at alluxio.AbstractClient.retryRPC(AbstractClient.java:324)
at alluxio.client.file.FileSystemMasterClient.loadMetadata(FileSystemMasterClient.java:293)
at alluxio.client.file.BaseFileSystem.loadMetadata(BaseFileSystem.java:207)
at alluxio.client.file.BaseFileSystem.loadMetadata(BaseFileSystem.java:199)
at alluxio.client.UfsUtils.loadUfs(UfsUtils.java:201)
at alluxio.client.UfsUtils.loadUfs(UfsUtils.java:86)
at alluxio.client.UfsUtils.main(UfsUtils.java:229)

example 2:

  • ./alluxio loadufs alluxio://<ip>:19998/<path> s3n://<bucket_name>/<subdirectory>/

alluxio.exception.InvalidPathException: Path alluxio://<ip>:19998/<path>/<subdirerectory>/<subdirectory>/<subdirectory>/<filename> is invalid
at sun.reflect.GeneratedConstructorAccessor18.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at alluxio.exception.AlluxioException.from(AlluxioException.java:72)
at alluxio.AbstractClient.retryRPC(AbstractClient.java:324)
at alluxio.client.file.FileSystemMasterClient.loadMetadata(FileSystemMasterClient.java:293)
at alluxio.client.file.BaseFileSystem.loadMetadata(BaseFileSystem.java:207)
at alluxio.client.file.BaseFileSystem.loadMetadata(BaseFileSystem.java:199)
at alluxio.client.UfsUtils.loadUfs(UfsUtils.java:171)
at alluxio.client.UfsUtils.loadUfs(UfsUtils.java:86)
at alluxio.client.UfsUtils.main(UfsUtils.java:229)

Environment

None

Assignee

Calvin Jia

Reporter

Gene Pang

Labels

Components

Fix versions

Affects versions

Priority

Major
Configure