File like "a?b=c" in mounted s3 file system will lead to the restart of master fail

Description

I have a path test on Amazon S3 with a file named "xx?a=b.rtf" in the folder. Now I will list the step of how to reproduce the issue.

1. mount the S3 path like this

1 /bin/alluxio fs mount --readonly --option aws.accessKeyId=<keyid> --option aws.secretKey=<key> /tmp/aws s3a://<bucket>/test

2. list the files of mounted path

1 ./bin/alluxio fs ls /tmp/aws

3. stop the master, and then start it.

The master will be failed with error like this

I found the reason why restarting the master failed. In the 2nd step above, Alluxio will generate an inode file journal entry with file name "xx?a=b.rtf", and then a complete file journal entry. When processing the complete entry in the restart process of the master, InodeTree.lockFullInodePath will be called, the codes is listed below.

1 2 3 4 5 6 7 8 9 10 11 public LockedInodePath lockFullInodePath(AlluxioURI path, LockMode lockMode) throws InvalidPathException, FileDoesNotExistException { TraversalResult traversalResult = traverseToInode(PathUtils.getPathComponents(path.getPath()), lockMode, null); if (!traversalResult.isFound()) { traversalResult.getInodeLockList().close(); throw new FileDoesNotExistException(ExceptionMessage.PATH_DOES_NOT_EXIST.getMessage(path)); } return new MutableLockedInodePath(path, traversalResult.getInodes(), traversalResult.getInodeLockList(), lockMode); }

path.getPath() will return "/tmp/aws/xx", and obviously this path can not be found in the inode tree.

Fixing only this point is very easy. But AlluxioURI.getPath is used more than 200 times, I'm doubting that other place may be not correct also.

Further more, we can not claim the under file system for special characters like "?" does not exist in file name.

Environment

None

Status

Assignee

Andrew Audibert

Reporter

Hector Zhang

Labels

None

Components

Fix versions

Affects versions

1.7.0
1.8.0

Priority

Blocker
Configure