When a dir is created concurrently, DefaultFileSystemMaster#createDirectory throws FileDoesNotExistException.

Description

[2018-11-06 12:14:30.828 GMT+0800][DefaultFileSystemMasterool-25-thread-60][Failed to Create directory: ]
alluxio.exception.FileDoesNotExistException: Path "/some/path" does not exist.
at alluxio.master.file.meta.LockedInodePath.getInode(LockedInodePath.java:88)
at alluxio.master.file.DefaultFileSystemMaster.createDirectoryInternal(DefaultFileSystemMaster.java:2047)
at alluxio.master.file.DefaultFileSystemMaster.createDirectoryAndJournal(DefaultFileSystemMaster.java:2025)
at alluxio.master.file.DefaultFileSystemMaster.createDirectory(DefaultFileSystemMaster.java:1995)
at alluxio.master.file.FileSystemMasterClientServiceHandler.lambda$createDirectory$2(FileSystemMasterClientServiceHandler.java:136)
at alluxio.RpcUtils.call(RpcUtils.java:217)
at alluxio.RpcUtils.call(RpcUtils.java:189)
at alluxio.master.file.FileSystemMasterClientServiceHandler.createDirectory(FileSystemMasterClientServiceHandler.java:135)
at alluxio.thrift.FileSystemMasterClientService$Processor$createDirectory.getResult(FileSystemMasterClientService.java:1453)
at alluxio.thrift.FileSystemMasterClientService$Processor$createDirectory.getResult(FileSystemMasterClientService.java:1437)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at alluxio.master.file.FileSystemMasterClientServiceProcessor.process(FileSystemMasterClientServiceProcessor.java:78)
at org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

Environment

None

Activity

Show:
Gene Pang
November 15, 2018, 5:08 PM

Could you describe the scenario when this happens? What is the operation (or operations) that are happening for this to happen? What is the desired behavior?

Bin Fan
November 15, 2018, 6:42 PM
ligq
November 16, 2018, 8:14 AM

The bugs occurs when multiple processes create the same dir concurrently(spark insert dynamic partition)

Gene Pang
November 16, 2018, 6:47 PM

I see. If there are multiple creates of the same dir, then why does a FileDoesNotExistException error happen? Should the fix be to figure out why a DNE error happen when there are concurrent creates?

ligq
November 17, 2018, 5:24 AM

It seems to be related to the unlockLast call in the method InodeTree

#createPath.

Assignee

Gene Pang

Reporter

ligq

Labels

None

Components

Affects versions

Priority

Major
Configure