To reproduce, set
then launch a worker/master and run "runTests" three times. Afterwards, restart the master. The second master will fail to start.
The root problem is that running "runTests" multiple times causes us to delete a persisted inode, and during journal replay of persisted inode deletion, we attempt to query the UFS to make sure it's in sync before deleting the inode. The UFS isn't set up yet during journal replay, so we get an NPE. This issue has been fixed in branch-1.6 and master by https://github.com/Alluxio/alluxio/pull/5962