Add a parallelism factor to the persist command

Description

Currently the persist command can take a file or directory argument. In the case of a directory, it will persist files in the directory sequentially. Since the persist is actually handled by Alluxio workers, the calls can be parallelized without any additional load on the client. We can add a parallelism factor argument to the command so users can effectively trade off between speed and additional load on the workers.

Environment

None

Status

Assignee

Calvin Jia

Reporter

Calvin Jia

Labels

None

Components

Fix versions

Affects versions

Priority

Major
Configure