Under file system storage of local file system by SSH protocol

Description

I am using Alluxio as default file system of Spark on Yarn big data processing platform. Sometimes, the user of my platform want to process files on their local file using Spark DataFrams, these local files may be the logs from some server or a training dataset download from web. Currently, the user of my platform must upload these files to Alluxio or HDFS, which must be operated on the user's computer and requiring a java environment, the workflow is not smooth.

With the support of local file under storage by SSH, the user of my platform can mount their folder to Alluxio, and process the files by Spark directly.

Environment

None

Status

Assignee

Hector Zhang

Reporter

Hector Zhang

Labels

None

Components

Affects versions

master

Priority

Major
Configure