Improve logging when job master is at full capacity

Description

When the job master is at full capacity and there are a large number of jobs which are attempting to be scheduled, the logs get clogged with a warning message which looks something like:

2018-10-26 23:52:45,359 WARN job.JobMasterClientServiceHandler (RpcUtils.java:call) - Exit (Error): Run: jobConfig=java.nio.HeapByteBuffer[pos=178 lim=178 cap=178], options=RunTOptions(), Error=alluxio.exception.status.ResourceExhaustedException: Job master is at full capacity

There are enough of these printed that it causes the job master log to roll. The frequency of this logging could be improved.

To reproduce, get a directory with a large number of files (50-100k+) and use ./bin/alluxio fs SetReplication -min 3 /path/to/files in order to trigger a large number of replication jobs, then watch the job_master.log

Environment

None

Status

Assignee

Zac Blanco

Reporter

Zac Blanco

Labels

None

Components

Fix versions

Affects versions

Priority

Minor
Configure