Support storing master metadata in RocksDB

Description

Currently, all master metadata is stored on the Java heap. This causes issues once you start to scale past 50 million files. The goal of this ticket is to refactor master metadata so that it can be stored externally, and try storing it in RocksDB as an alternative to on-heap.

Environment

None

Status

Assignee

Andrew Audibert

Reporter

Andrew Audibert

Labels

None

Components

Affects versions

2.0.0

Priority

Major
Configure