InodeDirectory consume too much memory

Description

With an alluxio system managing over 50M file/dirs, master can consume more than 100G RAM (The number is different with configuration/workload). After looking at the heap dump, we found that InodeDirctory consume quit a bit memory, further, the mChildren hashmap will allocate 8K slot with each 8bytes, which will consume 64K memory for directory without any children.

This should be investigated to see whether the mChildren is needed to allocated and kept all the time, and if there is some more efficient mechanism.

Environment

None

Status

Assignee

Unassigned

Reporter

Chao Guang Li

Labels

None

Components

Fix versions

Affects versions

1.8.0

Priority

Critical