Alluxio integration with Mapd

Description

alluxio need to integrate with gpu enabled Mapd as mapd

OmniSci Core, an open-source, GPU-accelerated database
https://www.omnisci.com/docs/latest/1_overview.html
https://www.omnisci.com/cloud/

Environment

None

Activity

Show:
Lu
October 9, 2018, 6:16 AM

When trying use Alluxio Fuse mount, occurs error:

Error trying to create file '/mapd-storage/data/mapd_data/table_0_0/epoch', file size 0 does not equal requestedFileSize 4

Vikash Singh
October 11, 2018, 9:22 AM

here id debug logs for Alluxio-fuse when i am executing write command on mounted directory

unique: 79, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 6149
getattr /
unique: 79, success, outsize: 120
unique: 80, opcode: LOOKUP (1), nodeid: 1, insize: 54, pid: 6149
LOOKUP /mapd_catalogs
getattr /mapd_catalogs
NODEID: 7
unique: 80, success, outsize: 144
unique: 81, opcode: OPENDIR (27), nodeid: 7, insize: 48, pid: 6149
unique: 81, success, outsize: 32
unique: 82, opcode: READDIR (28), nodeid: 7, insize: 80, pid: 6149
readdir[0] from 0
unique: 82, success, outsize: 80
unique: 83, opcode: READDIR (28), nodeid: 7, insize: 80, pid: 6149
unique: 83, success, outsize: 16
unique: 84, opcode: RELEASEDIR (29), nodeid: 7, insize: 64, pid: 0
unique: 84, success, outsize: 16
unique: 85, opcode: RMDIR (11), nodeid: 1, insize: 54, pid: 6149
rmdir /mapd_catalogs
unique: 85, success, outsize: 16
unique: 86, opcode: FORGET (2), nodeid: 7, insize: 48, pid: 0
FORGET 7/2
unique: 87, opcode: LOOKUP (1), nodeid: 1, insize: 50, pid: 6149
DELETE: 7
LOOKUP /mapd_data
getattr /mapd_data
NODEID: 9
unique: 87, success, outsize: 144
unique: 88, opcode: OPENDIR (27), nodeid: 9, insize: 48, pid: 6149
unique: 88, success, outsize: 32
unique: 89, opcode: READDIR (28), nodeid: 9, insize: 80, pid: 6149
readdir[0] from 0
unique: 89, success, outsize: 120
unique: 90, opcode: LOOKUP (1), nodeid: 9, insize: 50, pid: 6149
LOOKUP /mapd_data/table_0_0
getattr /mapd_data/table_0_0
NODEID: 10
unique: 90, success, outsize: 144
unique: 91, opcode: OPENDIR (27), nodeid: 10, insize: 48, pid: 6149
unique: 91, success, outsize: 32
unique: 92, opcode: READDIR (28), nodeid: 10, insize: 80, pid: 6149
readdir[0] from 0
unique: 92, success, outsize: 112
unique: 93, opcode: LOOKUP (1), nodeid: 10, insize: 46, pid: 6149
LOOKUP /mapd_data/table_0_0/epoch
getattr /mapd_data/table_0_0/epoch
NODEID: 11
unique: 93, success, outsize: 144
unique: 94, opcode: UNLINK (10), nodeid: 10, insize: 46, pid: 6149
unlink /mapd_data/table_0_0/epoch
unique: 94, success, outsize: 16
unique: 95, opcode: READDIR (28), nodeid: 10, insize: 80, pid: 6149
unique: 95, success, outsize: 16
unique: 96, opcode: FORGET (2), nodeid: 11, insize: 48, pid: 0
FORGET 11/2
DELETE: 11
unique: 97, opcode: RELEASEDIR (29), nodeid: 10, insize: 64, pid: 0
unique: 97, success, outsize: 16
unique: 98, opcode: RMDIR (11), nodeid: 9, insize: 50, pid: 6149
rmdir /mapd_data/table_0_0
unique: 98, success, outsize: 16
unique: 99, opcode: READDIR (28), nodeid: 9, insize: 80, pid: 6149
unique: 99, success, outsize: 16
unique: 100, opcode: FORGET (2), nodeid: 10, insize: 48, pid: 0
FORGET 10/2
DELETE: 10
unique: 101, opcode: RELEASEDIR (29), nodeid: 9, insize: 64, pid: 0
unique: 101, success, outsize: 16
unique: 102, opcode: RMDIR (11), nodeid: 1, insize: 50, pid: 6149
rmdir /mapd_data
unique: 102, success, outsize: 16
unique: 103, opcode: LOOKUP (1), nodeid: 1, insize: 52, pid: 6149
LOOKUP /mapd_export
getattr /mapd_export
unique: 104, opcode: FORGET (2), nodeid: 9, insize: 48, pid: 0
FORGET 9/2
DELETE: 9
NODEID: 8
unique: 103, success, outsize: 144
unique: 105, opcode: OPENDIR (27), nodeid: 8, insize: 48, pid: 6149
unique: 105, success, outsize: 32
unique: 106, opcode: READDIR (28), nodeid: 8, insize: 80, pid: 6149
readdir[0] from 0
unique: 106, success, outsize: 80
unique: 107, opcode: READDIR (28), nodeid: 8, insize: 80, pid: 6149
unique: 107, success, outsize: 16
unique: 108, opcode: RELEASEDIR (29), nodeid: 8, insize: 64, pid: 0
unique: 109, opcode: RMDIR (11), nodeid: 1, insize: 52, pid: 6149
rmdir /mapd_export
unique: 108, success, outsize: 16
unique: 109, success, outsize: 16
unique: 110, opcode: FORGET (2), nodeid: 8, insize: 48, pid: 0
FORGET 8/2
DELETE: 8
unique: 111, opcode: LOOKUP (1), nodeid: 1, insize: 54, pid: 6149
LOOKUP /mapd_catalogs
getattr /mapd_catalogs
unique: 111, error: -2 (No such file or directory), outsize: 16
unique: 112, opcode: MKDIR (9), nodeid: 1, insize: 62, pid: 6149
mkdir /mapd_catalogs 0755 umask=0022
getattr /mapd_catalogs
NODEID: 12
unique: 112, success, outsize: 144
unique: 113, opcode: LOOKUP (1), nodeid: 1, insize: 52, pid: 6149
LOOKUP /mapd_export
getattr /mapd_export
unique: 113, error: -2 (No such file or directory), outsize: 16
unique: 114, opcode: MKDIR (9), nodeid: 1, insize: 60, pid: 6149
mkdir /mapd_export 0755 umask=0022
getattr /mapd_export
NODEID: 13
unique: 114, success, outsize: 144
unique: 115, opcode: LOOKUP (1), nodeid: 1, insize: 50, pid: 6149
LOOKUP /mapd_data
getattr /mapd_data
unique: 115, error: -2 (No such file or directory), outsize: 16
unique: 116, opcode: LOOKUP (1), nodeid: 1, insize: 50, pid: 6149
LOOKUP /mapd_data
getattr /mapd_data
unique: 116, error: -2 (No such file or directory), outsize: 16
unique: 117, opcode: MKDIR (9), nodeid: 1, insize: 58, pid: 6149
mkdir /mapd_data 0755 umask=0022
getattr /mapd_data
NODEID: 14
unique: 117, success, outsize: 144
unique: 118, opcode: LOOKUP (1), nodeid: 14, insize: 50, pid: 6149
LOOKUP /mapd_data/table_0_0
getattr /mapd_data/table_0_0
unique: 118, error: -2 (No such file or directory), outsize: 16
unique: 119, opcode: LOOKUP (1), nodeid: 14, insize: 50, pid: 6149
LOOKUP /mapd_data/table_0_0
getattr /mapd_data/table_0_0
unique: 119, error: -2 (No such file or directory), outsize: 16
unique: 120, opcode: MKDIR (9), nodeid: 14, insize: 58, pid: 6149
mkdir /mapd_data/table_0_0 0755 umask=0022
getattr /mapd_data/table_0_0
NODEID: 15
unique: 120, success, outsize: 144
unique: 121, opcode: LOOKUP (1), nodeid: 15, insize: 46, pid: 6149
LOOKUP /mapd_data/table_0_0/epoch
getattr /mapd_data/table_0_0/epoch
unique: 121, error: -2 (No such file or directory), outsize: 16
unique: 122, opcode: LOOKUP (1), nodeid: 15, insize: 46, pid: 6149
LOOKUP /mapd_data/table_0_0/epoch
getattr /mapd_data/table_0_0/epoch
unique: 122, error: -2 (No such file or directory), outsize: 16
unique: 123, opcode: CREATE (35), nodeid: 15, insize: 62, pid: 6149
create flags: 0x8242 /mapd_data/table_0_0/epoch 0100644 umask=0022
create[1] flags: 0x8242 /mapd_data/table_0_0/epoch
getattr /mapd_data/table_0_0/epoch
NODEID: 16
unique: 123, success, outsize: 160
unique: 124, opcode: READ (15), nodeid: 16, insize: 80, pid: 6149
read[1] 3 bytes from 0 flags: 0x8002
unique: 124, error: -77 (File descriptor in bad state), outsize: 16
unique: 125, opcode: WRITE (16), nodeid: 16, insize: 81, pid: 6149
write[1] 1 bytes to 3 flags: 0x8002
write[1] 1 bytes to 3
unique: 125, success, outsize: 24
unique: 126, opcode: GETATTR (3), nodeid: 16, insize: 56, pid: 6149
getattr /mapd_data/table_0_0/epoch
unique: 126, success, outsize: 120
unique: 127, opcode: FLUSH (25), nodeid: 16, insize: 64, pid: 6149
flush[1]
unique: 127, success, outsize: 16
unique: 128, opcode: RELEASE (18), nodeid: 16, insize: 64, pid: 0
release[1] flags: 0x8002
unique: 128, success, outsize: 16

Assignee

Bin Fan

Reporter

Vikash Singh

Labels

None

Components

Affects versions

Priority

Major
Configure