About Hops File metadata update?

HopsFS
#1

If I create or delete a file, did the changes sync to NDB? or Just keep in a cache and then update to DB in batches?

#2 
EntityManager.add(node);

Will this function update to the database synchronously?

#3

Each RPC is executed in an atomic fashion and the state is persisted in NDB. Depending on the scenario multiple operations within a single RPC call are batched. Check the EnitityManager.begin and commit functions for more information.

#4

Thanks. I think what you mean is that whether it’s in batches or a single update, all updates will be synchronized to the database before commit. If so, why HopsFS still need an extra LogmetadataEvent for each operation? NDB or any Database system has its internal logging method to address durability issues. Looks an extra logmetadataEvent is unnecessary.

#5

logMetadataEvent is part of our provenance system and not related to persistence. It’s in fact still stored in the same database: https://github.com/hopshadoop/hops/blob/master/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/XAttrStorage.java#L127

Anything specific you are looking for here?

#6

No, I just feel that this will make the operation slower. I see. Thank you.