[FEATURE] Log replication

[jmderuty]

First, not sure if feature request is the right category, it's possible the feature already exist, and if not but it wouldn't be too complex to implement, I'm ready to lend an hand :)

I'm looking for a transaction log library I could use in my distributed gameserver project to internally store cluster related data and ZoneTree looked interesting, so I'm considering it. However to do that I'll need to be able to integrate the library in our log replication system. We use the raft algorithm, so we need to have some control on when entries are appended or committed to the log in order to synchronize logs between primaries and replica.

In particular we need to:

• Query the log for last appended and last applied log entry ids
• Intercept log entries to be appended for replication.
• Control when entries are committed.
• Manually append log entries.
• Overwrite/delete uncommitted entries.

Is it something that would be doable with ZoneTree ?

Thank you!

[koculu]

Hi @jmderuty
ZoneTree can be used by any algorithm including Raft.
The operations you have listed can be managed outside of the ZoneTree source code.
Hence, I don't see a particular enhancement point in ZoneTree to achive your goal.

[jmderuty]

Thank you for the quick answer,

In your opinion, how should I proceed to do that with the lib? (not asking for a detailed step by step, if you could just point me at the right extension points I could use to intercept update commands and work on the transaction log.)

[koculu]

You can create your own Transactional Tree by utilizing the public API provided by ZoneTree.

The OptimisticZoneTree classs represents the implementation of the optimistic transactional ZoneTree, serving as a helpful starting point.

As this will be an independent project, there is no requirement for any extension points, as you will need to construct the transactional log from the ground up.

You could find the implementation details at the following GitHub link:
https://github.com/koculu/ZoneTree/blob/main/src/ZoneTree/Transactional/OptimisticZoneTree.cs

[jmderuty]

Thank you, going to try that

[koculu]

Thank you too.