Implement Snapshot Config

[rafikc30]

Labels are used to tag different object, and perform
collective operation based on label configuration values

More information on issue #1094

Signed-off-by: Mohammed Rafi KC [email protected]

[prashanthpai]

As discussed offline, I expected some discussion on the issue first. There's clearly a overlap between metadata and labels. See these issues for some previous discussions:

#462
#1018
#1018

[rafikc30]

@prashanthpai , As far as I understand Metadata can only use as a tag. It cannot have different states, or in other words, it can not perform collective actions of its own. Using metadata, We can list all snapshots, delete all snapshots. But it cannot take intelligent decisions like when should we delete.

For that reason, extending metadata won't be a feasible solution at the moment.
Thoughts ?

[JohnStrunk]

"tags" should be key=value pairs. While they aren't to be interpreted by "core", it's fair game for plugins and peripheral entities to use them. For example, the operator will communicate w/ IVP and auto volume balancing (I need a new name) via the value of node-level tags.

Is key=value sufficient for your purposes?

[prashanthpai]

@rafikc30 What is generally referred as tags or labels in other projects, we call them metadata (key-value pairs). They are not meant to be interpreted by core. There is no inherent meaning (or actions) attached to these arbitrary strings in the core glusterd2.

If you wish to implement snapshot scheduler as an external entity/daemon or a glusterd2 plugin, you can use the metadata infrastructure.

But it cannot take intelligent decisions like when should we delete

The core is not supposed to - regardless of whether you call them labels, tags or metadata.

On the issue you mention:

The first consumer of labels will be Snapshot. We can use labels to list snapshot, delete the snapshot and perform other high-level actions.

You can use metadata for all of these usecases. These higher level actions are performed by user or some external entity. For example:

• list all snapshots that have has a label/tag backup
• delete all snapshots that have a label/tag thumbnails
• delete all snapshots that have timestamp label/tag set to a value older than certain date

For example, In a customer setup, a snapshot can schedule based on different labels like Monthly, Hourly, Yearly, Daily etc. Each one of this can have different configuration values. In a typical set up, the maximum number of a snapshot for the Hourly label can set to 24, while the same value will be 12 for the Monthly snapshot. So performing auto-delete based on this max values will be based on label values.

The core of glusterd2 will not be doing these automatic operations. You can do it as an external daemon or a plugin.

As snapshot is considered core in glusterd2, if you want these automatic operations to be done as core, you might as well make these operations concrete with a well-defined API to schedule+prune snapshots and not use metadata which is arbitrary.

What other specific and real-world usecase does your labels proposal solve beyond snapshot for it to be a generic infrastructure ?

As pointed out earlier, this discussion is best done over the issue.

[atinmu]

@aravindavk @kshlm Would like to get this picked up for GCS/0.4 milestone

[atinmu]

@kshlm can you please add this to your review backlog ?

[kshlm]

@rafikc30 was there any further discussion about metadata vs labels between you and @prashanthpai? The conversation seems to have stopped in between.

[rafikc30]

The discussion were moved to the issue #1094. Initially the PR was submitted as labels being a general use case api where it can be used for volumes,snapshots, etc. But then we decided to move to snapshot alone (please see the issue 1094), because at the moment snapshot were the only consumer.