Adjust staging environment configuration

[bdach]

No description provided.

[bdach]

This turns off purge requests to beatmap mirrors - see implementation:

osu-server-beatmap-submission/osu.Server.BeatmapSubmission/Services/MirrorService.cs

Line 27 in 82190f3

if (await performMirrorAction(mirror, "purge", new Dictionary<string, string> { ["s"] = beatmapSetId.ToString() }) != "1")

Wasn't sure whether it was fine to keep this from production config or not, so I turned it off. Whether that is a valid setup depends on how the beatmap mirror setup is going to work. If the local beatmap storage is going to use the mirror's cache directory as the storage path, then this is required to make that setup work; otherwise, it depends on the specifics of how the mirror is going to function.

[ThePooN]

Hmm I think it's better if applications have a single deployment config for consistency with everything else. More env vars can be used for further tuning (eg. turn off purging).

I'm not sure if a variable to turn off purging is required though, as that should never happen when using local storage (as that would always be a single shared volume), right?

[bdach]

I'm not sure if a variable to turn off purging is required though, as that should never happen when using local storage (as that would always be a single shared volume), right?

Not completely sure what "single shared volume" you're referring to there... Maybe I'll elaborate further on the logic here:

The purge operation on the beatmap mirror removes the .osz from the beatmapCache folder in the mirror folder structure (see https://github.com/peppy/osu-web-10/blob/00da7635a49a0edb7048a2ed924d726c5b8a7e56/www/web/d.php#L60-L68). Now if that cache folder is independent from whatever the primary/master beatmap storage is (which is S3 in production), then it's whatever, that call can be executed no problem, nothing needs to be disabled.

However, if for whatever reason (convenience, storage savings) the primary/master beatmap storage is symlinked/aliased/the same location as the beatmap cache, then the purge logic must be disabled, because every map would just get deleted from master after it's uploaded.

It's all a question of how you'd see the mirror functioning on staging I guess.

[ThePooN]

There is no S3 in staging. How would the beatmap mirror retrieve the osz file if it's not in the same volume?

[bdach]

How would the beatmap mirror retrieve the osz file if it's not in the same volume?

Yes well that is the question here kinda isn't it 😅

There are three options I can think of:

1. Modify d.php on the staging mirror so that it just serves beatmaps from osu-server-beatmap-submission's local storage every time and delete all of the caching logic. In that case the purge calls will become no-ops on the mirror side, so they may as well stay.
2. Modify d.php just for staging so that it can hit osu-server-beatmap-submission's local storage for the .osz rather than s3, and copy it from there instead of downloading it from s3. In that case the purge calls can stay, but there's a bit of a hitch in that there are now potentially 2 copies of the .osz on staging. Maybe fine, maybe not.
3. Delete s3 logic from d.php completely and just have www/beatmapCache of the mirror be the same directory as osu-server-beatmap-submission's local storage via mounting / symlinking / whatever. The local beatmap storage has a file structure compatible with the beatmap cache's, so if you can point both osu-server-beatmap-submission's local storage and the beatmap mirror's cache at the same directory, things will just work without copying. But if you do that, then the purging logic has to be turned off for reasons described above.

[ThePooN]

I'll have a think about this.

[ThePooN]

Hmm I think it's better if applications have a single deployment config for consistency with everything else. More env vars can be used for further tuning (eg. turn off purging).

I'm not sure if a variable to turn off purging is required though, as that should never happen when using local storage (as that would always be a single shared volume), right?

[ThePooN]

We don't use ddog on staging 💸

[bdach]

Roger, made optional in 8d398e1. Note that after that, if the envvar is not set, ddog will not be set up:

osu-server-beatmap-submission/osu.Server.BeatmapSubmission/Program.cs

Lines 116 to 128 in 82190f3

	if (AppSettings.DatadogAgentHost != null)
	{
	DogStatsd.Configure(new StatsdConfig
	{
	StatsdServerName = AppSettings.DatadogAgentHost,
	Prefix = "osu.server.beatmap-submission",
	ConstantTags = new[]
	{
	$@"hostname:{Dns.GetHostName()}",
	$@"startup:{DateTimeOffset.UtcNow.ToUnixTimeSeconds()}"
	}
	});
	}