k8s.io/kubernetes/pkg/master TestValidOpenAPISpec

[0xmichalis]

Haven't seen this one before and I am not sure it has any relation with the PR that hit it.
https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/pr-logs/pull/36243/pull-kubernetes-unit/12124/

@kubernetes/sig-testing-misc

[fraenkel]

From what I can gather, it is being killed due to taking over 5 minutes which is roughly how long it takes (~300s) on successful unit builds.

[deads2k]

bumping priority. I'm seeing this pretty frequently this morning.

@mbohlool I see OpenAPI and associate you. You know something about this one?

[mbohlool]

@deads2k Yes, I wrote this test. It was failing (timing out) for quite some time on jenkins but passing quite fast locally. I've disabled it but after a while it started passing and I assumed it was a configuration change on test machines. I will spend some time on it again to see if I can find the root cause and will disable it otherwise.

@@kubernetes/sig-testing-misc, This test is passing locally in about 15 second. Do you have any idea why it is timing out (or passing near to 300s) on jenkins machines?

@fraenkel where do you see the successful test stats?

[fraenkel]

@mbohlool I see what you do. Locally it takes 25s, with bots it is 300s. My guess is that we are constrained by CPU and all the encode/decode is causing strain either on CPU or GC. The bots fail about 75% for me so I just keep retrying until success. Just had 2 this morning pass at 296s.

[spxtr]

Curiously, it's taking about 20 seconds with Bazel. @rmmh, this might be worth looking at.

[ixdy]

I think we run tests in Jenkins with -race, which might explain the slowness?

[spxtr]

Nevermind, we hit it in a bazel run: https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/pr-logs/pull/36128/pull-kubernetes-unit/12297/

[fraenkel]

-race does kill the performance, 25s -> 166s which is on par with bazel.

[mbohlool]

is it possible to disable -race for this test? and should we do that if it is possible?

[deads2k]

Now blocking the submit queue.

[mbohlool]

I think I am missing something here. I've checked some of the attached PRs. Links to the pkg/master folder in master branch has the change (new file called master_openapi_test.go:

https://github.com/kubernetes/kubernetes/blob/master/pkg/master/master_openapi_test.go

but non of the recent PRs I checked has the file in their branch:

https://github.com/deads2k/kubernetes/tree/generic-17-move-tests/pkg/master
https://github.com/rf232/kubernetes/tree/dashboard-v1.5.1/pkg/master
https://github.com/kargakis/kubernetes/tree/unit-tests-for-the-d-controller/pkg/master

That means they didn't sync with master? What am I missing here?

[mbohlool]

OK. I see, the tests are patching the change of the PR into the master, so everybody should get that if they retest, but it look like my test is not blocking submit queue. there should be another issue.