Add integration-test in the Build

[qu1queee]

This issue is about adding support for integration tests in the Build.

Currently we have two types of testing(unit and e2e tests):

1. unit-test allow us to mock the behaviors of the controllers during Reconciliation. For example if the BuildRun controller triggers a reconcile because a new BuildRun resource was created, the unit-tests allow us to ensure the code we have during reconciliation is doing what is expected to do. Unit-tests also help us to mock the output of some of our public functions, e.g. when we create a TaskRun object. What we cannot do with unit-tests is to test continuous interaction of several components, for example creating a Build -> creating a BuildRun -> creating a TaskRun -> deleting a Build. Or the behaviour of the BuildRuns during deletion when we set a particular annotation in the Build. In general all features that can be executed via kubectl.

2. e2e tests are already in place to test most of the behaviors from creation of a Build until a docker image is pushed to X registry(end-to-end behaviour of the system), also should be the last step to ensure stability on the system. e2e should stay the way they are implemented, by only testing the whole flow. What is required to cover in e2e tests is the ability to build images with different strategies(we already do this) and the ability to build images with particular strategies but using different source code examples(we already do this) . While the idea of e2e tests is to be really close to a production environment, we might consider to modify our current implementation to use a more ad-hoc deployment of the operator via our yamls(we actually do not use this file https://github.com/redhat-developer/build/tree/master/deploy to start the operator), but we rather start the operator directly by starting the operator locally. We might also consider to extend e2e to also cover the part of using the build image to deploy a pod spec or knative svc.

From the above test approaches, we need a more fine-grained way of testing most of our main features on a Kubernetes cluster, which can be features that the user can enable/disable by triggering kubectl commands. For example:

A user applies a Build YAML, that specifies that during the Build resource deletion, all referenced BuildRuns should go away, same for the TaskRuns.

The above is a use case that should not belong to e2e tests(this usecase is only about testing the interactions between components) , and that would be hard to properly test with unit-tests(not ideal while we want to use a k8s based env for testing).

A set of features we need to start testing via integration tests would be:

• Ensure a BuildRun is deleted if a Build specifies the build.build.dev/build-run-deletion annotation
• Ensure that a Build is only deleted after all BuildRuns are deleted, if the Build specifies the build.build.dev/build-run-deletion annotation.
• Ensure that the ownerReferences from the BuildRun to the TaskRun are functional. This basically means that after you delete a BuildRun a TaskRun is deleted.
• Ensure that a BuildRun gets all of the TaskRun conditions after creation. Currently a TaskRun will generate multiple Reconcile events on the BuildRun when the Conditions are being change on the TaskRun.
• Ensure that a BuildRun gets an STARTIME and a COMPLETIONTIME.
• Validate the resources on the TaskRUn pods, based on how Tekton modifies the container´s resource.requests, this can help to spot breaking changes, or validate assumptions.

Also, any new upcoming feature:

• Allowing a Build to automatically generate a BuildRun, should be easy to cover via integration-tests.

Around the integration-test implementation:

• I would like to have each test running on a separate namespace(per ginkgo node) with an instance of the build operator. And run all of them in parallel.
• We should delete all namespaces after each test is done.
• We could also dump logs if a test failed, to debug issues.
• We should use the go client to generate API calls and for polling when waiting for resources.
• we should use the catalog to define test objects.

@sbose78 during last community meeting I briefly mentioned this topic, please take a look.

[qu1queee]

I will be working on this soonish!