Centralize building of Vector in one step on GH actions

[binarylogic]

Currently, we build Vector as part of testing Kubernetes, and we have plans to move all of our test to GH actions. It would be nice if we could build Vector once for all of our tests. Or, centralize building as much as possible. Building Vector takes a considerable amount of time and resources, so minimizing this is to our advantage.

[binarylogic]

@a-rodin @MOZGIII I've assigned both of you to this because it overlaps with both the "Move to Github Actions" project and the "Kubernetes Integration" project. Both need to build Vector for tests, so it would be nice if they could use the same binary. I'm not sure if this is possible, but I wanted to include it in #2222.

[MOZGIII]

Ideally, we want to have a tree-like pipeline, with the step at the root that builds the binaries, and the following steps that run them.

I'm thinking about a flow like this:

• For each PR/branch push:

  • "build" step prepares vector binary in a release build mode, and all the test binaries, and a .deb package.

  • Then, in parallel:

    • test binaries are run, utilizing pre-built test binaries
    • k8s integration tests are run, utilizing pre-built vector binary

  • By request, after "build" step completes:

    • test harness runs, utilizing pre-built .deb file

While the "build" and the steps that run afterwards are possible to automate within a single docker-compose container run with a shared file system, it's not going to work for test harness invocation.

We can still build a .deb at "build" stage and upload it to some artifact storage location, like S3. Github Actions have their own artifact storage, but that probably won't work with the issue_comment events that we trigger test harness Github Action on.

[binarylogic]

I'm closing this. You can see in #2352 we're compiling Vector with different features depending on the tests. I don't mind this, especially since we're using Github's free hosted runners. To speed this up we should consider caching. (#1591)