Parallelize Provisioning Automation

[slickwarren]

It would be nice to have things run in parallel to speed up the time of running tests. Most of provisioning is serially ran right now, but some good examples of things running in parallel are:

• custom cluster nodes currently run in 'parallel' meaning that they are created all at once, and there's a separate part of the func that waits on them to register / come up according to rancher

Some areas that need to be reworked are:

• airgap provisioning
  • corral currently uses cmd.CombinedOutput which waits on the command to complete. This blocks us from running other corrals to provision + register concurrently
• General provisioning tests
  • inside the s.Run() of Permutations -> this will run everything in the 'for' loops (k8s, cni, provider) in parallel
  • at the root level of each Test -> this will run each test in the suite in parallel
  • we would need to investigate how this would affect running multiple suites in the same run

[slickwarren]

Unfortunately the solution I currently have has 1 crux; cleanup of clients before we expect them to. It appears this is a fundamental limitation in Testify: stretchr/testify#1139 (comment)

[slickwarren]

we will have to use native goRoutines, ripping out testify from our provisioning tests if we want this done. putting this back in queue for triage later.