airspec: #2637 Support selecting nested test specs

[xerial]

Support selecting nested test cases with (test name)/(child test name)/... syntax.

Could you take a look? @takezoe @exoego

[codecov]

Codecov Report

Merging #2651 (2fe2414) into master (6b766a6) will increase coverage by 0.00%.
The diff coverage is n/a.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #2651   +/-   ##
=======================================
  Coverage   82.12%   82.13%           
=======================================
  Files         334      334           
  Lines       14030    14030           
  Branches     2182     2182           
=======================================
+ Hits        11522    11523    +1     
+ Misses       2508     2507    -1     

Impacted Files	Coverage Δ
...rx/src/main/scala/wvlet/airframe/rx/RxRunner.scala	95.27% <0.00%> (+0.39%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6b766a6...2fe2414. Read the comment docs.

[exoego]

I think slash separator is a bit awkward, but may be fine for AirSpec's nesting test blocks.

---

MUnit also use test block to declare test, but does not support nested tests, AFAIK.
If we declare a nested test block, assertion in nested block is not executed.

BTW, one can select tests with the following format.

sbt> testOnly -- "*migration v2*"
org.scalasteward.core.BuiltinConfigFilesTest:
  + migration v2 config is valid 0.649s
org.scalasteward.core.vcs.gitlab.GitLabApiAlgTest:

---

Some styles of ScalaTest supports nested tests.

class FooSpec extends org.scalatest.funspec.AnyFunSpec {
  describe("A Set") {
    describe("when empty") {
      it("should have size 0") {
        assert(Set.empty.size == 0)
      }

      it("should produce NoSuchElementException when head is invoked") {
        assertThrows[NoSuchElementException] {
          Set.empty.head
        }
      }
    }
  }
}

then one can select child test cases with -z option and space separators.

sbt> testOnly -- -z "A Set when empty should have size"
[info] FooSpec:
[info] A Set
[info]   when empty
[info]   - should have size 0

I think whitespace separator is more reasonable in ~~Spec style since it looks like a natural paragraph.

[takezoe]

I think whitespace separator is more reasonable in ~~Spec style since it looks like a natural paragraph.

Interesting. We might want to allow both whitespace and slash as a separator because we often group test cases just by the category name when we write test cases in AirSpec.

[xerial]

Thanks.

AirSpec allows partial match of the test names so that the user doesn't always need to specify the full test names with wildcard(*) like test name*. You only need to pick some keywords from individual levels of tests like (keyword1)/(keyword 2)... As test spec names may contain white space characters, I thought having some explicit separator is useful so that we can specify test hierarchies like "test name/child test name" without ambiguity. Other options are :, ., but those symbols are frequently used for test names and regex too, so I avoided them.

ScalaTest -z option is interesting, but if the same keyword appears multiple times like

• test A/test A and B
• test A/test A and C
• test B/test A and test D

In this case, it's a bit difficult to specify test paths like `testA/* because -z "test A" will run everything in this example as the keyword "test A" appears in all of the child paths.

[xerial]

Let me try this /-slash separator first and figure out if it's better to support other pattern matches like -z :)