tfversion: Treat Terraform CLI prerelease versions as equal to patch versions in SkipBelow #316
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…versions in SkipBelow Reference: #303 Reference: #308 This change is mainly due to the internals of how github.com/hashicorp/go-version treats version comparisons when there is prerelease data. When the running Terraform CLI is a prerelease version and the given version is a patch version, `SkipBelow` will currently skip the test. However, Terraform CLI prerelease versions are semantically treated as candidates for the associated patch version and therefore should be tested. This adjusts `SkipBelow` for that intended behavior. In the unexpected use case that prerelease version checking is still needed, callers can (still) opt into giving a prerelease version, which will perform the check including prerelease data. The unit testing for the `tfversion` package skip functionality is still manual because `github.com/mitchellh/go-testing-interface` will not immediately stop the test logic Goroutine when the equivalent of `(*testing.T).Skip()` is called nor does it provide helpful troubleshooting information should a test fail as it raises a panic with a generic error message. Future changes could switch to using a different testing interface, however they would require a breaking change to the exported API of this Go module, so that design and effort is being done separately. With the addition of the new tests, but no logic changes: ``` === RUN Test_SkipBelow_SkipTest skip_below_test.go:23: Terraform CLI version 1.0.7 is below minimum version 1.1.0: skipping test --- SKIP: Test_SkipBelow_SkipTest (3.91s) === RUN Test_SkipBelow_RunTest --- PASS: Test_SkipBelow_RunTest (3.32s) === RUN Test_SkipBelow_Prerelease_EqualCoreVersion skip_below_test.go:77: Terraform CLI version 1.8.0-rc1 is below minimum version 1.8.0: skipping test --- SKIP: Test_SkipBelow_Prerelease_EqualCoreVersion (4.23s) === RUN Test_SkipBelow_Prerelease_HigherCoreVersion skip_below_test.go:101: Terraform CLI version 1.7.0-rc1 is below minimum version 1.8.0: skipping test --- SKIP: Test_SkipBelow_Prerelease_HigherCoreVersion (4.40s) === RUN Test_SkipBelow_Prerelease_HigherPrerelease skip_below_test.go:122: Terraform CLI version 1.7.0-rc1 is below minimum version 1.7.0-rc2: skipping test --- SKIP: Test_SkipBelow_Prerelease_HigherPrerelease (3.32s) === RUN Test_SkipBelow_Prerelease_LowerCoreVersion --- PASS: Test_SkipBelow_Prerelease_LowerCoreVersion (3.93s) === RUN Test_SkipBelow_Prerelease_LowerPrerelease --- PASS: Test_SkipBelow_Prerelease_LowerPrerelease (3.38s) ``` After logic changes (note only difference is `Test_SkipBelow_Prerelease_EqualCoreVersion`): ``` === RUN Test_SkipBelow_SkipTest skip_below_test.go:22: Terraform CLI version 1.0.7 is below minimum version 1.1.0: skipping test --- SKIP: Test_SkipBelow_SkipTest (4.17s) === RUN Test_SkipBelow_RunTest --- PASS: Test_SkipBelow_RunTest (3.21s) === RUN Test_SkipBelow_Prerelease_EqualCoreVersion --- PASS: Test_SkipBelow_Prerelease_EqualCoreVersion (3.21s) === RUN Test_SkipBelow_Prerelease_HigherCoreVersion skip_below_test.go:99: Terraform CLI version 1.7.0-rc1 is below minimum version 1.8.0: skipping test --- SKIP: Test_SkipBelow_Prerelease_HigherCoreVersion (3.18s) === RUN Test_SkipBelow_Prerelease_HigherPrerelease skip_below_test.go:120: Terraform CLI version 1.7.0-rc1 is below minimum version 1.7.0-rc2: skipping test --- SKIP: Test_SkipBelow_Prerelease_HigherPrerelease (3.21s) === RUN Test_SkipBelow_Prerelease_LowerCoreVersion --- PASS: Test_SkipBelow_Prerelease_LowerCoreVersion (3.04s) === RUN Test_SkipBelow_Prerelease_LowerPrerelease --- PASS: Test_SkipBelow_Prerelease_LowerPrerelease (3.41s) ```
|
This type of change should probably also occur on the other functions, but I wanted to first gauge feelings on the highest impact function, since there is expected to be a number of provider-facing core features in the coming releases. |
austinvalle
approved these changes
Mar 27, 2024
SBGoods
approved these changes
Mar 27, 2024
|
Great idea, @SBGoods! I'm going to merge this in since I'm not sure when I'm going to be able to do the rest, but will keep that in mind. 👍 |
bflad
added a commit
that referenced
this pull request
Mar 28, 2024
Reference: #303 Reference: #308 Reference: #316 These changes fully update the rest of the `tfversion` built-in checks for treating prereleases as semantically equal to the patch version. It also includes a fix for `RequireBelow` to handle the exactly equal condition. Previously, `RequireBelow` when given 1.8.0 would still run on 1.8.0 (and prereleases variants). Prior to logic changes: ``` === RUN Test_RequireAbove --- PASS: Test_RequireAbove (3.01s) === RUN Test_RequireAbove_Error --- PASS: Test_RequireAbove_Error (3.24s) === RUN Test_RequireAbove_Prerelease_EqualCoreVersion require_above_test.go:77: expected Terraform CLI version above 1.8.0 but detected version is 1.8.0-rc1 --- FAIL: Test_RequireAbove_Prerelease_EqualCoreVersion (3.97s) === RUN Test_RequireAbove_Prerelease_HigherCoreVersion --- PASS: Test_RequireAbove_Prerelease_HigherCoreVersion (3.17s) === RUN Test_RequireAbove_Prerelease_HigherPrerelease --- PASS: Test_RequireAbove_Prerelease_HigherPrerelease (3.03s) === RUN Test_RequireAbove_Prerelease_LowerCoreVersion --- PASS: Test_RequireAbove_Prerelease_LowerCoreVersion (3.13s) === RUN Test_RequireAbove_Prerelease_LowerPrerelease --- PASS: Test_RequireAbove_Prerelease_LowerPrerelease (3.14s) === RUN Test_RequireBelow --- PASS: Test_RequireBelow (2.94s) === RUN Test_RequireBelow_Error --- PASS: Test_RequireBelow_Error (2.59s) === RUN Test_RequireBelow_Prerelease_EqualCoreVersion require_below_test.go:80: expected t.Fatal(), got none --- FAIL: Test_RequireBelow_Prerelease_EqualCoreVersion (3.29s) === RUN Test_RequireBelow_Prerelease_HigherCoreVersion --- PASS: Test_RequireBelow_Prerelease_HigherCoreVersion (3.04s) === RUN Test_RequireBelow_Prerelease_HigherPrerelease --- PASS: Test_RequireBelow_Prerelease_HigherPrerelease (3.22s) === RUN Test_RequireBelow_Prerelease_LowerCoreVersion --- PASS: Test_RequireBelow_Prerelease_LowerCoreVersion (2.74s) === RUN Test_RequireBelow_Prerelease_LowerPrerelease --- PASS: Test_RequireBelow_Prerelease_LowerPrerelease (2.80s) === RUN Test_RequireBetween --- PASS: Test_RequireBetween (2.69s) === RUN Test_RequireBetween_Error_BelowMin --- PASS: Test_RequireBetween_Error_BelowMin (2.57s) === RUN Test_RequireBetween_Error_EqToMax --- PASS: Test_RequireBetween_Error_EqToMax (2.29s) === RUN Test_RequireBetween_Prerelease_MaxEqualCoreVersion require_between_test.go:114: expected t.Fatal(), got none --- FAIL: Test_RequireBetween_Prerelease_MaxEqualCoreVersion (3.24s) === RUN Test_RequireBetween_Prerelease_MinEqualCoreVersion require_between_test.go:141: expected Terraform CLI version between 1.8.0 and 1.9.0 but detected version is 1.8.0-rc1 --- FAIL: Test_RequireBetween_Prerelease_MinEqualCoreVersion (2.83s) === RUN Test_RequireBetween_Prerelease_MaxHigherCoreVersion --- PASS: Test_RequireBetween_Prerelease_MaxHigherCoreVersion (3.17s) === RUN Test_RequireBetween_Prerelease_MinHigherCoreVersion --- PASS: Test_RequireBetween_Prerelease_MinHigherCoreVersion (3.09s) === RUN Test_RequireBetween_Prerelease_MaxHigherPrerelease --- PASS: Test_RequireBetween_Prerelease_MaxHigherPrerelease (3.84s) === RUN Test_RequireBetween_Prerelease_MinHigherPrerelease --- PASS: Test_RequireBetween_Prerelease_MinHigherPrerelease (3.02s) === RUN Test_RequireBetween_Prerelease_MaxLowerCoreVersion --- PASS: Test_RequireBetween_Prerelease_MaxLowerCoreVersion (2.84s) === RUN Test_RequireBetween_Prerelease_MinLowerCoreVersion --- PASS: Test_RequireBetween_Prerelease_MinLowerCoreVersion (4.07s) === RUN Test_RequireBetween_Prerelease_MaxLowerPrerelease --- PASS: Test_RequireBetween_Prerelease_MaxLowerPrerelease (2.79s) === RUN Test_RequireBetween_Prerelease_MinLowerPrerelease --- PASS: Test_RequireBetween_Prerelease_MinLowerPrerelease (3.25s) === RUN Test_RequireNot --- PASS: Test_RequireNot (3.70s) === RUN Test_RequireNot_Error --- PASS: Test_RequireNot_Error (2.76s) === RUN Test_RequireNot_Prerelease_EqualCoreVersion require_not_test.go:73: expected t.Fatal(), got none --- FAIL: Test_RequireNot_Prerelease_EqualCoreVersion (3.06s) === RUN Test_RequireNot_Prerelease_HigherCoreVersion --- PASS: Test_RequireNot_Prerelease_HigherCoreVersion (3.19s) === RUN Test_RequireNot_Prerelease_HigherPrerelease --- PASS: Test_RequireNot_Prerelease_HigherPrerelease (3.25s) === RUN Test_RequireNot_Prerelease_LowerCoreVersion --- PASS: Test_RequireNot_Prerelease_LowerCoreVersion (5.37s) === RUN Test_RequireNot_Prerelease_LowerPrerelease --- PASS: Test_RequireNot_Prerelease_LowerPrerelease (3.13s) === RUN Test_SkipAbove_SkipTest skip_above_test.go:22: Terraform CLI version 1.3.0 is above maximum version 1.2.9: skipping test --- SKIP: Test_SkipAbove_SkipTest (2.34s) === RUN Test_SkipAbove_RunTest --- PASS: Test_SkipAbove_RunTest (2.51s) === RUN Test_SkipAbove_Prerelease_EqualCoreVersion --- PASS: Test_SkipAbove_Prerelease_EqualCoreVersion (2.96s) === RUN Test_SkipAbove_Prerelease_HigherCoreVersion --- PASS: Test_SkipAbove_Prerelease_HigherCoreVersion (3.22s) === RUN Test_SkipAbove_Prerelease_HigherPrerelease --- PASS: Test_SkipAbove_Prerelease_HigherPrerelease (3.29s) === RUN Test_SkipAbove_Prerelease_LowerCoreVersion skip_above_test.go:145: Terraform CLI version 1.8.0-rc1 is above maximum version 1.7.0: skipping test --- SKIP: Test_SkipAbove_Prerelease_LowerCoreVersion (2.68s) === RUN Test_SkipAbove_Prerelease_LowerPrerelease skip_above_test.go:166: Terraform CLI version 1.8.0-rc1 is above maximum version 1.8.0-beta1: skipping test --- SKIP: Test_SkipAbove_Prerelease_LowerPrerelease (2.62s) === RUN Test_SkipBetween_SkipTest skip_between_test.go:24: Terraform CLI version 1.2.0 is between 1.2.0 and 1.3.0: skipping test. --- SKIP: Test_SkipBetween_SkipTest (2.70s) === RUN Test_SkipBetween_RunTest_AboveMax --- PASS: Test_SkipBetween_RunTest_AboveMax (2.71s) === RUN Test_SkipBetween_RunTest_EqToMin --- PASS: Test_SkipBetween_RunTest_EqToMin (2.76s) === RUN Test_SkipBetween_Prerelease_MaxEqualCoreVersion skip_between_test.go:106: Terraform CLI version 1.8.0-rc1 is between 1.7.0 and 1.8.0: skipping test. --- SKIP: Test_SkipBetween_Prerelease_MaxEqualCoreVersion (3.04s) === RUN Test_SkipBetween_Prerelease_MinEqualCoreVersion --- PASS: Test_SkipBetween_Prerelease_MinEqualCoreVersion (3.36s) === RUN Test_SkipBetween_Prerelease_MaxHigherCoreVersion skip_between_test.go:154: Terraform CLI version 1.7.0-rc1 is between 1.6.0 and 1.8.0: skipping test. --- SKIP: Test_SkipBetween_Prerelease_MaxHigherCoreVersion (2.82s) === RUN Test_SkipBetween_Prerelease_MinHigherCoreVersion --- PASS: Test_SkipBetween_Prerelease_MinHigherCoreVersion (3.81s) === RUN Test_SkipBetween_Prerelease_MaxHigherPrerelease skip_between_test.go:198: Terraform CLI version 1.7.0-rc1 is between 1.6.0 and 1.7.0-rc2: skipping test. --- SKIP: Test_SkipBetween_Prerelease_MaxHigherPrerelease (3.37s) === RUN Test_SkipBetween_Prerelease_MinHigherPrerelease --- PASS: Test_SkipBetween_Prerelease_MinHigherPrerelease (4.33s) === RUN Test_SkipBetween_Prerelease_MaxLowerCoreVersion --- PASS: Test_SkipBetween_Prerelease_MaxLowerCoreVersion (3.33s) === RUN Test_SkipBetween_Prerelease_MinLowerCoreVersion skip_between_test.go:261: Terraform CLI version 1.8.0-rc1 is between 1.7.0 and 1.9.0: skipping test. --- SKIP: Test_SkipBetween_Prerelease_MinLowerCoreVersion (3.12s) === RUN Test_SkipBetween_Prerelease_MaxLowerPrerelease --- PASS: Test_SkipBetween_Prerelease_MaxLowerPrerelease (3.08s) === RUN Test_SkipBetween_Prerelease_MinLowerPrerelease skip_between_test.go:303: Terraform CLI version 1.8.0-rc1 is between 1.8.0-beta1 and 1.9.0: skipping test. --- SKIP: Test_SkipBetween_Prerelease_MinLowerPrerelease (2.86s) === RUN Test_SkipIf_SkipTest skip_if_test.go:24: Terraform CLI version is 1.4.3: skipping test. --- SKIP: Test_SkipIf_SkipTest (2.52s) === RUN Test_SkipIf_RunTest --- PASS: Test_SkipIf_RunTest (3.15s) === RUN Test_SkipIf_Prerelease_EqualCoreVersion --- PASS: Test_SkipIf_Prerelease_EqualCoreVersion (4.58s) === RUN Test_SkipIf_Prerelease_HigherCoreVersion --- PASS: Test_SkipIf_Prerelease_HigherCoreVersion (3.46s) === RUN Test_SkipIf_Prerelease_HigherPrerelease --- PASS: Test_SkipIf_Prerelease_HigherPrerelease (3.28s) === RUN Test_SkipIf_Prerelease_LowerCoreVersion --- PASS: Test_SkipIf_Prerelease_LowerCoreVersion (3.10s) === RUN Test_SkipIf_Prerelease_LowerPrerelease --- PASS: Test_SkipIf_Prerelease_LowerPrerelease (3.06s) ``` Now (only showing skip testing as require testing will fail if incorrect): ``` === RUN Test_SkipAbove_SkipTest skip_above_test.go:22: Terraform CLI version 1.3.0 is above maximum version 1.2.9: skipping test --- SKIP: Test_SkipAbove_SkipTest (2.87s) === RUN Test_SkipAbove_RunTest --- PASS: Test_SkipAbove_RunTest (2.66s) === RUN Test_SkipAbove_Prerelease_EqualCoreVersion --- PASS: Test_SkipAbove_Prerelease_EqualCoreVersion (2.98s) === RUN Test_SkipAbove_Prerelease_HigherCoreVersion --- PASS: Test_SkipAbove_Prerelease_HigherCoreVersion (3.19s) === RUN Test_SkipAbove_Prerelease_HigherPrerelease --- PASS: Test_SkipAbove_Prerelease_HigherPrerelease (3.29s) === RUN Test_SkipAbove_Prerelease_LowerCoreVersion skip_above_test.go:145: Terraform CLI version 1.8.0-rc1 is above maximum version 1.7.0: skipping test --- SKIP: Test_SkipAbove_Prerelease_LowerCoreVersion (3.32s) === RUN Test_SkipAbove_Prerelease_LowerPrerelease skip_above_test.go:166: Terraform CLI version 1.8.0-rc1 is above maximum version 1.8.0-beta1: skipping test --- SKIP: Test_SkipAbove_Prerelease_LowerPrerelease (2.72s) === RUN Test_SkipBetween_SkipTest skip_between_test.go:24: Terraform CLI version 1.2.0 is between 1.2.0 and 1.3.0: skipping test. --- SKIP: Test_SkipBetween_SkipTest (2.97s) === RUN Test_SkipBetween_RunTest_AboveMax --- PASS: Test_SkipBetween_RunTest_AboveMax (2.69s) === RUN Test_SkipBetween_RunTest_EqToMin --- PASS: Test_SkipBetween_RunTest_EqToMin (2.47s) === RUN Test_SkipBetween_Prerelease_MaxEqualCoreVersion --- PASS: Test_SkipBetween_Prerelease_MaxEqualCoreVersion (3.65s) === RUN Test_SkipBetween_Prerelease_MinEqualCoreVersion skip_between_test.go:131: Terraform CLI version 1.8.0-rc1 is between 1.8.0 and 1.9.0: skipping test. --- SKIP: Test_SkipBetween_Prerelease_MinEqualCoreVersion (3.10s) === RUN Test_SkipBetween_Prerelease_MaxHigherCoreVersion skip_between_test.go:154: Terraform CLI version 1.7.0-rc1 is between 1.6.0 and 1.8.0: skipping test. --- SKIP: Test_SkipBetween_Prerelease_MaxHigherCoreVersion (2.86s) === RUN Test_SkipBetween_Prerelease_MinHigherCoreVersion --- PASS: Test_SkipBetween_Prerelease_MinHigherCoreVersion (3.46s) === RUN Test_SkipBetween_Prerelease_MaxHigherPrerelease skip_between_test.go:198: Terraform CLI version 1.7.0-rc1 is between 1.6.0 and 1.7.0-rc2: skipping test. --- SKIP: Test_SkipBetween_Prerelease_MaxHigherPrerelease (3.04s) === RUN Test_SkipBetween_Prerelease_MinHigherPrerelease --- PASS: Test_SkipBetween_Prerelease_MinHigherPrerelease (3.39s) === RUN Test_SkipBetween_Prerelease_MaxLowerCoreVersion --- PASS: Test_SkipBetween_Prerelease_MaxLowerCoreVersion (3.01s) === RUN Test_SkipBetween_Prerelease_MinLowerCoreVersion skip_between_test.go:261: Terraform CLI version 1.8.0-rc1 is between 1.7.0 and 1.9.0: skipping test. --- SKIP: Test_SkipBetween_Prerelease_MinLowerCoreVersion (2.66s) === RUN Test_SkipBetween_Prerelease_MaxLowerPrerelease --- PASS: Test_SkipBetween_Prerelease_MaxLowerPrerelease (3.24s) === RUN Test_SkipBetween_Prerelease_MinLowerPrerelease skip_between_test.go:303: Terraform CLI version 1.8.0-rc1 is between 1.8.0-beta1 and 1.9.0: skipping test. --- SKIP: Test_SkipBetween_Prerelease_MinLowerPrerelease (2.91s) === RUN Test_SkipIf_SkipTest skip_if_test.go:24: Terraform CLI version is 1.4.3: skipping test. --- SKIP: Test_SkipIf_SkipTest (2.51s) === RUN Test_SkipIf_RunTest --- PASS: Test_SkipIf_RunTest (3.12s) === RUN Test_SkipIf_Prerelease_EqualCoreVersion skip_if_test.go:70: Terraform CLI version is 1.8.0: skipping test. --- SKIP: Test_SkipIf_Prerelease_EqualCoreVersion (2.51s) === RUN Test_SkipIf_Prerelease_HigherCoreVersion --- PASS: Test_SkipIf_Prerelease_HigherCoreVersion (3.22s) === RUN Test_SkipIf_Prerelease_HigherPrerelease --- PASS: Test_SkipIf_Prerelease_HigherPrerelease (3.27s) === RUN Test_SkipIf_Prerelease_LowerCoreVersion --- PASS: Test_SkipIf_Prerelease_LowerCoreVersion (3.03s) === RUN Test_SkipIf_Prerelease_LowerPrerelease --- PASS: Test_SkipIf_Prerelease_LowerPrerelease (3.15s) ```
bflad
added a commit
that referenced
this pull request
Mar 29, 2024
Reference: #303 Reference: #308 Reference: #316 These changes fully update the rest of the `tfversion` built-in checks for treating prereleases as semantically equal to the patch version. It also includes a fix for `RequireBelow` to handle the exactly equal condition. Previously, `RequireBelow` when given 1.8.0 would still run on 1.8.0 (and prereleases variants). Prior to logic changes: ``` === RUN Test_RequireAbove --- PASS: Test_RequireAbove (3.01s) === RUN Test_RequireAbove_Error --- PASS: Test_RequireAbove_Error (3.24s) === RUN Test_RequireAbove_Prerelease_EqualCoreVersion require_above_test.go:77: expected Terraform CLI version above 1.8.0 but detected version is 1.8.0-rc1 --- FAIL: Test_RequireAbove_Prerelease_EqualCoreVersion (3.97s) === RUN Test_RequireAbove_Prerelease_HigherCoreVersion --- PASS: Test_RequireAbove_Prerelease_HigherCoreVersion (3.17s) === RUN Test_RequireAbove_Prerelease_HigherPrerelease --- PASS: Test_RequireAbove_Prerelease_HigherPrerelease (3.03s) === RUN Test_RequireAbove_Prerelease_LowerCoreVersion --- PASS: Test_RequireAbove_Prerelease_LowerCoreVersion (3.13s) === RUN Test_RequireAbove_Prerelease_LowerPrerelease --- PASS: Test_RequireAbove_Prerelease_LowerPrerelease (3.14s) === RUN Test_RequireBelow --- PASS: Test_RequireBelow (2.94s) === RUN Test_RequireBelow_Error --- PASS: Test_RequireBelow_Error (2.59s) === RUN Test_RequireBelow_Prerelease_EqualCoreVersion require_below_test.go:80: expected t.Fatal(), got none --- FAIL: Test_RequireBelow_Prerelease_EqualCoreVersion (3.29s) === RUN Test_RequireBelow_Prerelease_HigherCoreVersion --- PASS: Test_RequireBelow_Prerelease_HigherCoreVersion (3.04s) === RUN Test_RequireBelow_Prerelease_HigherPrerelease --- PASS: Test_RequireBelow_Prerelease_HigherPrerelease (3.22s) === RUN Test_RequireBelow_Prerelease_LowerCoreVersion --- PASS: Test_RequireBelow_Prerelease_LowerCoreVersion (2.74s) === RUN Test_RequireBelow_Prerelease_LowerPrerelease --- PASS: Test_RequireBelow_Prerelease_LowerPrerelease (2.80s) === RUN Test_RequireBetween --- PASS: Test_RequireBetween (2.69s) === RUN Test_RequireBetween_Error_BelowMin --- PASS: Test_RequireBetween_Error_BelowMin (2.57s) === RUN Test_RequireBetween_Error_EqToMax --- PASS: Test_RequireBetween_Error_EqToMax (2.29s) === RUN Test_RequireBetween_Prerelease_MaxEqualCoreVersion require_between_test.go:114: expected t.Fatal(), got none --- FAIL: Test_RequireBetween_Prerelease_MaxEqualCoreVersion (3.24s) === RUN Test_RequireBetween_Prerelease_MinEqualCoreVersion require_between_test.go:141: expected Terraform CLI version between 1.8.0 and 1.9.0 but detected version is 1.8.0-rc1 --- FAIL: Test_RequireBetween_Prerelease_MinEqualCoreVersion (2.83s) === RUN Test_RequireBetween_Prerelease_MaxHigherCoreVersion --- PASS: Test_RequireBetween_Prerelease_MaxHigherCoreVersion (3.17s) === RUN Test_RequireBetween_Prerelease_MinHigherCoreVersion --- PASS: Test_RequireBetween_Prerelease_MinHigherCoreVersion (3.09s) === RUN Test_RequireBetween_Prerelease_MaxHigherPrerelease --- PASS: Test_RequireBetween_Prerelease_MaxHigherPrerelease (3.84s) === RUN Test_RequireBetween_Prerelease_MinHigherPrerelease --- PASS: Test_RequireBetween_Prerelease_MinHigherPrerelease (3.02s) === RUN Test_RequireBetween_Prerelease_MaxLowerCoreVersion --- PASS: Test_RequireBetween_Prerelease_MaxLowerCoreVersion (2.84s) === RUN Test_RequireBetween_Prerelease_MinLowerCoreVersion --- PASS: Test_RequireBetween_Prerelease_MinLowerCoreVersion (4.07s) === RUN Test_RequireBetween_Prerelease_MaxLowerPrerelease --- PASS: Test_RequireBetween_Prerelease_MaxLowerPrerelease (2.79s) === RUN Test_RequireBetween_Prerelease_MinLowerPrerelease --- PASS: Test_RequireBetween_Prerelease_MinLowerPrerelease (3.25s) === RUN Test_RequireNot --- PASS: Test_RequireNot (3.70s) === RUN Test_RequireNot_Error --- PASS: Test_RequireNot_Error (2.76s) === RUN Test_RequireNot_Prerelease_EqualCoreVersion require_not_test.go:73: expected t.Fatal(), got none --- FAIL: Test_RequireNot_Prerelease_EqualCoreVersion (3.06s) === RUN Test_RequireNot_Prerelease_HigherCoreVersion --- PASS: Test_RequireNot_Prerelease_HigherCoreVersion (3.19s) === RUN Test_RequireNot_Prerelease_HigherPrerelease --- PASS: Test_RequireNot_Prerelease_HigherPrerelease (3.25s) === RUN Test_RequireNot_Prerelease_LowerCoreVersion --- PASS: Test_RequireNot_Prerelease_LowerCoreVersion (5.37s) === RUN Test_RequireNot_Prerelease_LowerPrerelease --- PASS: Test_RequireNot_Prerelease_LowerPrerelease (3.13s) === RUN Test_SkipAbove_SkipTest skip_above_test.go:22: Terraform CLI version 1.3.0 is above maximum version 1.2.9: skipping test --- SKIP: Test_SkipAbove_SkipTest (2.34s) === RUN Test_SkipAbove_RunTest --- PASS: Test_SkipAbove_RunTest (2.51s) === RUN Test_SkipAbove_Prerelease_EqualCoreVersion --- PASS: Test_SkipAbove_Prerelease_EqualCoreVersion (2.96s) === RUN Test_SkipAbove_Prerelease_HigherCoreVersion --- PASS: Test_SkipAbove_Prerelease_HigherCoreVersion (3.22s) === RUN Test_SkipAbove_Prerelease_HigherPrerelease --- PASS: Test_SkipAbove_Prerelease_HigherPrerelease (3.29s) === RUN Test_SkipAbove_Prerelease_LowerCoreVersion skip_above_test.go:145: Terraform CLI version 1.8.0-rc1 is above maximum version 1.7.0: skipping test --- SKIP: Test_SkipAbove_Prerelease_LowerCoreVersion (2.68s) === RUN Test_SkipAbove_Prerelease_LowerPrerelease skip_above_test.go:166: Terraform CLI version 1.8.0-rc1 is above maximum version 1.8.0-beta1: skipping test --- SKIP: Test_SkipAbove_Prerelease_LowerPrerelease (2.62s) === RUN Test_SkipBetween_SkipTest skip_between_test.go:24: Terraform CLI version 1.2.0 is between 1.2.0 and 1.3.0: skipping test. --- SKIP: Test_SkipBetween_SkipTest (2.70s) === RUN Test_SkipBetween_RunTest_AboveMax --- PASS: Test_SkipBetween_RunTest_AboveMax (2.71s) === RUN Test_SkipBetween_RunTest_EqToMin --- PASS: Test_SkipBetween_RunTest_EqToMin (2.76s) === RUN Test_SkipBetween_Prerelease_MaxEqualCoreVersion skip_between_test.go:106: Terraform CLI version 1.8.0-rc1 is between 1.7.0 and 1.8.0: skipping test. --- SKIP: Test_SkipBetween_Prerelease_MaxEqualCoreVersion (3.04s) === RUN Test_SkipBetween_Prerelease_MinEqualCoreVersion --- PASS: Test_SkipBetween_Prerelease_MinEqualCoreVersion (3.36s) === RUN Test_SkipBetween_Prerelease_MaxHigherCoreVersion skip_between_test.go:154: Terraform CLI version 1.7.0-rc1 is between 1.6.0 and 1.8.0: skipping test. --- SKIP: Test_SkipBetween_Prerelease_MaxHigherCoreVersion (2.82s) === RUN Test_SkipBetween_Prerelease_MinHigherCoreVersion --- PASS: Test_SkipBetween_Prerelease_MinHigherCoreVersion (3.81s) === RUN Test_SkipBetween_Prerelease_MaxHigherPrerelease skip_between_test.go:198: Terraform CLI version 1.7.0-rc1 is between 1.6.0 and 1.7.0-rc2: skipping test. --- SKIP: Test_SkipBetween_Prerelease_MaxHigherPrerelease (3.37s) === RUN Test_SkipBetween_Prerelease_MinHigherPrerelease --- PASS: Test_SkipBetween_Prerelease_MinHigherPrerelease (4.33s) === RUN Test_SkipBetween_Prerelease_MaxLowerCoreVersion --- PASS: Test_SkipBetween_Prerelease_MaxLowerCoreVersion (3.33s) === RUN Test_SkipBetween_Prerelease_MinLowerCoreVersion skip_between_test.go:261: Terraform CLI version 1.8.0-rc1 is between 1.7.0 and 1.9.0: skipping test. --- SKIP: Test_SkipBetween_Prerelease_MinLowerCoreVersion (3.12s) === RUN Test_SkipBetween_Prerelease_MaxLowerPrerelease --- PASS: Test_SkipBetween_Prerelease_MaxLowerPrerelease (3.08s) === RUN Test_SkipBetween_Prerelease_MinLowerPrerelease skip_between_test.go:303: Terraform CLI version 1.8.0-rc1 is between 1.8.0-beta1 and 1.9.0: skipping test. --- SKIP: Test_SkipBetween_Prerelease_MinLowerPrerelease (2.86s) === RUN Test_SkipIf_SkipTest skip_if_test.go:24: Terraform CLI version is 1.4.3: skipping test. --- SKIP: Test_SkipIf_SkipTest (2.52s) === RUN Test_SkipIf_RunTest --- PASS: Test_SkipIf_RunTest (3.15s) === RUN Test_SkipIf_Prerelease_EqualCoreVersion --- PASS: Test_SkipIf_Prerelease_EqualCoreVersion (4.58s) === RUN Test_SkipIf_Prerelease_HigherCoreVersion --- PASS: Test_SkipIf_Prerelease_HigherCoreVersion (3.46s) === RUN Test_SkipIf_Prerelease_HigherPrerelease --- PASS: Test_SkipIf_Prerelease_HigherPrerelease (3.28s) === RUN Test_SkipIf_Prerelease_LowerCoreVersion --- PASS: Test_SkipIf_Prerelease_LowerCoreVersion (3.10s) === RUN Test_SkipIf_Prerelease_LowerPrerelease --- PASS: Test_SkipIf_Prerelease_LowerPrerelease (3.06s) ``` Now (only showing skip testing as require testing will fail if incorrect): ``` === RUN Test_SkipAbove_SkipTest skip_above_test.go:22: Terraform CLI version 1.3.0 is above maximum version 1.2.9: skipping test --- SKIP: Test_SkipAbove_SkipTest (2.87s) === RUN Test_SkipAbove_RunTest --- PASS: Test_SkipAbove_RunTest (2.66s) === RUN Test_SkipAbove_Prerelease_EqualCoreVersion --- PASS: Test_SkipAbove_Prerelease_EqualCoreVersion (2.98s) === RUN Test_SkipAbove_Prerelease_HigherCoreVersion --- PASS: Test_SkipAbove_Prerelease_HigherCoreVersion (3.19s) === RUN Test_SkipAbove_Prerelease_HigherPrerelease --- PASS: Test_SkipAbove_Prerelease_HigherPrerelease (3.29s) === RUN Test_SkipAbove_Prerelease_LowerCoreVersion skip_above_test.go:145: Terraform CLI version 1.8.0-rc1 is above maximum version 1.7.0: skipping test --- SKIP: Test_SkipAbove_Prerelease_LowerCoreVersion (3.32s) === RUN Test_SkipAbove_Prerelease_LowerPrerelease skip_above_test.go:166: Terraform CLI version 1.8.0-rc1 is above maximum version 1.8.0-beta1: skipping test --- SKIP: Test_SkipAbove_Prerelease_LowerPrerelease (2.72s) === RUN Test_SkipBetween_SkipTest skip_between_test.go:24: Terraform CLI version 1.2.0 is between 1.2.0 and 1.3.0: skipping test. --- SKIP: Test_SkipBetween_SkipTest (2.97s) === RUN Test_SkipBetween_RunTest_AboveMax --- PASS: Test_SkipBetween_RunTest_AboveMax (2.69s) === RUN Test_SkipBetween_RunTest_EqToMin --- PASS: Test_SkipBetween_RunTest_EqToMin (2.47s) === RUN Test_SkipBetween_Prerelease_MaxEqualCoreVersion --- PASS: Test_SkipBetween_Prerelease_MaxEqualCoreVersion (3.65s) === RUN Test_SkipBetween_Prerelease_MinEqualCoreVersion skip_between_test.go:131: Terraform CLI version 1.8.0-rc1 is between 1.8.0 and 1.9.0: skipping test. --- SKIP: Test_SkipBetween_Prerelease_MinEqualCoreVersion (3.10s) === RUN Test_SkipBetween_Prerelease_MaxHigherCoreVersion skip_between_test.go:154: Terraform CLI version 1.7.0-rc1 is between 1.6.0 and 1.8.0: skipping test. --- SKIP: Test_SkipBetween_Prerelease_MaxHigherCoreVersion (2.86s) === RUN Test_SkipBetween_Prerelease_MinHigherCoreVersion --- PASS: Test_SkipBetween_Prerelease_MinHigherCoreVersion (3.46s) === RUN Test_SkipBetween_Prerelease_MaxHigherPrerelease skip_between_test.go:198: Terraform CLI version 1.7.0-rc1 is between 1.6.0 and 1.7.0-rc2: skipping test. --- SKIP: Test_SkipBetween_Prerelease_MaxHigherPrerelease (3.04s) === RUN Test_SkipBetween_Prerelease_MinHigherPrerelease --- PASS: Test_SkipBetween_Prerelease_MinHigherPrerelease (3.39s) === RUN Test_SkipBetween_Prerelease_MaxLowerCoreVersion --- PASS: Test_SkipBetween_Prerelease_MaxLowerCoreVersion (3.01s) === RUN Test_SkipBetween_Prerelease_MinLowerCoreVersion skip_between_test.go:261: Terraform CLI version 1.8.0-rc1 is between 1.7.0 and 1.9.0: skipping test. --- SKIP: Test_SkipBetween_Prerelease_MinLowerCoreVersion (2.66s) === RUN Test_SkipBetween_Prerelease_MaxLowerPrerelease --- PASS: Test_SkipBetween_Prerelease_MaxLowerPrerelease (3.24s) === RUN Test_SkipBetween_Prerelease_MinLowerPrerelease skip_between_test.go:303: Terraform CLI version 1.8.0-rc1 is between 1.8.0-beta1 and 1.9.0: skipping test. --- SKIP: Test_SkipBetween_Prerelease_MinLowerPrerelease (2.91s) === RUN Test_SkipIf_SkipTest skip_if_test.go:24: Terraform CLI version is 1.4.3: skipping test. --- SKIP: Test_SkipIf_SkipTest (2.51s) === RUN Test_SkipIf_RunTest --- PASS: Test_SkipIf_RunTest (3.12s) === RUN Test_SkipIf_Prerelease_EqualCoreVersion skip_if_test.go:70: Terraform CLI version is 1.8.0: skipping test. --- SKIP: Test_SkipIf_Prerelease_EqualCoreVersion (2.51s) === RUN Test_SkipIf_Prerelease_HigherCoreVersion --- PASS: Test_SkipIf_Prerelease_HigherCoreVersion (3.22s) === RUN Test_SkipIf_Prerelease_HigherPrerelease --- PASS: Test_SkipIf_Prerelease_HigherPrerelease (3.27s) === RUN Test_SkipIf_Prerelease_LowerCoreVersion --- PASS: Test_SkipIf_Prerelease_LowerCoreVersion (3.03s) === RUN Test_SkipIf_Prerelease_LowerPrerelease --- PASS: Test_SkipIf_Prerelease_LowerPrerelease (3.15s) ```
|
I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reference: #303
Reference: #308
This change is mainly due to the internals of how github.com/hashicorp/go-version treats version comparisons when there is prerelease data. When the running Terraform CLI is a prerelease version and the given version is a patch version,
SkipBelowwill currently skip the test. However, Terraform CLI prerelease versions are semantically treated as candidates for the associated patch version and therefore should be tested. This adjustsSkipBelowfor that intended behavior. In the unexpected use case that prerelease version checking is still needed, callers can (still) opt into giving a prerelease version, which will perform the check including prerelease data.The unit testing for the
tfversionpackage skip functionality is still manual becausegithub.conef.uk/mitchellh/go-testing-interfacewill not immediately stop the test logic Goroutine when the equivalent of(*testing.T).Skip()is called nor does it provide helpful troubleshooting information should a test fail as it raises a panic with a generic error message. Future changes could switch to using a different testing interface, however they would require a breaking change to the exported API of this Go module, so that design and effort is being done separately.With the addition of the new tests, but no logic changes:
After logic changes (note only difference is
Test_SkipBelow_Prerelease_EqualCoreVersion):