-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Stop using ctest #5169
Merged
Merged
Stop using ctest #5169
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
CaseyCarter
commented
Dec 6, 2024
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as resolved.
This comment was marked as resolved.
CaseyCarter
force-pushed
the
ctest-no-more
branch
3 times, most recently
from
December 7, 2024 02:59
69e92c1
to
bfd25fb
Compare
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as resolved.
This comment was marked as resolved.
CaseyCarter
commented
Dec 7, 2024
It's pointless to use ctest to run one of two different LIT commands when we can simply run LIT directly. This will make it easier to add future tests: we can simply add more targets, we won't need to jump through hoops to tell ctest not to run the new test all the time. This change will ease adding a new EDG-only test target for the IntelliSense team. Detailed changes: * In `tests/CMakeLists.txt`: * Replace `add_test` targets with `add_custom_target`s. The `stl` and `stlasan` tests are now `test` and `extra-asan-tests` targets, respectively. * Replace `list(APPEND unset_variable_name a b c d)` with the equivalent `set(unset_variable_name a b c d)` which makes it clear that we expect `unset_variable_name` to be undefined. * Merge `${Python_EXECUTABLE}` into `STL_LIT_COMMAND`, and extract `${STL_LIT_TEST_DIRS}`. The makes it easy to reuse `${STL_LIT_COMMAND}` and `${STL_LIT_TEST_DIRS}` with target-specific LIT options. * Note that `USES_TERMINAL` in the `add_custom_target`s makes ninja run those targets serially, maintaining the observable effects of the removed `set_tests_properties(stl stlasan PROPERTIES RUN_SERIAL ON)`. * Remove `enable_testing` from the top-level `CMakeLists.txt`; it does nothing now that we have no `add_test`s. We still use `BUILD_TESTING` to control including the test targets in the generated build system, but I'm not certain there's a good reason to do so. Maybe to enable folks without python to build but not test the STL? * In `run-tests.yml`, replace `ctestOptions` parameter with a `testTargets` parameter, a list of targets to build via `cmake --build meow --target woof quack`. This needs to be set in `asan-pipeline.yml` and plumbed through `build-and-test.yml` with a default of `test` to run the vanilla STL tests. * Remove/Replace the `ctest` discussion in `README.md`. * Drive-by: Report that the `psutil` python module is unavailable at most once per `stl-lit` run. Seeing three repetitions in the CI logs bothered me.
CaseyCarter
force-pushed
the
ctest-no-more
branch
from
December 7, 2024 21:02
b31e90a
to
bfd4cb5
Compare
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as resolved.
This comment was marked as resolved.
StephanTLavavej
requested changes
Dec 8, 2024
|
This comment was marked as resolved.
This comment was marked as resolved.
StephanTLavavej
approved these changes
Dec 9, 2024
I'm mirroring this to the MSVC-internal repo - please notify me if any further changes are pushed. |
🛑 😻 💚 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
It's pointless to use ctest to run one of two different LIT commands when we can simply run LIT directly. This will make it easier to add future tests: we can simply add more targets, we won't need to jump through hoops to tell ctest not to run the new test all the time. This change will ease adding a new EDG-only test target for the IntelliSense team.
Detailed changes:
tests/CMakeLists.txt
:add_test
targets withadd_custom_target
s. Thestl
andstlasan
tests are nowtest
andextra-asan-tests
targets, respectively.list(APPEND unset_variable_name a b c d)
with the equivalentset(unset_variable_name a b c d)
which makes it clear that we expectunset_variable_name
to be undefined.${Python_EXECUTABLE}
intoSTL_LIT_COMMAND
, and extract${STL_LIT_TEST_DIRS}
. The makes it easy to reuse${STL_LIT_COMMAND}
and${STL_LIT_TEST_DIRS}
with target-specific LIT options.USES_TERMINAL
in theadd_custom_target
s makes ninja run those targets serially, maintaining the observable effects of the removedset_tests_properties(stl stlasan PROPERTIES RUN_SERIAL ON)
.enable_testing
from the top-levelCMakeLists.txt
; it does nothing now that we have noadd_test
s. We still useBUILD_TESTING
to control including the test targets in the generated build system, but I'm not certain there's a good reason to do so. Maybe to enable folks without python to build but not test the STL?run-tests.yml
, replacectestOptions
parameter with atestTargets
parameter, a list of targets to build viacmake --build meow --target woof quack
. This needs to be set inasan-pipeline.yml
and plumbed throughbuild-and-test.yml
with a default oftest
to run the vanilla STL tests.ctest
discussion inREADME.md
.psutil
python module is unavailable at most once perstl-lit
run. Seeing three repetitions in the CI logs bothered me.