Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test_bump_recipe_cli() for multi-sourced recipes dramatically slows tests down #265

Open
schuylermartin45 opened this issue Dec 5, 2024 · 3 comments
Open

test_bump_recipe_cli() for multi-sourced recipes dramatically slows tests down #265

schuylermartin45 opened this issue Dec 5, 2024 · 3 comments
Assignees
@schuylermartin45
Labels
bug Something isn't working to sort Needs additional investigation
Milestone
Project: Recipe B...

Comments

@schuylermartin45
Copy link
Collaborator

As described/discovered/introduced in #260/#264, concurrently acquiring multiple source artifacts causes a significant slow down in the unit tests. The two tests that fall into this category reference:

  • gsm-amzn2-aarch64_version_bump.yaml
  • libprotobuf_version_bump.yaml

On occasion, these two tests now take about 30 seconds each. Most of this is mitigated by xdist parallelization, but on occassion if both tests are run sequentially, the tests take 60 seconds to complete.

As far as I can currently tell, it seems like pyfakefs is the cause of this slow down, but it is unclear why. Disabling pyfakefs usage seems to remove the issue. Not all runs suffer from this slow down. Presumably pyfakefs is causing some kind of contention that does not result in a complete deadlock.
@schuylermartin45 schuylermartin45 added bug Something isn't working to sort Needs additional investigation labels Dec 5, 2024
@schuylermartin45 schuylermartin45 self-assigned this Dec 5, 2024
@schuylermartin45 schuylermartin45 mentioned this issue Dec 5, 2024
Merged
@schuylermartin45 schuylermartin45 moved this from Ready to In progress in Conda Recipe Manager Dec 9, 2024
@schuylermartin45
Copy link
Collaborator Author

Today on my local machine I've had the tests take 7+ minutes, which is a dramatic and very annoying change. Still stumped at the root cause.

@schuylermartin45
Copy link
Collaborator Author

Found this: https://github.com/pytest-dev/pyfakefs/blob/main/docs/troubleshooting.rst#modules-not-working-with-pyfakefs

@schuylermartin45
Copy link
Collaborator Author

Asked for clarification here: pytest-dev/pyfakefs#1102

schuylermartin45 added a commit that referenced this issue Dec 16, 2024
@schuylermartin45
Adds unit tests for the partial save feature for bump recipe. Warning…
66674de 
…: the new test takes an incredibly long time due to the issue tracked by #265
schuylermartin45 added a commit that referenced this issue Dec 19, 2024
@schuylermartin45
244 bump recipe should exit and save gracefully on some failures (#274)
63b33b1 
* Adds a new save-on-failure flag

* Adds new _CliArgs struct that simplifies CLI argument passing. This should prevent future painful refactors

* Starts wiring work on save-on-failure feature

* Adds unit tests for the partial save feature for bump recipe. Warning: the new test takes an incredibly long time due to the issue tracked by #265

* Fixes a very dumb mistake that caused some tests to take 7+ minutes to complete. I forgot to add the interval timer control for the retry mechanism in a purposefully failing test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@schuylermartin45 schuylermartin45

Labels
bug Something isn't working to sort Needs additional investigation
Projects
Conda Recipe Manager
Status: In progress
Milestone
Project: Recipe Bumper
Development

No branches or pull requests
1 participant
@schuylermartin45