-
-
Notifications
You must be signed in to change notification settings - Fork 3.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Fix #1924] Use a cache to speed things up #2153
Conversation
Great feature! I had totally forgot about this discussion. A see a few small issues right now:
|
@@ -57,6 +57,9 @@ def validate_compatibility | |||
(@options[:only] & %w(Lint/UnneededDisable UnneededDisable)).any? | |||
fail ArgumentError, 'Lint/UnneededDisable can not be used with --only.' | |||
end | |||
if @options.key?(:cache) && @options.key?(:auto_correct) | |||
fail ArgumentError, '--cache can not be used with --auto-correct.' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why so? Can't we just invalidate the things we auto-correct?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, probably. Either way, it makes no sense to have this check as caching could also be turned on in configuration.
Good feedback. I'll make it on by default, but then I also need to add a |
Working on a solution that addresses the issues raised here. I have also identified a couple of more problems.
|
Pushed some fixes. |
|
||
Large projects containing hundreds or even thousands of files can take | ||
a really long time to inspect, but RuboCop has functionality to | ||
mitigate this problem. A result caching mechanism is activated if the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the term "result" is kind of confusing/unclear. I'd suggest elaborating on this.
Pushed some new updates. Please check again. |
Looks fine to me. |
Btw, one idea for optimization for dealing with config changes or CLI option changes would be to also cache the ASTs, so we can reuse them directly in such scenarios. |
Add `--cache` option and `UseCache` configuration parameter. Store a cache of inspection results at `/tmp/rubocop_cache/`. Clean up when it gets too big (governed by the configuration parameter `MaxFilesInCache`).
e0d699b
to
498580c
Compare
I haven't tested caching the ASTs, so I can't say for sure, but I suspect that it won't do much for performance. So if we push that aside, maybe we're ready to merge? |
[Fix #1924] Use a cache to speed things up
Fair enough. Let's merge this! |
👍 Can't wait to use this! |
I've been working on this for a while, and feel ready to make a pull request. The performance of RuboCop has been discussed lately, so I guess this would be welcomed by some users.
Cache invalidation is generally a hard problem, but I think this solution should work (knock on wood). It uses the source code of the
rubocop
executable, a selection of the passed options, the inspected file, and its configuration as basis.Add
--cache
option andAlwaysRunWithCache
configuration parameter (false
by default). Store a cache of inspection results at/tmp/rubocop_cache/
(or whatever the temporary directory is). Clean up when it gets too big (governed by the configuration parameterMaxFilesInCache
).Performace
There's a slight performance penalty for storing cached results while inspecting files, around 2%, which is why I chose to only store in the cache if the option is given. The execution when there are cached results to reuse is typically between 10 and 30 times faster.