Migrate tree-sitter support to MODULE.bazel #4783
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The WORKSPACE file is deprecated; support is already off by default, and it'll be removed in the next major bazel release. Our main dependency is tree-sitter, and I'm trying to address that here.
We're currently using https://github.com/elliottt/rules_tree_sitter, but that hasn't been updated in a couple years, meaning it lacks MODULE.bazel support. In the registry, there's https://registry.bazel.build/modules/tree-sitter-bazel, but this is only the parser libraries of tree-sitter, not the generator. I'm using it for that much, at least.
For the generator, which transforms grammar.js to parser.c/h, I'm just requiring a non-hermetic invocation (i.e., people who want to work on it will need to install tree-sitter; see the README.md updates). I tried running it manually, but parser.c is about 600 KB; pre-commit rejects files that large and I don't think an exception makes sense to override for this (it'd probably also grow substantially if the grammar were updated to cover more syntax). In order to make the non-hermetic call not break "bazel build //..." for most developers, I'm marking most targets in the package as manual.
Note, I did look long and hard at using
aspect_rules_js
/rules_nodejs
to invoke npm. This took a lot of time, and I have a commit that's mostly working, except I hit a point where it usesdeclare_symlink
which we disallow for compatibility reasons (commit "Lots of work for figuring out rule_js uses declare_symlink" on the PR). As a consequence, I think we can't use the primary supported ways to have hermetic npm calls.Also,
treesitter
->tree_sitter
because it's generally calledtree-sitter
, two words. We even had atreesitter/src/tree_sitter
directory so it's a bit inconsistent.As far as bugs here, the parser library breaks bazel queries, e.g. the error:
I'm just excluding tree_sitter from queries where I can to work around the error.