Michael Davis c9fa54f2eb highlighter: Tombstone finished layers in the QueryIter
This fixes a bug where a layer could be removed from `QueryIter`'s
`active_layers` because it had no more captures and then reinitialized
later, restarting the query cursor and producing duplicate captures.

The new test case exhibits this behavior. Consider this markdown.

    ```rust
    /// `Something`
    /// Anything
    ```

This consists of four layers:

* The root layer, the entire markdown document
* The Rust injection in the codefence
* Two markdown injections within the Rust doc comments. These injections
  are tagged with `injection.combined` so they form one markdown layer.
* Two markdown.inline injections within those markdown injections. Again
  these injections are combined so they form one layer.

Layers are removed from `QueryIter` when they have no more captures. In
this example code the "`Something`" was the only capture in the two
injections of the markdown.inline layer, so after the first injection
was finished, the layer would be removed. Once the `QueryIter` reached
the "Anything" region, though, it would create a new `ActiveLayer` for
the markdown.inline layer and recreate its query cursor. Then the query
cursor would emit the same highlights as it did previously for
"`Something`".

This breaks an invariant that the `Highlighter::next_event_offset` must
be monotonically increasing. The added test case failed with:

    thread 'tests::codefence_rust_doc_comments' panicked at /.../ropey-1.6.1/src/slice.rs:703:23:
    byte_slice(): Invalid byte range 27..12: start must be <= end

Also see helix-editor/helix#13569.

The solution is to "tombstone" any layers which are finished in a new
`HashSet<Layer>` and avoid recreating query cursors for any finished
layers. Outside of this bug, this also prevents us from needlessly
creating query cursors which we know will yield nothing. Query cursors
are reused in a thread-local cache, so by not creating a query cursor we
allow another caller to use the cached cursor.
2025-06-01 10:28:19 -04:00
2025-05-29 09:23:29 -04:00
2025-02-05 15:45:25 -05:00
2025-05-14 16:53:18 -04:00
2024-08-09 02:54:16 +02:00
2024-07-28 02:50:13 +02:00
2024-07-28 02:50:13 +02:00
2025-02-12 20:47:11 -05:00

tree-house

This repository contains a number of crates used by the Helix editor for integration with the tree-sitter C library.

  • bindings/ contains the tree-house-bindings crate which provides Rust bindings over the C library.
  • highlighter/ contains the tree-house crate which exposes a robust highlighter and query iterator for working across [injections].
  • skidder/ contains the skidder crate which exposes utilities for building a package repository for tree-sitter grammars.
  • cli/ contains the skidder-cli crate which wraps skidder in a command line interface.
Description
Cozy Rust bindings to the tree-sitter C library and a robust highlighter
Readme MPL-2.0 1.2 MiB
Languages
Rust 98.4%
Nix 0.9%
Shell 0.7%