Neopolitan tree-sitter Notes Scratchpad

Notes

For some reason the inline code sample highlighting doesn't work with the CLI tool when I was testing it. It works in Neovim though, so I'm running with it.
Adding parsers:
https://github.com/nvim-treesitter/nvim-treesitter#adding-parsers
Adding queries:
https://github.com/nvim-treesitter/nvim-treesitter#adding-queries
Adding colors with this in the nvim.lua file like
`vim.api.nvim_set_hl(0, "@neo_attr_key", { fg = "#667733" })``
Docs for the various color options
https://neovim.io/doc/user/api.html#nvim_set_hl()
I expect there's a better way to set colors via a colorsheme or similar. Something to investigate
Had to copy (or symbolically link during dev) the files "highlights.scm" and "injections.scm" into the dir `~/.config/nvim/queries/neopolitan/`
Had to add the filetype in _~/.config/nvim/init.lua__ with:

Code

vim.filetype.add({
  extension = {
    neo = 'neopolitan'
  }
})

Setup the neopolitan highligting for .org files as well with this in the config:

Code

vim.treesitter.language.register('neopolitan', 'org')

In order for nvim-treesitter to work with the lanauge I had to lowercase the language name in the `grammar.js`` file. That is, this didn't work:
`name: 'Neopolitan'``
But this did:
`name: 'neopolitan'``
Prior to that I got an error like:
Error executing lua Failed to load parser: uv_dlsym: dlsym(0x200f3fde0, tree_sitter_neopolitan): symbol not found
During initail dev, the html_body content wasn't getting highlighed via the injection. Running tree-sitter dump-languages showed that the only one the CLI was connected to was the neopolitan dev code. I think I need to explicity install (or download and point the config to) the HTML parser for that to work from the command line.
This is the isssue where I figured out that problem:
https://github.com/nvim-treesitter/nvim-treesitter/issues/3515
The tree-sitter parser doesn't handle look-ahead regex. I'm making an "external scanner" for tokens that need that functionality.
Example external scanner here:
https://github.com/tlaplus-community/tree-sitter-tlaplus/blob/5b35152fcca505290c1a99a37f7e7f700a3d267d/src/scanner.cc#L358

Reference

nvim-treesitter

This is what I'm using to wire tree-sitter up to neovim. I think I originally got it from the kickstarter.nvim config file

Reference

kickstarter.nvim config

TODO: Link this up

This is where I figured out enough C to get make the parser. (It's not elegent code. It works great though)

Reference

Scanner Example

Another external scanner that I looked at a little to make sure I was on the right path

Reference

tree-sitter regex lookahead issue #1252

This is where I confirmed that lookahead regex expressions don't work in the built in tree-sitter parser. If you want that functionality, you have have to kick out to an exteranl scanner.

Reference

tree-sitter CLI README

Reference

tree-sitter-markdown query docs

Used these a little to help figure out how to do the highlighting and injection highlighting

Reference

tree-sitter: External Scanners Docs

Reference

tree-sitter-html repo

I had to pull this down for the command line tool to get the html injection highlighting working

Reference

GraphViz

This is what tree-sitter uses to generate web page debugging. I had to dig into the docs a little to figure out how to change the colors of the output to more of a dark mode.

Reference

nvim-treesitter

This is what I use to get tree-sitter running inside neovim

Reference

tree-sitter-cli

The command line too version of tree-sitter. It's the way to generate parsers once you've got the code in place. It'll also show you preview highlight in the terminal and generate an HTML video of the parse tree. Both of which are super helpful

Reference

tree-sitter-rust

The rust tree-sitter code. Had to download it too in order to get injection syntax highlighting to show up