home ~ projects ~ socials

Neopolitan tree-sitter Notes Scratchpad

Notes

  • For some reason the inline code sample highlighting doesn't work with the CLI tool when I was testing it. It works in Neovim though, so I'm running with it.

  • Adding parsers:

    https://github.com/nvim-treesitter/nvim-treesitter#adding-parsers

  • Adding queries:

    https://github.com/nvim-treesitter/nvim-treesitter#adding-queries

  • Adding colors with this in the nvim.lua file like

    vim.api.nvim_set_hl(0, "@neo_attr_key", { fg = "#667733" })

  • Docs for the various color options

    https://neovim.io/doc/user/api.html#nvim_set_hl()

  • I expect there's a better way to set colors via a colorsheme or similar. Something to investigate
  • Had to copy (or symbolically link during dev) the files "highlights.scm" and "injections.scm" into the dir ~/.config/nvim/queries/neopolitan/
  • Had to add the filetype in ~/.config/nvim/init.lua with:
vim.filetype.add({
  extension = {
    neo = 'neopolitan'
  }
})
  • Setup the neopolitan highligting for .org files as well with this in the config:
vim.treesitter.language.register('neopolitan', 'org')

  • In order for nvim-treesitter to work with the lanauge I had to lowercase the language name in the grammar.js file. That is, this didn't work:

    name: 'Neopolitan'

    But this did:

    name: 'neopolitan'

    Prior to that I got an error like:

    Error executing lua Failed to load parser: uv_dlsym: dlsym(0x200f3fde0, tree_sitter_neopolitan): symbol not found

  • During initail dev, the html_body content wasn't getting highlighed via the injection. Running tree-sitter dump-languages showed that the only one the CLI was connected to was the neopolitan dev code. I think I need to explicity install (or download and point the config to) the HTML parser for that to work from the command line.

  • This is the isssue where I figured out that problem:

    https://github.com/nvim-treesitter/nvim-treesitter/issues/3515

  • The tree-sitter parser doesn't handle look-ahead regex. I'm making an "external scanner" for tokens that need that functionality.

  • Example external scanner here:

    https://github.com/tlaplus-community/tree-sitter-tlaplus/blob/5b35152fcca505290c1a99a37f7e7f700a3d267d/src/scanner.cc#L358

-- end of line --

References

This is what I'm using to wire tree-sitter up to neovim. I think I originally got it from the kickstarter.nvim config file

kickstarter.nvim config

TODO: Link this up

This is where I figured out enough C to get make the parser. (It's not elegent code. It works great though)

Another external scanner that I looked at a little to make sure I was on the right path

This is where I confirmed that lookahead regex expressions don't work in the built in tree-sitter parser. If you want that functionality, you have have to kick out to an exteranl scanner.

Used these a little to help figure out how to do the highlighting and injection highlighting

I had to pull this down for the command line tool to get the html injection highlighting working

This is what tree-sitter uses to generate web page debugging. I had to dig into the docs a little to figure out how to change the colors of the output to more of a dark mode.

This is what I use to get tree-sitter running inside neovim

The command line too version of tree-sitter. It's the way to generate parsers once you've got the code in place. It'll also show you preview highlight in the terminal and generate an HTML video of the parse tree. Both of which are super helpful

The rust tree-sitter code. Had to download it too in order to get injection syntax highlighting to show up