home ~ projects ~ socials

Use nom Parser's recognize Function To Convert A char To A &str In Rust

20 Years To An Overnight Success

This one took forever to figure out. Like, I've been fighting it on and off for years. Ever since I started working on my own file format1 and website builder2. I've always ended up doing hacky stuff to work around it. Today, I found a solution.

Specifically, this is how to use the nom3 recognize4 function to convert a char result from a parser into a &str:

---
[dependencies]
anyhow = "1.0.98"
nom = "8.0.0"
---

use anyhow::Result;
use nom::IResult;
use nom::combinator::recognize;
use nom::character::complete::one_of;
use nom::Parser;

fn main() -> Result<()> {
  let result = parse("alfa")?;
  dbg!(result);
  Ok(())
}

fn parse(
    source: &str,
) -> IResult<&str, &str> {
  let (source, result) = recognize(one_of("123abc")).parse(source)?;
  Ok((source, result))
}
Output:
[/Users/alan/.cargo/target/4b/0baba35be9214e/_active_nvim_run:16:3] result = (
    "lfa",
    "a",
)

What Doesn't Work

Here's an example of the error that happens without recognize:

---
[dependencies]
anyhow = "1.0.98"
nom = "8.0.0"
---

use anyhow::Result;
use nom::IResult;
use nom::character::complete::one_of;
use nom::Parser;


fn main() -> Result<()> {
  let result = parse("alfa");
  dbg!(result);
  Ok(())
}

fn parse(
    source: &str,
) -> IResult<&str, &str> {
  let (source, result) = one_of("123abc").parse(source)?;
  Ok((source, result))
}
Output:
error[E0308]: mismatched types
  --> /Users/alan/.cargo/target/4b/0baba35be9214e/_active_nvim_run:23:15
   |
23 |   Ok((source, result))
   |               ^^^^^^ expected `&str`, found `char`

For more information about this error, try `rustc --explain E0308`.
error: could not compile `_active_nvim_run` (bin "_active_nvim_run") due to 1 previous error

It tries to send back a char instead of a &str That pukes because of Rust's type system. In the past, I've ended up converting everything to capital "s" String types. Doing that cascades through the entire parsing app, though. It always felt a little gross5.

A Real World Example

It also made it way harder to work with when using things like alt() to let multiple child parsers attempt to parse a string. For example, this breaks and throws a huge error message:

---
[dependencies]
anyhow = "1.0.98"
nom = "8.0.0"
---

use anyhow::Result;
use nom::IResult;
use nom::branch::alt;
use nom::character::complete::alpha1;
use nom::character::complete::one_of;
use nom::Parser;

fn main() -> Result<()> {
  let result = parse("br4vo")?;
  dbg!(result);
  Ok(())
}

fn parse(
    source: &str,
) -> IResult<&str, &str> {
  let (source, result) = alt((
    one_of("123abc"),
    alpha1
  ))
  .parse(source)?;

  Ok((source, result))
}
Output:
error[E0277]: the trait bound `char: Input` is not satisfied
  --> /Users/alan/.cargo/target/4b/0baba35be9214e/_active_nvim_run:27:4
   |
27 |   .parse(source)?;
   |    ^^^^^ the trait `Input` is not implemented for `char`
   |
   = help: the following other types implement trait `Input`:
             &'a [u8]
             &'a str

error[E0277]: the trait bound `char: Input` is not satisfied
  --> /Users/alan/.cargo/target/4b/0baba35be9214e/_active_nvim_run:24:5
   |
24 |     one_of("123abc"),
   |     ^^^^^^^^^^^^^^^^ the trait `Input` is not implemented for `char`
   |
   = help: the following other types implement trait `Input`:
             &'a [u8]
             &'a str
note: required by a bound in `nom::character::complete::one_of`
  --> /Users/alan/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/nom-8.0.0/src/character/complete.rs:81:6
   |
79 | pub fn one_of<I, T, Error: ParseError<I>>(list: T) -> impl FnMut(I) -> IResult<I, char, Error>
   |        ------ required by a bound in this function
80 | where
81 |   I: Input,
   |      ^^^^^ required by this bound in `one_of`

error[E0277]: the trait bound `char: Input` is not satisfied
   --> /Users/alan/.cargo/target/4b/0baba35be9214e/_active_nvim_run:25:5
    |
25  |     alpha1
    |     ^^^^^^ the trait `Input` is not implemented for `char`
    |
    = help: the following other types implement trait `Input`:
              &'a [u8]
              &'a str
note: required by a bound in `nom::character::complete::alpha1`
   --> /Users/alan/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/nom-8.0.0/src/character/complete.rs:343:6
    |
341 | pub fn alpha1<T, E: ParseError<T>>(input: T) -> IResult<T, T, E>
    |        ------ required by a bound in this function
342 | where
343 |   T: Input,
    |      ^^^^^ required by this bound in `alpha1`

error[E0308]: mismatched types
   --> /Users/alan/.cargo/target/4b/0baba35be9214e/_active_nvim_run:27:10
    |
27  |   .parse(source)?;
    |    ----- ^^^^^^ expected `char`, found `&str`
    |    |
    |    arguments to this method are incorrect
    |
help: the return type of this call is `&str` due to the type of the argument passed
   --> /Users/alan/.cargo/target/4b/0baba35be9214e/_active_nvim_run:23:26
    |
23  |     let (source, result) = alt((
    |  __________________________^
24  | |     one_of("123abc"),
25  | |     alpha1
26  | |   ))
27  | |   .parse(source)?;
    | |__________------^
    |            |
    |            this argument influences the return type of `parse`
note: method defined here
   --> /Users/alan/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/nom-8.0.0/src/internal.rs:412:6
    |
412 |   fn parse(&mut self, input: Input) -> IResult<Input, Self::Output, Self::Error> {
    |      ^^^^^

error[E0277]: the trait bound `nom::error::Error<&str>: ParseError<char>` is not satisfied
  --> /Users/alan/.cargo/target/4b/0baba35be9214e/_active_nvim_run:24:5
   |
24 |     one_of("123abc"),
   |     ^^^^^^^^^^^^^^^^ the trait `ParseError<char>` is not implemented for `nom::error::Error<&str>`
   |
   = help: the trait `ParseError<char>` is not implemented for `nom::error::Error<&str>`
           but trait `ParseError<&str>` is implemented for it
   = help: for that trait implementation, expected `&str`, found `char`
note: required by a bound in `nom::character::complete::one_of`
  --> /Users/alan/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/nom-8.0.0/src/character/complete.rs:79:28
   |
79 | pub fn one_of<I, T, Error: ParseError<I>>(list: T) -> impl FnMut(I) -> IResult<I, char, Error>
   |                            ^^^^^^^^^^^^^ required by this bound in `one_of`

error[E0277]: the trait bound `nom::error::Error<&str>: ParseError<char>` is not satisfied
   --> /Users/alan/.cargo/target/4b/0baba35be9214e/_active_nvim_run:23:26
    |
23  |   let (source, result) = alt((
    |                          ^^^ the trait `ParseError<char>` is not implemented for `nom::error::Error<&str>`
    |
    = help: the trait `ParseError<char>` is not implemented for `nom::error::Error<&str>`
            but trait `ParseError<&str>` is implemented for it
    = help: for that trait implementation, expected `&str`, found `char`
note: required by a bound in `nom::character::complete::alpha1`
   --> /Users/alan/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/nom-8.0.0/src/character/complete.rs:341:21
    |
341 | pub fn alpha1<T, E: ParseError<T>>(input: T) -> IResult<T, T, E>
    |                     ^^^^^^^^^^^^^ required by this bound in `alpha1`

error[E0277]: the trait bound `nom::error::Error<&str>: ParseError<char>` is not satisfied
  --> /Users/alan/.cargo/target/4b/0baba35be9214e/_active_nvim_run:27:10
   |
27 |   .parse(source)?;
   |    ----- ^^^^^^ the trait `ParseError<char>` is not implemented for `nom::error::Error<&str>`
   |    |
   |    required by a bound introduced by this call
   |
   = help: the trait `ParseError<char>` is not implemented for `nom::error::Error<&str>`
           but trait `ParseError<&str>` is implemented for it
   = help: for that trait implementation, expected `&str`, found `char`
   = note: required for `Choice<(impl FnMut(char) -> Result<(char, char), Err<...>>, ...)>` to implement `Parser<char>`
   = note: the full name for the type has been written to '/Users/alan/.cargo/target/4b/0baba35be9214e/debug/deps/_active_nvim_run-987d237a674aaabb.long-type-7003196113168646821.txt'
   = note: consider using `--verbose` to print the full type name to the console

error[E0277]: the trait bound `nom::error::Error<&str>: ParseError<char>` is not satisfied
   --> /Users/alan/.cargo/target/4b/0baba35be9214e/_active_nvim_run:25:5
    |
25  |     alpha1
    |     ^^^^^^ the trait `ParseError<char>` is not implemented for `nom::error::Error<&str>`
    |
    = help: the trait `ParseError<char>` is not implemented for `nom::error::Error<&str>`
            but trait `ParseError<&str>` is implemented for it
    = help: for that trait implementation, expected `&str`, found `char`
note: required by a bound in `nom::character::complete::alpha1`
   --> /Users/alan/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/nom-8.0.0/src/character/complete.rs:341:21
    |
341 | pub fn alpha1<T, E: ParseError<T>>(input: T) -> IResult<T, T, E>
    |                     ^^^^^^^^^^^^^ required by this bound in `alpha1`

error[E0308]: mismatched types
  --> /Users/alan/.cargo/target/4b/0baba35be9214e/_active_nvim_run:29:7
   |
29 |   Ok((source, result))
   |       ^^^^^^ expected `&str`, found `char`

error[E0308]: mismatched types
  --> /Users/alan/.cargo/target/4b/0baba35be9214e/_active_nvim_run:29:15
   |
29 |   Ok((source, result))
   |               ^^^^^^ expected `&str`, found `char`

Some errors have detailed explanations: E0277, E0308.
For more information about an error, try `rustc --explain E0277`.
error: could not compile `_active_nvim_run` (bin "_active_nvim_run") due to 10 previous errors

But, if you add recognize, it works fine:

---
[dependencies]
anyhow = "1.0.98"
nom = "8.0.0"
---

use anyhow::Result;
use nom::IResult;
use nom::branch::alt;
use nom::character::complete::alpha1;
use nom::character::complete::one_of;
use nom::combinator::recognize;
use nom::Parser;

fn main() -> Result<()> {
  let result = parse("br4vo")?;
  dbg!(result);
  Ok(())
}

fn parse(
    source: &str,
) -> IResult<&str, &str> {
  let (source, result) = alt((
    recognize(one_of("123abc")),
    alpha1
  ))
  .parse(source)?;

  Ok((source, result))
}
Output:
[/Users/alan/.cargo/target/4b/0baba35be9214e/_active_nvim_run:17:3] result = (
    "r4vo",
    "b",
)

Success!

Nom is pretty amazing. It's the foundation for parsing my files. I don't know what I would have done without it7. The learning curve is pretty steep. Figuring this out put me a lot closer to the top of it.

-a

-- end of line --

Footnotes

1 ⤴
2 ⤴
3 ⤴
4 ⤴
5 ⤴

Basically all the examples show using &str directly. It always seemed like there should have been a way to get to that from char. I didn't find it until today.

7 ⤴

Sure, there are other parsers out there. I tried a couple. I expect I could have gotten them to work. Nom always felt like the right play though. And, except for this one thing, it was great. Now, there's no caveat.