☰

# std/nre

Theme:

- Manual
- Standard library
- Index
- Compiler docs
- Fusion docs
- devel, stable

Search:
Source Edit

# What is NRE?

A regular expression library for Nim using PCRE to do the hard work.

For documentation on how to write patterns, there exists the official PCRE pattern documentation. You can also search the internet for a wide variety of third-party documentation and tools.

Warning: If you love `sequtils.toSeq` we have bad news for you. This library doesn't work with it due to documented compiler limitations. As a workaround, use this:

Example:

    import std/nre
    # either `import std/nre except toSeq` or fully qualify `sequtils.toSeq`:
    import std/sequtils
    iterator iota(n: int): int =
      for i in 0..<n: yield i
    assert sequtils.toSeq(iota(3)) == @[0, 1, 2]

Note: There are also alternative nimble packages such as tinyre and regex.

## Licencing

PCRE has some additional terms that you must agree to in order to use this module.

Example:

    import std/nre
    import std/sugar
    let vowels = re"[aeoui]"
    let bounds = collect:
      for match in "moiga".findIter(vowels): match.matchBounds
    assert bounds == @[1 .. 1, 2 .. 2, 4 .. 4]
    from std/sequtils import toSeq
    let s = sequtils.toSeq("moiga".findIter(vowels))
      # fully qualified to avoid confusion with nre.toSeq
    assert s.len == 3

    let firstVowel = "foo".find(vowels)
    let hasVowel = firstVowel.isSome()
    assert hasVowel
    let matchBounds = firstVowel.get().captureBounds[-1]
    assert matchBounds.a == 1

    # as with module `re`, unless specified otherwise, `start` parameter in each
    # proc indicates where the scan starts, but outputs are relative to the start
    # of the input string, not to `start`:
    assert find("uxabc", re"(?<=x|y)ab", start = 1).get.captures[-1] == "ab"
    assert find("uxabc", re"ab", start = 3).isNone

# Imports

pcre, util, tables, strutils, options, unicode, assertions

# Types

    CaptureBounds = distinct RegexMatch

  Source Edit

    Captures = distinct RegexMatch

  Source Edit

    InvalidUnicodeError = ref object of RegexError
      pos*: int                  ## the location of the invalid unicode in bytes

  Thrown when matching fails due to invalid unicode in strings Source Edit

    Regex = ref RegexDesc

  Represents the pattern that things are matched against, constructed with `re(string)`. Examples: `re"foo"`, `re(r"(*ANYCRLF)(?x)foo # comment".`
  `pattern: string`
    the string that was used to create the pattern. For details on how to write a pattern, please see the official PCRE pattern documentation.
  `captureCount: int`
    the number of captures that the pattern has.
  `captureNameId: Table[string, int]`
    a table from the capture names to their numeric id.

  ### Options

  The following options may appear anywhere in the pattern, and they affect the rest of it.

  - `(?i)` - case insensitive
  - `(?m)` - multi-line: `^` and `$` match the beginning and end of lines, not of the subject string
  - `(?s)` - `.` also matches newline (dotall)
  - `(?U)` - expressions are not greedy by default. `?` can be added to a qualifier to make it greedy
  - `(?x)` - whitespace and comments (`#`) are ignored (extended)
  - `(?X)` - character escapes without special meaning (`\w` vs. `\a`) are errors (extra)

  One or a combination of these options may appear only at the beginning of the pattern:

  - `(*UTF8)` - treat both the pattern and subject as UTF-8
  - `(*UCP)` - Unicode character properties; `\w` matches `я`
  - `(*U)` - a combination of the two options above
  - `(*FIRSTLINE*)` - fails if there is not a match on the first line
  - `(*NO_AUTO_CAPTURE)` - turn off auto-capture for groups; `(?<name>...)` can be used to capture
  - `(*CR)` - newlines are separated by `\r`
  - `(*LF)` - newlines are separated by `\n` (UNIX default)
  - `(*CRLF)` - newlines are separated by `\r\n` (Windows default)
  - `(*ANYCRLF)` - newlines are separated by any of the above

  - `(*ANY)` - newlines are separated by any of the above and Unicode newlines:

    single characters VT (vertical tab, U+000B), FF (form feed, U+000C), NEL (next line, U+0085), LS (line separator, U+2028), and PS (paragraph separator, U+2029). For the 8-bit library, the last two are recognized only in UTF-8 mode. — man pcre

  - `(*JAVASCRIPT_COMPAT)` - JavaScript compatibility
  - `(*NO_STUDY)` - turn off studying; study is enabled by default

  For more details on the leading option groups, see the Option Setting and the Newline Convention sections of the PCRE syntax manual.

  Some of these options are not part of PCRE and are converted by nre into PCRE flags. These include `NEVER_UTF`, `ANCHORED`, `DOLLAR_ENDONLY`, `FIRSTLINE`, `NO_AUTO_CAPTURE`, `JAVASCRIPT_COMPAT`, `U`, `NO_STUDY`. In other PCRE wrappers, you will need to pass these as separate flags to PCRE.

  Source Edit

    RegexDesc = object
      pattern*: string

  Source Edit

    RegexError = ref object of CatchableError

  Source Edit

    RegexInternalError = ref object of RegexError

  Internal error in the module, this probably means that there is a bug Source Edit

    RegexMatch = object
      pattern*: Regex            ## The regex doing the matching.
                                 ## Not nil.
      str*: string               ## The string that was matched against.

  Usually seen as OptionRegexMatch, it represents the result of an execution. On failure, it is none, on success, it is some.
  `pattern: Regex`
    the pattern that is being matched
  `str: string`
    the string that was matched against
  `captures[]: string`
    the string value of whatever was captured at that id. If the value is invalid, then behavior is undefined. If the id is `-1`, then the whole match is returned. If the given capture was not matched, `nil` is returned. See examples for `match`.
  `captureBounds[]: HSlice[int, int]`
    gets the bounds of the given capture according to the same rules as the above. If the capture is not filled, then `None` is returned. The bounds are both inclusive. See examples for `match`.
  `match: string`
    the full text of the match.
  `matchBounds: HSlice[int, int]`
    the bounds of the match, as in `captureBounds[]`
  `(captureBounds|captures).toTable`
    returns a table with each named capture as a key.
  `(captureBounds|captures).toSeq`
    returns all the captures by their number.
  `$: string`
    same as `match`
  Source Edit

    StudyError = ref object of RegexError

  Thrown when studying the regular expression fails for whatever reason. The message contains the error code. Source Edit

    SyntaxError = ref object of RegexError
      pos*: int                  ## the location of the syntax error in bytes
      pattern*: string           ## the pattern that caused the problem

  Thrown when there is a syntax error in the regular expression string passed in Source Edit

# Procs

    proc `$`(pattern: RegexMatch): string {....raises: [], tags: [], forbids: [].}

  Source Edit

    proc `==`(a, b: Regex): bool {....raises: [], tags: [], forbids: [].}

  Source Edit

    proc `==`(a, b: RegexMatch): bool {....raises: [], tags: [], forbids: [].}

  Source Edit

    func `[]`(pattern: CaptureBounds; i: int): HSlice[int, int] {....raises: [],
        tags: [], forbids: [].}

  Source Edit

    func `[]`(pattern: CaptureBounds; name: string): HSlice[int, int] {.
        ...raises: [KeyError], tags: [], forbids: [].}

  Source Edit

    func `[]`(pattern: Captures; i: int): string {....raises: [], tags: [], forbids: [].}

  Source Edit

    func `[]`(pattern: Captures; name: string): string {....raises: [KeyError],
        tags: [], forbids: [].}

  Source Edit

    func captureBounds(pattern: RegexMatch): CaptureBounds {....raises: [], tags: [],
        forbids: [].}

  Source Edit

    proc captureCount(pattern: Regex): int {....raises: [ValueError], tags: [],
        forbids: [].}

  Source Edit

    proc captureNameId(pattern: Regex): Table[string, int] {....raises: [], tags: [],
        forbids: [].}

  Source Edit

    func captures(pattern: RegexMatch): Captures {....raises: [], tags: [], forbids: [].}

  Source Edit

    func contains(pattern: CaptureBounds; i: int): bool {....raises: [], tags: [],
        forbids: [].}

  Source Edit

    func contains(pattern: CaptureBounds; name: string): bool {....raises: [KeyError],
        tags: [], forbids: [].}

  Source Edit

    func contains(pattern: Captures; i: int): bool {....raises: [], tags: [],
        forbids: [].}

  Source Edit

    func contains(pattern: Captures; name: string): bool {....raises: [KeyError],
        tags: [], forbids: [].}

  Source Edit

    proc contains(str: string; pattern: Regex; start = 0; endpos = int.high): bool {.
        ...raises: [ValueError, RegexInternalError, InvalidUnicodeError], tags: [],
        forbids: [].}

  Determine if the string contains the given pattern between the end and start positions: This function is equivalent to `isSome(str.find(pattern, start, endpos))`.

  Example:

      assert "abc".contains(re"bc")
      assert not "abc".contains(re"cd")
      assert not "abc".contains(re"a", start = 1)

  Source Edit

    proc escapeRe(str: string): string {....gcsafe, raises: [], tags: [], forbids: [].}

  Escapes the string so it doesn't match any special characters. Incompatible with the Extra flag (`X`).

  Escaped char: `\ + * ? [ ^ ] $ ( ) { } = ! < > | : -`

  Example:

      assert escapeRe("fly+wind") == "fly\\+wind"
      assert escapeRe("!") == "\\!"
      assert escapeRe("nim*") == "nim\\*"

  Source Edit

    proc find(str: string; pattern: Regex; start = 0; endpos = int.high): Option[
        RegexMatch] {....raises: [ValueError, RegexInternalError, InvalidUnicodeError],
                      tags: [], forbids: [].}

  Finds the given pattern in the string between the end and start positions.
  `start`
    The start point at which to start matching. `|abc` is `0`; `a|bc` is `1`
  `endpos`
    The maximum index for a match; `int.high` means the end of the string, otherwise it’s an inclusive upper bound.
  Source Edit

    proc findAll(str: string; pattern: Regex; start = 0; endpos = int.high): seq[
        string] {....raises: [ValueError, RegexInternalError, InvalidUnicodeError],
                  tags: [], forbids: [].}

  Source Edit

    func match(pattern: RegexMatch): string {....raises: [], tags: [], forbids: [].}

  Source Edit

    proc match(str: string; pattern: Regex; start = 0; endpos = int.high): Option[
        RegexMatch] {....raises: [ValueError, RegexInternalError, InvalidUnicodeError],
                      tags: [], forbids: [].}

  Like find(...), but anchored to the start of the string.

  Example:

      assert "foo".match(re"f").isSome
      assert "foo".match(re"o").isNone

      assert "abc".match(re"(\w)").get.captures[0] == "a"
      assert "abc".match(re"(?<letter>\w)").get.captures["letter"] == "a"
      assert "abc".match(re"(\w)\w").get.captures[-1] == "ab"

      assert "abc".match(re"(\w)").get.captureBounds[0] == 0 .. 0
      assert 0 in "abc".match(re"(\w)").get.captureBounds
      assert "abc".match(re"").get.captureBounds[-1] == 0 .. -1
      assert "abc".match(re"abc").get.captureBounds[-1] == 0 .. 2

  Source Edit

    func matchBounds(pattern: RegexMatch): HSlice[int, int] {....raises: [], tags: [],
        forbids: [].}

  Source Edit

    proc re(pattern: string): Regex {....raises: [KeyError, SyntaxError, StudyError,
        ValueError], tags: [], forbids: [].}

  Source Edit

    proc replace(str: string; pattern: Regex; sub: string): string {.
        ...raises: [ValueError, RegexInternalError, InvalidUnicodeError, KeyError],
        tags: [], forbids: [].}

  Source Edit

    proc replace(str: string; pattern: Regex;
                 subproc: proc (match: RegexMatch): string): string {.
        ...raises: [ValueError, RegexInternalError, InvalidUnicodeError, Exception],
        tags: [RootEffect], forbids: [].}

  Replaces each match of Regex in the string with `subproc`, which should never be or return `nil`.

  If `subproc` is a `proc (RegexMatch): string`, then it is executed with each match and the return value is the replacement value.

  If `subproc` is a `proc (string): string`, then it is executed with the full text of the match and the return value is the replacement value.

  If `subproc` is a string, the syntax is as follows:

  - `$$` - literal `$`
  - `$123` - capture number `123`
  - `$foo` - named capture `foo`
  - `${foo}` - same as above
  - `$1$#` - first and second captures
  - `$#` - first capture
  - `$0` - full match

  If a given capture is missing, `IndexDefect` thrown for un-named captures and `KeyError` for named captures.

  Source Edit

    proc replace(str: string; pattern: Regex; subproc: proc (match: string): string): string {.
        ...raises: [ValueError, RegexInternalError, InvalidUnicodeError, Exception],
        tags: [RootEffect], forbids: [].}

  Source Edit

    proc split(str: string; pattern: Regex; maxSplit = -1; start = 0): seq[string] {.
        ...raises: [ValueError, RegexInternalError, InvalidUnicodeError], tags: [],
        forbids: [].}

  Splits the string with the given regex. This works according to the rules that Perl and Javascript use.

  `start` behaves the same as in find(...).

  Example:

      # -  If the match is zero-width, then the string is still split:
      assert "123".split(re"") == @["1", "2", "3"]

      # -  If the pattern has a capture in it, it is added after the string
      #    split:
      assert "12".split(re"(\d)") == @["", "1", "", "2", ""]

      # -  If `maxsplit != -1`, then the string will only be split
      #    `maxsplit - 1` times. This means that there will be `maxsplit`
      #    strings in the output seq.
      assert "1.2.3".split(re"\.", maxsplit = 2) == @["1", "2.3"]

  Source Edit

    proc toSeq(pattern: CaptureBounds; default = none(HSlice[int, int])): seq[
        Option[HSlice[int, int]]] {....raises: [ValueError], tags: [], forbids: [].}

  Source Edit

    proc toSeq(pattern: Captures; default: Option[string] = none(string)): seq[
        Option[string]] {....raises: [ValueError], tags: [], forbids: [].}

  Source Edit

    func toTable(pattern: CaptureBounds): Table[string, HSlice[int, int]] {.
        ...raises: [KeyError], tags: [], forbids: [].}

  Source Edit

    func toTable(pattern: Captures): Table[string, string] {....raises: [KeyError],
        tags: [], forbids: [].}

  Source Edit

# Iterators

    iterator findIter(str: string; pattern: Regex; start = 0; endpos = int.high): RegexMatch {.
        ...raises: [ValueError, RegexInternalError, InvalidUnicodeError], tags: [],
        forbids: [].}

  Works the same as find(...), but finds every non-overlapping match:

  Example:

      import std/sugar
      assert collect(for a in "2222".findIter(re"22"): a.match) == @["22", "22"]
       # not @["22", "22", "22"]

  Arguments are the same as find(...)

  Variants:

  - `proc findAll(...)` returns a `seq[string]`
  Source Edit

    iterator items(pattern: CaptureBounds; default = none(HSlice[int, int])): Option[
        HSlice[int, int]] {....raises: [ValueError], tags: [], forbids: [].}

  Source Edit

    iterator items(pattern: Captures; default: Option[string] = none(string)): Option[
        string] {....raises: [ValueError], tags: [], forbids: [].}

  Source Edit

# Exports

$, none, flatten, none, ==, get, map, get, map, UnpackDefect, Option, get, unsafeGet, option, UnpackError, flatMap, isSome, some, filter, isNone