# `highlight` module

The highlight module contains classes and functions for displaying short excerpts from hit documents in the search results you present to the user, with query terms highlighted.

The highlighting system has four main elements.

- Fragmenters chop up the original text into __fragments__, based on the locations of matched terms in the text.
- Scorers assign a score to each fragment, allowing the system to rank the best fragments by whatever criterion.
- Order functions control in what order the top-scoring fragments are presented to the user. For example, you can show the fragments in the order they appear in the document (FIRST) or show higher-scoring fragments first (SCORE)
- Formatters turn the fragment objects into human-readable output, such as an HTML string.

See How to create highlighted search result excerpts for more information.

See how to highlight terms in search results.

## Manual highlighting

class `whoosh.highlight.``Highlighter`(fragmenter=None, scorer=None, formatter=None, always_retokenize=False, order=<function FIRST>)

`whoosh.highlight.``highlight`(text, terms, analyzer, fragmenter, formatter, top=3, scorer=None, minscore=1, order=<function FIRST>, mode='query')

## Fragmenters

class `whoosh.highlight.``Fragmenter`
  `fragment_matches`(text, matched_tokens)

    Yields `Fragment` objects based on the text and the matched terms.

    Parameters:

    - text – the string being highlighted.
    - matched_tokens – a list of `analysis.Token` objects representing the term matches in the string.

  `fragment_tokens`(text, all_tokens)

    Yields `Fragment` objects based on the tokenized text.

    Parameters:

    - text – the string being highlighted.
    - all_tokens – an iterator of `analysis.Token` objects from the string.

  `must_retokenize`()

    Returns True if this fragmenter requires retokenized text.

    If this method returns True, the fragmenter’s `fragment_tokens` method will be called with an iterator of ALL tokens from the text, with the tokens for matched terms having the `matched` attribute set to True.

    If this method returns False, the fragmenter’s `fragment_matches` method will be called with a LIST of matching tokens.

class `whoosh.highlight.``WholeFragmenter`(charlimit=32768)

  Doesn’t fragment the token stream. This object just returns the entire entire stream as one “fragment”. This is useful if you want to highlight the entire text.

  Note that even if you use the WholeFragmenter, the highlight code will return no fragment if no terms matched in the given field. To return the whole fragment even in that case, call highlights() with minscore=0:

      # Query where no terms match in the "text" field
      q = query.Term("tag", "new")

      r = mysearcher.search(q)
      r.fragmenter = highlight.WholeFragmenter()
      r.formatter = highlight.UppercaseFormatter()
      # Since no terms in the "text" field matched, we get no fragments back
      assert r[0].highlights("text") == ""

      # If we lower the minimum score to 0, we get a fragment even though it
      # has no matching terms
      assert r[0].highlights("text", minscore=0) == "This is the text field."

class `whoosh.highlight.``SentenceFragmenter`(maxchars=200, sentencechars='.!?', charlimit=32768)

  Breaks the text up on sentence end punctuation characters (“.”, “!”, or “?”). This object works by looking in the original text for a sentence end as the next character after each token’s ‘endchar’.

  When highlighting with this fragmenter, you should use an analyzer that does NOT remove stop words, for example:

      sa = StandardAnalyzer(stoplist=None)

  Parameters: | maxchars – The maximum number of characters allowed in a fragment.

class `whoosh.highlight.``ContextFragmenter`(maxchars=200, surround=20, charlimit=32768)

  Looks for matched terms and aggregates them with their surrounding context.

  Parameters:

  - maxchars – The maximum number of characters allowed in a fragment.
  - surround – The number of extra characters of context to add both before the first matched term and after the last matched term.

class `whoosh.highlight.``PinpointFragmenter`(maxchars=200, surround=20, autotrim=False, charlimit=32768)

  This is a NON-RETOKENIZING fragmenter. It builds fragments from the positions of the matched terms.

  Parameters:

  - maxchars – The maximum number of characters allowed in a fragment.
  - surround – The number of extra characters of context to add both before the first matched term and after the last matched term.
  - autotrim – automatically trims text before the first space and after the last space in the fragments, to try to avoid truncated words at the start and end. For short fragments or fragments with long runs between spaces this may give strange results.

## Scorers

class `whoosh.highlight.``FragmentScorer`

class `whoosh.highlight.``BasicFragmentScorer`

## Formatters

class `whoosh.highlight.``UppercaseFormatter`(between='...')

  Returns a string in which the matched terms are in UPPERCASE.

  Parameters: | between – the text to add between fragments.

class `whoosh.highlight.``HtmlFormatter`(tagname='strong', between='...', classname='match', termclass='term', maxclasses=5, attrquote='"')

  Returns a string containing HTML formatting around the matched terms.

  This formatter wraps matched terms in an HTML element with two class names. The first class name (set with the constructor argument `classname`) is the same for each match. The second class name (set with the constructor argument `termclass` is different depending on which term matched. This allows you to give different formatting (for example, different background colors) to the different terms in the excerpt.

      >>> hf = HtmlFormatter(tagname="span", classname="match", termclass="term")
      >>> hf(mytext, myfragments)
      "The <span class="match term0">template</span> <span class="match term1">geometry</span> is..."

  This object maintains a dictionary mapping terms to HTML class names (e.g. `term0` and `term1` above), so that multiple excerpts will use the same class for the same term. If you want to re-use the same HtmlFormatter object with different searches, you should call HtmlFormatter.clear() between searches to clear the mapping.

  Parameters:

  - tagname – the tag to wrap around matching terms.
  - between – the text to add between fragments.
  - classname – the class name to add to the elements wrapped around matching terms.
  - termclass – the class name prefix for the second class which is different for each matched term.
  - maxclasses – the maximum number of term classes to produce. This limits the number of classes you have to define in CSS by recycling term class names. For example, if you set maxclasses to 3 and have 5 terms, the 5 terms will use the CSS classes `term0`, `term1`, `term2`, `term0`, `term1`.

class `whoosh.highlight.``GenshiFormatter`(qname='strong', between='...')

  Returns a Genshi event stream containing HTML formatting around the matched terms.

  Parameters:

  - qname – the QName for the tag to wrap around matched terms.
  - between – the text to add between fragments.

## Utility classes

class `whoosh.highlight.``Fragment`(text, matches, startchar=0, endchar=-1)

  Represents a fragment (extract) from a hit document. This object is mainly used to keep track of the start and end points of the fragment and the “matched” character ranges inside; it does not contain the text of the fragment or do much else.

  The useful attributes are:

  `Fragment.text`
    The entire original text from which this fragment is taken.
  `Fragment.matches`
    An ordered list of objects representing the matched terms in the fragment. These objects have `startchar` and `endchar` attributes.
  `Fragment.startchar`
    The index of the first character in the fragment.
  `Fragment.endchar`
    The index of the last character in the fragment.
  `Fragment.matched_terms`
    A `set` of the `text` of the matched terms in the fragment (if available).

  Parameters:

  - text – the source text of the fragment.
  - matches – a list of objects which have `startchar` and `endchar` attributes, and optionally a `text` attribute.
  - startchar – the index into `text` at which the fragment starts. The default is 0.
  - endchar – the index into `text` at which the fragment ends. The default is -1, which is interpreted as the length of `text`.