# API reference

## parsel.csstranslator

class parsel.csstranslator.GenericTranslator

  Bases: `TranslatorMixin`, `GenericTranslator`

  css_to_xpath(css: str, prefix: str = 'descendant-or-self::') → str

    Translate a group of selectors to XPath.

    Pseudo-elements are not supported here since XPath only knows about “real” elements.

    Parameters:

      - css – A group of selectors as a string.

      - prefix – This string is prepended to the XPath expression for each selector. The default makes selectors scoped to the context node’s subtree.

    Raises:

      `SelectorSyntaxError` on invalid selectors, `ExpressionError` on unknown/unsupported selectors, including pseudo-elements.

    Returns:

      The equivalent XPath 1.0 expression as a string.

class parsel.csstranslator.HTMLTranslator(xhtml: bool = False)

  Bases: `TranslatorMixin`, `HTMLTranslator`

  css_to_xpath(css: str, prefix: str = 'descendant-or-self::') → str

    Translate a group of selectors to XPath.

    Pseudo-elements are not supported here since XPath only knows about “real” elements.

    Parameters:

      - css – A group of selectors as a string.

      - prefix – This string is prepended to the XPath expression for each selector. The default makes selectors scoped to the context node’s subtree.

    Raises:

      `SelectorSyntaxError` on invalid selectors, `ExpressionError` on unknown/unsupported selectors, including pseudo-elements.

    Returns:

      The equivalent XPath 1.0 expression as a string.

class parsel.csstranslator.TranslatorMixin

  Bases: `object`

  This mixin adds support to CSS pseudo elements via dynamic dispatch.

  Currently supported pseudo-elements are `::text` and `::attr(ATTR_NAME)`.

  xpath_attr_functional_pseudo_element(xpath: XPathExpr, function: FunctionalPseudoElement) → XPathExpr

    Support selecting attribute values using ::attr() pseudo-element

  xpath_element(selector: Element) → XPathExpr
  xpath_pseudo_element(xpath: XPathExpr, pseudo_element: FunctionalPseudoElement | str) → XPathExpr

    Dispatch method that transforms XPath to support pseudo-element

  xpath_text_simple_pseudo_element(xpath: XPathExpr) → XPathExpr

    Support selecting text nodes using ::text pseudo-element

class parsel.csstranslator.TranslatorProtocol(*args, **kwargs)

  Bases: `Protocol`

  css_to_xpath(css: str, prefix: str = Ellipsis) → str
  xpath_element(selector: Element) → XPathExpr

class parsel.csstranslator.XPathExpr(path: str = '', element: str = '*', condition: str = '', star_prefix: bool = False)

  Bases: `XPathExpr`

  attribute: str | None = None
  classmethod from_xpath(xpath: OriginalXPathExpr, textnode: bool = False, attribute: str | None = None) → Self
  join(combiner: str, other: OriginalXPathExpr, *args: Any, **kwargs: Any) → Self
  textnode: bool = False

parsel.csstranslator.css2xpath(query: str) → str

  Return translated XPath version of a given CSS query

## parsel.selector

XPath and JMESPath selectors based on the lxml and jmespath Python packages.

class parsel.selector.CTGroupValue

  Bases: `TypedDict`

exception parsel.selector.CannotDropElementWithoutParent

  Bases: `CannotRemoveElementWithoutParent`

exception parsel.selector.CannotRemoveElementWithoutParent

  Bases: `Exception`

exception parsel.selector.CannotRemoveElementWithoutRoot

  Bases: `Exception`

class parsel.selector.SafeXMLParser(*args: Any, **kwargs: Any)

  Bases: `XMLParser`

class parsel.selector.Selector(text: str | None = None, type: str | None = None, body: bytes | bytearray = b'', encoding: str = 'utf-8', namespaces: Mapping[str, str] | None = None, root: Any | None = <object object>, base_url: str | None = None, _expr: str | None = None, huge_tree: bool = True)

  Bases: `object`

  Wrapper for input data in HTML, JSON, or XML format, that allows selecting parts of it using selection expressions.

  You can write selection expressions in CSS or XPath for HTML and XML inputs, or in JMESPath for JSON inputs.

  `text` is an `str` object.

  `body` is a `bytes` object. It can be used together with the `encoding` argument instead of the `text` argument.

  `type` defines the selector type. It can be `"html"` (default), `"json"`, or `"xml"`.

  `base_url` allows setting a URL for the document. This is needed when looking up external entities with relative paths. See the documentation for `lxml.etree.fromstring()` for more information.

  `huge_tree` controls the lxml/libxml2 feature that forbids parsing certain large documents to protect from possible memory exhaustion. The argument is `True` by default if the installed lxml version supports it, which disables the protection to allow parsing such documents. Set it to `False` if you want to enable the protection. See this lxml FAQ entry for more information.

  property attrib: dict[str, str]

    Return the attributes dictionary for underlying element. For JSON selectors, return an empty dict.

  body
  css(query: str) → SelectorList[Self]

    Apply the given CSS selector and return a `SelectorList` instance.

    `query` is a string containing the CSS selector to apply.

    In the background, CSS queries are translated into XPath queries using cssselect library and run `.xpath()` method.

  drop() → None

    Drop matched nodes from the parent element.

  extract() → Any

    Serialize and return the matched nodes.

    For HTML and XML, the result is always a string, and percent-encoded content is unquoted.

  get() → Any

    Serialize and return the matched nodes.

    For HTML and XML, the result is always a string, and percent-encoded content is unquoted.

  getall() → list[str]

    Serialize and return the matched node in a 1-element list of strings.

  jmespath(query: str, **kwargs: Any) → SelectorList[Self]

    Find objects matching the JMESPath `query` and return the result as a `SelectorList` instance with all elements flattened. List elements implement `Selector` interface too.

    `query` is a string containing the JMESPath query to apply.

    Any additional named arguments are passed to the underlying `jmespath.search` call, e.g.:

        selector.jmespath('author.name', options=jmespath.Options(dict_cls=collections.OrderedDict))

  namespaces
  re(regex: str | Pattern[str], replace_entities: bool = True) → list[str]

    Apply the given regex and return a list of strings with the matches.

    `regex` can be either a compiled regular expression or a string which will be compiled to a regular expression using `re.compile(regex)`.

    By default, character entity references are replaced by their corresponding character (except for `&amp;` and `&lt;`). Passing `replace_entities` as `False` switches off these replacements.

  re_first(regex: str | Pattern[str], default: None = None, replace_entities: bool = True) → str | None
  re_first(regex: str | Pattern[str], default: str, replace_entities: bool = True) → str

    Apply the given regex and return the first string which matches. If there is no match, return the default value (`None` if the argument is not provided).

    By default, character entity references are replaced by their corresponding character (except for `&amp;` and `&lt;`). Passing `replace_entities` as `False` switches off these replacements.

  register_namespace(prefix: str, uri: str) → None

    Register the given namespace to be used in this `Selector`. Without registering namespaces you can’t select or extract data from non-standard namespaces. See Working on XML (and namespaces).

  remove_namespaces() → None

    Remove all namespaces, allowing to traverse the document using namespace-less xpaths. See Removing namespaces. For JSON selectors, this method does nothing.

  root: Any
  selectorlist_cls

    alias of `SelectorList`[Selector]

  type
  xpath(query: str, namespaces: Mapping[str, str] | None = None, **kwargs: Any) → SelectorList[Self]

    Find nodes matching the xpath `query` and return the result as a `SelectorList` instance with all elements flattened. List elements implement `Selector` interface too.

    `query` is a string containing the XPATH query to apply.

    `namespaces` is an optional `prefix: namespace-uri` mapping (dict) for additional prefixes to those registered with `register_namespace(prefix, uri)`. Contrary to `register_namespace()`, these prefixes are not saved for future calls.

    Any additional named arguments can be used to pass values for XPath variables in the XPath expression, e.g.:

        selector.xpath('//a[href=$url]', url="http://www.example.com")

class parsel.selector.SelectorList(iterable=(), /)

  Bases: `list`[`_SelectorType`]

  The `SelectorList` class is a subclass of the builtin `list` class, which provides a few additional methods.

  property attrib: Mapping[str, str]

    Return the attributes dictionary for the first element. If the list is empty, return an empty dict.

  css(query: str) → SelectorList[_SelectorType]

    Call the `.css()` method for each element in this list and return their results flattened as another `SelectorList`.

    `query` is the same argument as the one in `Selector.css()`

  drop() → None

    Drop matched nodes from the parent for each element in this list.

  extract() → list[str]

    Call the `.get()` method for each element is this list and return their results flattened, as a list of strings.

  extract_first(default: str | None = None) → Any

    Return the result of `.get()` for the first element in this list. If the list is empty, return the default value.

  get(default: None = None) → str | None
  get(default: str) → str

    Return the result of `.get()` for the first element in this list. If the list is empty, return the default value.

  getall() → list[str]

    Call the `.get()` method for each element is this list and return their results flattened, as a list of strings.

  jmespath(query: str, **kwargs: Any) → SelectorList[_SelectorType]

    Call the `.jmespath()` method for each element in this list and return their results flattened as another `SelectorList`.

    `query` is the same argument as the one in `Selector.jmespath()`.

    Any additional named arguments are passed to the underlying `jmespath.search` call, e.g.:

        selector.jmespath('author.name', options=jmespath.Options(dict_cls=collections.OrderedDict))

  re(regex: str | Pattern[str], replace_entities: bool = True) → list[str]

    Call the `.re()` method for each element in this list and return their results flattened, as a list of strings.

    By default, character entity references are replaced by their corresponding character (except for `&amp;` and `&lt;`. Passing `replace_entities` as `False` switches off these replacements.

  re_first(regex: str | Pattern[str], default: None = None, replace_entities: bool = True) → str | None
  re_first(regex: str | Pattern[str], default: str, replace_entities: bool = True) → str

    Call the `.re()` method for the first element in this list and return the result in an string. If the list is empty or the regex doesn’t match anything, return the default value (`None` if the argument is not provided).

    By default, character entity references are replaced by their corresponding character (except for `&amp;` and `&lt;`. Passing `replace_entities` as `False` switches off these replacements.

  xpath(xpath: str, namespaces: Mapping[str, str] | None = None, **kwargs: Any) → SelectorList[_SelectorType]

    Call the `.xpath()` method for each element in this list and return their results flattened as another `SelectorList`.

    `xpath` is the same argument as the one in `Selector.xpath()`

    `namespaces` is an optional `prefix: namespace-uri` mapping (dict) for additional prefixes to those registered with `register_namespace(prefix, uri)`. Contrary to `register_namespace()`, these prefixes are not saved for future calls.

    Any additional named arguments can be used to pass values for XPath variables in the XPath expression, e.g.:

        selector.xpath('//a[href=$url]', url="http://www.example.com")

parsel.selector.create_root_node(text: str, parser_cls: type[XMLParser | HTMLParser], base_url: str | None = None, huge_tree: bool = True, body: bytes = b'', encoding: str = 'utf-8') → _Element

  Create root node for text using given parser class.

## parsel.utils

parsel.utils.extract_regex(regex: str | Pattern[str], text: str, replace_entities: bool = True) → list[str]

  Extract a list of strings from the given text/encoding using the following policies: * if the regex contains a named group called “extract” that will be returned * if the regex contains multiple numbered groups, all those will be returned (flattened) * if the regex doesn’t contain any group the entire regex matching is returned

parsel.utils.flatten(sequence) → list

  Returns a single, flat list which contains all elements retrieved from the sequence and all recursively contained sub-sequences (iterables). Examples: >>> [1, 2, [3,4], (5,6)] [1, 2, [3, 4], (5, 6)] >>> flatten([[[1,2,3], (42,None)], [4,5], [6], 7, (8,9,10)]) [1, 2, 3, 42, None, 4, 5, 6, 7, 8, 9, 10] >>> flatten([“foo”, “bar”]) [‘foo’, ‘bar’] >>> flatten([“foo”, [“baz”, 42], “bar”]) [‘foo’, ‘baz’, 42, ‘bar’]

parsel.utils.iflatten(sequence) → Iterator

  Similar to `.flatten()`, but returns iterator instead Examples: >>> list(iflatten([[1, 2], (3, 4)])) [1, 2, 3, 4]

parsel.utils.shorten(text: str, width: int, suffix: str = '...') → str

  Truncate the given text to fit in the given width.