# tld

Extract the top level domain (TLD) from the URL given. List of TLD names is taken from Public Suffix.

Optionally raises exceptions on non-existing TLDs or silently fails (if `fail_silently` argument is set to True).

## Prerequisites

- Python 3.7, 3.8, 3.9, 3.10 or 3.11.

## Documentation

Documentation is available on Read the Docs.

## Installation

Latest stable version on PyPI:

    pip install tld

Or latest stable version from GitHub:

    pip install https://github.com/barseghyanartur/tld/archive/stable.tar.gz

## Usage examples

In addition to examples below, see the jupyter notebook workbook file.

### Get the TLD name as string from the URL given

    from tld import get_tld

    get_tld("http://www.google.co.uk")
    # 'co.uk'

    get_tld("http://www.google.idontexist", fail_silently=True)
    # None

### Get the TLD as an object

    from tld import get_tld

    res = get_tld("http://some.subdomain.google.co.uk", as_object=True)

    res
    # 'co.uk'

    res.subdomain
    # 'some.subdomain'

    res.domain
    # 'google'

    res.tld
    # 'co.uk'

    res.fld
    # 'google.co.uk'

    res.parsed_url
    # SplitResult(
    #     scheme='http',
    #     netloc='some.subdomain.google.co.uk',
    #     path='',
    #     query='',
    #     fragment=''
    # )

### Get TLD name, ignoring the missing protocol

    from tld import get_tld, get_fld

    get_tld("www.google.co.uk", fix_protocol=True)
    # 'co.uk'

    get_fld("www.google.co.uk", fix_protocol=True)
    # 'google.co.uk'

### Return TLD parts as tuple

    from tld import parse_tld

    parse_tld('http://www.google.com')
    # 'com', 'google', 'www'

### Get the first level domain name as string from the URL given

    from tld import get_fld

    get_fld("http://www.google.co.uk")
    # 'google.co.uk'

    get_fld("http://www.google.idontexist", fail_silently=True)
    # None

### Check if some tld is a valid tld

    from tld import is_tld

    is_tld('co.uk)
    # True

    is_tld('uk')
    # True

    is_tld('tld.doesnotexist')
    # False

    is_tld('www.google.com')
    # False

## Custom TLD parsers

By default list of TLD names is taken from Mozilla. Parsing implemented in the `tld.utils.MozillaTLDSourceParser` class. If you want to use another parser, subclass the `tld.base.BaseTLDSourceParser`, provide `uid`, `source_url`, `local_path` and implement the `get_tld_names` method. Take the `tld.utils.MozillaTLDSourceParser` as a good example of such implementation. You could then use `get_tld` (as well as other `tld` module functions) as shown below:

    from tld import get_tld
    from some.module import CustomTLDSourceParser

    get_tld(
        "http://www.google.co.uk",
        parser_class=CustomTLDSourceParser
    )

## Custom list of TLD names

You could maintain your own custom version of the TLD names list (even multiple ones) and use them simultaneously with built in TLD names list.

You would then store them locally and provide a path to it as shown below:

    from tld import get_tld
    from tld.utils import BaseMozillaTLDSourceParser

    class CustomBaseMozillaTLDSourceParser(BaseMozillaTLDSourceParser):

        uid: str = 'custom_mozilla'
        local_path: str = 'tests/res/effective_tld_names_custom.dat.txt'

    get_tld(
        "http://www.foreverchild",
        parser_class=CustomBaseMozillaTLDSourceParser
    )
    # 'foreverchild'

Same goes for first level domain names:

    from tld import get_fld

    get_fld(
        "http://www.foreverchild",
        parser_class=CustomBaseMozillaTLDSourceParser
    )
    # 'www.foreverchild'

Note, that in both examples shown above, there the original TLD names file has been modified in the following way:

    ...
    // ===BEGIN ICANN DOMAINS===

    // This one actually does not exist, added for testing purposes
    foreverchild
    ...

## Free up resources

To free up memory occupied by loading of custom TLD names, use `reset_tld_names` function with `tld_names_local_path` parameter.

    from tld import get_tld, reset_tld_names

    # Get TLD from a custom TLD names parser
    get_tld(
        "http://www.foreverchild",
        parser_class=CustomBaseMozillaTLDSourceParser
    )

    # Free resources occupied by the custom TLD names list
    reset_tld_names("tests/res/effective_tld_names_custom.dat.txt")

## Troubleshooting

If somehow domain names listed here are not recognised, make sure you have the most recent version of TLD names in your virtual environment:

    update-tld-names

To update TLD names list for a single parser, specify it as an argument:

    update-tld-names mozilla

## Testing

Simply type:

    pytest

Or use tox:

    tox

Or use tox to check specific env:

    tox -e py39

## Writing documentation

Keep the following hierarchy.

    =====
    title
    =====

    header
    ======

    sub-header
    ----------

    sub-sub-header
    ~~~~~~~~~~~~~~

    sub-sub-sub-header
    ^^^^^^^^^^^^^^^^^^

    sub-sub-sub-sub-header
    ++++++++++++++++++++++

    sub-sub-sub-sub-sub-header
    **************************

## License

MPL-1.1 OR GPL-2.0-only OR LGPL-2.1-or-later

## Support

For security issues contact me at the e-mail given in the Author section.

For overall issues, go to GitHub.

## Author

Artur Barseghyan <artur.barseghyan@gmail.com>

## Project documentation

Contents:

- Security Policy
  - Reporting a Vulnerability
  - Supported Versions
- Contributor Covenant Code of Conduct
  - Our Pledge
  - Our Standards
  - Enforcement Responsibilities
  - Scope
  - Enforcement
  - Enforcement Guidelines
    - 1. Correction
    - 2. Warning
    - 3. Temporary Ban
    - 4. Permanent Ban
  - Attribution
- Contributor guidelines
  - Developer prerequisites
    - pre-commit
  - Code standards
  - Requirements
  - Virtual environment
  - Documentation
  - Testing
  - Pull requests
  - Questions
  - Issues
- Release history and notes
  - 0.13
  - 0.12.7
  - 0.12.6
  - 0.12.5
  - 0.12.4
  - 0.12.3
  - 0.12.2
  - 0.12.1
  - 0.12
  - 0.11.11
  - 0.11.10
  - 0.11.9
  - 0.11.8
  - 0.11.7
  - 0.11.6
  - 0.11.5
  - 0.11.4
  - 0.11.3
  - 0.11.2
  - 0.11.1
  - 0.11
  - 0.10
  - 0.9.8
  - 0.9.7
  - 0.9.6
  - 0.9.5
  - 0.9.4
  - 0.9.3
  - 0.9.2
  - 0.9.1
  - 0.9
  - 0.8
  - 0.7.10
  - 0.7.9
  - 0.7.8
  - 0.7.7
  - 0.7.6
  - 0.7.5
  - 0.7.4
  - 0.7.3
  - 0.7.2
  - 0.7.1
  - 0.7
  - 0.6.4
  - 0.6.3
  - 0.6.2
  - 0.6.1
  - 0.6
  - 0.5
  - 0.4
- tld package
  - Submodules
  - tld.base module
    - `BaseTLDSourceParser`
      - `BaseTLDSourceParser.get_tld_names()`
      - `BaseTLDSourceParser.include_private`
      - `BaseTLDSourceParser.local_path`
      - `BaseTLDSourceParser.source_url`
      - `BaseTLDSourceParser.uid`
      - `BaseTLDSourceParser.update_tld_names()`
      - `BaseTLDSourceParser.validate()`
    - `Registry`
      - `Registry.REGISTRY`
      - `Registry.get()`
      - `Registry.items()`
      - `Registry.reset()`
  - tld.conf module
    - `get_setting()`
    - `reset_settings()`
    - `set_setting()`
  - tld.defaults module
  - tld.exceptions module
    - `TldBadUrl`
    - `TldDomainNotFound`
    - `TldIOError`
    - `TldImproperlyConfigured`
  - tld.helpers module
    - `PROJECT_DIR()`
    - `project_dir()`
  - tld.registry module
    - `Registry`
      - `Registry.REGISTRY`
      - `Registry.get()`
      - `Registry.items()`
      - `Registry.reset()`
  - tld.result module
    - `Result`
      - `Result.domain`
      - `Result.extension`
      - `Result.fld`
      - `Result.parsed_url`
      - `Result.subdomain`
      - `Result.suffix`
      - `Result.tld`
  - tld.trie module
    - `Trie`
      - `Trie.add()`
    - `TrieNode`
      - `TrieNode.children`
      - `TrieNode.exception`
      - `TrieNode.leaf`
      - `TrieNode.private`
  - tld.utils module
    - `BaseMozillaTLDSourceParser`
      - `BaseMozillaTLDSourceParser.get_tld_names()`
      - `BaseMozillaTLDSourceParser.local_path`
      - `BaseMozillaTLDSourceParser.source_url`
    - `MozillaPublicOnlyTLDSourceParser`
      - `MozillaPublicOnlyTLDSourceParser.include_private`
      - `MozillaPublicOnlyTLDSourceParser.local_path`
      - `MozillaPublicOnlyTLDSourceParser.source_url`
      - `MozillaPublicOnlyTLDSourceParser.uid`
    - `MozillaTLDSourceParser`
      - `MozillaTLDSourceParser.local_path`
      - `MozillaTLDSourceParser.source_url`
      - `MozillaTLDSourceParser.uid`
    - `Result`
      - `Result.domain`
      - `Result.extension`
      - `Result.fld`
      - `Result.parsed_url`
      - `Result.subdomain`
      - `Result.suffix`
      - `Result.tld`
    - `get_fld()`
    - `get_tld()`
    - `get_tld_names()`
    - `get_tld_names_container()`
    - `is_tld()`
    - `parse_tld()`
    - `pop_tld_names_container()`
    - `process_url()`
    - `reset_tld_names()`
    - `update_tld_names()`
    - `update_tld_names_cli()`
    - `update_tld_names_container()`
  - Module contents
    - `Result`
      - `Result.domain`
      - `Result.extension`
      - `Result.fld`
      - `Result.parsed_url`
      - `Result.subdomain`
      - `Result.suffix`
      - `Result.tld`
    - `get_fld()`
    - `get_tld()`
    - `get_tld_names()`
    - `is_tld()`
    - `parse_tld()`
    - `update_tld_names()`

## Indices and tables

- Index

- Module Index

- Search Page