# transliterate

Bi-directional transliterator for Python. Transliterates (unicode) strings according to the rules specified in the language packs (source script <-> target script).

Comes with language packs for the following languages (listed in alphabetical order):

- Armenian
- Bulgarian (beta)
- Georgian
- Greek
- Macedonian (alpha)
- Mongolian (alpha)
- Russian
- Serbian (alpha)
- Ukrainian (beta)

There are also a number of useful tools included, such as:

- Simple lorem ipsum generator, which allows lorem ipsum generation in the language chosen.
- Language detection for the text (if appropriate language pack is available).
- Slugify function for non-latin texts.

## Prerequisites

- Python >=2.7, >=3.4, PyPy

## Installation

Install with latest stable version from PyPI.

    pip install transliterate

or install the latest stable version from BitBucket:

    pip install https://bitbucket.org/barseghyanartur/transliterate/get/stable.tar.gz

or install the latest stable version from GitHub:

    pip install https://github.com/barseghyanartur/transliterate/archive/stable.tar.gz

That’s all. See the Usage and examples section for more.

## Usage and examples

### Simple usage

Required imports

    from transliterate import translit, get_available_language_codes

Original text

    text = "Lorem ipsum dolor sit amet"

Transliteration to Armenian

    print(translit(text, 'hy'))

    # Լօրեմ իպսում դօլօր սիտ ամետ

Transliteration to Georgian

    print(translit(text, 'ka'))

    # ლორემ იპსუმ დოლორ სით ამეთ

Transliteration to Greek

    print(translit(text, 'el'))

    # Λορεμ ιψθμ δολορ σιτ αμετ

Transliteration to Russian

    print(translit(text, 'ru'))

    # Лорем ипсум долор сит амет

List of available (registered) languages

    print(get_available_language_codes())

    # ['el', 'hy', 'ka', 'ru']

Reversed transliterations are transliterations made from target language to source language (in terms they are defined in language packs). In case of reversed transliterations, you may leave out the `language_code` attribute, although if you know it on beforehand, specify it since it works faster that way.

Reversed transliteration from Armenian

    print(translit(u"Լօրեմ իպսում դօլօր սիտ ամետ", 'hy', reversed=True))

    # Lorem ipsum dolor sit amet

Reversed transliteration from Armenian with `language_code` argument left out

    print(translit(u"Լօրեմ իպսում դօլօր սիտ ամետ", reversed=True))

    # Lorem ipsum dolor sit amet

Reversed transliteration from Georgian

    print(translit(u"ლორემ იპსუმ დოლორ სით ამეთ", 'ka', reversed=True))

    # Lorem ipsum dolor sit amet

Reversed transliteration from Georgian with `language_code` argument left out

    print(translit(u"ლორემ იპსუმ დოლორ სით ამეთ", reversed=True))

    # Lorem ipsum dolor sit amet

Reversed transliteration from Greek

    print(translit(u"Λορεμ ιψθμ δολορ σιτ αμετ", 'el', reversed=True))

    # Lorem ipsum dolor sit amet

Reversed transliteration from Greek with `language_code` argument left out

    print(translit(u"Λορεμ ιψθμ δολορ σιτ αμετ", reversed=True))

    # Lorem ipsum dolor sit amet

Reversed transliteration from Russian (Cyrillic)

    print(translit(u"Лорем ипсум долор сит амет", 'ru', reversed=True))

    # Lorеm ipsum dolor sit amеt

Reversed transliteration from Russian (Cyrillic) with `language_code` argument left out

    print(translit(u"Лорем ипсум долор сит амет", reversed=True))

    # Lorem ipsum dolor sit amet

Testing the decorator

    from transliterate.decorators import transliterate_function

    @transliterate_function(language_code='hy')
    def decorator_test(text):
        return text

    print(decorator_test(u"Lorem ipsum dolor sit amet"))

    # Լօրեմ իպսում դօլօր սիտ ամետ

### Working with large amounts of data

If you know which language pack shall be used for transliteration, especially when working with large amounts of data, it makes sense to get the transliteration function in the following way:

    from transliterate import get_translit_function

    translit_hy = get_translit_function('hy')

    print(translit_hy(u"Լօրեմ իպսում դօլօր սիտ ամետ", reversed=True))
    # Lorem ipsum dolor sit amet

    print(translit_hy(u"Lorem ipsum dolor sit amet"))
    # Լօրեմ իպսում դօլօր սիտ ամետ

### Registering a custom language pack

#### Basics

Make sure to call the `autodiscover` function before registering your own language packs if you want to use the bundled language packs along with your own custom ones.

    from transliterate.discover import autodiscover
    autodiscover()

Then the custom language pack part comes.

    from transliterate.base import TranslitLanguagePack, registry

    class ExampleLanguagePack(TranslitLanguagePack):
        language_code = "example"
        language_name = "Example"
        mapping = (
            u"abcdefghij",
            u"1234567890",
        )

    registry.register(ExampleLanguagePack)

    print(get_available_language_codes())

    # ['el', 'hy', 'ka', 'ru', 'example']

    print(translit(text, 'example'))

    # Lor5m 9psum 4olor s9t 1m5t

It’s possible to replace existing language packs with your own ones. By default, existing language packs are not force-installed.

To force install a language pack, set the `force` argument to True when registering a language pack. In that case, if a language pack with same language code has already been registered, it will be replaced; otherwise, if language pack didn’t exist in the registry, it will be just registered.

    registry.register(ExampleLanguagePack, force=True)

Forced language packs can’t be replaced or unregistered.

#### API in depth

There are 7 class properties that you could/should be using in your language pack, of which 4 are various sorts of mappings.

##### Mappings

- `mapping` (tuple): A tuple of two strings, that simply represent the mapping of characters from the source language to the target language. For example, if your source language is Latin and you want to convert “a”, “b”, “c”, “d” and “e” characters to appropriate characters in Russian Cyrillic, your mapping would look as follows:

      mapping = (u"abcde", u"абцде")

  Example (taken from the Greek language pack).

      mapping = (
          u"abgdezhiklmnxoprstyfwuABGDEZHIKLMNXOPRSTYFWU",
          u"αβγδεζηικλμνξοπρστυφωθΑΒΓΔΕΖΗΙΚΛΜΝΞΟΠΡΣΤΥΦΩΘ",
      )

- `reversed_specific_mapping` (tuple): When making reversed translations, the `mapping` property is still used, but in some cases you need to provide additional rules. This property (`reversed_specific_mapping`) is meant for such cases. Further, is alike the `mapping`.

  Example (taken from the Greek language pack).

      reversed_specific_mapping = (
          u"θΘ",
          u"uU"
      )

- `pre_processor_mapping` (dict): A dictionary of mapping from source language to target language. Use this only in cases if a single character in source language shall be represented by more than one character in the target language.

  Example (taken from the Greek language pack).

      pre_processor_mapping = {
          u"th": u"θ",
          u"ch": u"χ",
          u"ps": u"ψ",
          u"TH": u"Θ",
          u"CH": u"Χ",
          u"PS": u"Ψ",
      }

- `reversed_specific_pre_processor_mapping`: Same as `pre_processor_mapping`, but used in reversed translations.

  Example (taken from the Armenian language pack)

      reversed_specific_pre_processor_mapping = {
          u"ու": u"u",
          u"Ու": u"U"
      }

##### Additional

- `character_ranges` (tuple): A tuple of character ranges (unicode table). Used in language detection. Works only if `detectable` property is set to True. Be aware, that language (or shall I better be saying - script) detection is very basic and is based on characters only.
- `detectable` (bool): If set to True, language pack would be used for automatic language detection.

### Using the lorem ipsum generator

Note, that due to incompatibility of the original lorem-ipsum-generator package with Python 3, when used with Python 3 transliterate uses its’ own simplified fallback lorem ipsum generator (which still does the job).

Required imports

    from transliterate.contrib.apps.translipsum import TranslipsumGenerator

Generating paragraphs in Armenian

    g_am = TranslipsumGenerator(language_code='hy')
    print(g_am.generate_paragraph())

    # Մագնա տրիստիքուե ֆաուցիբուս ֆամես նետուս նետուս օրցի մաուրիս,
    # սուսցիպիտ. Դապիբուս րիսուս սեդ ադիպիսցինգ դիցտում. Ֆերմենտում ուրնա
    # նատօքուե ատ. Uլտրիցես եգետ, տացիտի. Լիտօրա ցլասս ցօնուբիա պօսուերե
    # մալեսուադա ին իպսում իդ պեր վե.

Generating sentense in Georgian

    g_ka = TranslipsumGenerator(language_code='ka')
    print(g_ka.generate_sentence())

    # გგეთ ყუამ არსუ ვულფუთათე რუთრუმ აუთორ.

Generating sentense in Greek

    g_el = TranslipsumGenerator(language_code='el')
    print(g_el.generate_sentence())

    # Νεc cρασ αμετ, ελιτ vεστιβθλθμ εθ, αενεαν ναμ, τελλθσ vαριθσ.

Generating sentense in Russian (Cyrillic)

    g_ru = TranslipsumGenerator(language_code='ru')
    print(g_ru.generate_sentence())

    # Рисус cонсеcтетуер, фусcе qуис лаореет ат ерос пэдэ фелис магна.

### Language detection

Required imports

    from transliterate import detect_language

Detect Armenian text

    detect_language(u'Լօրեմ իպսում դօլօր սիտ ամետ')

    # hy

Detect Georgian text

    detect_language(u'ლორემ იპსუმ დოლორ სით ამეთ')

    # ka

Detect Greek text

    detect_language(u'Λορεμ ιψθμ δολορ σιτ αμετ')

    # el

Detect Russian (Cyrillic) text

    detect_language(u'Лорем ипсум долор сит амет')

    # ru

### Slugify

Required imports

    from transliterate import slugify

Slugify Armenian text

    slugify(u'Լօրեմ իպսում դօլօր սիտ ամետ')

    # lorem-ipsum-dolor-sit-amet

Slugify Georgian text

    slugify(u'ლორემ იპსუმ დოლორ სით ამეთ')

    # lorem-ipsum-dolor-sit-amet

Slugify Greek text

    slugify(u'Λορεμ ιψθμ δολορ σιτ αμετ')

    # lorem-ipsum-dolor-sit-amet

Slugify Russian (Cyrillic) text

    slugify(u'Лорем ипсум долор сит амет')

    # lorem-ipsum-dolor-sit-amet

## Missing a language pack?

Missing a language pack for your own language? Contribute to the project by making one and it will appear in a new version (which will be released very quickly).

## Writing documentation

Keep the following hierarchy.

    =====
    title
    =====

    header
    ======

    sub-header
    ----------

    sub-sub-header
    ~~~~~~~~~~~~~~

    sub-sub-sub-header
    ^^^^^^^^^^^^^^^^^^

    sub-sub-sub-sub-header
    ++++++++++++++++++++++

    sub-sub-sub-sub-sub-header
    **************************

## License

GPL 2.0/LGPL 2.1

## Support

For any issues contact me at the e-mail given in the Author section.

## Author

Artur Barseghyan <artur.barseghyan@gmail.com>

## Documentation

Contents:

- transliterate
  - Prerequisites
  - Installation
  - Usage and examples
    - Simple usage
    - Working with large amounts of data
    - Registering a custom language pack
      - Basics
      - API in depth
        - Mappings
        - Additional
    - Using the lorem ipsum generator
    - Language detection
    - Slugify
  - Missing a language pack?
  - Writing documentation
  - License
  - Support
  - Author
  - Documentation
    - transliterate package
      - Subpackages
        - transliterate.contrib package
          - Subpackages
            - transliterate.contrib.apps package
              - Subpackages
                - transliterate.contrib.apps.translipsum package
                  - Submodules
                  - transliterate.contrib.apps.translipsum.utils module
                  - Module contents
              - Module contents
            - transliterate.contrib.languages package
              - Subpackages
                - transliterate.contrib.languages.bg package
                  - Subpackages
                    - transliterate.contrib.languages.bg.data package
                      - Submodules
                      - transliterate.contrib.languages.bg.data.default module
                      - Module contents
                  - Submodules
                  - transliterate.contrib.languages.bg.translit_language_pack module
                  - Module contents
                - transliterate.contrib.languages.el package
                  - Subpackages
                    - transliterate.contrib.languages.el.data package
                      - Submodules
                      - transliterate.contrib.languages.el.data.default module
                      - Module contents
                  - Submodules
                  - transliterate.contrib.languages.el.translit_language_pack module
                  - Module contents
                - transliterate.contrib.languages.he package
                  - Submodules
                  - transliterate.contrib.languages.he.translit_language_pack module
                  - Module contents
                - transliterate.contrib.languages.hi package
                  - Submodules
                  - transliterate.contrib.languages.hi.translit_language_pack module
                  - Module contents
                - transliterate.contrib.languages.hy package
                  - Subpackages
                    - transliterate.contrib.languages.hy.data package
                      - Submodules
                      - transliterate.contrib.languages.hy.data.default module
                      - Module contents
                  - Submodules
                  - transliterate.contrib.languages.hy.translit_language_pack module
                  - Module contents
                - transliterate.contrib.languages.ka package
                  - Subpackages
                    - transliterate.contrib.languages.ka.data package
                      - Submodules
                      - transliterate.contrib.languages.ka.data.default module
                      - Module contents
                  - Submodules
                  - transliterate.contrib.languages.ka.translit_language_pack module
                  - Module contents
                - transliterate.contrib.languages.l1 package
                  - Subpackages
                    - transliterate.contrib.languages.l1.data package
                      - Submodules
                      - transliterate.contrib.languages.l1.data.default module
                      - Module contents
                  - Submodules
                  - transliterate.contrib.languages.l1.translit_language_pack module
                  - Module contents
                - transliterate.contrib.languages.mk package
                  - Subpackages
                    - transliterate.contrib.languages.mk.data package
                      - Submodules
                      - transliterate.contrib.languages.mk.data.default module
                      - Module contents
                  - Submodules
                  - transliterate.contrib.languages.mk.translit_language_pack module
                  - Module contents
                - transliterate.contrib.languages.mn package
                  - Subpackages
                    - transliterate.contrib.languages.mn.data package
                      - Submodules
                      - transliterate.contrib.languages.mn.data.default module
                      - Module contents
                  - Submodules
                  - transliterate.contrib.languages.mn.translit_language_pack module
                  - Module contents
                - transliterate.contrib.languages.ru package
                  - Subpackages
                    - transliterate.contrib.languages.ru.data package
                      - Submodules
                      - transliterate.contrib.languages.ru.data.default module
                      - Module contents
                  - Submodules
                  - transliterate.contrib.languages.ru.translit_language_pack module
                  - Module contents
                - transliterate.contrib.languages.sr package
                  - Subpackages
                    - transliterate.contrib.languages.sr.data package
                      - Submodules
                      - transliterate.contrib.languages.sr.data.default module
                      - Module contents
                  - Submodules
                  - transliterate.contrib.languages.sr.translit_language_pack module
                  - Module contents
                - transliterate.contrib.languages.uk package
                  - Subpackages
                    - transliterate.contrib.languages.uk.data package
                      - Submodules
                      - transliterate.contrib.languages.uk.data.default module
                      - Module contents
                  - Submodules
                  - transliterate.contrib.languages.uk.translit_language_pack module
                  - Module contents
              - Module contents
          - Module contents
        - transliterate.tests package
          - Subpackages
            - transliterate.tests.data package
              - Submodules
              - transliterate.tests.data.default module
              - Module contents
          - Submodules
          - transliterate.tests.base module
          - transliterate.tests.defaults module
          - transliterate.tests.helpers module
          - transliterate.tests.test_transliterate module
          - Module contents
      - Submodules
      - transliterate.base module
      - transliterate.conf module
      - transliterate.decorators module
      - transliterate.defaults module
      - transliterate.discover module
      - transliterate.exceptions module
      - transliterate.helpers module
      - transliterate.utils module
      - Module contents
    - Release history
      - 1.10.2
      - 1.10.1
      - 1.10
      - 1.9
      - 1.8.1
      - 1.8
      - 1.7.6
      - 1.7.5
      - 1.7.4
      - 1.7.3
      - 1.7.2
      - 1.7.1
      - 1.7
      - 1.6.1
      - 1.6
      - 1.5
      - 1.4
      - 1.3
      - 1.2
      - 1.1
      - 1.0
      - 0.9
      - 0.8
      - 0.7
      - 0.6
      - 0.5
      - 0.4
      - 0.3
      - 0.2
      - 0.1
  - Indices and tables
- transliterate package
  - Subpackages
    - transliterate.contrib package
      - Subpackages
        - transliterate.contrib.apps package
          - Subpackages
            - transliterate.contrib.apps.translipsum package
              - Submodules
              - transliterate.contrib.apps.translipsum.utils module
              - Module contents
          - Module contents
        - transliterate.contrib.languages package
          - Subpackages
            - transliterate.contrib.languages.bg package
              - Subpackages
                - transliterate.contrib.languages.bg.data package
                  - Submodules
                  - transliterate.contrib.languages.bg.data.default module
                  - Module contents
              - Submodules
              - transliterate.contrib.languages.bg.translit_language_pack module
              - Module contents
            - transliterate.contrib.languages.el package
              - Subpackages
                - transliterate.contrib.languages.el.data package
                  - Submodules
                  - transliterate.contrib.languages.el.data.default module
                  - Module contents
              - Submodules
              - transliterate.contrib.languages.el.translit_language_pack module
              - Module contents
            - transliterate.contrib.languages.he package
              - Submodules
              - transliterate.contrib.languages.he.translit_language_pack module
              - Module contents
            - transliterate.contrib.languages.hi package
              - Submodules
              - transliterate.contrib.languages.hi.translit_language_pack module
              - Module contents
            - transliterate.contrib.languages.hy package
              - Subpackages
                - transliterate.contrib.languages.hy.data package
                  - Submodules
                  - transliterate.contrib.languages.hy.data.default module
                  - Module contents
              - Submodules
              - transliterate.contrib.languages.hy.translit_language_pack module
              - Module contents
            - transliterate.contrib.languages.ka package
              - Subpackages
                - transliterate.contrib.languages.ka.data package
                  - Submodules
                  - transliterate.contrib.languages.ka.data.default module
                  - Module contents
              - Submodules
              - transliterate.contrib.languages.ka.translit_language_pack module
              - Module contents
            - transliterate.contrib.languages.l1 package
              - Subpackages
                - transliterate.contrib.languages.l1.data package
                  - Submodules
                  - transliterate.contrib.languages.l1.data.default module
                  - Module contents
              - Submodules
              - transliterate.contrib.languages.l1.translit_language_pack module
              - Module contents
            - transliterate.contrib.languages.mk package
              - Subpackages
                - transliterate.contrib.languages.mk.data package
                  - Submodules
                  - transliterate.contrib.languages.mk.data.default module
                  - Module contents
              - Submodules
              - transliterate.contrib.languages.mk.translit_language_pack module
              - Module contents
            - transliterate.contrib.languages.mn package
              - Subpackages
                - transliterate.contrib.languages.mn.data package
                  - Submodules
                  - transliterate.contrib.languages.mn.data.default module
                  - Module contents
              - Submodules
              - transliterate.contrib.languages.mn.translit_language_pack module
              - Module contents
            - transliterate.contrib.languages.ru package
              - Subpackages
                - transliterate.contrib.languages.ru.data package
                  - Submodules
                  - transliterate.contrib.languages.ru.data.default module
                  - Module contents
              - Submodules
              - transliterate.contrib.languages.ru.translit_language_pack module
              - Module contents
            - transliterate.contrib.languages.sr package
              - Subpackages
                - transliterate.contrib.languages.sr.data package
                  - Submodules
                  - transliterate.contrib.languages.sr.data.default module
                  - Module contents
              - Submodules
              - transliterate.contrib.languages.sr.translit_language_pack module
              - Module contents
            - transliterate.contrib.languages.uk package
              - Subpackages
                - transliterate.contrib.languages.uk.data package
                  - Submodules
                  - transliterate.contrib.languages.uk.data.default module
                  - Module contents
              - Submodules
              - transliterate.contrib.languages.uk.translit_language_pack module
              - Module contents
          - Module contents
      - Module contents
    - transliterate.tests package
      - Subpackages
        - transliterate.tests.data package
          - Submodules
          - transliterate.tests.data.default module
          - Module contents
      - Submodules
      - transliterate.tests.base module
      - transliterate.tests.defaults module
      - transliterate.tests.helpers module
      - transliterate.tests.test_transliterate module
      - Module contents
  - Submodules
  - transliterate.base module
  - transliterate.conf module
  - transliterate.decorators module
  - transliterate.defaults module
  - transliterate.discover module
  - transliterate.exceptions module
  - transliterate.helpers module
  - transliterate.utils module
  - Module contents
- Release history
  - 1.10.2
  - 1.10.1
  - 1.10
  - 1.9
  - 1.8.1
  - 1.8
  - 1.7.6
  - 1.7.5
  - 1.7.4
  - 1.7.3
  - 1.7.2
  - 1.7.1
  - 1.7
  - 1.6.1
  - 1.6
  - 1.5
  - 1.4
  - 1.3
  - 1.2
  - 1.1
  - 1.0
  - 0.9
  - 0.8
  - 0.7
  - 0.6
  - 0.5
  - 0.4
  - 0.3
  - 0.2
  - 0.1

## Indices and tables

- Index
- Module Index
- Search Page