.. currentmodule:: parfive

=======
Parfive
=======

Parfive is a small library for downloading files, its objective is to provide a simple API for queuing files for download and then providing excellent feedback to the user about the in progress downloads.
It also aims to provide a clear interface for inspecting any failed downloads.

The parfive package was motivated by the needs of `SunPy's <https://sunpy.org>`__ ``net`` submodule, but should be generally applicable to anyone who wants a user friendly way of downloading multiple files in parallel.
Parfive supports downloading files over either HTTP or FTP using `aiohttp <http://aiohttp.readthedocs.io/>`__ and `aioftp <https://aioftp.readthedocs.io/>`__ ``aioftp`` is an optional dependency, which does not need to be installed to download files over HTTP.

Installation
------------

parfive can be installed via pip::

  pip install parfive

or with FTP support::

  pip install parfive[ftp]

or with conda from conda-forge::

  conda install -c conda-forge parfive

or from `GitHub <https://github.com/Cadair/parfive>`__.

Usage
-----

parfive works by creating a downloader object, queuing downloads with it and then running the download.
parfive has a synchronous API, but uses `asyncio` to parallelise downloading the files.

A simple example is::

  from parfive import Downloader
  dl = Downloader()
  dl.enqueue_file("http://data.sunpy.org/sample-data/predicted-sunspot-radio-flux.txt", path="./")
  files = dl.download()

It's also possible to download a list of URLs to a single destination using the `parfive.Downloader.simple_download` method::

  from parfive import Downloader
  files = Downloader.simple_download(['http://212.183.159.230/5MB.zip' 'http://212.183.159.230/10MB.zip'], path="./")

Parfive also bundles a CLI. The following example will download the two files concurrently::

  $ parfive 'http://212.183.159.230/5MB.zip' 'http://212.183.159.230/10MB.zip'
  $ parfive --help
  usage: parfive [-h] [--max-conn MAX_CONN] [--overwrite] [--no-file-progress] [--directory DIRECTORY] [--print-filenames] URLS [URLS ...]

  Parfive, the python asyncio based downloader

  positional arguments:
    URLS                  URLs of files to be downloaded.

  optional arguments:
    -h, --help            show this help message and exit
    --max-conn MAX_CONN   Number of maximum connections.
    --overwrite           Overwrite if the file exists.
    --no-file-progress    Show progress bar for each file.
    --directory DIRECTORY
                          Directory to which downloaded files are saved.
    --print-filenames     Print successfully downloaded files's names to stdout.

.. automodapi:: parfive
   :no-heading:
   :no-main-docstr:

Environment Variables
---------------------

To facilitate debugging parfive reads the following environment variables:

* ``PARFIVE_SINGLE_DOWNLOAD`` - If set (to any value) this variable sets ``max_conn`` and ``max_splits`` to one; meaning that no parallelisation of the downloads will occur.
* ``PARFIVE_DISABLE_RANGE`` - If set (to any value) this variable will set ``max_splits`` to one; meaning that each file downloaded will only have one concurrent connection, although multiple files may be downloaded simultaneously.
* ``PARFIVE_OVERWRITE_ENABLE_AIOFILES`` - If set (to any value) and aiofiles is installed in the system, aiofiles will be used to write files to disk.
* ``PARFIVE_DEBUG`` - If set (to any value) will configure the built-in Python logger to log to stderr and set parfive, aiohttp and aioftp to debug levels.

Changelog
---------

See `GitHub Releases <https://github.com/Cadair/parfive/releases>`__ for the release history and changelog.
