# Parsl - Parallel Scripting Library

Parsl extends parallelism in Python beyond a single computer.

You can use Parsl just like Python’s parallel executors but across multiple cores and nodes. However, the real power of Parsl is in expressing multi-step workflows of functions. Parsl lets you chain functions together and will launch each function as inputs and computing resources are available.

    import parsl
    from parsl import python_app

    # Start Parsl on a single computer
    parsl.load()

    # Make functions parallel by decorating them
    @python_app
    def f(x):
        return x + 1

    @python_app
    def g(x, y):
        return x + y

    # These functions now return Futures, and can be chained
    future = f(1)
    assert future.result() == 2

    # Functions run concurrently, can be chained
    f_a, f_b = f(2), f(3)
    future = g(f_a, f_b)
    assert future.result() == 7

Start with the configuration quickstart to learn how to tell Parsl how to use your computing resource, see if a template configuration for your supercomputer is already available, then explore the parallel computing patterns to determine how to use parallelism best in your application.

Parsl is an open-source code, and available on GitHub: https://github.com/parsl/parsl/

## Why use Parsl?

### Parsl is Python

Everything about a Parsl program is written in Python. Parsl follows Python’s native parallelization approach and functions, how they combine into workflows, and where they run are all described in Python.

### Parsl works everywhere

Parsl can run parallel functions on a laptop and the world’s fastest supercomputers. Scaling from laptop to supercomputer is often as simple as changing the resource configuration. Parsl is tested on many of the top supercomputers.

### Parsl is flexible

Parsl supports many kinds of applications. Functions can be pure Python or invoke external codes, be single- or multi-threaded or GPUs.

### Parsl handles data

Parsl has first-class support for workflows involving files. Data will be automatically moved between workers, even if they reside on different filesystems.

### Parsl is fast

Parsl was built for speed. Workflows can manage tens of thousands of parallel tasks and process thousands of tasks per second.

### Parsl is a community

Parsl is part of a large, experienced community.

The Parsl Project was launched by researchers with decades of experience in workflows as part of a National Science Foundation project to create sustainable research software.

The Parsl team is guided by the community through its GitHub, conversations on Slack, Bi-Weekly developer calls, and engagement with the Workflows Community Initiative.

#### Table of Contents

- Quickstart
  - Installation
  - Getting started
  - Tutorial
  - Usage Tracking
  - For Developers
- User guide
  - Overview
  - Writing Parsl Apps
  - Configuring Parsl
  - Running Workflows
  - Advanced Topics
- Glossary
  - App
  - AppFuture
  - Bash App
  - Block
  - Checkpointing
  - Concurrency
  - Configuration
  - DataFuture
  - DataFlowKernel (DFK)
  - Elasticity
  - Execution Provider
  - Executor
  - Future
  - Job
  - Launcher
  - Manager
  - Memoization
  - MPI App
  - Node
  - Parallelism
  - Parsl Script
  - Plugin
  - Python App
  - Resource
  - Serialization
  - Staging
  - Task
  - Thread
  - Worker
  - Workflow
- FAQ
  - How can I debug a Parsl script?
  - How can I view outputs and errors from apps?
  - How can I make an App dependent on multiple inputs?
  - Can I pass any Python object between apps?
  - How do I specify where apps should be run?
  - Workers do not connect back to Parsl
  - parsl.errors.ConfigurationError
  - Remote execution fails with SystemError(unknown opcode)
  - Parsl complains about missing packages
  - How do I run code that uses Python2.X?
  - Parsl hangs
  - How can I start a Jupyter notebook over SSH?
  - How can I sync my conda environment and Jupyter environment?
  - Addressing SerializationError
  - How do I cite Parsl?
  - How can my tasks survive `WorkerLost` and `ManagerLost` at the end of a batch job?
- API Reference guide
  - Core
  - Configuration
  - Channels
  - Data management
  - Executors
  - Manager Selectors
  - Launchers
  - Providers
  - Batch jobs
  - Exceptions
  - Internal
  - Task Vine configuration
- Developer documentation
  - Contributing
  - Roadmap
  - Packaging
  - Doc Docs
- Historical Documents
  - Historical: Changelog
  - Historical: Libsubmit Changelog
  - Historical: Swift vs Parsl
  - Historical: Performance and Scalability

## Indices and tables

- Index

- Module Index

- Search Page