Skip to main content
`Ctrl`+`K`

# Backends

NetworkX can be configured to use separate thrid-party backends to improve performance and add functionality. Backends are optional, installed separately, and can be enabled either directly in the user’s code or through environment variables.

Note

The interface used by developers creating custom NetworkX backends is receiving frequent updates and improvements. Participating in weekly NetworkX dispatch meetings is an excellent way to stay updated and contribute to the ongoing discussions.

## Docs for backend users

NetworkX utilizes a plugin-dispatch architecture. A valid NetworkX backend specifies entry points, named `networkx.backends` and an optional `networkx.backend_info` when it is installed (not imported). This allows NetworkX to dispatch (redirect) function calls to the backend so the execution flows to the designated backend implementation. This design enhances flexibility and integration, making NetworkX more adaptable and efficient.

NetworkX can dispatch to backends explicitly (this requires changing code) or automatically (this requires setting configuration or environment variables). The best way to use a backend depends on the backend, your use case, and whether you want to automatically convert to or from backend graphs. Automatic conversions of graphs is always opt-in.

To explicitly dispatch to a backend, use the `backend=` keyword argument in a dispatchable function. This will convert (and cache by default) input NetworkX graphs to backend graphs and call the backend implementation. Another explicit way to use a backend is to create a backend graph directly–for example, perhaps the backend has its own functions for loading data and creating graphs–and pass that graph to a dispatchable function, which will then call the backend implementation without converting.

Using automatic dispatch requires setting configuration options. Every NetworkX configuration may also be set from an environment variable and are processed at the time networkx is imported. The following configuration variables are supported:

- `nx.config.backend_priority` (`NETWORKX_BACKEND_PRIORITY` env var), a list of backends, controls dispatchable functions that don’t return graphs such as e.g. `nx.pagerank`. When one of these functions is called with NetworkX graphs as input, the dispatcher iterates over the backends listed in this backend_priority config and will use the first backend that implements this function. The input NetworkX graphs are converted (and cached by default) to backend graphs. Using this configuration can allow you to use the full flexibility of NetworkX graphs and the performance of backend implementations, but possible downsides are that creating NetworkX graphs, converting to backend graphs, and caching backend graphs may all be expensive.

- `nx.config.backend_priority.algos` (`NETWORKX_BACKEND_PRIORITY_ALGOS` env var), can be used instead of `nx.config.backend_priority` (`NETWORKX_BACKEND_PRIORITY` env var) to emphasize that the setting only affects the dispatching of algorithm functions as described above.

- `nx.config.backend_priority.generators` (`NETWORKX_BACKEND_PRIORITY_GENERATORS` env var), a list of backends, controls dispatchable functions that return graphs such as nx.from_pandas_edgelist and nx.empty_graph. When one of these functions is called, the first backend listed in this backend_priority config that implements this function will be used and will return a backend graph. When this backend graph is passed to other dispatchable NetworkX functions, it will use the backend implementation if it exists or raise by default unless nx.config.fallback_to_nx is True (default is False). Using this configuration avoids creating NetworkX graphs, which subsequently avoids the need to convert to and cache backend graphs as when using nx.config.backend_priority.algos, but possible downsides are that the backend graph may not behave the same as a NetworkX graph and the backend may not implement all algorithms that you use, which may break your workflow.

- `nx.config.fallback_to_nx` (`NETWORKX_FALLBACK_TO_NX` env var), a boolean (default False), controls what happens when a backend graph is passed to a dispatchable function that is not implemented by that backend. The default behavior when False is to raise. If True, then the backend graph will be converted (and cached by default) to a NetworkX graph and will run with the default NetworkX implementation. Enabling this configuration can allow workflows to complete if the backend does not implement all algorithms used by the workflow, but a possible downside is that it may require converting the input backend graph to a NetworkX graph, which may be expensive. If a backend graph is duck-type compatible as a NetworkX graph, then the backend may choose not to convert to a NetworkX graph and use the incoming graph as-is.

- `nx.config.cache_converted_graphs` (`NETWORKX_CACHE_CONVERTED_GRAPHS` env var), a boolean (default True), controls whether graph conversions are cached to G.__networkx_cache__ or not. Caching can improve performance by avoiding repeated conversions, but it uses more memory.

Note

Backends should follow the NetworkX backend naming convention. For example, if a backend is named `parallel` and specified using `backend=parallel` or `NETWORKX_BACKEND_PRIORITY=parallel`, the package installed is `nx-parallel`, and we would use `import nx_parallel` if we were to import the backend package directly.

Backends are encouraged to document how they recommend to be used and whether their graph types are duck-type compatible as NetworkX graphs. If backend graphs are NetworkX-compatible and you want your workflow to automatically “just work” with a backend–converting and caching if necessary–then use all of the above configurations. Automatically converting graphs is opt-in, and configuration gives the user control.

### Examples:

Use the `cugraph` backend for every algorithm function it supports. This will allow for fall back to the default NetworkX implementations for algorithm calls not supported by cugraph because graph generator functions are still returning NetworkX graphs.

    bash> NETWORKX_BACKEND_PRIORITY=cugraph python my_networkx_script.py

Explicitly use the `parallel` backend for a function call.

    nx.betweenness_centrality(G, k=10, backend="parallel")

Explicitly use the `parallel` backend for a function call by passing an instance of the backend graph type to the function.

    H = nx_parallel.ParallelGraph(G)
    nx.betweenness_centrality(H, k=10)

Explicitly use the `parallel` backend and pass additional backend-specific arguments. Here, `get_chunks` is an argument unique to the `parallel` backend.

    nx.betweenness_centrality(G, k=10, backend="parallel", get_chunks=get_chunks)

Automatically dispatch the `cugraph` backend for all NetworkX algorithms and generators, and allow the backend graph object returned from generators to be passed to NetworkX functions the backend does not support.

    bash> NETWORKX_BACKEND_PRIORITY_ALGOS=cugraph \
          NETWORKX_BACKEND_PRIORITY_GENERATORS=cugraph \
          NETWORKX_FALLBACK_TO_NX=True \
          python my_networkx_script.py

### How does this work?

If you’ve looked at functions in the NetworkX codebase, you might have seen the `@nx._dispatchable` decorator on most of the functions. This decorator allows the NetworkX function to dispatch to the corresponding backend function if available. When the decorated function is called, it first checks for a backend to run the function, and if no appropriate backend is specified or available, it runs the NetworkX version of the function.

#### Backend Keyword Argument

When a decorated function is called with the `backend` kwarg provided, it checks if the specified backend is installed, and loads it. Next it checks whether to convert input graphs by first resolving the backend of each input graph by looking for an attribute named `__networkx_backend__` that holds the backend name for that graph type. If all input graphs backend matches the `backend` kwarg, the backend’s function is called with the original inputs. If any of the input graphs do not match the `backend` kwarg, they are converted to the backend graph type before calling. Exceptions are raised if any step is not possible, e.g. if the backend does not implement this function.

#### Finding a Backend

When a decorated function is called without a `backend` kwarg, it tries to find a dispatchable backend function. The backend type of each input graph parameter is resolved (using the `__networkx_backend__` attribute) and if they all agree, that backend’s function is called if possible. Otherwise the backends listed in the config `backend_priority` are considered one at a time in order. If that backend supports the function and can convert the input graphs to its backend type, that backend function is called. Otherwise the next backend is considered.

During this process, the backends can provide helpful information to the dispatcher via helper methods in the backend’s interface. Backend methods `can_run` and `should_run` are used by the dispatcher to determine whether to use the backend function. If the number of nodes is small, it might be faster to run the NetworkX version of the function. This is how backends can provide info about whether to run.

#### Falling Back to NetworkX

If none of the backends are appropriate, we “fall back” to the NetworkX function. That means we resolve the backends of all input graphs and if all are NetworkX graphs we call the NetworkX function. If any are not NetworkX graphs, we raise an exception unless the `fallback_to_nx` config is set. If it is, we convert all graph types to NetworkX graph types before calling the NetworkX function.

#### Functions that mutate the graph

Any function decorated with the option that indicates it mutates the graph goes through a slightly different path to automatically find backends. These functions typically generate a graph, or add attributes or change the graph structure. The config `backend_priority.generators` holds a list of backend names similar to the config `backend_priority`. The process is similar for finding a matching backend. Once found, the backend function is called and a backend graph is returned (instead of a NetworkX graph). You can then use this backend graph in any function supported by the backend. And you can use it for functions not supported by the backend if you set the config `fallback_to_nx` to allow it to convert the backend graph to a NetworkX graph before calling the function.

#### Optional keyword arguments

Backends can add optional keyword parameters to NetworkX functions to allow you to control aspects of the backend algorithm. Thus the function signatures can be extended beyond the NetworkX function signature. For example, the `parallel` backend might have a parameter to specify how many CPUs to use. These parameters are collected by the dispatchable decorator code at the start of the function call and used when calling the backend function.

#### Existing Backends

NetworkX does not know all the backends that have been created. In fact, the NetworkX library does not need to know that a backend exists for it to work. As long as the backend package creates the `entry_point`, and provides the correct interface, it will be called when the user requests it using one of the three approaches described above. Some backends have been working with the NetworkX developers to ensure smooth operation.

Refer to the Backends section to see a list of available backends known to work with the current stable release of NetworkX.

### Introspection and Logging

Introspection techniques aim to demystify dispatching and backend graph conversion behaviors.

The primary way to see what the dispatch machinery is doing is by enabling logging. This can help you verify that the backend you specified is being used. You can enable NetworkX’s backend logger to print to `sys.stderr` like this:

    import logging
    nxl = logging.getLogger("networkx")
    nxl.addHandler(logging.StreamHandler())
    nxl.setLevel(logging.DEBUG)

And you can disable it by running this:

    nxl.setLevel(logging.CRITICAL)

Refer to :external+python:mod:`logging` to learn more about the logging facilities in Python.

By looking at the `.backends` attribute, you can get the set of all currently installed backends that implement a particular function. For example:

    >>> nx.betweenness_centrality.backends
    {'parallel'}

The function docstring will also show which installed backends support it along with any backend-specific notes and keyword arguments:

    >>> help(nx.betweenness_centrality)
    ...
    Backends
    --------
    parallel : Parallel backend for NetworkX algorithms
      The parallel computation is implemented by dividing the nodes into chunks
      and computing betweenness centrality for each chunk concurrently.
    ...

The NetworkX documentation website also includes info about trusted backends of NetworkX in function references. For example, see `all_pairs_bellman_ford_path_length()`.

Introspection capabilities are currently limited, but we are working to improve them. We plan to make it easier to answer questions such as:

- What happened (and why)?

- What will happen (and why)?

- Where was time spent (including conversions)?

- What is in the cache and how much memory is it using?

Transparency is essential to allow for greater understanding, debug-ability, and customization. After all, NetworkX dispatching is extremely flexible and can support advanced workflows with multiple backends and fine-tuned configuration, but introspection can be helpful by describing when and how to evolve your workflow to meet your needs. If you have suggestions for how to improve introspection, please let us know!

## Docs for backend developers

### Creating a custom backend

1. Defining a `BackendInterface` object:

   Note that the `BackendInterface` doesn’t need to must be a class. It can be an instance of a class, or a module as well. You can define the following methods or functions in your backend’s `BackendInterface` object.:

   1. `convert_from_nx` and `convert_to_nx` methods or functions are required for backend dispatching to work. The arguments to `convert_from_nx` are:

      - `G` : NetworkX Graph

      - `edge_attrs` dict, optional

          Dictionary mapping edge attributes to default values if missing in `G`. If None, then no edge attributes will be converted and default may be 1.

      - `node_attrs`: dict, optional

          Dictionary mapping node attributes to default values if missing in `G`. If None, then no node attributes will be converted.

      - `preserve_edge_attrs` bool

          Whether to preserve all edge attributes.

      - `preserve_node_attrs` bool

          Whether to preserve all node attributes.

      - `preserve_graph_attrs` bool

          Whether to preserve all graph attributes.

      - `preserve_all_attrs` bool

          Whether to preserve all graph, node, and edge attributes.

      - `name` str

          The name of the algorithm.

      - `graph_name` str

          The name of the graph argument being converted.

   2. `can_run` (Optional):

        If your backend only partially implements an algorithm, you can define a `can_run(name, args, kwargs)` function in your `BackendInterface` object that returns True or False indicating whether the backend can run the algorithm with the given arguments or not. Instead of a boolean you can also return a string message to inform the user why that algorithm can’t be run.

   3. `should_run` (Optional):

        A backend may also define `should_run(name, args, kwargs)` that is similar to `can_run`, but answers whether the backend should be run. `should_run` is only run when performing backend graph conversions. Like `can_run`, it receives the original arguments so it can decide whether it should be run by inspecting the arguments. `can_run` runs before `should_run`, so `should_run` may assume `can_run` is True. If not implemented by the backend, `can_run``and ``should_run` are assumed to always return True if the backend implements the algorithm.

   4. `on_start_tests` (Optional):

        A special `on_start_tests(items)` function may be defined by the backend. It will be called with the list of NetworkX tests discovered. Each item is a test object that can be marked as xfail if the backend does not support the test using `item.add_marker(pytest.mark.xfail(reason=...))`.

2. Adding entry points

   To be discoverable by NetworkX, your package must register an entry-point `networkx.backends` in the package’s metadata, with a key pointing to your dispatch object . For example, if you are using `setuptools` to manage your backend package, you can add the following to your pyproject.toml file:

       [project.entry-points."networkx.backends"]
       backend_name = "your_backend_interface_object"

   You can also add the `backend_info` entry-point. It points towards the `get_info` function that returns all the backend information, which is then used to build the “Additional Backend Implementation” box at the end of algorithm’s documentation page. Note that the `get_info` function shouldn’t import your backend package.:

       [project.entry-points."networkx.backend_info"]
       backend_name = "your_get_info_function"

   The `get_info` should return a dictionary with following key-value pairs:
     - `backend_name` str or None

         It is the name passed in the `backend` kwarg.

     - `project` str or None

         The name of your backend project.

     - `package` str or None

         The name of your backend package.

     - `url` str or None

         This is the url to either your backend’s codebase or documentation, and will be displayed as a hyperlink to the `backend_name`, in the “Additional backend implementations” section.

     - `short_summary` str or None

         One line summary of your backend which will be displayed in the “Additional backend implementations” section.

     - `default_config` dict

         A dictionary mapping the backend config parameter names to their default values. This is used to automatically initialize the default configs for all the installed backends at the time of networkx’s import.

         See also

         `Config`

     - `functions` dict or None

         A dictionary mapping function names to a dictionary of information about the function. The information can include the following keys:

         - `url` : str or None The url to `function`’s source code or documentation.

         - `additional_docs` : str or None A short description or note about the backend function’s implementation.

         - `additional_parameters` : dict or None A dictionary mapping additional parameters headers to their short descriptions. For example:

               "additional_parameters": {
                   'param1 : str, function (default = "chunks")' : "...",
                   'param2 : int' : "...",
               }

         If any of these keys are not present, the corresponding information will not be displayed in the “Additional backend implementations” section on NetworkX docs website.

     Note that your backend’s docs would only appear on the official NetworkX docs only if your backend is a trusted backend of NetworkX, and is present in the `circleci/config.yml` and `github/workflows/deploy-docs.yml` files in the NetworkX repository.

3. Defining a Backend Graph class

   The backend must create an object with an attribute `__networkx_backend__` that holds a string with the entry point name:

       class BackendGraph:
           __networkx_backend__ = "backend_name"
           ...

   A backend graph instance may have a `G.__networkx_cache__` dict to enable caching, and care should be taken to clear the cache when appropriate.

### Testing the Custom backend

To test your custom backend, you can run the NetworkX test suite on your backend. This also ensures that the custom backend is compatible with NetworkX’s API. The following steps will help you run the tests:

1. Setting Backend Environment Variables:

     - `NETWORKX_TEST_BACKEND` : Setting this to your backend’s `backend_name` will let NetworkX’s dispatch machinery to automatically convert a regular NetworkX `Graph`, `DiGraph`, `MultiGraph`, etc. to their backend equivalents, using `your_backend_interface_object.convert_from_nx(G, ...)` function.

     - `NETWORKX_FALLBACK_TO_NX` (default=False) : Setting this variable to `True` will instruct tests to use a NetworkX `Graph` for algorithms not implemented by your custom backend. Setting this to `False` will only run the tests for algorithms implemented by your custom backend and tests for other algorithms will `xfail`.

2. Running Tests:

     You can invoke NetworkX tests for your custom backend with the following commands:

         NETWORKX_TEST_BACKEND=<backend_name>
         NETWORKX_FALLBACK_TO_NX=True # or False
         pytest --pyargs networkx

### How tests are run?

1. While dispatching to the backend implementation the `_convert_and_call` function is used and while testing the `_convert_and_call_for_tests` function is used. Other than testing it also checks for functions that return numpy scalars, and for functions that return graphs it runs the backend implementation and the networkx implementation and then converts the backend graph into a NetworkX graph and then compares them, and returns the networkx graph. This can be regarded as (pragmatic) technical debt. We may replace these checks in the future.

2. Conversions while running tests:

     - Convert NetworkX graphs using `<your_backend_interface_object>.convert_from_nx(G, ...)` into the backend graph.

     - Pass the backend graph objects to the backend implementation of the algorithm.

     - Convert the result back to a form expected by NetworkX tests using `<your_backend_interface_object>.convert_to_nx(result, ...)`.

     - For nx_loopback, the graph is copied using the dispatchable metadata

3. Dispatchable algorithms that are not implemented by the backend will cause a `pytest.xfail`, when the `NETWORKX_FALLBACK_TO_NX` environment variable is set to `False`, giving some indication that not all tests are running, while avoiding causing an explicit failure.

`_dispatchable`([func, name, graphs, ...]) | A decorator function that is used to redirect the execution of `func` function to its backend implementation.