Skip to main content
`Ctrl`+`K`

# Text Tables (`astropy.io.ascii`)

`astropy.io.ascii` provides methods for reading and writing a wide range of text data table formats via built-in Extension Reader Classes. The emphasis is on flexibility and convenience of use, although readers can optionally use a less flexible C-based engine for reading and writing for improved performance.

Note

It is strongly encouraged to use the Unified I/O Text Tables interface rather than using `astropy.io.ascii` directly.

For reading large CSV files, the astropy PyArrow CSV reader is the fastest option, while for writing large data tables to CSV, the Table - Pandas interface is an option to consider.

Additional information is available in the Unified I/O and Unified I/O Table Data pages.

The following shows a few of the text formats that are available, while the section on Supported formats contains the full list.

- `Basic`: basic table with customizable delimiters and header configurations

- `Cds`: CDS format table (also Vizier)

- `Daophot`: table from the IRAF DAOphot package

- `Ecsv`: ECSV Format for lossless round-trip of data tables (recommended)

- `FixedWidth`: table with fixed-width columns (see also Fixed-Width Gallery)

- `Ipac`: IPAC format table

- `HTML`: HTML format table contained in a <table> tag

- `Latex`: LaTeX table with datavalue in the `tabular` environment

- `Mesa`: MESA stellar evolution code output format

- `Mrt`: AAS Machine-Readable Tables (MRT))

- `SExtractor`: SExtractor format table

The strength of `astropy.io.ascii` is the support for astronomy-specific formats (often with metadata) and specialized data types such as SkyCoord, Time, and Quantity.

## Getting Started

### Reading Tables

The majority of commonly encountered text tables can be read with the `read()` function. Assume you have a file named `sources.dat` with the following contents:

    obsid redshift  X      Y     object
    3102  0.32      4167  4085   Q1250+568-A
    877   0.22      4378  3892   "Source 82"

This table can be read with the following (assuming that the path to the data directory is set like this: `datadir=Path('path/to/my/data')`):

    >>> from astropy.io import ascii
    >>> data = ascii.read(datadir / "sources.dat")
    >>> print(data)
    obsid redshift  X    Y      object
    ----- -------- ---- ---- -----------
     3102     0.32 4167 4085 Q1250+568-A
      877     0.22 4378 3892   Source 82

The first argument to the `read()` function can be the name of a file, a string representation of a table, or a list of table lines. The return value (`data` in this case) is a Table object.

By default, `read()` will try to guess the table format by trying most of the supported formats.

Warning

Guessing the file format might be convenient, but has two disadvantages:

- It is often slow for large files because the reader tries parsing the file with every allowed format until one succeeds.

- Tables sometimes match multiple formats and the first one that succeeds might not be the one you expected (example).

Thus, it is recommended to disable guessing with `guess=False` and explicitly give the table format (e.g. `format='csv'`) whenever possible.

If guessing the format does not work, as in the case for unusually formatted tables, you may need to give `astropy.io.ascii` additional hints about the format.

To specify specific data types for one or more columns, use the `converters` argument (see Converters for Specifying Dtype for details). For instance if the `obsid` is actually a string identifier (instead of an integer) you can read the table with the code below. This also illustrates using the preferred Table interface for reading:

    >>> from astropy.table import Table
    >>> sources = """
    ... target observatory obsid
    ... TW_Hya Chandra     22178
    ... MP_Mus XMM         0406030101"""
    >>> data = Table.read(sources, format='ascii', converters={'obsid': str})
    >>> data
    <Table length=2>
    target observatory   obsid
     str6      str7      str10
    ------ ----------- ----------
    TW_Hya     Chandra      22178
    MP_Mus         XMM 0406030101

### Writing Tables

The `write()` function provides a way to write a data table as a formatted text table. Most of the input table Supported Formats for reading are also available for writing. This provides a great deal of flexibility in the format for writing.

The following shows how to write a formatted text table using the `write()` function:

    >>> import numpy as np
    >>> from astropy.io import ascii
    >>> from astropy.table import Table
    >>> data = Table()
    >>> data['x'] = np.array([1, 2, 3], dtype=np.int32)
    >>> data['y'] = data['x'] ** 2
    >>> ascii.write(data, 'values.dat', overwrite=True)

The `values.dat` file will then contain:

    x y
    1 1
    2 4
    3 9

It is also possible and encouraged to use the write functionality from `astropy.io.ascii` through a higher level interface in the Data Tables package (see High-level Unified File I/O for more details). For example:

    >>> data.write('values.dat', format='ascii', overwrite=True)

Attention

ECSV is recommended

For a reproducible text version of your table, we recommend using the ECSV Format. This stores all the table meta-data (in particular the column types and units) to a comment section at the beginning while maintaining compatibility with most plain CSV readers. It also allows storing richer data like `SkyCoord` or multidimensional or variable-length columns. ECSV is also supported in Java by |STIL| and |TOPCAT| (see ECSV Format).

To write our simple example table to ECSV we use:

    >>> data.write('values.ecsv', overwrite=True)

The `.ecsv` extension is recognized and implies using ECSV (equivalent to `format='ascii.ecsv'`). The `values.ecsv` file will then contain:

    # %ECSV 1.0
    # ---
    # datatype:
    # - {name: x, datatype: int32}
    # - {name: y, datatype: int32}
    # schema: astropy-2.0
    x y
    1 1
    2 4
    3 9

## Supported Formats

A full list of the supported `format` values and corresponding format types for text tables is given below. The `Write` column indicates which formats support write functionality, and the `Fast` column indicates which formats are compatible with the fast Cython/C engine for reading and writing.

Format | Write | Fast | Description
--- | --- | --- | ---
`aastex` | Yes |  | `AASTex`: AASTeX deluxetable used for AAS journals
`basic` | Yes | Yes | `Basic`: Basic table with custom delimiters
`cds` | Yes |  | `Cds`: CDS format table
`commented_header` | Yes | Yes | `CommentedHeader`: Column names in a commented line
`csv` | Yes | Yes | `Csv`: Basic table with comma-separated values
`daophot` |  |  | `Daophot`: IRAF DAOphot format table
`ecsv` | Yes |  | `Ecsv`: Enhanced CSV format (recommended)
`fixed_width` | Yes |  | `FixedWidth`: Fixed width
`fixed_width_no_header` | Yes |  | `FixedWidthNoHeader`: Fixed-width with no header
`fixed_width_two_line` | Yes |  | `FixedWidthTwoLine`: Fixed-width with second header line
`html` | Yes |  | `HTML`: HTML format table
`ipac` | Yes |  | `Ipac`: IPAC format table
`latex` | Yes |  | `Latex`: LaTeX table
`mesa` | No |  | `Mesa`: MESA stellar evolution code format
`mrt` | Yes |  | `Mrt`: AAS Machine-Readable Table format
`no_header` | Yes | Yes | `NoHeader`: Basic table with no headers
`qdp` | Yes |  | `QDP`: Quick and Dandy Plotter files
`rdb` | Yes | Yes | `Rdb`: Tab-separated with a type definition header line
`rst` | Yes |  | `RST`: reStructuredText simple format table
`sextractor` |  |  | `SExtractor`: SExtractor format table
`tab` | Yes | Yes | `Tab`: Basic table with tab-separated values
`tdat` | Yes |  | `Tdat`: Transportable Database Aggregate Table format

## Getting Help

Some formats have additional options that can be set to control the behavior of the reader or writer. For more information on these options, you can either see the documentation for the specific format class (e.g. `HTML`) or use the `help` function of the `read` or `write` functions. For example:

    >>> ascii.read.help()  # Common help for all formats
    >>> ascii.read.help("html")  # Common help plus "html" format-specific args
    >>> ascii.write.help("latex")  # Common help plus "latex" format-specific args

## Using `astropy.io.ascii`

The details of using `astropy.io.ascii` are provided in the following sections:

### Reading tables

- Reading Tables
  - Parameters for `read()`
  - Specifying Header and Data Location
  - Bad or Missing Values
  - Guess Table Format
  - Comments and Metadata
  - Converters for Specifying Dtype
  - Fortran-Style Exponents
  - Advanced Customization
  - Reading Large Tables in Chunks
  - How to Find and Fix Problems Reading a Table
  - Reading Gaia Data Tables

### Writing tables

- Writing Tables
  - Examples
  - Parameters for `write()`
  - Machine-Readable Table Format

### ECSV Format

- ECSV Format
  - Usage
  - Simple Table
  - Masked Data
  - Mixin Columns
  - Multidimensional Columns
  - Variable-length arrays
  - Object arrays
  - Meta data

### Fixed-Width Gallery

- Fixed-Width Gallery
  - Reading
  - Writing
  - Custom Header Rows

### Fast ASCII Engine

- Fast ASCII I/O
  - Examples
  - Fast Conversion Options
  - Fast Converter
  - Reading Large Tables

### Base Class Elements

- Base Class Elements

### Extension Reader Classes

- Extension Reader Classes

## Performance Tips

By default, when trying to read a file the reader will guess the format, which involves trying to read it with many different readers. For better performance when dealing with large tables, it is recommended to specify the format and any options explicitly, and turn off guessing as well.

### Example

If you are reading a simple CSV file with a one-line header with column names, the following:

    read('example.csv', format='basic', delimiter=',', guess=False)  # doctest: +SKIP

can be at least an order of magnitude faster than:

    read('example.csv')  # doctest: +SKIP

## Reference/API

- Reference/API
  - astropy.io.ascii Package