astro py:docs

- Index
- Modules

### Navigation

- next »
- « previous |
- Astropy v0.3 »
- Virtual Observatory Access (`astropy.vo`) »
- VO Simple Cone Search »
- Using `astropy.vo.client`

# Using `astropy.vo.client`

This subpackage contains modules supporting VO client-side operations.

## General VO Services Access

`astropy.vo.client.vos_catalog` contains common utilities for accessing simple VO services.

### Configurable Items

These parameters are set via Configuration system (astropy.config):

- `astropy.io.votable.table.PEDANTIC`
- `astropy.utils.data.REMOTE_TIMEOUT`
- `astropy.vo.client.vos_catalog.BASEURL`

### Examples

    >>> from astropy.vo.client import vos_catalog

Get all catalogs from a database named `'conesearch_good'` (this contains cone search services that cleanly passed daily validations; also see Cone Search Examples):

    >>> my_db = vos_catalog.get_remote_catalog_db('conesearch_good')
    Downloading http://stsdas.stsci.edu/astrolib/vo_databases/conesearch_good.json
    |============================================|  56/ 56k (100.00%)        00s
    >>> my_db
    <astropy.vo.client.vos_catalog.VOSDatabase at 0x2b0f3d0>
    >>> print(my_db)
    Guide Star Catalog 2.3 1
    SDSS DR7 - Sloan Digital Sky Survey Data Release 7 1
    SDSS DR7 - Sloan Digital Sky Survey Data Release 7 2
    # ...
    USNO-A2 Catalogue 1
    USNO-A2.0 1

If you get timeout error, you need to use a custom timeout as follows:

    >>> from astropy.utils.data import REMOTE_TIMEOUT
    >>> with REMOTE_TIMEOUT.set_temp(30):
    ...     my_db = vos_catalog.get_remote_catalog_db('conesearch_good')

Find catalog names containing `'usno*a2'`:

    >>> my_db.list_catalogs(pattern='usno*a2')
    [u'The USNO-A2.0 Catalogue (Monet+ 1998) 1', u'USNO-A2 Catalogue 1']

Get information for a catalog titled `'USNO-A2 Catalogue 1'`:

    >>> my_cat = my_db.get_catalog('USNO-A2 Catalogue 1')
    >>> my_cat
    <astropy.vo.client.vos_catalog.VOSCatalog at 0x1f78150>
    >>> print(my_cat)
    title: USNO-A2 Catalogue
    url: http://www.nofs.navy.mil/cgi-bin/vo_cone.cgi?CAT=USNO-A2&
    >>> print(my_cat.dumps())
    {
        "capabilityClass": "ConeSearch",
        "capabilityStandardID": "ivo://ivoa.net/std/ConeSearch",
        "capabilityValidationLevel": "",
        "contentLevel": "#University#Research#Amateur#",
        # ...
        "version": "",
        "waveband": "#Optical#"
    }
    >>> my_cat.keys()
    [u'validate_network_error',
     u'capabilityClass',
     u'updated',
     # ...
     u'identifier',
     u'validate_xmllint']
    >>> my_cat['url']
    u'http://www.nofs.navy.mil/cgi-bin/vo_cone.cgi?CAT=USNO-A2&'
    >>> my_cat['maxRadius']
    1.0

One can also get information for a catalog using its URL. If a URL yields multiple catalogs (this can happen when the service provider re-register the URL with a different title), only the first match is returned:

    >>> my_cat2 = my_db.get_catalog_by_url(
    ...     'http://vizier.u-strasbg.fr/viz-bin/votable/-A?-source=I/255/out&')
    >>> print(my_cat2.dumps())
    {
        "capabilityClass": "ConeSearch",
        "capabilityStandardID": "ivo://ivoa.net/std/ConeSearch",
        "capabilityValidationLevel": "",
        "contentLevel": "#Research#",
        # ...
        "version": "15-Sep-1999",
        "waveband": "#Optical#"
    }

To see validation warnings generated by Validation for Simple Cone Search for the catalog above:

    >>> for w in my_cat2['validate_warnings']:
    ...     print(w)
    /.../vo.xml:13:0: W22: The DEFINITIONS element is deprecated in VOTable 1.1...

To get all the matching catalogs by URL:

    >>> matched_cats = [cat for key, cat in my_db.get_catalogs_by_url(
    ...     'http://vizier.u-strasbg.fr/viz-bin/votable/-A?-source=I/255/out&')]
    >>> for c in matched_cats:
    ...     print(str(c))
    title: The HST Guide Star Catalog, Version GSC-ACT (Lasker+ 1996-99)
    url: http://vizier.u-strasbg.fr/viz-bin/votable/-A?-source=I/255/out&

To get all catalogs in the database:

    >>> all_cats = [cat for key, cat in my_db.get_catalogs()]

By default, pedantic is `False`:

    >>> from astropy.io.votable.table import PEDANTIC
    >>> PEDANTIC()
    False

To call a given VO service; In this case, a Cone Search (also see Cone Search Examples):

    >>> from astropy import coordinates as coord
    >>> from astropy import units as u
    >>> c = coord.ICRS.from_name('47 Tuc')
    >>> c
    <ICRS RA=6.02233 deg, Dec=-72.08144 deg>
    >>> sr = 0.5 * u.degree
    >>> sr
    <Quantity 0.5 deg>
    >>> result = vos_catalog.call_vo_service(
    ...     'conesearch_good',
    ...     kwargs={'RA': c.ra.degree, 'DEC': c.dec.degree, 'SR': sr},
    ...     catalog_db='The PMM USNO-A1.0 Catalogue (Monet 1997) 1')
    Trying http://vizier.u-strasbg.fr/viz-bin/votable/-A?-source=I/243/out&
    Downloading ...
    WARNING: W22: ... The DEFINITIONS element is deprecated in VOTable 1.1...
    WARNING: W03: ... Implictly generating an ID from a name 'RA(ICRS)'...
    WARNING: W03: ... Implictly generating an ID from a name 'DE(ICRS)'...
    >>> result
    <astropy.io.votable.tree.Table at 0x3cc4850>

To repeat the above and suppress all the screen outputs (not recommended):

    >>> import warnings
    >>> with warnings.catch_warnings():
    ...     warnings.simplefilter('ignore')
    ...     result = vos_catalog.call_vo_service(
    ...         'conesearch_good',
    ...         kwargs={'RA': c.ra.degree, 'DEC': c.dec.degree, 'SR': sr},
    ...         catalog_db='The PMM USNO-A1.0 Catalogue (Monet 1997) 1',
    ...         verbose=False)

## Simple Cone Search

`astropy.vo.client.conesearch` supports VO Simple Cone Search capabilities.

Available databases are generated on the server-side hosted by STScI using Validation for Simple Cone Search. The database used is controlled by `astropy.vo.client.conesearch.CONESEARCH_DBNAME`, which can be changed in Configurable Items below. Here are the available options:

1. `'conesearch_good'`

     Default. Passed validation without critical warnings and exceptions.

2. `'conesearch_warn'`

     Has critical warnings but no exceptions. Use at your own risk.

3. `'conesearch_exception'`

     Has some exceptions. Never use this.

4. `'conesearch_error'`

     Has network connection error. Never use this.

In the default setting, it searches the good Cone Search services one by one, stops at the first one that gives non-zero match(es), and returns the result. Since the list of services are extracted from a Python dictionary, the search order might differ from call to call.

There are also functions, both synchronously and asynchronously, available to return all the Cone Search query results. However, this is not recommended unless one knows what one is getting into, as it could potentially take up significant run time and computing resources.

Examples below show how to use non-default search behaviors, where the user has more control of which catalog(s) to search, et cetera.

Note

Most services currently fail to parse when `pedantic=True`.

Warning

When Cone Search returns warnings, user should decide whether the results are reliable by inspecting the warning codes in `astropy.io.votable.exceptions`.

### Configurable Items

These parameters are set via Configuration system (astropy.config):

- `astropy.utils.data.REMOTE_TIMEOUT`
- `astropy.vo.client.conesearch.CONESEARCH_DBNAME`

Also depends on General VO Services Access Configurable Items.

### Examples

    >>> from astropy.vo.client import conesearch

Shows a sorted list of Cone Search services to be searched (to inspect them in detail, see General VO Services Access):

    >>> conesearch.list_catalogs()
    [u'Guide Star Catalog 2.3 1',
     u'SDSS DR7 - Sloan Digital Sky Survey Data Release 7 1',
     u'SDSS DR7 - Sloan Digital Sky Survey Data Release 7 2',
     u'SDSS DR7 - Sloan Digital Sky Survey Data Release 7 3',
     u'SDSS DR7 - Sloan Digital Sky Survey Data Release 7 4',
     u'SDSS DR8 - Sloan Digital Sky Survey Data Release 8 1',
     u'SDSS DR8 - Sloan Digital Sky Survey Data Release 8 2',
     u'The HST Guide Star Catalog, Version 1.1 (Lasker+ 1992) 1',
     u'The HST Guide Star Catalog, Version 1.2 (Lasker+ 1996) 1',
     u'The HST Guide Star Catalog, Version GSC-ACT (Lasker+ 1996-99) 1',
     u'The PMM USNO-A1.0 Catalogue (Monet 1997) 1',
     u'The USNO-A2.0 Catalogue (Monet+ 1998) 1',
     u'Two Micron All Sky Survey (2MASS) 1',
     u'Two Micron All Sky Survey (2MASS) 2',
     u'USNO-A2 Catalogue 1',
     u'USNO-A2.0 1']

Select a catalog to search:

    >>> my_catname = 'The PMM USNO-A1.0 Catalogue (Monet 1997) 1'

By default, pedantic is `False`:

    >>> from astropy.io.votable.table import PEDANTIC
    >>> PEDANTIC()
    False

Perform Cone Search in the selected catalog above for 0.5 degree radius around 47 Tucanae with minimum verbosity, if supported. The `catalog_db` keyword gives control over which catalog(s) to use. If running this for the first time, a copy of the catalogs database will be downloaded to local cache. To run this again without using cached data, set `cache=False`:

    >>> from astropy import coordinates as coord
    >>> from astropy import units as u
    >>> c = coord.ICRS.from_name('47 Tuc')
    >>> c
    <ICRS RA=6.02233 deg, Dec=-72.08144 deg>
    >>> sr = 0.5 * u.degree
    >>> sr
    <Quantity 0.5 deg>
    >>> result = conesearch.conesearch(c, sr, catalog_db=my_catname)
    Trying http://vizier.u-strasbg.fr/viz-bin/votable/-A?-source=I/243/out&
    Downloading ...
    WARNING: W22: ... The DEFINITIONS element is deprecated in VOTable 1.1...

To run the command above using custom timeout of 30 seconds for each Cone Search service query:

    >>> from astropy.utils.data import REMOTE_TIMEOUT
    >>> with REMOTE_TIMEOUT.set_temp(30):
    ...     result = conesearch.conesearch(c, sr, catalog_db=my_catname)

To suppress all the screen outputs (not recommended):

    >>> import warnings
    >>> with warnings.catch_warnings():
    ...     warnings.simplefilter('ignore')
    ...     result = conesearch.conesearch(c, sr, catalog_db=my_catname,
    ...                                    verbose=False)

Extract Numpy array containing the matched objects. See `numpy` for available operations:

    >>> cone_arr = result.array.data
    >>> cone_arr
    array([(0.499298, 4.403473, -72.124045, '0150-00088188'),
           (0.499075, 4.403906, -72.122762, '0150-00088198'),
           (0.499528, 4.404531, -72.045198, '0150-00088210'), ...,
           (0.4988, 7.641731, -72.113156, '0150-00225965'),
           (0.499554, 7.645489, -72.103167, '0150-00226134'),
           (0.499917, 7.6474, -72.0876, '0150-00226223')],
          dtype=[('_r', '<f8'), ('_RAJ2000', '<f8'), ('_DEJ2000', '<f8'),
                 ('USNO-A1.0', '|S13')])
    >>> cone_arr.dtype.names
    ('_r', '_RAJ2000', '_DEJ2000', 'USNO-A1.0')
    >>> cone_arr.size
    36184
    >>> ra_list = cone_arr['_RAJ2000']
    >>> ra_list
    array([ 4.403473,  4.403906,  4.404531, ...,  7.641731,  7.645489,  7.6474  ])
    >>> cone_arr[0]  # First row
    (0.499298, 4.403473, -72.124045, '0150-00088188')
    >>> cone_arr[-1]  # Last row
    (0.499917, 7.6474, -72.0876, '0150-00226223')
    >>> cone_arr[:10]  # First 10 rows
    array([(0.499298, 4.403473, -72.124045, '0150-00088188'),
           (0.499075, 4.403906, -72.122762, '0150-00088198'),
           (0.499528, 4.404531, -72.045198, '0150-00088210'),
           (0.497252, 4.406078, -72.095045, '0150-00088245'),
           (0.499739, 4.406462, -72.139545, '0150-00088254'),
           (0.496312, 4.410623, -72.110492, '0150-00088372'),
           (0.49473, 4.415053, -72.071217, '0150-00088494'),
           (0.494171, 4.415939, -72.087512, '0150-00088517'),
           (0.493722, 4.417678, -72.0972, '0150-00088572'),
           (0.495147, 4.418262, -72.047142, '0150-00088595')],
          dtype=[('_r', '<f8'), ('_RAJ2000', '<f8'), ('_DEJ2000', '<f8'),
                 ('USNO-A1.0', '|S13')])

Sort the matched objects by angular separation in ascending order:

    >>> import numpy as np
    >>> sep = cone_arr['_r']
    >>> i_sorted = np.argsort(sep)
    >>> cone_arr[i_sorted]
    array([(0.081971, 5.917787, -72.006075, '0150-00145335'),
           (0.083181, 6.020339, -72.164623, '0150-00149799'),
           (0.089166, 5.732798, -72.077698, '0150-00137181'), ...,
           (0.499981, 7.024962, -72.477503, '0150-00198745'),
           (0.499987, 6.423773, -71.597364, '0150-00168596'),
           (0.499989, 6.899589, -72.5043, '0150-00192872')],
          dtype=[('_r', '<f8'), ('_RAJ2000', '<f8'), ('_DEJ2000', '<f8'),
                 ('USNO-A1.0', '|S13')])

Result can also be manipulated as VOTable XML handling (astropy.io.votable) and its unit can be manipulated as Units and Quantities (astropy.units). In this example, we convert RA values from degree to arcsec:

    >>> from astropy import units as u
    >>> ra_field = result.get_field_by_id('_RAJ2000')
    >>> ra_field.title
    u'Right ascension (FK5, Equinox=J2000.0) (computed by VizieR, ...)'
    >>> ra_field.unit
    Unit("deg")
    >>> ra_field.unit.to(u.arcsec) * ra_list
    array([ 15852.5028,  15854.0616,  15856.3116, ...,  27510.2316,
            27523.7604,  27530.64  ])

Perform the same Cone Search as above but asynchronously using `AsyncConeSearch`. Queries to individual Cone Search services are still governed by `astropy.utils.data.REMOTE_TIMEOUT`. Cone Search is forced to run in silent mode asynchronously, but warnings are still controlled by `warnings`:

    >>> async_search = conesearch.AsyncConeSearch(c, sr, catalog_db=my_catname)

Check asynchronous search status:

    >>> async_search.running()
    True
    >>> async_search.done()
    False

Get search results after a 30-second wait (not to be confused with `astropy.utils.data.REMOTE_TIMEOUT` that governs individual Cone Search queries). If search is still not done after 30 seconds, `TimeoutError` is raised. Otherwise, Cone Search result is returned and can be manipulated as above. If no `timeout` keyword given, it waits until completion:

    >>> async_result = async_search.get(timeout=30)
    >>> cone_arr = async_result.array.data
    >>> cone_arr.size
    36184

Estimate the execution time and the number of objects for the Cone Search service URL from above. The prediction naively assumes a linear model, which might not be accurate for some cases. It also uses the normal `conesearch()`, not the asynchronous version. This example uses a custom timeout of 30 seconds and runs silently (except for warnings):

    >>> result.url
    u'http://vizier.u-strasbg.fr/viz-bin/votable/-A?-source=I/243/out&'
    >>> with REMOTE_TIMEOUT.set_temp(30):
    ...     t_est, n_est = conesearch.predict_search(
    ...         result.url, c, sr, verbose=False, plot=True)
    WARNING: W22: ... The DEFINITIONS element is deprecated in VOTable 1.1...
    # ...
    >>> t_est  # Predicted execution time
    19.272144977377838
    >>> n_est  # Predicted number of objects
    37340

For debugging purpose, one can obtain the actual execution time and number of objects, and compare them with the predicted values above. The INFO message shown in controlled by `astropy.logger`. Keep in mind that running this for every prediction would defeat the purpose of the prediction itself:

    >>> t_real, tab = conesearch.conesearch_timer(
    ...     c, sr, catalog_db=result.url, verbose=False)
    INFO: conesearch_timer took 11.5103080273 s on AVERAGE for 1 call(s). [...]
    >>> t_real  # Actual execution time
    11.510308027267456
    >>> tab.array.size  # Actual number of objects
    36184

One can also search in a list of catalogs instead of a single one. In this example, we look for all catalogs containing `'guide*star'` in their titles and only perform Cone Search using those services. The first catalog in the list to successfully return non-zero result is used. Therefore, the order of catalog names given in `catalog_db` is important:

    >>> gsc_cats = conesearch.list_catalogs(pattern='guide*star')
    >>> gsc_cats
    [u'Guide Star Catalog 2.3 1',
     u'The HST Guide Star Catalog, Version 1.1 (Lasker+ 1992) 1',
     u'The HST Guide Star Catalog, Version 1.2 (Lasker+ 1996) 1',
     u'The HST Guide Star Catalog, Version GSC-ACT (Lasker+ 1996-99) 1']
    >>> gsc_result = conesearch.conesearch(c, sr, catalog_db=gsc_cats)
    Trying http://gsss.stsci.edu/webservices/vo/ConeSearch.aspx?CAT=GSC23&
    WARNING: W25: ... failed with: timed out [...]
    Trying http://vizier.u-strasbg.fr/viz-bin/votable/-A?-source=I/220/out&
    Downloading ...
    WARNING: W22: ... The DEFINITIONS element is deprecated in VOTable 1.1...
    >>> gsc_result.array.size
    2997
    >>> gsc_result.url
    u'http://vizier.u-strasbg.fr/viz-bin/votable/-A?-source=I/220/out&'

To repeat the Cone Search above with the services listed in a different order:

    >>> gsc_cats_reordered = [gsc_cats[i] for i in (3, 1, 2, 0)]
    >>> gsc_cats_reordered
    [u'The HST Guide Star Catalog, Version GSC-ACT (Lasker+ 1996-99) 1',
     u'The HST Guide Star Catalog, Version 1.1 (Lasker+ 1992) 1',
     u'The HST Guide Star Catalog, Version 1.2 (Lasker+ 1996) 1',
     u'Guide Star Catalog 2.3 1']
    >>> gsc_result = conesearch.conesearch(c, sr, catalog_db=gsc_cats_reordered)
    Trying http://vizier.u-strasbg.fr/viz-bin/votable/-A?-source=I/255/out&
    Downloading ...
    WARNING: W22: ... The DEFINITIONS element is deprecated in VOTable 1.1...
    >>> gsc_result.array.size
    2997
    >>> gsc_result.url
    u'http://vizier.u-strasbg.fr/viz-bin/votable/-A?-source=I/255/out&'

To obtain results from all the services above:

    >>> all_gsc_results = conesearch.search_all(c, sr, catalog_db=gsc_cats)
    Trying http://gsss.stsci.edu/webservices/vo/ConeSearch.aspx?CAT=GSC23&
    WARNING: W25: ... failed with: <urlopen error timed out> [...]
    Trying http://vizier.u-strasbg.fr/viz-bin/votable/-A?-source=I/220/out&
    Downloading ...
    WARNING: W22: ... The DEFINITIONS element is deprecated in VOTable 1.1...
    Trying http://vizier.u-strasbg.fr/viz-bin/votable/-A?-source=I/254/out&
    Downloading ...
    WARNING: W22: ... The DEFINITIONS element is deprecated in VOTable 1.1...
    Trying http://vizier.u-strasbg.fr/viz-bin/votable/-A?-source=I/255/out&
    Downloading ...
    WARNING: W22: ... The DEFINITIONS element is deprecated in VOTable 1.1...
    >>> all_gsc_results
    {u'http://.../220/out&': <astropy.io.votable.tree.Table at 0x2988e10>,
     u'http://.../254/out&': <astropy.io.votable.tree.Table at 0x2993350>,
     u'http://.../255/out&': <astropy.io.votable.tree.Table at 0x298de90>}
    >>> for url, tab in all_gsc_results.items():
    ...     print('{0} has {1} results'.format(url[-17:], tab.array.size))
    source=I/254/out& has 2998 results
    source=I/255/out& has 2997 results
    source=I/220/out& has 2997 results

To repeat the above asynchronously:

    >>> async_search_all = conesearch.AsyncSearchAll(c, sr, catalog_db=gsc_cats)
    >>> async_search_all.running()
    True
    >>> async_search_all.done()
    False
    >>> all_gsc_results = async_search_all.get()

If one is unable to obtain any results using the default Cone Search database, `'conesearch_good'`, that only contains sites that cleanly passed validation, one can use Configuration system (astropy.config) to use another database, `'conesearch_warn'`, containing sites with validation warnings. One should use these sites with caution:

    >>> conesearch.CONESEARCH_DBNAME.set('conesearch_warn')
    >>> conesearch.list_catalogs()
    Downloading http://stsdas.stsci.edu/astrolib/vo_databases/conesearch_warn.json
    |===========================================|  87k/ 87k (100.00%)        00s
    [u'2MASS All-Sky Catalog of Point Sources (Cutri+ 2003) 1',
     u'2MASS All-Sky Point Source Catalog 1',
     u'Data release 7 of Sloan Digital Sky Survey catalogs 1',
     u'Data release 7 of Sloan Digital Sky Survey catalogs 2',
     u'Data release 7 of Sloan Digital Sky Survey catalogs 3',
     u'Data release 7 of Sloan Digital Sky Survey catalogs 4',
     u'Data release 7 of Sloan Digital Sky Survey catalogs 5',
     u'Data release 7 of Sloan Digital Sky Survey catalogs 6',
     u'The 2MASS All-Sky Catalog 1',
     u'The 2MASS All-Sky Catalog 2',
     u'The USNO-B1.0 Catalog (Monet+ 2003) 1',
     u'The USNO-B1.0 Catalog 1',
     u'USNO-A V2.0, A Catalog of Astrometric Standards 1',
     u'USNO-B1 Catalogue 1']