Programmatic Access - Gaia Users
Help supportShould you have any question, please check the Gaia FAQ section or contact the Gaia Helpdesk |
Python Access
ESA Gaia Archive provides a Python Astroquery (Astropy) package for easy access to the ESA Gaia Archive: astroquery.gaia.
Also, a generic package is provided for accessing any TAP and TAP+ compliant service: astroquery.utils.tap.
See a Python tutorial for Jupyter Notebook example here.
Command line access
The entry point is a TAP (Table Access Protocol) server.
You may use HTTP protocol to execute TAP requests at https://gea.esac.esa.int/tap-server/tap. TAP provides two operation modes: Synchronous and Asynchronous:
- Synchronous: the response to the request will be generated as soon as the request received by the server.
- Asynchronous: the server will start a job that will execute the request. The first response to the request is the required information (a link) to obtain the job status. Once the job is finished, the results can be retrieved.
Our TAP server provides two access mode: public and authenticated:
- Public: this is the standard TAP access. A user can execute ADQL queries and upload tables to be used in a query 'on-the-fly' (these tables will be removed once the query is executed). The results are available to any other user and they will remain in the server for a limited space of time.
- Authenticated: some functionalities are restricted to authenticated users only The results are saved in a private user space and they will remain in the server for ever (they can be removed by the user).
- ADQL queries and results are saved in a user private area.
- Cross-match operations: a catalogue cross-match operation can be executed. Cross-match operations results are saved in a user private area.
- Persintance of uploaded tables: a user can upload a table in a private space. These tables can be used in queries as well as in cross-matches operations.
Here you can find some examples about how to interact with a TAP server (we are using the curl tool):
1. Non authenticated access
1.1. Getting all public tables
curl "https://gea.esac.esa.int/tap-server/tap/tables"
1.2. Synchronous query
curl "https://gea.esac.esa.int/tap-server/tap/sync?REQUEST=doQuery&LANG=ADQL&FORMAT=votable&QUERY=SELECT+TOP+5+source_id,ra,dec+FROM+gaiadr1.gaia_source"
The retrieved results is a VO table by default (see '3.2. Synchronous Queries' section parameters to specify a different output format). The results can be saved in a file and inspected using any analysis tool like TOPCAT, for instance.
1.3. Synchronous query on an 'on-the-fly' uploaded table
curl --form UPLOAD="table_c,param:table1" --form table1=@test_ra_dec.vot --form LANG=ADQL --form REQUEST=doQuery --form QUERY="select top 5 * from tap_upload.table_c" https://gea.esac.esa.int/tap-server/tap/sync
Where 'test_ra_dec.vot'
is a file that contains the VOTable to be uploaded (in order to be used by the query)
The retrieved results is a VO table by default (see '3.2. Synchronous Queries' section parameters to specify a different output format). The results can be saved in a file and inspected using any analysis tool like TOPCAT, for instance.
1.4. Asynchronous query
curl -i -X POST --data "PHASE=run&LANG=ADQL&LANG=ADQL&REQUEST=doQuery&QUERY=select+top+5+*+from+gaiadr1.gaia_source" "https://gea.esac.esa.int/tap-server/tap/async"
Note that there is the possibility to use the optional parameters "JOBNAME" to assign a name to the job and "JOBDESCRIPTION" to add a description:
curl -i -X POST --data "PHASE=run&LANG=ADQL&JOBNAME=optionalJobName&JOBDESCRIPTION=optionalDescription&LANG=ADQL&REQUEST=doQuery&QUERY=select+top+5+*+from+gaiadr1.gaia_source" "https://gea.esac.esa.int/tap-server/tap/async"
The response will contain the URL of the job running at server side (see Location header):
HTTP/1.1 303 See Other Date: Mon, 30 Jun 2014 14:44:39 GMT Server: Apache-Coyote/1.1 Location: https://gea.esac.esa.int/tap-server/tap/async/1404139480755A Content-Type: application/x-www-form-urlencoded Connection: close Transfer-Encoding: chunked
To obtain the status of the running job:
curl "https://gea.esac.esa.int/tap-server/tap/async/1404139480755A"
The status response is something like:
<?xml version="1.0" encoding="UTF-8"?> <uws:job xmlns:uws="http://www.ivoa.net/xml/UWS/v1.0" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <uws:jobId><![CDATA[1404139480755A]]></uws:jobId> <uws:runId xsi:nil="true" /> <uws:ownerId><![CDATA[anonymous]]></uws:ownerId> <uws:phase>COMPLETED</uws:phase> <uws:quote xsi:nil="true" /> <uws:startTime>2014-06-30T16:44:40.766+0200</uws:startTime> <uws:endTime>2014-06-30T16:44:40.830+0200</uws:endTime> <uws:executionDuration>0</uws:executionDuration> <uws:destruction>2014-07-07T16:44:40.754+0200</uws:destruction> <uws:parameters> <uws:parameter id="maxRec"><![CDATA[100000]]></uws:parameter> <uws:parameter id="query"><![CDATA[select top 5 * from gaiadr1.gaia_source]]></uws:parameter> <uws:parameter id="request"><![CDATA[doQuery]]></uws:parameter> <uws:parameter id="format"><![CDATA[votable]]></uws:parameter> <uws:parameter id="keepAuthenticatedUserJobs"><![CDATA[true]]></uws:parameter> <uws:parameter id="lang"><![CDATA[ADQL]]></uws:parameter> <uws:parameter id="version"><![CDATA[1.0]]></uws:parameter> </uws:parameters> <uws:results> <uws:result id="result" xlink:type="simple" xlink:href="http%3A%2F%2Fgea.esac.esa.int%2Ftap-server%2Ftap%2Fasync%2F1404139480755A%2Fresults%2Fresult" mime="application/x-votable+xml" size="8863" rows="6" /> </uws:results> <uws:errorSummary xsi:nil="true" />
To obtain the results of the job (once the job is finished):
curl "https://gea.esac.esa.int/tap-server/tap/async/1404139480755A/results/result"
The retrieved results is a VO table by default (see '3.3 Asynchronous Queries' section parameters to specify a different output format).
The results can be saved in a file and inspected using any analysis tool like TOPCAT, for instance.
1.5. Python script to execute an asynchronous query and wait until the results are available
#ASYNCHRONOUS REQUEST #Python 2 #import httplib #import urllib #Python 3 import http.client as httplib import urllib.parse as urllib import time from xml.dom.minidom import parseString host = "gea.esac.esa.int" port = 443 pathinfo = "/tap-server/tap/async" #------------------------------------- #Create job params = urllib.urlencode({\ "REQUEST": "doQuery", \ "LANG": "ADQL", \ "FORMAT": "votable_plain", \ "PHASE": "RUN", \ "JOBNAME": "Any name (optional)", \ "JOBDESCRIPTION": "Any description (optional)", \ "QUERY": "SELECT DISTANCE(POINT('ICRS',ra,dec), POINT('ICRS',266.41683,-29.00781)) AS dist, * FROM gaiadr1.gaia_source WHERE 1=CONTAINS(POINT('ICRS',ra,dec),CIRCLE('ICRS',266.41683,-29.00781, 0.08333333)) ORDER BY dist ASC" }) headers = {\ "Content-type": "application/x-www-form-urlencoded", \ "Accept": "text/plain" \ } connection = httplib.HTTPSConnection(host, port) connection.request("POST",pathinfo,params,headers) #Status response = connection.getresponse() print ("Status: " +str(response.status), "Reason: " + str(response.reason)) #Server job location (URL) location = response.getheader("location") print ("Location: " + location) #Jobid jobid = location[location.rfind('/')+1:] print ("Job id: " + jobid) connection.close() #------------------------------------- #Check job status, wait until finished while True: connection = httplib.HTTPSConnection(host, port) connection.request("GET",pathinfo+"/"+jobid) response = connection.getresponse() data = response.read() #XML response: parse it to obtain the current status #(you may use pathinfo/jobid/phase entry point to avoid XML parsing) dom = parseString(data) phaseElement = dom.getElementsByTagName('uws:phase')[0] phaseValueElement = phaseElement.firstChild phase = phaseValueElement.toxml() print ("Status: " + phase) #Check finished if phase == 'COMPLETED': break #wait and repeat time.sleep(0.2) connection.close() #------------------------------------- #Get results connection = httplib.HTTPSConnection(host, port) connection.request("GET",pathinfo+"/"+jobid+"/results/result") response = connection.getresponse() data = response.read().decode('iso-8859-1') #print(type(data)) #print(data) outputFileName = "example_votable_output.vot" outputFile = open(outputFileName, "w") outputFile.write(data) outputFile.close() connection.close() print ("Data saved in: " + outputFileName)
The saved file is a VO table (by default, see '3.3 Asynchronous Queries' section parameters to specify a different output format). The file can be inspected using any analysis tool like TOPCAT, for instance.
2. Authenticated access
2.1. Login
curl -k -c cookies.txt -X POST -d username=USERNAME -d password=PASSWORD -L "https://gea.esac.esa.int/tap-server/login"
2.2. Logout
curl -k -b cookies.txt -X POST -d -L "https://gea.esac.esa.int/tap-server/logout"
2.3. Getting public and user tables
curl -k -b cookies.txt -X POST -L "https://gea.esac.esa.int/tap-server/tap/tables"
2.4. Upload user table
curl -k -b cookies.txt -X POST -F FILE=@file.name -F TABLE_NAME=table_name "https://gea.esac.esa.int/tap-server/Upload"
Where 'file.name'
is a file that contains the VOTable to be uploaded.
TABLE_DESC
parameter can be used to specify a table description.
FORMAT
parameter can be used to specify the table data format. Available formats: 'VOTable'
, 'CSV'
and 'ASCII'
: see section 3.7 for more details.
2.5. Delete user table
curl -k -b cookies.txt -X POST -F TABLE_NAME=table_name -F DELETE=TRUE "https://gea.esac.esa.int/tap-server/Upload"
Where 'table.name'
is the argument used when the table was uploaded (argument TABLE_NAME=table_name).
To force a table removal, use FORCE_REMOVAL=TRUE flag:
curl -k -b cookies.txt -X POST -F TABLE_NAME=table_name -F DELETE=TRUE -F FORCE_REMOVAL=TRUE "https://gea.esac.esa.int/tap-server/Upload"
2.6. Asynchronous query
curl -k -b cookies.txt -i -X POST --data "PHASE=run&LANG=ADQL&REQUEST=doQuery&QUERY=select+top+5+*+from+gaiadr1.gaia_source" "https://gea.esac.esa.int/tap-server/tap/async"

curl -k -b cookies.txt -i -X POST --data "PHASE=run&LANG=ADQL&JOBNAME=optionalJobName&JOBDESCRIPTION=optionalDescription&REQUEST=doQuery&QUERY=select+top+5+*+from+gaiadr1.gaia_source" "https://gea.esac.esa.int/tap-server/tap/async"
The response will contain the URL of the job running at server side:
HTTP/1.1 303 See Other Date: Mon, 30 Jun 2014 15:02:00 GMT Server: Apache-Coyote/1.1 Location: https://gea.esac.esa.int/tap-server/tap/async/1404140520859A Content-Type: application/x-www-form-urlencoded Connection: close Transfer-Encoding: chunked
To obtain the status of the running job:
curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/async/1404140520859A"
The status response is something like:
<?xml version="1.0" encoding="UTF-8"?> <uws:job xmlns:uws="http://www.ivoa.net/xml/UWS/v1.0" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <uws:jobId><![CDATA[1404141177261A]]></uws:jobId> <uws:runId xsi:nil="true" /> <uws:ownerId><![CDATA[USERNAME]]></uws:ownerId> <uws:phase>COMPLETED</uws:phase> <uws:quote xsi:nil="true" /> <uws:startTime>2014-06-30T17:12:57.273+0200</uws:startTime> <uws:endTime>2014-06-30T17:12:57.864+0200</uws:endTime> <uws:executionDuration>0</uws:executionDuration> <uws:destruction>2014-07-07T17:23:31.362+0200</uws:destruction> <uws:parameters> <uws:parameter id="jobdescription"><![CDATA[]]></uws:parameter> <uws:parameter id="jobname"><![CDATA[]]></uws:parameter> <uws:parameter id="session"><![CDATA[1404141103551]]></uws:parameter> <uws:parameter id="maxRec"><![CDATA[100000]]></uws:parameter> <uws:parameter id="query"><![CDATA[SELECT DISTANCE(POINT('ICRS',ra,dec), POINT('ICRS',266.41683,-29.00781)) AS dist, * FROM gaiadr1.gaia_source WHERE 1=CONTAINS(POINT('ICRS',ra,dec),CIRCLE('ICRS',266.41683,-29.00781, 0.08333333)) ORDER BY dist ASC]]></uws:parameter> <uws:parameter id="request"><![CDATA[doQuery]]></uws:parameter> <uws:parameter id="keepAuthenticatedUserJobs"><![CDATA[true]]></uws:parameter> <uws:parameter id="format"><![CDATA[votable]]></uws:parameter> <uws:parameter id="lang"><![CDATA[ADQL]]></uws:parameter> <uws:parameter id="version"><![CDATA[1.0]]></uws:parameter> </uws:parameters> <uws:results> <uws:result id="result" xlink:type="simple" xlink:href="http%3A%2F%2Fgea.esac.esa.int%2Ftap-server%2Ftap%2Fasync%2F1404141177261A%2Fresults%2Fresult" mime="application/x-votable+xml" size="2468741" rows="4734" /> </uws:results> <uws:errorSummary xsi:nil="true" />
To obtain the results of the job (once the job is finished):
curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/async/1404140520859A/results/result"
The retrieved results is a VO table by default (see '3.3 Asynchronous Queries' section parameters to specify a different output format). The results can be saved in a file and inspected using any analysis tool like TOPCAT, for instance.
2.7. Shared tables
Shared tables are accessible once the user is authenticated. You can use the tables in the same way you use your own tables. For instance, if you want to get the first 5 rows of the table 'shared_table' from user 'user_joe' (i.e. 'user_joe._shared_table') in a synchronous mode, you can type:
curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/sync?REQUEST=doQuery&LANG=ADQL&FORMAT=votable&QUERY=SELECT+TOP+5+*+FROM+user_joe.shared_table"
in asynchronous mode:
curl -k -b cookies.txt -i -X POST --data "PHASE=run&LANG=ADQL&REQUEST=doQuery&QUERY=select+top+5+*+from+user_joe.shared_table" "https://gea.esac.esa.int/tap-server/tap/async"
If you want to obtain a list of available tables and columns, you can type:
curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/tables?share_info=true&share_accessible=true"
If you want to obtain a list of available tables, you can type:
curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/tables?share_info=true&share_accessible=true&only_tables=true"
2.8. Share capability
You have to create a shared group in order to share a table.
The following example creates a group named 'new_group' and adds two users ('user_id_1' and 'user_id_2') to it:
curl -k -b cookies.txt -X POST --data "action=CreateOrUpdateGroup&title=my_new_group&description=descripion&users_list=user_id1,user_id2" "https://gea.esac.esa.int/tap-server/tap/share"
The response will contain the group identifier.
OK Group '<user_id>_1425480932331JC' successfully created.
Then, you can update a group (to add/remove users to a group, see 'Adding/Removing a user' below):
curl -k -b cookies.txt -X POST --data "action=CreateOrUpdateGroup&group_id=<group_identifier>&title=my_new_group&description=descripion&users_list=<new_users_list>" "https://gea.esac.esa.int/tap-server/tap/share"
To obtain a list of groups associated to the user (either the user is the owner or the user belongs to):
curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/share?action=GetGroups"
The response:
<?xml version="1.0" encoding="UTF-8"?> <sharedGroups> <sharedGroup id="user_id_1425482038968JC" owner="user_id"> <title><![CDATA[my_new_group]]></title> <description><![CDATA[descripion]]></description> </sharedGroup> <sharedGroup id="other_user_1417615009193I" owner="other_user"> <title><![CDATA[gacs]]></title> <description><![CDATA[group of gacs developers]]></description> </sharedGroup> </sharedGroups>
In this case, the user 'user_id' is the owner of the group 'user_id_1425482038968JC', while the user 'other_user' owns 'other_user_1417615009193I' group.
To remove a group:
curl -k -b cookies.txt -X POST --data "action=RemoveGroup&group_id=<group_identifier>" "https://gea.esac.esa.int/tap-server/tap/share"
Adding a user:
curl -k -b cookies.txt -X POST --data "action=CreateUserGroup&group_id=<group_identifier>&user_id=<user_id>" "https://gea.esac.esa.int/tap-server/tap/share"
Removing a user:
curl -k -b cookies.txt -X POST --data "action=RemoveUserGroup&group_id=<group_identifier>&user_id=<user_id>" "https://gea.esac.esa.int/tap-server/tap/share"
To share a table named 'user_schema.table1' (full qualified table name) you need to create a shared item. In this example, a new item is created and shared to 'my_new_group' group:
curl -k -b cookies.txt -X POST --data "action=CreateOrUpdateItem&resource_type=0&title=user_schema.table1&description=description&items_list=group_id|Group|Read" "https://gea.esac.esa.int/tap-server/tap/share"
The response will contain the shared resource identifier.
OK Resource '<user_id>_1425542442398JC' (type '0') successfully created.
Then, you can update a shared table:
curl -k -b cookies.txt -X POST --data "action=CreateOrUpdateItem&resource_id=<resource_identifier>&resource_type=0&title=user_schema.table1&description=description&items_list=group_id|Group|Read" "https://gea.esac.esa.int/tap-server/tap/share"
To obtain a list of shared tables:
curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/share?action=GetSharedItems"
The response
<?xml version="1.0" encoding="UTF-8"?> <sharedItems owner="user_id"> <sharedItem id="user_id_1425485873983JC" type="0"> <title><![CDATA[user_schema.table]]></title> <description><![CDATA[description]]></description> <sharedToItems> <sharedToItem shareTo="group_id_1425482038968JC" shareType="Group" shareMode="Read"/> </sharedToItems> </sharedItem> </sharedItems>
To remove a sharing (it does not remove the actual table):
curl -k -b cookies.txt -X POST --data "action=RemoveItem&resource_id=<resource_identifier>&resource_type=0" "https://gea.esac.esa.int/tap-server/tap/share"
2.9. Events capability
Each action (table creation/removal, sharing creation/removal, etc.) generates an event. Events are associated to a session and they are kept while the session (connecton) is open.
Events are grouped by types and each time has a 'latest modification time' (latest time that this type of event was generated).
The following example retrieves the events for a user:
curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/event"
The response will contain the events and times associated.
event_type1=timestamp1 event_type2=timestamp2 ...
2.10. Notifications capability
Notifications provides information about some activities. Notifications are available for authenticated users only and are kept for a month.
The following example retrieves the notifications for a user:
curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/notification?action=GetNotifications"
A response example:
notification_id1[type: notification_type1,notification_subtype1]=explanation1 notification_id2[type: notification_type2,notification_subtype2]=explanation2 ...
2.11. Table flags editing capability
Tables flags can be modified.
The following example explains how to set the flags of a table:
curl -k -b cookies.txt -X POST --data "ACTION=radec&TABLE_NAME=schema.table_name&RA=ra_column_name&DEC=dec_column_name" "https://gea.esac.esa.int/tap-server/TableTool"
Where schema.table_name is a full qualified table name (e.g. user_userid.table_name). ra_column specifies the ra column name. dec_column specifies the dec column name.
2.12. Listing jobs
Jobs can be listed using the following request:
curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/jobs/list?offset=index&limit=jobs_number&order=order"
Where list specifies the jobs list (e.g. 'sync' or 'async'). It is mandatory.
index specifies the number of jobs to skip before beginning to return the first job. By default, it is '0'.
limit specifies the number of jobs to be returned. No limit by default is set.
order specifies the order of the results.
Example
curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/jobs/async?offset=0&limit=20&order=CREATION_TIME:DESC"
2.13. Deleting jobs
Jobs can be deleted using their identifiers:
curl -k -b cookies.txt -X POST --data "JOB_IDS=job_id1,job_id2..." "https://gea.esac.esa.int/tap-server/tap/deletejobs"
2.14. Queries on jobs
A job can be used as a table in a query.
Launch an asynchronous job:
curl -i -X POST --data "PHASE=run&LANG=ADQL&LANG=ADQL&REQUEST=doQuery&QUERY=select+top+5+*+from+gaiadr1.gaia_source" "https://gea.esac.esa.int/tap-server/tap/async"
Response:
HTTP/1.1 303 See Other Date: Mon, 30 Jun 2014 14:44:39 GMT Server: Apache-Coyote/1.1 Location: https://gea.esac.esa.int/tap-server/tap/async/1404139480755A Content-Type: application/x-www-form-urlencoded Connection: close Transfer-Encoding: chunked
Use the job identifier (1404139480755A in this case) as a table:
curl "https://gea.esac.esa.int/tap-server/tap/sync?REQUEST=doQuery&LANG=ADQL&FORMAT=votable&QUERY=SELECT+TOP+5+source_id,ra,dec+FROM+JOB_UPLOAD.JOB1404139480755A"
NOTE always add 'JOB' to your job identifier, e.g. JOB1404139480755A
2.15. External TAP queries
A query can be executed in an external TAP server. When an external query is executed, an asynchronous job is launched to the remote TAP server. (See '3.15. TAP+ external TAP access capability' section parameters)
In synchronous mode, the results are served on the fly (in this example, we are accessing Herschel TAP service).
curl "https://gea.esac.esa.int/tap-server/tap/sync?REQUEST=external_tap&EXT_TAP_URL=http://archives.esac.esa.int/hsa/whsa-tap-server/tap&LANG=ADQL&FORMAT=votable_plain&QUERY=SELECT+TOP+5+observation_id,ra,dec+FROM+hsa.v_active_observation"
In asynchronous mode, the results are saved locally (in this example, we access to Vizier TAP service)
curl -i -X POST --data "REQUEST=external_tap&EXT_TAP_URL=http://tapvizier.u-strasbg.fr/TAPVizieR/tap&PHASE=run&LANG=ADQL&LANG=ADQL&QUERY=select+top+5+recno,lambda,flux+from+%22J/A%2BA/605/L8/quasar_a%22" "https://gea.esac.esa.int/tap-server/tap/async"
Response:
HTTP/1.1 303 See Other Date: Mon, 30 Jun 2014 14:44:39 GMT Server: Apache-Coyote/1.1 Location: https://gea.esac.esa.int/tap-server/tap/async/1404139480755A Content-Type: application/x-www-form-urlencoded Connection: close Transfer-Encoding: chunked
Now, you have a local job (1404139480755A in this case) with the results of your external TAP query.
2.16. Global Tap Schema (GloTS) search
A global tap schema search service is available.
Search for services that contains a table name or description.
curl "https://gea.esac.esa.int/tap-server/tap/glots?ACTION=search_tables&keyword=quasars+new&MODE=all&FORMAT=csv"
In this case, we are searching for the keywords 'quasars new'
Response:
accessurl,ivoid,table_name,table_desc,utype http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,J/A+AS/134/483/table3,"New QSOs from the Hamburg Quasar Survey Object ( Hagen H.-J., Engels D., Reimers D.)", http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,J/AJ/113/2000/table1,"New quasar radio detections ( Bischof O.B., Becker R.H.)", http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,J/AJ/122/518/table1,"New LBQS quasars ( Hewett P.C., Foltz C.B., Chaffee F.H.)", http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,J/AJ/142/78/table1,"Parameters of 14 new 2.2<z<3.0 quasars in S82 ( Wu X.-B., Wang R., Schmidt K.B., Bian F., Jiang L., Fan X.)", http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,J/AJ/145/159/table1,"Catalog of new quasars in the vicinity of M31 and M33 discovered by LAMOST in fields observed in 2010 and 2011 ( Huo Z.-Y., Liu X.-W., Xiang M.-S., et al.)", http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,J/AJ/151/24/table1,"Parameters of the 1180 New Quasars Discovered in LAMOST DR1 ( Ai Y.L., Wu X.-B., Yang J., Yang Q., Wang F., Guo R., Zuo W., Dong X., Zhang Y.-X., Yuan H.-L., Song Y.-H., Wang J., Dong X., Yang M., Wu H., Shen S.-Y., Shi J.-R., He B.-L., Lei Y.-J., Li Y.-B., Luo A.-L., Zhao Y.-H., Zhang H.-T.)", http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,J/ApJS/194/22/table1,"29 new, 12 plausible, and 3 previously known quasars behind the SMC ( Kozlowski S., Kochanek C. S., Udalski A.)", http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,J/ApJS/227/11/table5,"Spectroscopic observations of the new PS1 quasars (77 unique sources) ( Banados E., Venemans B.P., Decarli R., et al.)", http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,J/other/RAA/15.1438/table1,"Catalog of new quasars in fields in the vicinity of M31 and M33 discovered by LAMOST in the 2013 observational season ( Huo Z.-Y., Liu X.-W., Xiang M.-S., Shi J.-R., Yuan H.-B., Huang Y., Zhang Y., Hou Y.-H., Wang Y.-F., Yang M.)", http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,VII/182/table6,"new UV excess quasars (paper II) ( Warren S.J., Hewett P.C., Irwin M.J., Osmer P.S.)",
In order to retrieve all columns of a table, we have to provide the 'ivoid' service identifier and the table name:
curl "https://gea.esac.esa.int/tap-server/tap/glots?ACTION=get_columns&IVOID=ivo%3A%2F%2Fcds.vizier%2Ftap&TABLE_NAME=J%2FA%2BAS%2F134%2F483%2Ftable3&FORMAT=csv"
Response:
ivoid,table_name,column_name,description,unit,ucd,utype,datatype,ndhnpmiighpa,principal,indexed,std ivo://cds.vizier/tap,J/A+AS/134/483/table3,Bmag,B magnitude,mag,phot.mag;em.opt.B, ,DOUBLE,mt.size,0,0,1 ivo://cds.vizier/tap,J/A+AS/134/483/table3,DE2000,Declination (2000.0),deg,pos.eq.dec;meta.main, ,DOUBLE,mt.size,1,0,1 ivo://cds.vizier/tap,J/A+AS/134/483/table3,Name,Object name, ,meta.id;meta.main, ,VARCHAR,mt.size,0,0,1 ivo://cds.vizier/tap,J/A+AS/134/483/table3,RA2000,Right ascension (2000.0),deg,pos.eq.ra;meta.main, ,DOUBLE,mt.size,1,0,1 ivo://cds.vizier/tap,J/A+AS/134/483/table3,Run,Run number, ,meta.id;obs, ,SMALLINT,mt.size,0,0,1 ivo://cds.vizier/tap,J/A+AS/134/483/table3,l_Bmag,[~] Approximate flag on Bmag, ,meta.code.error, ,CHAR(1),mt.size,0,0,1 ivo://cds.vizier/tap,J/A+AS/134/483/table3,recno,Record number assigned by the VizieR team. Should Not be used for identification., ,meta.record, ,INTEGER,mt.size,0,0,1 ivo://cds.vizier/tap,J/A+AS/134/483/table3,z,Redshift, ,src.redshift, ,DOUBLE,mt.size,0,0,1
3. Interface
See the following specifications:
- TAP (Table Access Protocol) v1.0
- UWS (Universal Worker Service) v1.0
- ADQL (Astronomical Data Query Language) v2.0
3.1. TAP resources
https://gea.esac.esa.int/tap-server/tap/
Tables |
https://gea.esac.esa.int/tap-server/tap/tables |
|
Synchronous access |
https://gea.esac.esa.int/tap-server/tap/sync |
|
Asynchronous access |
https://gea.esac.esa.int/tap-server/tap/async |
|
Service availability |
https://gea.esac.esa.int/tap-server/tap/availability |
|
Sharing capability |
https://gea.esac.esa.int/tap-server/tap/share |
TAP+ |
Events capability |
https://gea.esac.esa.int/tap-server/tap/event |
TAP+ |
Notifications capability |
https://gea.esac.esa.int/tap-server/tap/notification |
TAP+ |
Jobs listing capability |
https://gea.esac.esa.int/tap-server/tap/jobs |
TAP+ |
Jobs removal capability |
https://gea.esac.esa.int/tap-server/tap/deletejobs |
TAP+ |
3.2. Synchronous Queries
Parameter |
Value |
Comments |
REQUEST |
doQuery |
Requests to execute the provided query |
LANG |
ADQL |
Query language |
FORMAT |
- votable |
Results output format |
QUERY |
ADQL query |
query |
3.3. Asynchronous Queries
Parameter |
Value |
Comments |
Same parameters as defined in 3.2 Synchronous Queries and |
||
PHASE |
run |
Query job initial phase |
The response header will contain the location of the job.
3.4. Query on an 'on-the-fly' uploaded table
Use a multipart/form-data (see IETF RFC 2388) HTTP POST
Parameter |
Value |
Comments |
Same parameters as defined in 3.2 Synchronous Queries and |
||
UPLOAD |
query_table,param:parameter_table_name |
query_table: the name of the table used in the query |
parameter_table_name |
file |
file name that contains the table to be uploaded |
For instance, in the following request:
curl --form UPLOAD="table_c,param:table1" --form table1=@test_ra_dec.vot --form LANG=ADQL --form REQUEST=doQuery --form QUERY="select top 5 * from tap_upload.table_c" https://gea.esac.esa.int/tap-server/tap/sync
table_c
is the name of the table used in the query: QUERY="select top 5 * from tap_upload.table_c"
,
table1
is the name of the HTTP parameter that provides the file: table1=@test_ra_dec.vot
and test_ra_dec.vot
is the file name that contains the table to be uploaded.
3.5. TAP+ login
Parameter |
Value |
Comments |
username |
user_name |
User name |
password |
user_password |
User password |
The response header will contain the session identifier.
3.6. TAP+ logout
Parameter |
Value |
Comments |
session identifier |
session identifier |
Session identifier provided by a login request |
3.7. TAP+ upload user table
Use a multipart/form-data (see IETF RFC 2388) HTTP POST
Parameter |
Value |
Comments |
session identifier |
session identifier |
Session identifier provided by a login request |
FILE |
file |
file name that contains the table to be uploaded |
TABLE_NAME |
table name |
table name associated to the uploaded table |
TABLE_DESC |
table description |
table description associated to the uploaded table |
FORMAT |
-VOTable (see file format) |
table data format |
3.8. TAP+ delete user table
Parameter |
Value |
Comments |
session identifier |
session identifier |
Session identifier provided by a login request |
TABLE_NAME |
table name |
table name associated to the uploaded table |
DELETE |
TRUE |
requested action |
FORCE_REMOVAL |
TRUE |
To force a table removal |
3.9. TAP+ sharing capability
3.9.1. Tables list
'table' TAP capability allows the user to retrieve shared items information. The following parameters can be used:
Parameter |
Value |
Comments |
session identifier |
session identifier |
Session identifier provided by a login request |
share_info |
TRUE |
provides information about shared items |
share_accessible |
TRUE |
includes tables that are accessible to the user |
3.9.2. Shared groups
Create shared group TAP capability is required in order to allow user to share tables. The following parameters can be used:
Parameter |
Value |
Comments |
session identifier |
session identifier |
Session identifier provided by a login request |
action |
CreateOrUpdateGroup |
Request to create or update a group |
group_id |
Group identifier |
If not provided: a new group is created and its identifier is returned as part of the response. |
title |
Group title |
Mandatory. |
description |
Group description |
Optional. |
user_list |
List of user identifiers |
Optional. |
To remove a shared group the following parameters must be used:
Parameter |
Value |
Comments |
session identifier |
session identifier |
Session identifier provided by a login request |
action |
RemoveGroup |
Request to remove a group |
group_id |
Group identifier |
Mandatory. |
3.9.3. Shared items
Create shared item TAP capability allows the user to share a table to one or more groups (user groups). The following parameters can be used:
Parameter |
Value |
Comments |
session identifier |
session identifier |
Session identifier provided by a login request |
action |
CreateOrUpdateItem |
Request to create or update a group |
resource_id |
Shared resource identifier |
If not provided: a new shared resource is created and its identifier is returned as part of the response. |
resource_type |
0 |
Mandatory. |
title |
Full qualified table name |
Mandatory. |
description |
Shared table description |
Optional. |
items_list |
List of groups identifiers that share this item |
Mandatory. |
To remove a shared table the following parameters must be used:
Parameter |
Value |
Comments |
session identifier |
session identifier |
Session identifier provided by a login request |
action |
RemoveItem |
Request to remove a shared item |
resource_id |
Shared item identifier |
Mandatory. |
resource_type |
0 |
Mandatory. |
3.9.4. Sharing: other actions
Other Shared TAP+ capabilities are:
Parameter |
Value |
Comments |
session identifier |
session identifier |
Session identifier provided by a login request |
action |
GetGroups |
Returns a list of groups where the user belongs to or created by the user |
GetGroupsDescriptionsOnly |
Returns a list of groups where the user belongs to or created by the user (only descriptions). |
|
GetSharedItems |
Returns a list of items shared by the user. |
|
GetSharedItemsDescriptionsOnly |
Returns a list of items shared by the user (only descriptions). |
|
include_users |
'true' or 'false' |
Valid for 'GetGroups' action only. |
groups_list |
list of groups ids |
Valid for 'GetGroups' action only. |
3.10. TAP+ tables capabilities
In addition to the standar 'tables' TAP capability, the following parameters can be used too:
Parameter |
Value |
Comments |
tables |
comma separated full qualified table names |
A lis of the specified tables will be returned |
schemas |
comma separated schema names |
A list of the specified schemas will be returned |
only_tables |
TRUE / FALSE (default: FALSE) |
TRUE: no columns information will be returned |
only_schemas |
TRUE / FALSE (default: FALSE) |
TRUE: no tables nor columns information will be returned |
These parameters are handled based on the following priorities
Priority |
Parameter |
Comments |
1. |
tables != null |
No more checks are performed (share_info and share_accessible are handled if present) |
2. |
tables == null (default) |
More checks are performed |
2.1. |
only_schemas = TRUE |
No more checks are performed |
2.2. |
only_schemas = FALSE (default) |
The following checks are performed |
2.2.1. |
schema_names != null |
The following parameters are applied to the specified schemas only |
2.2.2 |
only_tables = TRUE |
No columns data are generated |
2.2.3 |
only_tables = FALSE (default) |
Columns data are generated |
3.11. TAP+ events capabilities
Tables creation/removal/sharing actions generate events. Events are grouped by types and each type has a 'last modification time' associated. Those events and times can be retrieved using:
Parameter |
Value |
Comments |
id |
event type identifier |
Optional parameter. If not provided, all events are returned. If provided, only the requested type is returned. |
The current event types are:
Value |
Comments |
100 |
Job created |
101 |
Job updated |
102 |
Job removed |
210 |
Shared item created |
211 |
Shared item updated |
212 |
Shared item removed |
220 |
Shared group created |
221 |
Shared group updated |
222 |
Shared group removed |
230 |
Shared user created |
231 |
Shared user updated |
232 |
Shared user removed |
300 |
Log in |
301 |
Log out |
401 |
Database quota updated |
402 |
File quota updated |
500 |
Notification created |
501 |
Notification removed |
4000 |
Table created |
4001 |
Table updated |
4002 |
Table removed |
3.12. TAP+ notifications
Notifications are kept for one month. Each notification provides a message that explains the notification. Notifications are available for authenticated users only.
Parameter |
Value |
Comments |
session identifier |
session identifier |
Session identifier provided by a login request |
action |
GetNotifications |
Notificaions associated to the user |
3.13. Table flags editing
Parameter |
Value |
Comments |
session identifier |
session identifier |
Session identifier provided by a login request |
ACTION |
radec |
Requests a Ra/Dec update |
TABLE_NAME |
Full qualified table name |
|
RA |
ra column name |
|
DEC |
dec column name |
|
3.14. TAP+ jobs capabilities
Parameter |
Value |
Comments |
session identifier |
session identifier |
Session identifier provided by a login request |
list |
job list identifier |
Mandatory |
offset |
Number of jobs to skip |
Optional. '0' by default. |
limit |
Number of jobs to return |
Optinal. No limit by default. |
order |
List of column order records. |
Optional. |
3.15. TAP+ deletejobs capability
Parameter |
Value |
Comments |
session identifier |
session identifier |
Session identifier provided by a login request |
JOB_IDS |
comma separated job identifiers |
Mandatory |
3.16. TAP+ external TAP access capability
Synchronous mode
Parameter |
Value |
Comments |
Same parameters as defined in 3.2 Synchronous Queries and: |
||
REQUEST |
external_tap |
Request an external TAP access |
EXT_TAP_URL |
URL |
URL of the external TAP service |
Asynchronous mode
Parameter |
Value |
Comments |
Same parameters as defined in 3.3 Asynchronous Queries and: |
||
REQUEST |
external_tap |
Request an external TAP access |
EXT_TAP_URL |
URL |
URL of the external TAP service |
The response header will contain the location of the job.
3.17. Global Tap Schema (GloTS) search capability
Search by table or description
Parameter |
Value |
Comments |
ACTION |
search_table |
|
KEYWORD |
text |
Descriptons or table names |
MODE |
-all (default) |
-Search in both: table names and descriptions |
FORMAT |
-votable (default) |
Results output formt |
Retrieve table columns
Parameter |
Value |
Comments |
ACTION |
get_columns |
|
IVOID |
text |
Service descriptor identifier |
TABLE_NAME |
table name |
|
FORMAT |
-votable (default) |
Results output formt |