Help support

Should you have any question, please check the Gaia FAQ section or contact the Gaia Helpdesk

Python Access

 

ESA Gaia Archive provides a Python Astroquery (Astropy) package for easy access to the ESA Gaia Archive: astroquery.gaia.

Also, a generic package is provided for accessing any TAP and TAP+ compliant service: astroquery.utils.tap.

See a Python tutorial for Jupyter Notebook example here.

Command line access

 

The entry point is a TAP (Table Access Protocol) server.

You may use HTTP protocol to execute TAP requests at https://gea.esac.esa.int/tap-server/tap. TAP provides two operation modes: Synchronous and Asynchronous:

  • Synchronous: the response to the request will be generated as soon as the request received by the server.
  • Asynchronous: the server will start a job that will execute the request. The first response to the request is the required information (a link) to obtain the job status. Once the job is finished, the results can be retrieved.

Our TAP server provides two access mode: public and authenticated:

  • Public: this is the standard TAP access. A user can execute ADQL queries and upload tables to be used in a query 'on-the-fly' (these tables will be removed once the query is executed). The results are available to any other user and they will remain in the server for a limited space of time.
  • Authenticated: some functionalities are restricted to authenticated users only The results are saved in a private user space and they will remain in the server for ever (they can be removed by the user).
    • ADQL queries and results are saved in a user private area.
    • Cross-match operations: a catalogue cross-match operation can be executed. Cross-match operations results are saved in a user private area.
    • Persintance of uploaded tables: a user can upload a table in a private space. These tables can be used in queries as well as in cross-matches operations.

Here you can find some examples about how to interact with a TAP server (we are using the curl tool):

 

1. Non authenticated access

 

1.1. Getting all public tables​​​​

curl "https://gea.esac.esa.int/tap-server/tap/tables"	

 

1.2. Synchronous query

curl "https://gea.esac.esa.int/tap-server/tap/sync?REQUEST=doQuery&LANG=ADQL&FORMAT=votable&QUERY=SELECT+TOP+5+source_id,ra,dec+FROM+gaiadr1.gaia_source"

The retrieved results is a VO table by default (see '3.2. Synchronous Queries' section parameters to specify a different output format). The results can be saved in a file and inspected using any analysis tool like TOPCAT, for instance.

 

1.3. Synchronous query on an 'on-the-fly' uploaded table

curl --form UPLOAD="table_c,param:table1" --form table1=@test_ra_dec.vot --form LANG=ADQL --form REQUEST=doQuery --form QUERY="select top 5 * from tap_upload.table_c" https://gea.esac.esa.int/tap-server/tap/sync			

Where 'test_ra_dec.vot' is a file that contains the VOTable to be uploaded (in order to be used by the query)

The retrieved results is a VO table by default (see '3.2. Synchronous Queries' section parameters to specify a different output format). The results can be saved in a file and inspected using any analysis tool like TOPCAT, for instance.

 

1.4. Asynchronous query

curl -i -X POST --data "PHASE=run&LANG=ADQL&LANG=ADQL&REQUEST=doQuery&QUERY=select+top+5+*+from+gaiadr1.gaia_source" "https://gea.esac.esa.int/tap-server/tap/async"

Note that there is the possibility to use the optional parameters "JOBNAME" to assign a name to the job and "JOBDESCRIPTION" to add a description:

curl -i -X POST --data "PHASE=run&LANG=ADQL&JOBNAME=optionalJobName&JOBDESCRIPTION=optionalDescription&LANG=ADQL&REQUEST=doQuery&QUERY=select+top+5+*+from+gaiadr1.gaia_source" "https://gea.esac.esa.int/tap-server/tap/async"			

The response will contain the URL of the job running at server side (see Location header):

HTTP/1.1 303 See Other
Date: Mon, 30 Jun 2014 14:44:39 GMT
Server: Apache-Coyote/1.1
Location: https://gea.esac.esa.int/tap-server/tap/async/1404139480755A
Content-Type: application/x-www-form-urlencoded
Connection: close
Transfer-Encoding: chunked			

To obtain the status of the running job:

curl "https://gea.esac.esa.int/tap-server/tap/async/1404139480755A"

The status response is something like:

<?xml version="1.0" encoding="UTF-8"?>
<uws:job xmlns:uws="http://www.ivoa.net/xml/UWS/v1.0" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
        <uws:jobId><![CDATA[1404139480755A]]></uws:jobId>
        <uws:runId xsi:nil="true" />
        <uws:ownerId><![CDATA[anonymous]]></uws:ownerId>
        <uws:phase>COMPLETED</uws:phase>
        <uws:quote xsi:nil="true" />
        <uws:startTime>2014-06-30T16:44:40.766+0200</uws:startTime>
        <uws:endTime>2014-06-30T16:44:40.830+0200</uws:endTime>
        <uws:executionDuration>0</uws:executionDuration>
        <uws:destruction>2014-07-07T16:44:40.754+0200</uws:destruction>
        <uws:parameters>
                <uws:parameter id="maxRec"><![CDATA[100000]]></uws:parameter>
                <uws:parameter id="query"><![CDATA[select top 5 * from gaiadr1.gaia_source]]></uws:parameter>
                <uws:parameter id="request"><![CDATA[doQuery]]></uws:parameter>
                <uws:parameter id="format"><![CDATA[votable]]></uws:parameter>
                <uws:parameter id="keepAuthenticatedUserJobs"><![CDATA[true]]></uws:parameter>
                <uws:parameter id="lang"><![CDATA[ADQL]]></uws:parameter>
                <uws:parameter id="version"><![CDATA[1.0]]></uws:parameter>
        </uws:parameters>
        <uws:results>
                <uws:result id="result" xlink:type="simple" xlink:href="http%3A%2F%2Fgea.esac.esa.int%2Ftap-server%2Ftap%2Fasync%2F1404139480755A%2Fresults%2Fresult" mime="application/x-votable+xml" size="8863" rows="6" />
        </uws:results>
        <uws:errorSummary xsi:nil="true" />

To obtain the results of the job (once the job is finished):

curl "https://gea.esac.esa.int/tap-server/tap/async/1404139480755A/results/result"

The retrieved results is a VO table by default (see '3.3 Asynchronous Queries' section parameters to specify a different output format).

The results can be saved in a file and inspected using any analysis tool like TOPCAT, for instance.

 

1.5. Python script to execute an asynchronous query and wait until the results are available

#ASYNCHRONOUS REQUEST

#Python 2
#import httplib
#import urllib

#Python 3
import http.client as httplib
import urllib.parse as urllib
import time
from xml.dom.minidom import parseString

host = "gea.esac.esa.int"
port = 443
pathinfo = "/tap-server/tap/async"


#-------------------------------------
#Create job

params = urllib.urlencode({\
	"REQUEST": "doQuery", \
	"LANG":    "ADQL", \
	"FORMAT":  "votable_plain", \
	"PHASE":  "RUN", \
	"JOBNAME":  "Any name (optional)", \
	"JOBDESCRIPTION":  "Any description (optional)", \
	"QUERY":   "SELECT DISTANCE(POINT('ICRS',ra,dec), POINT('ICRS',266.41683,-29.00781)) AS dist, * FROM gaiadr1.gaia_source  WHERE 1=CONTAINS(POINT('ICRS',ra,dec),CIRCLE('ICRS',266.41683,-29.00781, 0.08333333)) ORDER BY dist ASC"
	})

headers = {\
	"Content-type": "application/x-www-form-urlencoded", \
	"Accept":       "text/plain" \
	}

connection = httplib.HTTPSConnection(host, port)
connection.request("POST",pathinfo,params,headers)

#Status
response = connection.getresponse()
print ("Status: " +str(response.status), "Reason: " + str(response.reason))

#Server job location (URL)
location = response.getheader("location")
print ("Location: " + location)

#Jobid
jobid = location[location.rfind('/')+1:]
print ("Job id: " + jobid)

connection.close()

#-------------------------------------
#Check job status, wait until finished

while True:
	connection = httplib.HTTPSConnection(host, port)
	connection.request("GET",pathinfo+"/"+jobid)
	response = connection.getresponse()
	data = response.read()
	#XML response: parse it to obtain the current status
	#(you may use pathinfo/jobid/phase entry point to avoid XML parsing)
	dom = parseString(data)
	phaseElement = dom.getElementsByTagName('uws:phase')[0]
	phaseValueElement = phaseElement.firstChild
	phase = phaseValueElement.toxml()
	print ("Status: " + phase)
	#Check finished
	if phase == 'COMPLETED': break
	#wait and repeat
	time.sleep(0.2)


connection.close()

#-------------------------------------
#Get results
connection = httplib.HTTPSConnection(host, port)
connection.request("GET",pathinfo+"/"+jobid+"/results/result")
response = connection.getresponse()
data = response.read().decode('iso-8859-1')
#print(type(data))
#print(data)
outputFileName = "example_votable_output.vot"
outputFile = open(outputFileName, "w")
outputFile.write(data)
outputFile.close()
connection.close()
print ("Data saved in: " + outputFileName)
				

The saved file is a VO table (by default, see '3.3 Asynchronous Queries' section parameters to specify a different output format). The file can be inspected using any analysis tool like TOPCAT, for instance.

 

2. Authenticated access

 

2.1. Login

curl -k -c cookies.txt -X POST -d username=USERNAME -d password=PASSWORD -L "https://gea.esac.esa.int/tap-server/login"

 

2.2. Logout

curl -k -b cookies.txt -X POST -d -L "https://gea.esac.esa.int/tap-server/logout"

 

2.3. Getting public and user tables

curl -k -b cookies.txt -X POST -L "https://gea.esac.esa.int/tap-server/tap/tables"

 

2.4. Upload user table

curl -k -b cookies.txt -X POST -F FILE=@file.name -F TABLE_NAME=table_name "https://gea.esac.esa.int/tap-server/Upload"

Where 'file.name' is a file that contains the VOTable to be uploaded.
TABLE_DESC parameter can be used to specify a table description.
FORMAT parameter can be used to specify the table data format. Available formats: 'VOTable', 'CSV' and 'ASCII': see section 3.7 for more details.

 

2.5. Delete user table

curl -k -b cookies.txt -X POST -F TABLE_NAME=table_name -F DELETE=TRUE "https://gea.esac.esa.int/tap-server/Upload"		


Where 'table.name' is the argument used when the table was uploaded (argument TABLE_NAME=table_name).

To force a table removal, use FORCE_REMOVAL=TRUE flag:

curl -k -b cookies.txt -X POST -F TABLE_NAME=table_name -F DELETE=TRUE -F FORCE_REMOVAL=TRUE "https://gea.esac.esa.int/tap-server/Upload"

 

2.6. Asynchronous query

curl -k -b cookies.txt -i -X POST --data "PHASE=run&LANG=ADQL&REQUEST=doQuery&QUERY=select+top+5+*+from+gaiadr1.gaia_source" "https://gea.esac.esa.int/tap-server/tap/async"
Note that there is the possibility to use the optional parameters "JOBNAME" to assign a name to the job and "JOBDESCRIPTION" to add a description:
curl -k -b cookies.txt -i -X POST --data "PHASE=run&LANG=ADQL&JOBNAME=optionalJobName&JOBDESCRIPTION=optionalDescription&REQUEST=doQuery&QUERY=select+top+5+*+from+gaiadr1.gaia_source" "https://gea.esac.esa.int/tap-server/tap/async"				

The response will contain the URL of the job running at server side:

HTTP/1.1 303 See Other
Date: Mon, 30 Jun 2014 15:02:00 GMT
Server: Apache-Coyote/1.1
Location: https://gea.esac.esa.int/tap-server/tap/async/1404140520859A
Content-Type: application/x-www-form-urlencoded
Connection: close
Transfer-Encoding: chunked

To obtain the status of the running job:

curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/async/1404140520859A"

The status response is something like:

<?xml version="1.0" encoding="UTF-8"?>
<uws:job xmlns:uws="http://www.ivoa.net/xml/UWS/v1.0" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
        <uws:jobId><![CDATA[1404141177261A]]></uws:jobId>
        <uws:runId xsi:nil="true" />
        <uws:ownerId><![CDATA[USERNAME]]></uws:ownerId>
        <uws:phase>COMPLETED</uws:phase>
        <uws:quote xsi:nil="true" />
        <uws:startTime>2014-06-30T17:12:57.273+0200</uws:startTime>
        <uws:endTime>2014-06-30T17:12:57.864+0200</uws:endTime>
        <uws:executionDuration>0</uws:executionDuration>
        <uws:destruction>2014-07-07T17:23:31.362+0200</uws:destruction>
        <uws:parameters>
                <uws:parameter id="jobdescription"><![CDATA[]]></uws:parameter>
                <uws:parameter id="jobname"><![CDATA[]]></uws:parameter>
                <uws:parameter id="session"><![CDATA[1404141103551]]></uws:parameter>
                <uws:parameter id="maxRec"><![CDATA[100000]]></uws:parameter>
                <uws:parameter id="query"><![CDATA[SELECT DISTANCE(POINT('ICRS',ra,dec), POINT('ICRS',266.41683,-29.00781)) AS dist, *
FROM gaiadr1.gaia_source
WHERE 1=CONTAINS(POINT('ICRS',ra,dec),CIRCLE('ICRS',266.41683,-29.00781, 0.08333333)) ORDER BY dist ASC]]></uws:parameter>
                <uws:parameter id="request"><![CDATA[doQuery]]></uws:parameter>
                <uws:parameter id="keepAuthenticatedUserJobs"><![CDATA[true]]></uws:parameter>
                <uws:parameter id="format"><![CDATA[votable]]></uws:parameter>
                <uws:parameter id="lang"><![CDATA[ADQL]]></uws:parameter>
                <uws:parameter id="version"><![CDATA[1.0]]></uws:parameter>
        </uws:parameters>
        <uws:results>
                <uws:result id="result" xlink:type="simple" xlink:href="http%3A%2F%2Fgea.esac.esa.int%2Ftap-server%2Ftap%2Fasync%2F1404141177261A%2Fresults%2Fresult" mime="application/x-votable+xml" size="2468741" rows="4734" />
        </uws:results>
        <uws:errorSummary xsi:nil="true" />

To obtain the results of the job (once the job is finished):

curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/async/1404140520859A/results/result"

The retrieved results is a VO table by default (see '3.3 Asynchronous Queries' section parameters to specify a different output format). The results can be saved in a file and inspected using any analysis tool like TOPCAT, for instance.

 

2.7. Shared tables

Shared tables are accessible once the user is authenticated. You can use the tables in the same way you use your own tables. For instance, if you want to get the first 5 rows of the table 'shared_table' from user 'user_joe' (i.e. 'user_joe._shared_table') in a synchronous mode, you can type:

curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/sync?REQUEST=doQuery&LANG=ADQL&FORMAT=votable&QUERY=SELECT+TOP+5+*+FROM+user_joe.shared_table"

in asynchronous mode:

curl -k -b cookies.txt -i -X POST --data "PHASE=run&LANG=ADQL&REQUEST=doQuery&QUERY=select+top+5+*+from+user_joe.shared_table" "https://gea.esac.esa.int/tap-server/tap/async"

If you want to obtain a list of available tables and columns, you can type:

curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/tables?share_info=true&share_accessible=true"					

If you want to obtain a list of available tables, you can type:

curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/tables?share_info=true&share_accessible=true&only_tables=true"

 

2.8. Share capability

You have to create a shared group in order to share a table.
The following example creates a group named 'new_group' and adds two users ('user_id_1' and 'user_id_2') to it:

curl -k -b cookies.txt -X POST --data "action=CreateOrUpdateGroup&title=my_new_group&description=descripion&users_list=user_id1,user_id2" "https://gea.esac.esa.int/tap-server/tap/share"

The response will contain the group identifier.

OK
Group '<user_id>_1425480932331JC' successfully created.

Then, you can update a group (to add/remove users to a group, see 'Adding/Removing a user' below):

curl -k -b cookies.txt -X POST --data "action=CreateOrUpdateGroup&group_id=<group_identifier>&title=my_new_group&description=descripion&users_list=<new_users_list>" "https://gea.esac.esa.int/tap-server/tap/share"

To obtain a list of groups associated to the user (either the user is the owner or the user belongs to):

curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/share?action=GetGroups"

The response:

<?xml version="1.0" encoding="UTF-8"?>
<sharedGroups>
<sharedGroup id="user_id_1425482038968JC" owner="user_id">
	<title><![CDATA[my_new_group]]></title>
	<description><![CDATA[descripion]]></description>
</sharedGroup>
<sharedGroup id="other_user_1417615009193I" owner="other_user">
	<title><![CDATA[gacs]]></title>
	<description><![CDATA[group of gacs developers]]></description>
</sharedGroup>
</sharedGroups>

In this case, the user 'user_id' is the owner of the group 'user_id_1425482038968JC', while the user 'other_user' owns 'other_user_1417615009193I' group.

To remove a group:

curl -k -b cookies.txt -X POST --data "action=RemoveGroup&group_id=<group_identifier>" "https://gea.esac.esa.int/tap-server/tap/share"

Adding a user:

curl -k -b cookies.txt -X POST --data "action=CreateUserGroup&group_id=<group_identifier>&user_id=<user_id>" "https://gea.esac.esa.int/tap-server/tap/share"

Removing a user:

curl -k -b cookies.txt -X POST --data "action=RemoveUserGroup&group_id=<group_identifier>&user_id=<user_id>" "https://gea.esac.esa.int/tap-server/tap/share"
						

To share a table named 'user_schema.table1' (full qualified table name) you need to create a shared item. In this example, a new item is created and shared to 'my_new_group' group:

curl -k -b cookies.txt -X POST --data "action=CreateOrUpdateItem&resource_type=0&title=user_schema.table1&description=description&items_list=group_id|Group|Read" "https://gea.esac.esa.int/tap-server/tap/share"

The response will contain the shared resource identifier.

OK
Resource '<user_id>_1425542442398JC' (type '0') successfully created.

Then, you can update a shared table:

curl -k -b cookies.txt -X POST --data "action=CreateOrUpdateItem&resource_id=<resource_identifier>&resource_type=0&title=user_schema.table1&description=description&items_list=group_id|Group|Read" "https://gea.esac.esa.int/tap-server/tap/share"	

To obtain a list of shared tables:

curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/share?action=GetSharedItems"						

The response

<?xml version="1.0" encoding="UTF-8"?>
<sharedItems owner="user_id">
<sharedItem id="user_id_1425485873983JC" type="0">
	<title><![CDATA[user_schema.table]]></title>
	<description><![CDATA[description]]></description>
	<sharedToItems>
<sharedToItem shareTo="group_id_1425482038968JC" shareType="Group" shareMode="Read"/>
	</sharedToItems>
</sharedItem>
</sharedItems>

To remove a sharing (it does not remove the actual table):

curl -k -b cookies.txt -X POST --data "action=RemoveItem&resource_id=<resource_identifier>&resource_type=0" "https://gea.esac.esa.int/tap-server/tap/share"

 

2.9. Events capability

Each action (table creation/removal, sharing creation/removal, etc.) generates an event. Events are associated to a session and they are kept while the session (connecton) is open.
Events are grouped by types and each time has a 'latest modification time' (latest time that this type of event was generated).
The following example retrieves the events for a user:

curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/event"

The response will contain the events and times associated.

event_type1=timestamp1
event_type2=timestamp2
...	

 

2.10. Notifications capability

Notifications provides information about some activities. Notifications are available for authenticated users only and are kept for a month.
The following example retrieves the notifications for a user:

curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/notification?action=GetNotifications"

A response example:

notification_id1[type: notification_type1,notification_subtype1]=explanation1
notification_id2[type: notification_type2,notification_subtype2]=explanation2
...	

 

2.11. Table flags editing capability

Tables flags can be modified.
The following example explains how to set the flags of a table:

curl -k -b cookies.txt -X POST --data "ACTION=radec&TABLE_NAME=schema.table_name&RA=ra_column_name&DEC=dec_column_name" "https://gea.esac.esa.int/tap-server/TableTool"

Where schema.table_name is a full qualified table name (e.g. user_userid.table_name). ra_column specifies the ra column name. dec_column specifies the dec column name.

 

2.12. Listing jobs

Jobs can be listed using the following request:

curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/jobs/list?offset=index&limit=jobs_number&order=order"

Where list specifies the jobs list (e.g. 'sync' or 'async'). It is mandatory.
index specifies the number of jobs to skip before beginning to return the first job. By default, it is '0'.
limit specifies the number of jobs to be returned. No limit by default is set.
order specifies the order of the results.

Example

curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/jobs/async?offset=0&limit=20&order=CREATION_TIME:DESC"

 

2.13. Deleting jobs

Jobs can be deleted using their identifiers:

curl -k -b cookies.txt -X POST --data "JOB_IDS=job_id1,job_id2..." "https://gea.esac.esa.int/tap-server/tap/deletejobs"		

 

2.14. Queries on jobs

A job can be used as a table in a query.

Launch an asynchronous job:

curl -i -X POST --data "PHASE=run&LANG=ADQL&LANG=ADQL&REQUEST=doQuery&QUERY=select+top+5+*+from+gaiadr1.gaia_source" "https://gea.esac.esa.int/tap-server/tap/async"

Response:

HTTP/1.1 303 See Other
Date: Mon, 30 Jun 2014 14:44:39 GMT
Server: Apache-Coyote/1.1
Location: https://gea.esac.esa.int/tap-server/tap/async/1404139480755A
Content-Type: application/x-www-form-urlencoded
Connection: close
Transfer-Encoding: chunked

Use the job identifier (1404139480755A in this case) as a table:

curl "https://gea.esac.esa.int/tap-server/tap/sync?REQUEST=doQuery&LANG=ADQL&FORMAT=votable&QUERY=SELECT+TOP+5+source_id,ra,dec+FROM+JOB_UPLOAD.JOB1404139480755A"

NOTE always add 'JOB' to your job identifier, e.g. JOB1404139480755A

 

2.15. External TAP queries

A query can be executed in an external TAP server. When an external query is executed, an asynchronous job is launched to the remote TAP server. (See '3.15. TAP+ external TAP access capability' section parameters)

In synchronous mode, the results are served on the fly (in this example, we are accessing Herschel TAP service).

curl "https://gea.esac.esa.int/tap-server/tap/sync?REQUEST=external_tap&EXT_TAP_URL=http://archives.esac.esa.int/hsa/whsa-tap-server/tap&LANG=ADQL&FORMAT=votable_plain&QUERY=SELECT+TOP+5+observation_id,ra,dec+FROM+hsa.v_active_observation"

In asynchronous mode, the results are saved locally (in this example, we access to Vizier TAP service)

curl -i -X POST --data "REQUEST=external_tap&EXT_TAP_URL=http://tapvizier.u-strasbg.fr/TAPVizieR/tap&PHASE=run&LANG=ADQL&LANG=ADQL&QUERY=select+top+5+recno,lambda,flux+from+%22J/A%2BA/605/L8/quasar_a%22" "https://gea.esac.esa.int/tap-server/tap/async"

Response:

HTTP/1.1 303 See Other
Date: Mon, 30 Jun 2014 14:44:39 GMT
Server: Apache-Coyote/1.1
Location: https://gea.esac.esa.int/tap-server/tap/async/1404139480755A
Content-Type: application/x-www-form-urlencoded
Connection: close
Transfer-Encoding: chunked

Now, you have a local job (1404139480755A in this case) with the results of your external TAP query.

 

2.16. Global Tap Schema (GloTS) search

A global tap schema search service is available.

Search for services that contains a table name or description.

curl "https://gea.esac.esa.int/tap-server/tap/glots?ACTION=search_tables&keyword=quasars+new&MODE=all&FORMAT=csv"

In this case, we are searching for the keywords 'quasars new'

Response:

accessurl,ivoid,table_name,table_desc,utype
http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,J/A+AS/134/483/table3,"New QSOs from the Hamburg Quasar Survey Object ( Hagen H.-J., Engels D., Reimers D.)",
http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,J/AJ/113/2000/table1,"New quasar radio detections ( Bischof O.B., Becker R.H.)",
http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,J/AJ/122/518/table1,"New LBQS quasars ( Hewett P.C., Foltz C.B., Chaffee F.H.)",
http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,J/AJ/142/78/table1,"Parameters of 14 new 2.2<z<3.0 quasars in S82 ( Wu X.-B., Wang R., Schmidt K.B., Bian F., Jiang L., Fan X.)",
http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,J/AJ/145/159/table1,"Catalog of new quasars in the vicinity of M31 and M33 discovered by LAMOST in fields observed in 2010 and 2011 ( Huo Z.-Y., Liu X.-W., Xiang M.-S., et al.)",
http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,J/AJ/151/24/table1,"Parameters of the 1180 New Quasars Discovered in LAMOST DR1 ( Ai Y.L., Wu X.-B., Yang J., Yang Q., Wang F., Guo R., Zuo W., Dong X., Zhang Y.-X., Yuan H.-L., Song Y.-H., Wang J., Dong X., Yang M., Wu H., Shen S.-Y., Shi J.-R., He B.-L., Lei Y.-J., Li Y.-B., Luo A.-L., Zhao Y.-H., Zhang H.-T.)",
http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,J/ApJS/194/22/table1,"29 new, 12 plausible, and 3 previously known quasars behind the SMC ( Kozlowski S., Kochanek C. S., Udalski A.)",
http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,J/ApJS/227/11/table5,"Spectroscopic observations of the new PS1 quasars (77 unique sources) ( Banados E., Venemans B.P., Decarli R., et al.)",
http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,J/other/RAA/15.1438/table1,"Catalog of new quasars in fields in the vicinity of M31 and M33 discovered by LAMOST in the 2013 observational season ( Huo Z.-Y., Liu X.-W., Xiang M.-S., Shi J.-R., Yuan H.-B., Huang Y., Zhang Y., Hou Y.-H., Wang Y.-F., Yang M.)",
http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,VII/182/table6,"new UV excess quasars (paper II) ( Warren S.J., Hewett P.C., Irwin M.J., Osmer P.S.)", 						

In order to retrieve all columns of a table, we have to provide the 'ivoid' service identifier and the table name:

curl "https://gea.esac.esa.int/tap-server/tap/glots?ACTION=get_columns&IVOID=ivo%3A%2F%2Fcds.vizier%2Ftap&TABLE_NAME=J%2FA%2BAS%2F134%2F483%2Ftable3&FORMAT=csv"

Response:

ivoid,table_name,column_name,description,unit,ucd,utype,datatype,ndhnpmiighpa,principal,indexed,std
ivo://cds.vizier/tap,J/A+AS/134/483/table3,Bmag,B magnitude,mag,phot.mag;em.opt.B, ,DOUBLE,mt.size,0,0,1
ivo://cds.vizier/tap,J/A+AS/134/483/table3,DE2000,Declination (2000.0),deg,pos.eq.dec;meta.main, ,DOUBLE,mt.size,1,0,1
ivo://cds.vizier/tap,J/A+AS/134/483/table3,Name,Object name, ,meta.id;meta.main, ,VARCHAR,mt.size,0,0,1
ivo://cds.vizier/tap,J/A+AS/134/483/table3,RA2000,Right ascension (2000.0),deg,pos.eq.ra;meta.main, ,DOUBLE,mt.size,1,0,1
ivo://cds.vizier/tap,J/A+AS/134/483/table3,Run,Run number, ,meta.id;obs, ,SMALLINT,mt.size,0,0,1
ivo://cds.vizier/tap,J/A+AS/134/483/table3,l_Bmag,[~] Approximate flag on Bmag, ,meta.code.error, ,CHAR(1),mt.size,0,0,1
ivo://cds.vizier/tap,J/A+AS/134/483/table3,recno,Record number assigned by the VizieR team. Should Not be used for identification., ,meta.record, ,INTEGER,mt.size,0,0,1
ivo://cds.vizier/tap,J/A+AS/134/483/table3,z,Redshift, ,src.redshift, ,DOUBLE,mt.size,0,0,1

 

3. Interface

 

See the following specifications:

 

3.1. TAP resources

https://gea.esac.esa.int/tap-server/tap/

Tables

https://gea.esac.esa.int/tap-server/tap/tables

 

Synchronous access

https://gea.esac.esa.int/tap-server/tap/sync

 

Asynchronous access

https://gea.esac.esa.int/tap-server/tap/async

 

Service availability

https://gea.esac.esa.int/tap-server/tap/availability

 

Sharing capability

https://gea.esac.esa.int/tap-server/tap/share

TAP+

Events capability

https://gea.esac.esa.int/tap-server/tap/event

TAP+

Notifications capability

https://gea.esac.esa.int/tap-server/tap/notification

TAP+

Jobs listing capability

https://gea.esac.esa.int/tap-server/tap/jobs

TAP+

Jobs removal capability

https://gea.esac.esa.int/tap-server/tap/deletejobs

TAP+

 

3.2. Synchronous Queries

Parameter

Value

Comments

REQUEST

doQuery

Requests to execute the provided query

LANG

ADQL

Query language

FORMAT

- votable
- votable_plain
- csv
- json
- fits

Results output format

QUERY

ADQL query

query

 

3.3. Asynchronous Queries

Parameter

Value

Comments

Same parameters as defined in 3.2 Synchronous Queries and

PHASE

run

Query job initial phase

The response header will contain the location of the job.

 

3.4. Query on an 'on-the-fly' uploaded table

Use a multipart/form-data (see IETF RFC 2388) HTTP POST

Parameter

Value

Comments

Same parameters as defined in 3.2 Synchronous Queries and

UPLOAD

query_table,param:parameter_table_name

query_table: the name of the table used in the query
parameter_table_name: HTTP parameter name that points to the table to be uploaded

parameter_table_name

file

file name that contains the table to be uploaded

 

For instance, in the following request:

curl --form UPLOAD="table_c,param:table1" --form table1=@test_ra_dec.vot --form LANG=ADQL --form REQUEST=doQuery --form QUERY="select top 5 * from tap_upload.table_c" https://gea.esac.esa.int/tap-server/tap/sync

table_c is the name of the table used in the query: QUERY="select top 5 * from tap_upload.table_c",
table1 is the name of the HTTP parameter that provides the file: table1=@test_ra_dec.vot and test_ra_dec.vot is the file name that contains the table to be uploaded.

 

3.5. TAP+ login

Parameter

Value

Comments

username

user_name

User name

password

user_password

User password

The response header will contain the session identifier.

 

3.6. TAP+ logout

Parameter

Value

Comments

session identifier

session identifier

Session identifier provided by a login request
Must be added to the HTTP header

 

3.7. TAP+ upload user table

Use a multipart/form-data (see IETF RFC 2388) HTTP POST

Parameter

Value

Comments

session identifier

session identifier

Session identifier provided by a login request
Must be added to the HTTP header

FILE

file

file name that contains the table to be uploaded

TABLE_NAME

table name

table name associated to the uploaded table

TABLE_DESC

table description

table description associated to the uploaded table

FORMAT

-VOTable (see file format)
-CSV (see file format)
-ASCII (see file format)

table data format
(Default: VOTable)

 

3.8. TAP+ delete user table

Parameter

Value

Comments

session identifier

session identifier

Session identifier provided by a login request
Must be added to the HTTP header

TABLE_NAME

table name

table name associated to the uploaded table

DELETE

TRUE

requested action

FORCE_REMOVAL

TRUE

To force a table removal
(Use this flag to remove the table ignoring any dependency on the table to be removed)

 

3.9. TAP+ sharing capability

3.9.1. Tables list

'table' TAP capability allows the user to retrieve shared items information. The following parameters can be used:

Parameter

Value

Comments

session identifier

session identifier

Session identifier provided by a login request
Must be added to the HTTP header

share_info

TRUE

provides information about shared items

share_accessible

TRUE

includes tables that are accessible to the user

 
3.9.2. Shared groups

Create shared group TAP capability is required in order to allow user to share tables. The following parameters can be used:

Parameter

Value

Comments

session identifier

session identifier

Session identifier provided by a login request
Must be added to the HTTP header

action

CreateOrUpdateGroup

Request to create or update a group

group_id

Group identifier

If not provided: a new group is created and its identifier is returned as part of the response.
If provided: an update action is requested.

title

Group title

Mandatory.

description

Group description

Optional.

user_list

List of user identifiers

Optional.
Updates the users that belong to this group.

To remove a shared group the following parameters must be used:

Parameter

Value

Comments

session identifier

session identifier

Session identifier provided by a login request
Must be added to the HTTP header

action

RemoveGroup

Request to remove a group

group_id

Group identifier

Mandatory.

 
3.9.3. Shared items

Create shared item TAP capability allows the user to share a table to one or more groups (user groups). The following parameters can be used:

Parameter

Value

Comments

session identifier

session identifier

Session identifier provided by a login request
Must be added to the HTTP header

action

CreateOrUpdateItem

Request to create or update a group

resource_id

Shared resource identifier

If not provided: a new shared resource is created and its identifier is returned as part of the response.
If provided: an update action is requested.

resource_type

0

Mandatory.

title

Full qualified table name

Mandatory.

description

Shared table description

Optional.

items_list

List of groups identifiers that share this item

Mandatory.
Updates the groups that can share this item. Each record composed of:
group_identifier|Group|Read

To remove a shared table the following parameters must be used:

Parameter

Value

Comments

session identifier

session identifier

Session identifier provided by a login request
Must be added to the HTTP header

action

RemoveItem

Request to remove a shared item

resource_id

Shared item identifier

Mandatory.

resource_type

0

Mandatory.

 
3.9.4. Sharing: other actions

Other Shared TAP+ capabilities are:

Parameter

Value

Comments

session identifier

session identifier

Session identifier provided by a login request
Must be added to the HTTP header

action

GetGroups

Returns a list of groups where the user belongs to or created by the user

GetGroupsDescriptionsOnly

Returns a list of groups where the user belongs to or created by the user (only descriptions).

GetSharedItems

Returns a list of items shared by the user.

GetSharedItemsDescriptionsOnly

Returns a list of items shared by the user (only descriptions).

include_users

'true' or 'false'

Valid for 'GetGroups' action only.
Default value: 'true'

groups_list

list of groups ids
(comma separated values)

Valid for 'GetGroups' action only.

 

3.10. TAP+ tables capabilities

In addition to the standar 'tables' TAP capability, the following parameters can be used too:

Parameter

Value

Comments

tables

comma separated full qualified table names

A lis of the specified tables will be returned

schemas

comma separated schema names

A list of the specified schemas will be returned

only_tables

TRUE / FALSE (default: FALSE)

TRUE: no columns information will be returned

only_schemas

TRUE / FALSE (default: FALSE)

TRUE: no tables nor columns information will be returned

These parameters are handled based on the following priorities

Priority

Parameter

Comments

1.

tables != null

No more checks are performed (share_info and share_accessible are handled if present)

2.

tables == null (default)

More checks are performed

2.1.

only_schemas = TRUE

No more checks are performed

2.2.

only_schemas = FALSE (default)

The following checks are performed

2.2.1.

schema_names != null

The following parameters are applied to the specified schemas only

2.2.2

only_tables = TRUE

No columns data are generated

2.2.3

only_tables = FALSE (default)

Columns data are generated

 

3.11. TAP+ events capabilities

Tables creation/removal/sharing actions generate events. Events are grouped by types and each type has a 'last modification time' associated. Those events and times can be retrieved using:

Parameter

Value

Comments

id

event type identifier

Optional parameter. If not provided, all events are returned. If provided, only the requested type is returned.

The current event types are:

Value

Comments

100

Job created

101

Job updated

102

Job removed

210

Shared item created

211

Shared item updated

212

Shared item removed

220

Shared group created

221

Shared group updated

222

Shared group removed

230

Shared user created

231

Shared user updated

232

Shared user removed

300

Log in

301

Log out

401

Database quota updated

402

File quota updated

500

Notification created

501

Notification removed

4000

Table created

4001

Table updated

4002

Table removed

 

3.12. TAP+ notifications

Notifications are kept for one month. Each notification provides a message that explains the notification. Notifications are available for authenticated users only.

Parameter

Value

Comments

session identifier

session identifier

Session identifier provided by a login request
Must be added to the HTTP header

action

GetNotifications

Notificaions associated to the user

 

3.13. Table flags editing

Parameter

Value

Comments

session identifier

session identifier

Session identifier provided by a login request
Must be added to the HTTP header

ACTION

radec

Requests a Ra/Dec update

TABLE_NAME

Full qualified table name

 

RA

ra column name

 

DEC

dec column name

 

 

3.14. TAP+ jobs capabilities

Parameter

Value

Comments

session identifier

session identifier

Session identifier provided by a login request
Must be added to the HTTP header

list

job list identifier
(e.g. 'sync'/'async')

Mandatory

offset

Number of jobs to skip

Optional. '0' by default.

limit

Number of jobs to return

Optinal. No limit by default.

order

List of column order records.

Optional.
Each record is composed of:
column_name:ASC|DESC

 

3.15. TAP+ deletejobs capability

Parameter

Value

Comments

session identifier

session identifier

Session identifier provided by a login request
Must be added to the HTTP header

JOB_IDS

comma separated job identifiers

Mandatory

 

3.16. TAP+ external TAP access capability

Synchronous mode

Parameter

Value

Comments

Same parameters as defined in 3.2 Synchronous Queries and:

REQUEST

external_tap

Request an external TAP access

EXT_TAP_URL

URL

URL of the external TAP service

Asynchronous mode

Parameter

Value

Comments

Same parameters as defined in 3.3 Asynchronous Queries and:

REQUEST

external_tap

Request an external TAP access

EXT_TAP_URL

URL

URL of the external TAP service

The response header will contain the location of the job.

 

3.17. Global Tap Schema (GloTS) search capability

Search by table or description

Parameter

Value

Comments

ACTION

search_table

 

KEYWORD

text

Descriptons or table names

MODE

-all (default)
-table_name
-table_description

-Search in both: table names and descriptions
-Search in table names only
-Search in descriptions only

FORMAT

-votable (default)
-csv

Results output formt

Retrieve table columns

Parameter

Value

Comments

ACTION

get_columns

 

IVOID

text

Service descriptor identifier

TABLE_NAME

table name

 

FORMAT

-votable (default)
-csv

Results output formt