Help support

Should you have any question, please check the Gaia FAQ section or contact the Gaia Helpdesk

Python Access: Astroquery

 

ESA Gaia Archive provides a Python Astroquery (Astropy) package for easy access to the ESA Gaia Archive: astroquery.gaiaThe latest Astroquery version can be instaled following the instructions given in the "How can I install the latest version of Astroquery.Gaia" FAQ.

Also, a generic package is provided for accessing any TAP and TAP+ compliant service: astroquery.utils.tap.

See a Python tutorial for Jupyter Notebook example here.

Command line access: TAP/TAP+

 

The entry point is a TAP (Table Access Protocol) server. You may use HTTP protocol to execute TAP requests at https://gea.esac.esa.int/tap-server/tap. TAP provides two operation modes: 

  • Synchronous: the response to the request will be generated as soon as the request received by the server.
  • Asynchronous: the server will start a job that will execute the request. The first response to the request is the required information (a link) to obtain the job status. Once the job is finished, the results can be retrieved.

Our TAP server provides two access modes: 

  • Public: this is the standard TAP access. A user can execute ADQL queries and upload tables to be used in a query 'on-the-fly' (these tables will be removed once the query is executed). The results are available to any other user and they will remain in the server for a limited space of time.
  • Authenticated: some functionalities are restricted to authenticated users only. The results are saved in a private user space and they will remain in the server for ever (they can be removed by the user).
    • ADQL queries and results are saved in a user private area.
    • Built-in cross-match: a catalogue cross-match operation can be executed. Cross-match operations results are saved in a user private area.
    • Persintance of uploaded tables: a user can upload a table in a private space. These tables can be used in queries as well as in cross-matches operations.

Here you can find some examples about how to interact with a TAP server (we are using the curl tool):

1. Non authenticated access
2. Authenticated access
3. Interface

 

1. Non authenticated access

Content:

1.1. Getting all public tables​​​​
1.2. Synchronous query
1.3. Synchronous query on an 'on-the-fly' uploaded table
1.4. Asynchronous query
1.5. Python low-level script to execute an asynchronous query and wait until the results are available.

 

1.1. Getting all public tables

curl "https://gea.esac.esa.int/tap-server/tap/tables"	

1.2. Synchronous query

curl "https://gea.esac.esa.int/tap-server/tap/sync?REQUEST=doQuery&LANG=ADQL&FORMAT=votable&QUERY=SELECT+TOP+5+source_id,ra,dec+FROM+gaiadr1.gaia_source"

The retrieved results is a VO table by default (see '3.2. Synchronous Queries' section parameters to specify a different output format). The results can be saved in a file and inspected using any analysis tool like TOPCAT, for instance.

1.3. Synchronous query on an 'on-the-fly' uploaded table

curl --form UPLOAD="table_c,param:table1" --form table1=@test_ra_dec.vot --form LANG=ADQL --form REQUEST=doQuery --form QUERY="select top 5 * from tap_upload.table_c" https://gea.esac.esa.int/tap-server/tap/sync			

Where 'test_ra_dec.vot' is a file that contains the VOTable to be uploaded (in order to be used by the query)

The retrieved results is a VO table by default (see '3.2. Synchronous Queries' section parameters to specify a different output format). The results can be saved in a file and inspected using any analysis tool like TOPCAT, for instance.

1.4. Asynchronous query

curl -i -X POST --data "PHASE=run&LANG=ADQL&LANG=ADQL&REQUEST=doQuery&QUERY=select+top+5+*+from+gaiadr1.gaia_source" "https://gea.esac.esa.int/tap-server/tap/async"

Note that there is the possibility to use the optional parameters "JOBNAME" to assign a name to the job and "JOBDESCRIPTION" to add a description:

curl -i -X POST --data "PHASE=run&LANG=ADQL&JOBNAME=optionalJobName&JOBDESCRIPTION=optionalDescription&LANG=ADQL&REQUEST=doQuery&QUERY=select+top+5+*+from+gaiadr1.gaia_source" "https://gea.esac.esa.int/tap-server/tap/async"			

The response will contain the URL of the job running at server side (see Location header):

HTTP/1.1 303 See Other
Date: Mon, 30 Jun 2014 14:44:39 GMT
Server: Apache-Coyote/1.1
Location: https://gea.esac.esa.int/tap-server/tap/async/1404139480755A
Content-Type: application/x-www-form-urlencoded
Connection: close
Transfer-Encoding: chunked			

To obtain the status of the running job:

curl "https://gea.esac.esa.int/tap-server/tap/async/1404139480755A"

The status response is something like:

<?xml version="1.0" encoding="UTF-8"?>
<uws:job xmlns:uws="http://www.ivoa.net/xml/UWS/v1.0" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
        <uws:jobId><![CDATA[1404139480755A]]></uws:jobId>
        <uws:runId xsi:nil="true" />
        <uws:ownerId><![CDATA[anonymous]]></uws:ownerId>
        <uws:phase>COMPLETED</uws:phase>
        <uws:quote xsi:nil="true" />
        <uws:startTime>2014-06-30T16:44:40.766+0200</uws:startTime>
        <uws:endTime>2014-06-30T16:44:40.830+0200</uws:endTime>
        <uws:executionDuration>0</uws:executionDuration>
        <uws:destruction>2014-07-07T16:44:40.754+0200</uws:destruction>
        <uws:parameters>
                <uws:parameter id="maxRec"><![CDATA[100000]]></uws:parameter>
                <uws:parameter id="query"><![CDATA[select top 5 * from gaiadr1.gaia_source]]></uws:parameter>
                <uws:parameter id="request"><![CDATA[doQuery]]></uws:parameter>
                <uws:parameter id="format"><![CDATA[votable]]></uws:parameter>
                <uws:parameter id="keepAuthenticatedUserJobs"><![CDATA[true]]></uws:parameter>
                <uws:parameter id="lang"><![CDATA[ADQL]]></uws:parameter>
                <uws:parameter id="version"><![CDATA[1.0]]></uws:parameter>
        </uws:parameters>
        <uws:results>
                <uws:result id="result" xlink:type="simple" xlink:href="http%3A%2F%2Fgea.esac.esa.int%2Ftap-server%2Ftap%2Fasync%2F1404139480755A%2Fresults%2Fresult" mime="application/x-votable+xml" size="8863" rows="6" />
        </uws:results>
        <uws:errorSummary xsi:nil="true" />

To obtain the results of the job (once the job is finished):

curl "https://gea.esac.esa.int/tap-server/tap/async/1404139480755A/results/result"

The retrieved results is a VO table by default (see '3.2 Synchronous Queries' section parameters to specify a different output format).

The results can be saved in a file and inspected using any analysis tool like TOPCAT, for instance.

1.5. Python low-level script to execute an asynchronous query and wait until the results are available.

It is also possible to access the Archive using a low-level Python script that does not require external libraries like Astroquery.

#ASYNCHRONOUS REQUEST

#Python 2
#import httplib
#import urllib

#Python 3
import http.client as httplib
import urllib.parse as urllib
import time
from xml.dom.minidom import parseString

host = "gea.esac.esa.int"
port = 443
pathinfo = "/tap-server/tap/async"


#-------------------------------------
#Create job

params = urllib.urlencode({\
	"REQUEST": "doQuery", \
	"LANG":    "ADQL", \
	"FORMAT":  "votable_plain", \
	"PHASE":  "RUN", \
	"JOBNAME":  "Any name (optional)", \
	"JOBDESCRIPTION":  "Any description (optional)", \
	"QUERY":   "SELECT DISTANCE(POINT('ICRS',ra,dec), POINT('ICRS',266.41683,-29.00781)) AS dist, * FROM gaiadr1.gaia_source  WHERE 1=CONTAINS(POINT('ICRS',ra,dec),CIRCLE('ICRS',266.41683,-29.00781, 0.08333333)) ORDER BY dist ASC"
	})

headers = {\
	"Content-type": "application/x-www-form-urlencoded", \
	"Accept":       "text/plain" \
	}

connection = httplib.HTTPSConnection(host, port)
connection.request("POST",pathinfo,params,headers)

#Status
response = connection.getresponse()
print ("Status: " +str(response.status), "Reason: " + str(response.reason))

#Server job location (URL)
location = response.getheader("location")
print ("Location: " + location)

#Jobid
jobid = location[location.rfind('/')+1:]
print ("Job id: " + jobid)

connection.close()

#-------------------------------------
#Check job status, wait until finished

while True:
	connection = httplib.HTTPSConnection(host, port)
	connection.request("GET",pathinfo+"/"+jobid)
	response = connection.getresponse()
	data = response.read()
	#XML response: parse it to obtain the current status
	#(you may use pathinfo/jobid/phase entry point to avoid XML parsing)
	dom = parseString(data)
	phaseElement = dom.getElementsByTagName('uws:phase')[0]
	phaseValueElement = phaseElement.firstChild
	phase = phaseValueElement.toxml()
	print ("Status: " + phase)
	#Check finished
	if phase == 'COMPLETED': break
	#wait and repeat
	time.sleep(0.2)


connection.close()

#-------------------------------------
#Get results
connection = httplib.HTTPSConnection(host, port)
connection.request("GET",pathinfo+"/"+jobid+"/results/result")
response = connection.getresponse()
data = response.read().decode('iso-8859-1')
#print(type(data))
#print(data)
outputFileName = "example_votable_output.vot"
outputFile = open(outputFileName, "w")
outputFile.write(data)
outputFile.close()
connection.close()
print ("Data saved in: " + outputFileName)	

The saved file is a VO table (by default, see '3.3 Asynchronous Queries' section parameters to specify a different output format). The file can be inspected using any analysis tool like TOPCAT, for instance.

 

2. Authenticated access

Content:

2.1. Login
2.2. Logout
2.3. Getting public and user tables
2.4. Upload user table
2.5. Delete user table
2.6. Asynchronous query
2.7. Shared tables
2.8. Share capability
2.9. Events capability
2.10. Notifications capability
2.11. Table flags editing capability
2.12. Listing jobs
2.13. Deleting jobs
2.14. Queries on jobs
2.15. External TAP queries
2.16. Global Tap Schema (GloTS) search

 

2.1. Login

curl -k -c cookies.txt -X POST -d username=USERNAME -d password=PASSWORD -L "https://gea.esac.esa.int/tap-server/login"

2.2. Logout

curl -k -b cookies.txt -X POST -d -L "https://gea.esac.esa.int/tap-server/logout"

2.3. Getting public and user tables

curl -k -b cookies.txt -X POST -L "https://gea.esac.esa.int/tap-server/tap/tables"

2.4. Upload user table

curl -k -b cookies.txt -X POST -F FILE=@file.name -F TABLE_NAME=table_name "https://gea.esac.esa.int/tap-server/Upload"

Where 'file.name' is a file that contains the VOTable to be uploaded. TABLE_DESC parameter can be used to specify a table description. FORMAT parameter can be used to specify the table data format. Available formats: 'VOTable', 'CSV' and 'ASCII': see section 3.7 for more details.

2.5. Delete user table

curl -k -b cookies.txt -X POST -F TABLE_NAME=table_name -F DELETE=TRUE "https://gea.esac.esa.int/tap-server/Upload"		

Where 'table.name' is the argument used when the table was uploaded (argument TABLE_NAME=table_name). To force a table removal, use FORCE_REMOVAL=TRUE flag:

curl -k -b cookies.txt -X POST -F TABLE_NAME=table_name -F DELETE=TRUE -F FORCE_REMOVAL=TRUE "https://gea.esac.esa.int/tap-server/Upload"

2.6. Asynchronous query

curl -k -b cookies.txt -i -X POST --data "PHASE=run&LANG=ADQL&REQUEST=doQuery&QUERY=select+top+5+*+from+gaiadr1.gaia_source" "https://gea.esac.esa.int/tap-server/tap/async"
Note that there is the possibility to use the optional parameters "JOBNAME" to assign a name to the job and "JOBDESCRIPTION" to add a description:
curl -k -b cookies.txt -i -X POST --data "PHASE=run&LANG=ADQL&JOBNAME=optionalJobName&JOBDESCRIPTION=optionalDescription&REQUEST=doQuery&QUERY=select+top+5+*+from+gaiadr1.gaia_source" "https://gea.esac.esa.int/tap-server/tap/async"				

The response will contain the URL of the job running at server side:

HTTP/1.1 303 See Other
Date: Mon, 30 Jun 2014 15:02:00 GMT
Server: Apache-Coyote/1.1
Location: https://gea.esac.esa.int/tap-server/tap/async/1404140520859A
Content-Type: application/x-www-form-urlencoded
Connection: close
Transfer-Encoding: chunked

To obtain the status of the running job:

curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/async/1404140520859A"

The status response is something like:

<?xml version="1.0" encoding="UTF-8"?>
<uws:job xmlns:uws="http://www.ivoa.net/xml/UWS/v1.0" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
        <uws:jobId><![CDATA[1404141177261A]]></uws:jobId>
        <uws:runId xsi:nil="true" />
        <uws:ownerId><![CDATA[USERNAME]]></uws:ownerId>
        <uws:phase>COMPLETED</uws:phase>
        <uws:quote xsi:nil="true" />
        <uws:startTime>2014-06-30T17:12:57.273+0200</uws:startTime>
        <uws:endTime>2014-06-30T17:12:57.864+0200</uws:endTime>
        <uws:executionDuration>0</uws:executionDuration>
        <uws:destruction>2014-07-07T17:23:31.362+0200</uws:destruction>
        <uws:parameters>
                <uws:parameter id="jobdescription"><![CDATA[]]></uws:parameter>
                <uws:parameter id="jobname"><![CDATA[]]></uws:parameter>
                <uws:parameter id="session"><![CDATA[1404141103551]]></uws:parameter>
                <uws:parameter id="maxRec"><![CDATA[100000]]></uws:parameter>
                <uws:parameter id="query"><![CDATA[SELECT DISTANCE(POINT('ICRS',ra,dec), POINT('ICRS',266.41683,-29.00781)) AS dist, *
FROM gaiadr1.gaia_source
WHERE 1=CONTAINS(POINT('ICRS',ra,dec),CIRCLE('ICRS',266.41683,-29.00781, 0.08333333)) ORDER BY dist ASC]]></uws:parameter>
                <uws:parameter id="request"><![CDATA[doQuery]]></uws:parameter>
                <uws:parameter id="keepAuthenticatedUserJobs"><![CDATA[true]]></uws:parameter>
                <uws:parameter id="format"><![CDATA[votable]]></uws:parameter>
                <uws:parameter id="lang"><![CDATA[ADQL]]></uws:parameter>
                <uws:parameter id="version"><![CDATA[1.0]]></uws:parameter>
        </uws:parameters>
        <uws:results>
                <uws:result id="result" xlink:type="simple" xlink:href="http%3A%2F%2Fgea.esac.esa.int%2Ftap-server%2Ftap%2Fasync%2F1404141177261A%2Fresults%2Fresult" mime="application/x-votable+xml" size="2468741" rows="4734" />
        </uws:results>
        <uws:errorSummary xsi:nil="true" />

To obtain the results of the job (once the job is finished) and store them locally in a file:

curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/async/1404140520859A/results/result" >query_out.vot.gzip

 

By default, the downloaded product is a compressed (.gzip) file in VOTable format. Other output formats are also possible (see '3.3 Asynchronous Queries' section parameters to specify a different output format). The following example shows how to retrieve the results in .fits format:

curl -k -b cookies.txt -i -X POST --data "PHASE=run&LANG=ADQL&REQUEST=doQuery&QUERY=select+top+5+*+from+gaiadr1.gaia_source&FORMAT=fits" "https://gea.esac.esa.int/tap-server/tap/async"

 

curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/async/<job_id>/results/result" >query_out.fits.gzip

 

2.7. Shared tables

Shared tables are accessible once the user is authenticated. You can use the tables in the same way you use your own tables. For instance, if you want to get the first 5 rows of the table 'shared_table' from user 'user_joe' (i.e. 'user_joe._shared_table') in a synchronous mode, you can type:

curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/sync?REQUEST=doQuery&LANG=ADQL&FORMAT=votable&QUERY=SELECT+TOP+5+*+FROM+user_joe.shared_table"

in asynchronous mode:

curl -k -b cookies.txt -i -X POST --data "PHASE=run&LANG=ADQL&REQUEST=doQuery&QUERY=select+top+5+*+from+user_joe.shared_table" "https://gea.esac.esa.int/tap-server/tap/async"

If you want to obtain a list of available tables and columns, you can type:

curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/tables?share_info=true&share_accessible=true"					

If you want to obtain a list of available tables, you can type:

curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/tables?share_info=true&share_accessible=true&only_tables=true"

2.8. Share capability

You have to create a shared group in order to share a table.
The following example creates a group named 'new_group' and adds two users ('user_id_1' and 'user_id_2') to it:

curl -k -b cookies.txt -X POST --data "action=CreateOrUpdateGroup&title=my_new_group&description=descripion&users_list=user_id1,user_id2" "https://gea.esac.esa.int/tap-server/tap/share"

The response will contain the group identifier.

OK
Group '<user_id>_1425480932331JC' successfully created.

Then, you can update a group (to add/remove users to a group, see 'Adding/Removing a user' below):

curl -k -b cookies.txt -X POST --data "action=CreateOrUpdateGroup&group_id=<group_identifier>&title=my_new_group&description=descripion&users_list=<new_users_list>" "https://gea.esac.esa.int/tap-server/tap/share"

To obtain a list of groups associated to the user (either the user is the owner or the user belongs to):

curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/share?action=GetGroups"

The response:

<?xml version="1.0" encoding="UTF-8"?>
<sharedGroups>
<sharedGroup id="user_id_1425482038968JC" owner="user_id">
	<title><![CDATA[my_new_group]]></title>
	<description><![CDATA[descripion]]></description>
</sharedGroup>
<sharedGroup id="other_user_1417615009193I" owner="other_user">
	<title><![CDATA[gacs]]></title>
	<description><![CDATA[group of gacs developers]]></description>
</sharedGroup>
</sharedGroups>

In this case, the user 'user_id' is the owner of the group 'user_id_1425482038968JC', while the user 'other_user' owns 'other_user_1417615009193I' group.

To remove a group:

curl -k -b cookies.txt -X POST --data "action=RemoveGroup&group_id=<group_identifier>" "https://gea.esac.esa.int/tap-server/tap/share"

Adding a user:

curl -k -b cookies.txt -X POST --data "action=CreateUserGroup&group_id=<group_identifier>&user_id=<user_id>" "https://gea.esac.esa.int/tap-server/tap/share"

Removing a user:

curl -k -b cookies.txt -X POST --data "action=RemoveUserGroup&group_id=<group_identifier>&user_id=<user_id>" "https://gea.esac.esa.int/tap-server/tap/share"

To share a table named 'user_schema.table1' (full qualified table name) you need to create a shared item. In this example, a new item is created and shared to 'my_new_group' group:

curl -k -b cookies.txt -X POST --data "action=CreateOrUpdateItem&resource_type=0&title=user_schema.table1&description=description&items_list=group_id|Group|Read" "https://gea.esac.esa.int/tap-server/tap/share"

The response will contain the shared resource identifier.

OK
Resource '<user_id>_1425542442398JC' (type '0') successfully created.

Then, you can update a shared table:

curl -k -b cookies.txt -X POST --data "action=CreateOrUpdateItem&resource_id=<resource_identifier>&resource_type=0&title=user_schema.table1&description=description&items_list=group_id|Group|Read" "https://gea.esac.esa.int/tap-server/tap/share"	

To obtain a list of shared tables:

curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/share?action=GetSharedItems"						

The response

<?xml version="1.0" encoding="UTF-8"?>
<sharedItems owner="user_id">
<sharedItem id="user_id_1425485873983JC" type="0">
	<title><![CDATA[user_schema.table]]></title>
	<description><![CDATA[description]]></description>
	<sharedToItems>
<sharedToItem shareTo="group_id_1425482038968JC" shareType="Group" shareMode="Read"/>
	</sharedToItems>
</sharedItem>
</sharedItems>

To remove a sharing (it does not remove the actual table):

curl -k -b cookies.txt -X POST --data "action=RemoveItem&resource_id=<resource_identifier>&resource_type=0" "https://gea.esac.esa.int/tap-server/tap/share"

2.9. Events capability

Each action (table creation/removal, sharing creation/removal, etc.) generates an event. Events are associated to a session and they are kept while the session (connecton) is open.
Events are grouped by types and each time has a 'latest modification time' (latest time that this type of event was generated).
The following example retrieves the events for a user:

curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/event"

The response will contain the events and times associated.

event_type1=timestamp1
event_type2=timestamp2
...	

2.10. Notifications capability

Notifications provides information about some activities. Notifications are available for authenticated users only and are kept for a month.
The following example retrieves the notifications for a user:

curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/notification?action=GetNotifications"

A response example:

notification_id1[type: notification_type1,notification_subtype1]=explanation1
notification_id2[type: notification_type2,notification_subtype2]=explanation2
...	

2.11. Table flags editing capability

Tables flags can be modified.
The following example explains how to set the flags of a table:

curl -k -b cookies.txt -X POST --data "ACTION=radec&TABLE_NAME=schema.table_name&RA=ra_column_name&DEC=dec_column_name" "https://gea.esac.esa.int/tap-server/TableTool"

Where schema.table_name is a full qualified table name (e.g. user_userid.table_name). ra_column specifies the ra column name. dec_column specifies the dec column name.

2.12. Listing jobs

Jobs can be listed using the following request:

curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/jobs/list?offset=index&limit=jobs_number&order=order"

Where list specifies the jobs list (e.g. 'sync' or 'async'). It is mandatory.
index specifies the number of jobs to skip before beginning to return the first job. By default, it is '0'.
limit specifies the number of jobs to be returned. No limit by default is set.
order specifies the order of the results.

Example:

curl -k -b cookies.txt "https://gea.esac.esa.int/tap-server/tap/jobs/async?offset=0&limit=20&order=CREATION_TIME:DESC"

2.13. Deleting jobs

Jobs can be deleted using their identifiers:

curl -k -b cookies.txt -X POST --data "JOB_IDS=job_id1,job_id2..." "https://gea.esac.esa.int/tap-server/tap/deletejobs"		

2.14. Queries on jobs

A job can be used as a table in a query. Launch an asynchronous job:

curl -i -X POST --data "PHASE=run&LANG=ADQL&LANG=ADQL&REQUEST=doQuery&QUERY=select+top+5+*+from+gaiadr1.gaia_source" "https://gea.esac.esa.int/tap-server/tap/async"

Response:

HTTP/1.1 303 See Other
Date: Mon, 30 Jun 2014 14:44:39 GMT
Server: Apache-Coyote/1.1
Location: https://gea.esac.esa.int/tap-server/tap/async/1404139480755A
Content-Type: application/x-www-form-urlencoded
Connection: close
Transfer-Encoding: chunked

Use the job identifier (1404139480755A in this case) as a table:

curl "https://gea.esac.esa.int/tap-server/tap/sync?REQUEST=doQuery&LANG=ADQL&FORMAT=votable&QUERY=SELECT+TOP+5+source_id,ra,dec+FROM+JOB_UPLOAD.JOB1404139480755A"

NOTE always add 'JOB' to your job identifier, e.g. JOB1404139480755A

2.15. External TAP queries

A query can be executed in an external TAP server. When an external query is executed, an asynchronous job is launched to the remote TAP server. (See '3.15. TAP+ external TAP access capability' section parameters)

In synchronous mode, the results are served on the fly (in this example, we are accessing Herschel TAP service).

curl "https://gea.esac.esa.int/tap-server/tap/sync?REQUEST=external_tap&EXT_TAP_URL=http://archives.esac.esa.int/hsa/whsa-tap-server/tap&LANG=ADQL&FORMAT=votable_plain&QUERY=SELECT+TOP+5+observation_id,ra,dec+FROM+hsa.v_active_observation"

In asynchronous mode, the results are saved locally (in this example, we access to Vizier TAP service)

curl -i -X POST --data "REQUEST=external_tap&EXT_TAP_URL=http://tapvizier.u-strasbg.fr/TAPVizieR/tap&PHASE=run&LANG=ADQL&LANG=ADQL&QUERY=select+top+5+recno,lambda,flux+from+%22J/A%2BA/605/L8/quasar_a%22" "https://gea.esac.esa.int/tap-server/tap/async"

Response:

HTTP/1.1 303 See Other
Date: Mon, 30 Jun 2014 14:44:39 GMT
Server: Apache-Coyote/1.1
Location: https://gea.esac.esa.int/tap-server/tap/async/1404139480755A
Content-Type: application/x-www-form-urlencoded
Connection: close
Transfer-Encoding: chunked

Now, you have a local job (1404139480755A in this case) with the results of your external TAP query.

2.16. Global Tap Schema (GloTS) search

A global tap schema search service is available.

Search for services that contains a table name or description.

curl "https://gea.esac.esa.int/tap-server/tap/glots?ACTION=search_tables&keyword=quasars+new&MODE=all&FORMAT=csv"

In this case, we are searching for the keywords 'quasars new'

Response:

accessurl,ivoid,table_name,table_desc,utype
http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,J/A+AS/134/483/table3,"New QSOs from the Hamburg Quasar Survey Object ( Hagen H.-J., Engels D., Reimers D.)",
http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,J/AJ/113/2000/table1,"New quasar radio detections ( Bischof O.B., Becker R.H.)",
http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,J/AJ/122/518/table1,"New LBQS quasars ( Hewett P.C., Foltz C.B., Chaffee F.H.)",
http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,J/AJ/142/78/table1,"Parameters of 14 new 2.2<z<3.0 quasars in S82 ( Wu X.-B., Wang R., Schmidt K.B., Bian F., Jiang L., Fan X.)",
http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,J/AJ/145/159/table1,"Catalog of new quasars in the vicinity of M31 and M33 discovered by LAMOST in fields observed in 2010 and 2011 ( Huo Z.-Y., Liu X.-W., Xiang M.-S., et al.)",
http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,J/AJ/151/24/table1,"Parameters of the 1180 New Quasars Discovered in LAMOST DR1 ( Ai Y.L., Wu X.-B., Yang J., Yang Q., Wang F., Guo R., Zuo W., Dong X., Zhang Y.-X., Yuan H.-L., Song Y.-H., Wang J., Dong X., Yang M., Wu H., Shen S.-Y., Shi J.-R., He B.-L., Lei Y.-J., Li Y.-B., Luo A.-L., Zhao Y.-H., Zhang H.-T.)",
http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,J/ApJS/194/22/table1,"29 new, 12 plausible, and 3 previously known quasars behind the SMC ( Kozlowski S., Kochanek C. S., Udalski A.)",
http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,J/ApJS/227/11/table5,"Spectroscopic observations of the new PS1 quasars (77 unique sources) ( Banados E., Venemans B.P., Decarli R., et al.)",
http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,J/other/RAA/15.1438/table1,"Catalog of new quasars in fields in the vicinity of M31 and M33 discovered by LAMOST in the 2013 observational season ( Huo Z.-Y., Liu X.-W., Xiang M.-S., Shi J.-R., Yuan H.-B., Huang Y., Zhang Y., Hou Y.-H., Wang Y.-F., Yang M.)",
http://tapvizier.u-strasbg.fr/TAPVizieR/tap,ivo://cds.vizier/tap,VII/182/table6,"new UV excess quasars (paper II) ( Warren S.J., Hewett P.C., Irwin M.J., Osmer P.S.)", 						

In order to retrieve all columns of a table, we have to provide the 'ivoid' service identifier and the table name:

curl "https://gea.esac.esa.int/tap-server/tap/glots?ACTION=get_columns&IVOID=ivo%3A%2F%2Fcds.vizier%2Ftap&TABLE_NAME=J%2FA%2BAS%2F134%2F483%2Ftable3&FORMAT=csv"

Response:

ivoid,table_name,column_name,description,unit,ucd,utype,datatype,ndhnpmiighpa,principal,indexed,std
ivo://cds.vizier/tap,J/A+AS/134/483/table3,Bmag,B magnitude,mag,phot.mag;em.opt.B, ,DOUBLE,mt.size,0,0,1
ivo://cds.vizier/tap,J/A+AS/134/483/table3,DE2000,Declination (2000.0),deg,pos.eq.dec;meta.main, ,DOUBLE,mt.size,1,0,1
ivo://cds.vizier/tap,J/A+AS/134/483/table3,Name,Object name, ,meta.id;meta.main, ,VARCHAR,mt.size,0,0,1
ivo://cds.vizier/tap,J/A+AS/134/483/table3,RA2000,Right ascension (2000.0),deg,pos.eq.ra;meta.main, ,DOUBLE,mt.size,1,0,1
ivo://cds.vizier/tap,J/A+AS/134/483/table3,Run,Run number, ,meta.id;obs, ,SMALLINT,mt.size,0,0,1
ivo://cds.vizier/tap,J/A+AS/134/483/table3,l_Bmag,[~] Approximate flag on Bmag, ,meta.code.error, ,CHAR(1),mt.size,0,0,1
ivo://cds.vizier/tap,J/A+AS/134/483/table3,recno,Record number assigned by the VizieR team. Should Not be used for identification., ,meta.record, ,INTEGER,mt.size,0,0,1
ivo://cds.vizier/tap,J/A+AS/134/483/table3,z,Redshift, ,src.redshift, ,DOUBLE,mt.size,0,0,1

 

3. Interface

See the following specifications:

Content:

3.1. TAP resources
3.2. Synchronous Queries
3.3. Asynchronous Queries
3.4. Query on an 'on-the-fly' uploaded table
3.5. TAP+ login
3.6. TAP+ logout
3.7. TAP+ upload user table
3.8. TAP+ delete user table
3.9. TAP+ sharing capability
3.10. TAP+ tables capabilities
3.11. TAP+ events capabilities
3.12. TAP+ notifications
3.13. Table flags editing
3.14. TAP+ jobs capabilities
3.15. TAP+ deletejobs capability
3.16. TAP+ external TAP access capability
3.17. Global Tap Schema (GloTS) search capability

 

3.1. TAP resources

https://gea.esac.esa.int/tap-server/tap/

Tables

https://gea.esac.esa.int/tap-server/tap/tables

 

Synchronous access

https://gea.esac.esa.int/tap-server/tap/sync

 

Asynchronous access

https://gea.esac.esa.int/tap-server/tap/async

 

Service availability

https://gea.esac.esa.int/tap-server/tap/availability

 

Sharing capability

https://gea.esac.esa.int/tap-server/tap/share

TAP+

Events capability

https://gea.esac.esa.int/tap-server/tap/event

TAP+

Notifications capability

https://gea.esac.esa.int/tap-server/tap/notification

TAP+

Jobs listing capability

https://gea.esac.esa.int/tap-server/tap/jobs

TAP+

Jobs removal capability

https://gea.esac.esa.int/tap-server/tap/deletejobs

TAP+

 

3.2. Synchronous Queries

Parameter

Value

Comments

REQUEST

doQuery

Requests to execute the provided query

LANG

ADQL

Query language

FORMAT

- votable
- votable_plain
- csv
- json
- fits

Results output format

QUERY

ADQL query

query

 

3.3. Asynchronous Queries

Parameter

Value

Comments

Same parameters as defined in 3.2 Synchronous Queries and

PHASE

run

Query job initial phase

The response header will contain the location of the job.

 

3.4. Query on an 'on-the-fly' uploaded table

Use a multipart/form-data (see IETF RFC 2388) HTTP POST

Parameter

Value

Comments

Same parameters as defined in 3.2 Synchronous Queries and

UPLOAD

query_table,param:parameter_table_name

query_table: the name of the table used in the query
parameter_table_name: HTTP parameter name that points to the table to be uploaded

parameter_table_name

file

file name that contains the table to be uploaded

 

For instance, in the following request:

curl --form UPLOAD="table_c,param:table1" --form table1=@test_ra_dec.vot --form LANG=ADQL --form REQUEST=doQuery --form QUERY="select top 5 * from tap_upload.table_c" https://gea.esac.esa.int/tap-server/tap/sync

table_c is the name of the table used in the query: QUERY="select top 5 * from tap_upload.table_c",
table1 is the name of the HTTP parameter that provides the file: table1=@test_ra_dec.vot and test_ra_dec.vot is the file name that contains the table to be uploaded.

3.5. TAP+ login

Parameter

Value

Comments

username

user_name

User name

password

user_password

User password

The response header will contain the session identifier.

 

3.6. TAP+ logout

Parameter

Value

Comments

session identifier

session identifier

Session identifier provided by a login request
Must be added to the HTTP header

 

3.7. TAP+ upload user table

Use a multipart/form-data (see IETF RFC 2388) HTTP POST

Parameter

Value

Comments

session identifier

session identifier

Session identifier provided by a login request
Must be added to the HTTP header

FILE

file

file name that contains the table to be uploaded

TABLE_NAME

table name

table name associated to the uploaded table

TABLE_DESC

table description

table description associated to the uploaded table

FORMAT

-VOTable (see file format)
-CSV (see file format)
-ASCII (see file format)

table data format
(Default: VOTable)

 

3.8. TAP+ delete user table

Parameter

Value

Comments

session identifier

session identifier

Session identifier provided by a login request
Must be added to the HTTP header

TABLE_NAME

table name

table name associated to the uploaded table

DELETE

TRUE

requested action

FORCE_REMOVAL

TRUE

To force a table removal
(Use this flag to remove the table ignoring any dependency on the table to be removed)

3.9. TAP+ sharing capability

3.9.1. Tables list

'table' TAP capability allows the user to retrieve shared items information. The following parameters can be used:

Parameter

Value

Comments

session identifier

session identifier

Session identifier provided by a login request
Must be added to the HTTP header

share_info

TRUE

provides information about shared items

share_accessible

TRUE

includes tables that are accessible to the user

3.9.2. Shared groups

Create shared group TAP capability is required in order to allow user to share tables. The following parameters can be used:

Parameter

Value

Comments

session identifier

session identifier

Session identifier provided by a login request
Must be added to the HTTP header

action

CreateOrUpdateGroup

Request to create or update a group

group_id

Group identifier

If not provided: a new group is created and its identifier is returned as part of the response.
If provided: an update action is requested.

title

Group title

Mandatory.

description

Group description

Optional.

user_list

List of user identifiers

Optional.
Updates the users that belong to this group.

To remove a shared group the following parameters must be used:

Parameter

Value

Comments

session identifier

session identifier

Session identifier provided by a login request
Must be added to the HTTP header

action

RemoveGroup

Request to remove a group

group_id

Group identifier

Mandatory.

3.9.3. Shared items

Create shared item TAP capability allows the user to share a table to one or more groups (user groups). The following parameters can be used:

Parameter

Value

Comments

session identifier

session identifier

Session identifier provided by a login request
Must be added to the HTTP header

action

CreateOrUpdateItem

Request to create or update a group

resource_id

Shared resource identifier

If not provided: a new shared resource is created and its identifier is returned as part of the response.
If provided: an update action is requested.

resource_type

0

Mandatory.

title

Full qualified table name

Mandatory.

description

Shared table description

Optional.

items_list

List of groups identifiers that share this item

Mandatory.
Updates the groups that can share this item. Each record composed of:
group_identifier|Group|Read

To remove a shared table the following parameters must be used:

Parameter

Value

Comments

session identifier

session identifier

Session identifier provided by a login request
Must be added to the HTTP header

action

RemoveItem

Request to remove a shared item

resource_id

Shared item identifier

Mandatory.

resource_type

0

Mandatory.

3.9.4. Sharing: other actions

Other Shared TAP+ capabilities are:

Parameter

Value

Comments

session identifier

session identifier

Session identifier provided by a login request
Must be added to the HTTP header

action

GetGroups

Returns a list of groups where the user belongs to or created by the user

GetGroupsDescriptionsOnly

Returns a list of groups where the user belongs to or created by the user (only descriptions).

GetSharedItems

Returns a list of items shared by the user.

GetSharedItemsDescriptionsOnly

Returns a list of items shared by the user (only descriptions).

include_users

'true' or 'false'

Valid for 'GetGroups' action only.
Default value: 'true'

groups_list

list of groups ids
(comma separated values)

Valid for 'GetGroups' action only.

 

3.10. TAP+ tables capabilities

In addition to the standar 'tables' TAP capability, the following parameters can be used too:

Parameter

Value

Comments

tables

comma separated full qualified table names

A lis of the specified tables will be returned

schemas

comma separated schema names

A list of the specified schemas will be returned

only_tables

TRUE / FALSE (default: FALSE)

TRUE: no columns information will be returned

only_schemas

TRUE / FALSE (default: FALSE)

TRUE: no tables nor columns information will be returned

These parameters are handled based on the following priorities

Priority

Parameter

Comments

1.

tables != null

No more checks are performed (share_info and share_accessible are handled if present)

2.

tables == null (default)

More checks are performed

2.1.

only_schemas = TRUE

No more checks are performed

2.2.

only_schemas = FALSE (default)

The following checks are performed

2.2.1.

schema_names != null

The following parameters are applied to the specified schemas only

2.2.2

only_tables = TRUE

No columns data are generated

2.2.3

only_tables = FALSE (default)

Columns data are generated

 

3.11. TAP+ events capabilities

Tables creation/removal/sharing actions generate events. Events are grouped by types and each type has a 'last modification time' associated. Those events and times can be retrieved using:

Parameter

Value

Comments

id

event type identifier

Optional parameter. If not provided, all events are returned. If provided, only the requested type is returned.

The current event types are:

Value

Comments

100

Job created

101

Job updated

102

Job removed

210

Shared item created

211

Shared item updated

212

Shared item removed

220

Shared group created

221

Shared group updated

222

Shared group removed

230

Shared user created

231

Shared user updated

232

Shared user removed

300

Log in

301

Log out

401

Database quota updated

402

File quota updated

500

Notification created

501

Notification removed

4000

Table created

4001

Table updated

4002

Table removed

 

3.12. TAP+ notifications

Notifications are kept for one month. Each notification provides a message that explains the notification. Notifications are available for authenticated users only.

Parameter

Value

Comments

session identifier

session identifier

Session identifier provided by a login request
Must be added to the HTTP header

action

GetNotifications

Notificaions associated to the user

 

3.13. Table flags editing

Parameter

Value

Comments

session identifier

session identifier

Session identifier provided by a login request
Must be added to the HTTP header

ACTION

radec

Requests a Ra/Dec update

TABLE_NAME

Full qualified table name

 

RA

ra column name

 

DEC

dec column name

 

 

3.14. TAP+ jobs capabilities

Parameter

Value

Comments

session identifier

session identifier

Session identifier provided by a login request
Must be added to the HTTP header

list

job list identifier
(e.g. 'sync'/'async')

Mandatory

offset

Number of jobs to skip

Optional. '0' by default.

limit

Number of jobs to return

Optinal. No limit by default.

order

List of column order records.

Optional.
Each record is composed of:
column_name:ASC|DESC

 

3.15. TAP+ deletejobs capability

Parameter

Value

Comments

session identifier

session identifier

Session identifier provided by a login request
Must be added to the HTTP header

JOB_IDS

comma separated job identifiers

Mandatory

 

3.16. TAP+ external TAP access capability

Synchronous mode

Parameter

Value

Comments

Same parameters as defined in 3.2 Synchronous Queries and:

REQUEST

external_tap

Request an external TAP access

EXT_TAP_URL

URL

URL of the external TAP service

Asynchronous mode

Parameter

Value

Comments

Same parameters as defined in 3.3 Asynchronous Queries and:

REQUEST

external_tap

Request an external TAP access

EXT_TAP_URL

URL

URL of the external TAP service

The response header will contain the location of the job.

 

3.17. Global Tap Schema (GloTS) search capability

Search by table or description

Parameter

Value

Comments

ACTION

search_table

 

KEYWORD

text

Descriptons or table names

MODE

-all (default)
-table_name
-table_description

-Search in both: table names and descriptions
-Search in table names only
-Search in descriptions only

FORMAT

-votable (default)
-csv

Results output formt

Retrieve table columns

Parameter

Value

Comments

ACTION

get_columns

 

IVOID

text

Service descriptor identifier

TABLE_NAME

table name

 

FORMAT

-votable (default)
-csv

Results output formt

Datalink: Command line access

Authors: Jose Osinde, Héctor Cánovas, and Jos de Bruijne

Introduction

The DataLink server (see also the DataLink service tutorial) provides a service listing all the available resources per source linked at the entry point:

gea.esac.esa.int/data-server/datalink/links?

To invoke the service, the Gaia source ID and its associated catalogue release (either RELEASE=Gaia+DR2 or RELEASE=Gaia+DR3) should be passed as follows:

curl "https://gea.esac.esa.int/data-server/datalink/links?ID=6680733225618222592&RELEASE=Gaia+DR2"

Alternatively, the Gaia source designation can be used:

curl "https://gea.esac.esa.int/data-server/datalink/links?ID=Gaia+DR3+30343944744320"

It is also possible to connect as a registered user by logging into the Gaia Archive (follow this link to create a Gaia Archive user account) and storing the credentials as follows:

# Login command:
curl -k -c cookies-datalink.txt -X POST -d username=<USERNAME> -d password=<PASSWORD> -L "https://gea.esac.esa.int/data-server/login"
curl -k -b cookies-datalink.txt "https://gea.esac.esa.int/data-server/datalink/links?ID=Gaia+DR3+30343944744320"

This option may be useful for users aiming to retrieve the DataLink products associated to the sources included in their jobs and/or user_tables (see Sect. 2 in the Command line access: TAP/TAP+ tutorial, and the Upload a User table & Advanced (ADQL) tab tutorials to learn more about managing the user space). It is recommended to wait a few seconds between the execution of the two commands listed above to allow the system to process the request.

The response is always a DataLink compliant XML file with content like:

<?xml version="1.0" encoding="UTF-8"?>
<VOTABLE version="1.4" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.ivoa.net/xml/VOTable/v1.3 http://www.ivoa.net/xml/VOTable/v1.3" xmlns="http://www.ivoa.net/xml/VOTable/v1.3">
    <RESOURCE type="results">
        <INFO name="standardID" value="ivo://ivoa.net/std/DataLink#links-1.0"/>
        <INFO name="QUERY_STATUS" value="OK" />
        <INFO name="JOBID" value="1654590222600ODL"><![CDATA[1654590222600ODL]]></INFO>
        <INFO name="JOBNAME" value=""><![CDATA[Datalink job for user hcanovas]]></INFO>
        <TABLE>
        <FIELD name="ID" ucd="meta.id;meta.main" datatype="char" arraysize="*"/>
        <FIELD name="access_url" ucd="meta.ref.url" datatype="char" arraysize="*"/>
        <FIELD name="service_def" ucd="meta.ref" datatype="char" arraysize="*"/>
        <FIELD name="error_message" ucd="meta.code.error" datatype="char" arraysize="*"/>
        <FIELD name="semantics" ucd="meta.code" datatype="char" arraysize="*"/>
        <FIELD name="description" ucd="meta.note" datatype="char" arraysize="*"/>
        <FIELD name="content_type" ucd="meta.code.mime" datatype="char" arraysize="*"/>
        <FIELD name="content_length" ucd="phys.size;meta.file" datatype="long" unit="byte"/>
        <DATA>
        <TABLEDATA>
        <TR>
        <TD>Gaia DR3 30343944744320</TD>
        <TD><![CDATA[https://gea.esac.esa.int/data-server/data?ID=Gaia+DR3+30343944744320&RETRIEVAL_TYPE=MCMC_MSC]]></TD>
        <TD></TD>
        <TD></TD>
        <TD>#this</TD>
        <TD><![CDATA[MCMC MSC, source Gaia DR3 30343944744320]]></TD>
        <TD>application/x-votable+xml</TD>
        <TD></TD>
        </TR>
        <TR>
        <TD>Gaia DR3 30343944744320</TD>
        <TD><![CDATA[https://gea.esac.esa.int/data-server/data?ID=Gaia+DR3+30343944744320&RETRIEVAL_TYPE=XP_SAMPLED]]></TD>
        <TD></TD>
        <TD></TD>
        <TD>#this</TD>
        <TD><![CDATA[XP mean sampled spectra, source Gaia DR3 30343944744320]]></TD>
        <TD>application/x-votable+xml</TD>
        <TD></TD>
        </TR>
        <TR>
        <TD>Gaia DR3 30343944744320</TD>
        <TD><![CDATA[https://gea.esac.esa.int/data-server/data?ID=Gaia+DR3+30343944744320&RETRIEVAL_TYPE=XP_CONTINUOUS]]></TD>
        <TD></TD>
        <TD></TD>
        <TD>#this</TD>
        <TD><![CDATA[XP mean continuous spectra, source Gaia DR3 30343944744320]]></TD>
        <TD>application/x-votable+xml</TD>
        <TD></TD>
        </TR>
        <TR>
        <TD>Gaia DR3 30343944744320</TD>
        <TD><![CDATA[https://gea.esac.esa.int/data-server/data?ID=Gaia+DR3+30343944744320&RETRIEVAL_TYPE=MCMC_GSPPHOT]]></TD>
        <TD></TD>
        <TD></TD>
        <TD>#this</TD>
        <TD><![CDATA[MCMC GSP-Phot, source Gaia DR3 30343944744320]]></TD>
        <TD>application/x-votable+xml</TD>
        <TD></TD>
        </TR>
        <TR>
        <TD>Gaia DR3 30343944744320</TD>
        <TD><![CDATA[https://gea.esac.esa.int/data-server/data?ID=Gaia+DR3+30343944744320&RETRIEVAL_TYPE=EPOCH_PHOTOMETRY]]></TD>
        <TD></TD>
        <TD></TD>
        <TD>#this</TD>
        <TD><![CDATA[Epoch photometry, source Gaia DR3 30343944744320]]></TD>
        <TD>application/x-votable+xml</TD>
        <TD></TD>
        </TR>
        <TR>
        <TD>Gaia DR3 30343944744320</TD>
        <TD><![CDATA[https://gea.esac.esa.int/data-server/data?ID=Gaia+DR3+30343944744320&RETRIEVAL_TYPE=RVS]]></TD>
        <TD></TD>
        <TD></TD>
        <TD>#this</TD>
        <TD><![CDATA[RVS mean spectra, source Gaia DR3 30343944744320]]></TD>
        <TD>application/x-votable+xml</TD>
        <TD></TD>
        </TR>
        </TABLEDATA>
        </DATA>
        </TABLE>
    </RESOURCE>
</VOTABLE>

This file, which is intended for IVOA-compliant clients like TOPCAT, contains the links to the DataLink products available for the requested source (Gaia DR3 30343944744320). The section below shows how to retrieve the different products. For details about them, please see the Datamodel description Chapter in the Gaia DR3 Documentation and the DataLink products serialisation tutorial.

 

data retrieval

The different DataLink products can be downloaded in a variety of output formats and data structures, as described in the DataLink products serialisation tutorial. The service entry point is:

https://gea.esac.esa.int/data-server/data?RETRIEVAL_TYPE=<retrieval_type>

The table below indicates the accepted values for all available parameters.

Parameter

Value

Definition

Mandatory

*RETRIEVAL_TYPE

EPOCH_PHOTOMETRY | XP_SAMPLED | XP_CONTINUOUS | MCMC_GSPPHOT | MCMC_MSC | RVS | ALL

DataLink product(s) to be retrieved. See DataLink products serialisation tutorial for details.

YES

ID

source_id(s) | designation(s)

Unique source identifier(s). The identifier(s) can be passed in different formats (see the input format table below).

YES

RELEASE

Gaia+DR2 | Gaia+DR3

Gaia data release associated to a source_id.

*YES if ID is set as source_id.

VALID_DATA

TRUE | FALSE

The default value is TRUE, which means that the epoch photometry service returns only valid data: all data rows where the flux is not null and the rejected_by_photometry flag is not True. 

NO

BAND

G | BP | RP

By default, the epoch photometry service returns all the available photometry bands for the requested source. This parameter allows to filter the output lightcurve by its band.

NO

FORMAT

VOTABLE | VOTABLE_PLAIN | FITS | CSV | ECSV

Output file format. The default format is a binary VOTable. 

NO

DATA_STRUCTURE

INDIVIDUAL | COMBINED | RAW

Output file data structure. The default data structure is individual (one source_id per file). 

NO

 

Notes:

  • Strictly speaking, RETRIEVAL_TYPE defines the service entry point so it is not a parameter. It is included in the table for convenience.
  • When retrieving all data products (RETRIEVAL_TYPE = ALL), the output file is always a .gzip compressed file that contains the individual DataLink products saved in a COMBINED data structure.
  • The epoch photometry product accepts two extra, optional parameters that are highlighted with a green background in the table above.
  • The RELEASE parameter is not strictly mandatory when using a source_id as input ID (there are some combinations that successfully retrieve DataLink products without setting this parameter). However, we recommend to include it in the request because the data release information is not stored in the retrieved products when downloading them in COMBINED data structure. Note that this parameter is set to Gaia+DR3 by default.

 

The table below summarises the accepted input formats to request individual or multiple ID's:

Format

Definition

Example

*source_id

Individual source ID. If this format is selected, then the release parameter MUST be provided.

1514169246023424

designation

Individual source designation.

Gaia+DR3+5912901375001820288

source_id1,source_id2,...

Comma-separated source ID list. If this format is selected, then the release parameter MUST be provided.

1514169246023424,16061459700373248,16061459700373248

source_id1-sourceid2

Source ID range. The service returns all source_id's fulfilling the condition: source_id <= source_id2 & source_id >= source_id1

720000000000000-1800000000000000

designation1,designation2,...

Comma-separated designation list.

Gaia+DR2+1514169246023424,Gaia+DR2+16061459700373248

designation1-designation2

Designation range.

Gaia+DR2+720000000000000-Gaia+DR2+1800000000000000

job:job_id.column_name

Result column from a job with the list of source id's/designations.

job:1530537474992O.source_id

user_schema.user_table.column_name

Column from a defined user table with the list of source id's/designations.

user_<user_name>.<table_name>.<column_name>

 

Notes:

  • Remember to include the RELEASE parameter when the input ID is provided as source_id (instead of a source designation).
  • The retrieved dataset is a compressed .gzip file if DATA_STRUCTURE = INDIVIDUAL (default) when downloading products for multiple sources.

 

Examples:

Any combination of the request parameters listed in the table above is allowed. The following examples show how to retrieve selected DataLink products for a single source:

curl -k -b cookies-datalink.txt "https://gea.esac.esa.int/data-server/data?RETRIEVAL_TYPE=EPOCH_PHOTOMETRY&ID=Gaia+DR3+30343944744320"
curl -k -b cookies-datalink.txt "https://gea.esac.esa.int/data-server/data?RETRIEVAL_TYPE=EPOCH_PHOTOMETRY&ID=30343944744320&RELEASE=Gaia+DR3&BAND=BP"
curl -k -b cookies-datalink.txt "https://gea.esac.esa.int/data-server/data?RETRIEVAL_TYPE=EPOCH_PHOTOMETRY&ID=30343944744320&RELEASE=Gaia+DR3&BAND=G&VALID_DATA=TRUE&FORMAT=ECSV"
curl -k -b cookies-datalink.txt "https://gea.esac.esa.int/data-server/data?RETRIEVAL_TYPE=RVS&ID=Gaia+DR3+30343944744320&format=VOTABLE_PLAIN"
curl -k -b cookies-datalink.txt "https://gea.esac.esa.int/data-server/data?RETRIEVAL_TYPE=ALL&ID=Gaia+DR3+30343944744320&format=FITS" > datalink_all.gzip

*Note: the use of the ">" bash command to write the output dataset to a file.

The examples below show how to download DataLink products for multiple sources (for the sake of simplicity, combinations of DATA_STRUCTURE and FORMAT have been avoided, but they can be included as well):

curl -k -b cookies-datalink.txt "https://gea.esac.esa.int/data-server/data?RETRIEVAL_TYPE=XP_SAMPLED&ID=5912901375001820288,5912586227479692800,5912437793408452608&RELEASE=Gaia+DR3" > xp_sampled_multi.gzip
curl -k -b cookies-datalink.txt "https://gea.esac.esa.int/data-server/data?RETRIEVAL_TYPE=MCMC_GSPPHOT&ID=Gaia+DR3+5912901375001820288,Gaia+DR3+5912586227479692800,Gaia+DR3+5912437793408452608" > mcmc_gspphot_multi_2.gzip
curl -k -b cookies-datalink.txt "https://gea.esac.esa.int/data-server/data?RETRIEVAL_TYPE=RVS&ID=Gaia+DR3+5912901375001820288,Gaia+DR3+5912586227479692800,Gaia+DR3+5912437793408452608&DATA_STRUCTURE=COMBINED" > rvs_multi.vot
curl -k -b cookies-datalink.txt "https://gea.esac.esa.int/data-server/data?RETRIEVAL_TYPE=EPOCH_PHOTOMETRY&ID=10655814178816-30343944744320&RELEASE=Gaia+DR3" > epoch_multi.gzip
curl -k -b cookies-datalink.txt "https://gea.esac.esa.int/data-server/data?RETRIEVAL_TYPE=EPOCH_PHOTOMETRY&ID=job:<job_id>.source_id"
curl -k -b cookies-datalink.txt "https://gea.esac.esa.int/data-server/data?RETRIEVAL_TYPE=XP_CONTINUOUS&ID=<user_schema>.<user_table>.source_id&RELEASE=Gaia+DR3"

 

Complex combinations for multiple requests are also possible:

curl -k -b cookies-datalink.txt "https://gea.esac.esa.int/data-server/data?RETRIEVAL_TYPE=EPOCH_PHOTOMETRY&ID=10655814178816-30343944744320,46316927345664,54528905041920-56732223835904&RELEASE=Gaia+DR3" > epoch_multi_2.gzip