Demos & Tutorials - Gaia Users
Help supportShould you have any question, please check the Gaia FAQ section or contact the Gaia Helpdesk |
Graphical User Interface
The Graphical User Interface (GUI) of the ESA Gaia Archive offers the possibility to carry out basic and advanced queries using ADQL (Astronomical Data Query Language). The outputs of these queries can be easily downloaded in a variety of formats, including VOTable, FITS and CSV. Below you can find a brief description of the main features offered by the Archive, as well as two video tutorials explaining how to use it.
|
Graphic User Interface main page.
|
|
Basic query form. This form allows to easily search for data in all the catalogues hosted by the Archive. Restrictions can be added to the query using the 'Extra conditions' wizard. The output fields can be selected by means of the 'Display columns' option panel. |
|
Advanced (ADQL) query form. This form allows to execute ADQL queries. Each query generates a job at server side. The jobs executed by the user can be inspected in the list provided in this page. All the public tables and the user-uploaded tables are visible in the left side of the web. |
|
Query Results.
The output of the queries are displayed in this window. The ADQL query that generated these results can be inspected by clicking on the 'Show query in ADQL form' link. |
Video tutorial: How to use the Archive Author: Deborah Baines |
Video tutorial: How to use the simple form of the Archive Author: Alcione Mora |
Tutorial on simple form (DR1)
Author: Alcione Mora
This tutorial was developed for Data Release 1. A new version will be available in the future. Even if the interface has evolved for eDR3, many changes are cosmetic and this introduction is still valuable for newcomers. Just keep that in mind while following it.
The main function of the Gaia Archive is to provide data to the astronomers. The easiest way to do it is the simple form, accessible through the web interface by clicking "Search" in the welcome page.
The main objectives for the simple form are to provide data ready to use in a way as friendly as possible, and to help users into the transition toward the "ADQL form" for advanced use cases, when necessary.
The "Search" page provides three different sub-tabs: "Simple form", "ADQL form" for advanced users and "Query results" for the output. The simple form is the default option, and the subject of this tutorial.
Search for a single source
The most basic use case could be formulated as "I want the most relevant Gaia results for a single object". It can be fulfilled typing the object identifier (e.g. UX Ori) in the "Name" box and clicking "Submit query"
The view now moves to the "Query results" sub-tab, where the results are displayed. In this example, a single Gaia source has been found.
The single object search launches a cone search around the celestial coordinates of the input object, which are provided by a name resolver. Currently, Simbad (default) and NED can be used. The object box turns green when the name has been successfully resolved. The search radius can be tuned using the controls on the right.
The cone is centred around the position provided by the name resolver without further conversion. The default radius might not be large enough for high proper motion stars. In addition, false matches can be found in crowded areas or for visual pairs.
The output is presented in tabular format, one possible match per row. The columns provide units when available. Further details on the meaning of each field and table can be found in the "Gaia Data Model", accessible in the bottom part.
This is the authoritative reference on the Gaia Data Release 1 contents. Detailed explanations can be obtained clicking on each table name. For example, within "tgas_source", the full explanation on column "source_id" can be retrieved
source_id includes geometrical information, encoded as a HEALPix index, and accessible via advanced ADQL user defined functions, see "Help" "ADQL syntax" for further details.
Additional resources and output filters
By default, the simple form explores the contents of "tgas_source", which includes the TGAS subset of about two million sources for which a five parameters astrometric solution has been derived (i.e. including parallaxes and proper motions). The full contents of the DR1 catalogue can be explored selecting "gaia_source" instead, which provides access to around 1.1 billion sources with precise positions and photometry in the sky.
There are additional resources (tables) in the Archive in addition to the two tables above. The simple form can explore other tables, provided they contain geometric information in the form of celestial coordinates.
The additional resources geometrically indexed are the ICRF2 sources table (see "Help" → "ICRF sources" tutorial), the external catalogues for which pre-computed cross-matches have been generated (e.g. WISE, 2MASS, UCAC4) and a number of auxiliary catalogues (e.g. Hipparcos, Hipparcos new reduction and Tycho2). Tables belonging to DR1 are prefixed by "gaiadr1", while purely external unmodified catalogues are in the "public" schema.
For example, if the same query is carried out in the Hipparcos catalogue
A single record is found, corresponding to the "UX Ori" identifier and coordinates in the Hipparcos catalogue.
Quick filters can be applied to trim the output in case it could grow too large. For instance, imagine a larger area of five arcminutes radius around UX Ori needs to be explored in the full "gaia_source" table.
A total of 127 sources have been found. Note the results are presented in groups of 20, which can be cycled using the left-right arrows in the bottom.
The results of a large query can be quickly trimmed applying filters. For example, imagine we are only interested in objects brighter than G ≤ 18. A condition on "phot_g_mean_mag", the field containing the G magnitude, can be easily placed.
Conditions can only be applied on indexed columns within the simple form for performance reasons. Advanced filtering is available in the "ADQL form" via where clauses.
A quick exploration of the Data Model description reveals the simple form is only providing a few output columns by default. This behaviour can be fully customized using the display columns box. For example, ecliptic coordinates (the last two columns) can be included in the query results.
=
The new query results show both the availability of these extra columns and a reduced number of rows (49 vs 129) due to the application of the minimum brightness condition.
Search for a list of sources
A common use case is "I want to look for Gaia counterparts on my list of known objects". The simple form allows to retrieve such information using the "File" tab instead of "Position". The input is a plain text file with one identifier per row. Additional input such as coordinates, magnitudes, etc. will produce errors and should be avoided.
Note all identifiers must be recognized by the name resolver or an error will be generated. Some information is displayed in case of a successful input file parsing.
The results are displayed in the same tabular format. Note the original input name is not shown in the output. This means additional analysis by the user is needed to identify which possible Gaia sources correspond to each parent target object.
Advanced cross-match features preserving the original name and providing a match distance are described in "Help" → "White dwarfs exploration" and "Cluster analysis" tutorials.
A cone search can be generated around any point on the sphere using the "Equatorial" radio button under the "Position" tab. For example, five arcminutes around the North Ecliptic Pole can be inspected as shown below.
Many coordinate input formats are supported. They can be revealed by clicking on the "RA" and "Dec" labels.
If the ecliptic coordinates additional output columns are still selected, it can be seen that the ecliptic latitudes are very high, as requested.
Box queries are also possible. For instance, a request for an area of approximately one square degree around the Pleiades location is shown below.
A box is defined as a spherical quadrilateral delimited by great circle segments. This can provide counter-intuitive results, and require extra care, when the area analysed is large or close to the celestial poles. The reason is parallels (constant declination loci) are not great circles, and thus unsupported by ADQL.
Advanced: retrieving the ADQL query
All simple form functionalities are based on ADQL queries. This means the same results can be retrieved running them under the advanced "ADQL form" tab. There are many resources available to learn this query language. One good starting point is the DPAC ADQL Cookbook.
The underlying ADQL queries can be retrieved in different ways. The first one consists of hovering on top of the different results tabs. After waiting a few seconds, a box is created displaying the query.
The query of choice can be directly shown in the "ADQL form" window by clicking on "Show query in ADQL form" in the bottom part of the screen,
which loads the query within the ADQL form tab.
The query is now available for full customisation. For example, the "TOP 500" output default limit could be removed when a large response is expected.
Finally, note the ADQL query can also be generated within the simple form itself, without the need to run it, by clicking on "Show Query".
ADQL syntax
ADQL (Astronomical Data Query Language) is based on SQL (Structured Query Language) which is a language to retrieve information from tables stored in a database.
(References: ADQL 2.0and SQL 92)
A very concise introduction to ADQL is provided in the next sections. There are a number of tutorials and resources providing a gentler learning curve for newcomers, specially those without previous experience with SQL or relational data bases. A small selection is provided below
- The DPAC ADQL Cookbook
- GAVO ADQL reference card
- GAVO ADQL short course for Gaia
- The Gaia Archive White dwarfs exploration and Cluster analysis tutorials
1. Basic syntax
In ADQL/SQL you write queries to the database. A query is compound of the table columns you want to retrieve (the SELECT part), the table or tables that store the data (the FROM part) and the conditions to restrict the data you obtain (the WHERE part). E.g.
SELECT <columns> FROM <tables> WHERE <conditions>
So, if you want to obtain Right Ascension and Declination of all items from the table gaia_source, you may write:
SELECT ra, dec FROM gaiadr1.gaia_source
ra
is the column name of Right Ascension in gaia_source table.
dec
is the column name of Declination in gaia_source table.
gaiadr1
is the database schema name where gaia_source table belongs to. It is a good practice to add schema names to tables to avoid name clashes.
Probably, you want to obtain also the object identification, so you can modify the query as follows:
SELECT source_id, ra, dec FROM gaiadr1.gaia_source
If you want to know all the column names associated to a table, you may use GACS GUI, clicking on the plus sign next to a table name.
|
|
|
Or, in TAP+, you may obtain all the columns and descriptions of a table using the following syntax:
curl "https://gea.esac.esa.int/tap-server/tap/tables?tables=gaiadr1.gaia_source"
Now, suppose you are interested in an specific region (e.g. ra=266.41683, dec=-29.00781, radius=0.083333 arc.min.). So you want to restrict the results to that region. In order to do that, you may execute a 'cone search' to obtain all the objects where ra,dec are inside a cone:
SELECT source_id, ra, dec FROM gaiadr1.gaia_source
WHERE 1=CONTAINS(POINT('ICRS',ra,dec),
CIRCLE('ICRS',266.41683,-29.00781, 0.08333333))
(You may read the ADQL recommendation to obtain the list of functions that can be used).
One way to create a complex query could be to use the 'Simple Form' page to create the basic query graphically. Then you should press 'Show query' button to show the query as ADQL in the 'ADQL Form'. And then, you can modify it.
2. Selecting complex data
You are not restricted to obtain column names only. You can obtain complex values too.
For instance, you may want to obtain the distance of each source to the center of an specific region. Then, you may type:
SELECT source_id, ra, dec, DISTANCE(POINT('ICRS',ra,dec),
POINT('ICRS',266.41683,-29.00781)) AS dist,
FROM gaiadr1.gaia_source
WHERE 1=CONTAINS(POINT('ICRS',ra,dec),CIRCLE('ICRS',266.41683,-29.00781, 0.08333333))
source_id
, ra
and dec
are the Source Identifier, Right ascension and Declination of each item of the gaia_source table.
'DISTANCE(POINT('ICRS',ra,dec), POINT('ICRS',266.41683,-29.00781)) AS dist' is a created on the fly column, with the name 'dist' that contains the distance of the item to the specified point.
3. Additional functions available
Apart from the standard ADQL functions, ESA Gaia TAP+ service offers the next functions:
Table Gaia TAP+ ADQL functions
Function |
Return Type |
Description |
Example |
Result |
STDDEV(expression) |
Numeric |
Standard deviation function |
STDEV(column) | |
GAIA_HEALPIX_INDEX(norder,source_id) |
bigint |
Returns the healpix index of the given.
|
GAIA_HEALPIX_INDEX(4, 2060294888487267584) | 914 |
GREATEST(v1,v2[,v3,..,vn]) |
Same as input |
Greatest value among the given arguments |
GREATEST(10.55, 9.12323, 11.2, 7.8) | 11.2 |
LEAST(v1,v2[,v3,..,vn]) |
Same as input |
Least value among the given arguments |
LEAST(10.55, 9.12323, 11.2, 7.8) | 7.8 |
SIGN(x) |
Integer |
Sign of the argument (-1, 0, +1) |
SIGN(-10.55) | -1 |
COALESCE(v1,[]v2,v3,..,vn]) |
Same as input |
Returns first argument that is not null. If all arguments are null, it will return null. |
COALESCE(NULL, 1, 2) | 1 |
NULLIF(v1,v2) |
Same as input |
Returns a null value if v1 equals to v2, otherwise it returns v1. |
NULLIF(1, 1) | NULL |
WIDTH_BUCKET(operand,min,max,buckets) |
Integer |
Returns the bucket number to which operand would be assigned in a histogram having count equal-width buckets spanning the range min to max; returns 0 or count+1 for an input outside the range |
WIDTH_BUCKET(5.35, 0.024, 10.06, 5) | 3 |
Also, the ESA Gaia TAP+ service offers functions for array handling:
Table Gaia TAP+ array handling functions
Function |
Description |
Example |
Result |
ARRAY_ELEMENT(array_column, index1 [,index2....,indexN]) |
Returns the requested element inside the array. Indexes beginning in 1 |
Example. Given the array [4,5,6] |
SELECT array_element(array,2) : 5
|
ARRAY_NDIMS(array_column) |
Returns the number of dimensions of the array (integer) |
Example. Given the array [[1,2,3], [4,5,6]] |
SELECT array_ndims(array) : 2
|
ARRAY_DIMS(array_column) |
Returns a text representation of array's dimensions. Initial and end index for each dimension are given |
Example. Given the array [[1,2,3], [4,5,6]] |
SELECT array_dims(array) : [1:2][1:3]
|
ARRAY_LENGTH(array_column, index) |
Returns the length of the requested array dimension (integer), 1 being the first index value |
Example. Given the array [1,2,3] |
SELECT array_length(array,1) : 3
|
CARDINALITY(array_column) |
Returns the total number of elements in the array (integer), or 0 if the array is empty |
Example. Given the array [[1,2],[3,4]] |
SELECT cardinality(array) : 4
|
4. ADQL mathematical functions
ADQL defines some mathematical functions. Also, other mathematical functions have been defined in our service to support scientific queries
Table Mathematical ADQL functions
Function |
Return Type |
Description |
Example |
Result |
ABS(x) |
Same as input |
Absolute value |
ABS(-19.4) | 19.4 |
CBRT(x) |
numeric |
Cube Root |
CBRT(27.0) | 3 |
DEGREES(x) |
numeric |
Radians to Degrees |
DEGREES(0.5) | 28.64788975654116 |
DIV(y,x) |
numeric |
Integer quotient of y/x |
DIV(9,4) | 2 |
EXP(x) |
same as input |
Exponential |
EXP(1.0) | 2.718281828459045 |
FLOOR(x) |
same as input |
Nearest integer less than or equal to argument |
FLOOR(-42.8) | -43 |
LOG(x) |
same as input |
Natural Logarithm |
LOG(2.0) | 0.6931471805599453 |
LOG(b,x) |
numeric |
Logarithm to base b |
LOG(2.0, 64.0) | 6.0000000000 |
LOG10(x) |
numeric |
Base 10 logarithm |
LOG10(100.0) | 2 |
MOD(y,x) |
same as arguments |
Remainder of y/x |
MOD(9, 4) | 1 |
PI() |
numeric |
Pi constant |
PI() | 3.141592653589793 |
POWER(x,y) |
numeric |
x raised to the power of y |
POWER(9.0, 3.0) | 729 |
RADIANS(x) |
numeric |
Degrees to radians |
RADIANS(45.0) | 0.7853981633974483 |
RAND(x) |
numeric |
Random number in the range 0<=x<1 |
RAND() | |
ROUND(x,s) |
numeric |
Round x to s decimal places, where s is an integer |
ROUND(45.2191,2) | 45.22 |
SQRT(x) |
numeric |
Square root |
SQRT(2.0) | 1.414213562373095 |
TRUNCATE(x) |
numeric |
Truncate toward zero |
TRUNCATE(48.8) | 48 |
TRUNCATE(x, s) |
numeric |
Truncate to s decimal places |
TRUNCATE(48.8328, 3) | 48.832 |
5. ADQL trigonometric functions
ADQL defines some trigonometric functions considered important for astronomical queries
Table trigonometric ADQL functions
Function |
Return Type |
Description |
Example |
Result |
ACOS(x) |
Numeric |
Inverse cosine or arc cosine |
ACOS(0.12582) | 1.4446419701843678 |
ASIN(x) |
Numeric |
Inverse sine or arc sine |
ASIN(0.12582) | 0.1261543566105288 |
ATAN(x) |
Numeric |
Inverse tangent or arc tangent |
ASIN(10.28527) | 1.4738745386849255 |
ATAN2(x,y) |
Numeric |
Inverse tangent of x/y |
ATAN2(10.28527,3.1) | 1.2780538751678443 |
COS(x) |
Numeric |
Cosine of x |
COS(10.28527) | -0.6520645009291157 |
SIN(x) |
Numeric |
Sine of x |
SIN(10.28527) | -0.7581634959743598 |
TAN(x) |
Numeric |
Tangent of x |
TAN(10.28527) | 1.1627124232251034 |
COT(x) |
Numeric |
Cotangent of x |
COT(0.785) | 1.000796644031489 |
6. Data type casting ADQL functions
Some User Defined Functions have been implemented in order to allow the casting of values between different data types. The casting functions relies in the underlying PostgreSQL functions so errors received during casting will be, in most of the cases, PosgreSQL errors.
Table Data type casting ADQL functions
Function |
Return Type |
Description |
Example |
Result |
TO_SMALLINT(x) |
int2 |
Conversion of valid values into smallint. From -2^15 to 2^15-1 |
TO_SMALLINT(17.4) | 17 |
TO_INTEGER(x) |
int4 |
Conversion of valid values into integer. From -2^31 to 2^31-1 |
TO_INTEGER(1713112213.4123) | 1713112213 |
TO_BIGINT(x) |
int8 |
Conversion of valid values into bigint. From -2^63 to 2^63-1 |
TO_BIGINT(1713112213.4123) | 1713112213 |
TO_REAL(x) |
float4 |
Conversion of valid values into real. 8 decimal digits precision |
TO_REAL(91323.1231) | 91323.125 |
TO_DOUBLE(x) |
float8 |
Conversion of valid values into double precision. 16 decimal digits precision |
TO_DOUBLE(91321213.112212231) | 91321213.11221223 |
TO_CHAR(x) |
char |
Convert valid values into char data type |
TO_CHAR(1123) | '1123' |
TO_CHAR(v1, v2) |
char |
Convert valid values into char data type, following the format defined in v2. For a full list of valid format, check Formats |
TO_CHAR(-125.8, '999D99S') | '125.80-' |
TO_BOOLEAN(v1) |
boolean |
Convert valid values into boolean data type |
TO_BOOLEAN(1) | true |
7. ADQL extension: conditional expressions
Some conditional expressions have been implemented as User Defined Functions.
Table Conditional Expressions ADQL functions
Function |
PostgreSQL expression replicated |
Example |
CASE_CONDITION(default_value, condition1, value1, condition2, value2, ...) |
CASE WHEN condition THEN result
[WHEN ...]
[ELSE result]
END
|
case_condition(astrometric_n_obs_al, dec < -40, -astrometric_n_obs_al, dec > 40, astrometric_n_obs_al / 2) |
CASE_EXPRESSION(input_column, default_value, target_value1, value1, target_value2, value2, ...) |
CASE a
WHEN 1 THEN 'one'
WHEN 2 THEN 'two'
ELSE 'other'
END
|
case_expression(astrometric_n_obs_al, 'unknown', 85, 'eighty five', 78, 'seventy eight', 228, 'two hundred and twenty eight', 3, 'three', 4, 'four') |
IF_THEN_ELSE(condition, value, [default_value]) |
CASE WHEN condition THEN result
[ELSE result]
END
|
if_then_else(dec < 0, astrometric_n_obs_al, -astrometric_n_obs_al) |
8. Deviations of ADQL functions implementation from standard
ADQL standard is implemented in compliance with IVOA ADQL 2.0 plus the ADQL Errata available by 16th April 2018. Some implementation limitations apply in the following functions:
- BOX(coordsys, longitudeCentre, latitudeCentre, longitudeExtent, latitudeExtent): it is interpreted as follows:
- As defined into the standard when arguments are fixed values (A cross at the central position with arms extending, parallel to the coordinate axes at the center position, for half the respective sizes on either side).
- When the arguments are variable (e.g. table columns), the sides of the box are line segments or great circles intersecting the arms of the cross in its end points at right angles with the arms.