Image of the Week

Interactive and statistical visualisation of Gaia DR1 with VAEX


The size of the Gaia DR1 datasets requires sophisticated analysis and plotting tools. For example, a simple plot of the positions of the stars on the sky in Gaia DR1 miserably fails, even if only 10 000 points or 1 000 000 points (top left and right panels, respectively) are shown. However, a density plot as that in the bottom adequately reveals the rich structure of the data, including all 1 142 679 769 sources in Gaia DR1, and can be generated in less than a second with VAEX. This density plot reveals, for example, structure in the galactic disk and artefacts due to the scanning nature of the observations performed with Gaia.


With the arrival of large catalogues such as the Gaia DR1, which contains more than a billion objects, new methods of handling and visualising these data volumes are needed. For many science cases, as well as for quality checks of the data, one needs to visualise all or large parts of the data. While scatter plots would suffice for small catalogues, it would not work for the full Gaia catalogue. Apart from the long time it takes to render each individual point as a glyph, overplotting makes the plot useless, as demonstrated in the figure above, presented by Maarten Breddels at the Astro-informatics IAU Symposium in 2016 at Sorrento, Italy.

This figure demonstrates how plotting a random subset of 10,000 stars (0.001% of the data, top left panel) shows structure in the galactic disk, while plotting just a million stars (0.1% of the data, top right panel), already starts to hide many of the structures present in the data. Instead, in the bottom panel of this figure, showing all data (more than a billion stars) in the form of a density plot reveals much more structure: dust lanes are clearly visible in the disk, our neighbouring galaxies (the Large and Small Magellanic Clouds) stand out clearly against the background, but also artefacts in the data due to the scanning law of the satellite become visible. In this plot, low densities correspond to black, and high densities to white, where the scaling is logarithmic.

To visualise and explore large catalogues such as Gaia DR1, Maarten Breddels from the Kapteyn Astronomical Institute (University of Groningen) developed a software package to perform the calculations needed for these visualisations efficiently. The calculations to compute the number of stars in each pixel take only about a second on a high-end desktop machine. Statistics, such as minimum, maximum, mean, moments, etc., can also be calculated efficiently in any number of dimensions.

The software packaged called VAEX exists of two parts. The first is a Python package, allowing fast calculations of statistics for any property of the data (or any mathematical operations on them), and their visualisation in, for instance, the Jupyter notebook. Built on top of this, is a graphical program for Linux and Mac OS X, enabling interactive exploration of the data including zooming, panning, and screen selections. VAEX is open source, available under a MIT License.

This work has been carried out in collaboration with Amina Helmi. It has been funded by a grant from the Netherlands Research School for Astronomy (NOVA), and a Vici grant from the Netherlands Organisation for Scientific Research (NWO).

Credits: ESA/Gaia/DPAC/CU9, Maarten Breddels, Amina Helmi

[Published: 23/12/2016]


Image of the Week Archive

24/04: Gaia reveals the composition of asteroids
20/04: Extra-galactic observations with Gaia
10/04: How faint are the faintest Gaia stars?
24/03: Pulsating stars to study Galactic structures
09/02: Known exoplanetary transits in Gaia data
31/01: Successful second DPAC Consortium Meeting
23/12: Interactive and statistical visualisation of Gaia DR1 with vaex
16/12: Standard uncertainties for the photometric data (in GDR1)
25/11: Signature of the rotation of the galactic bar uncovered
15/11: Successful first DR1 Workshop
27/10: Microlensing Follow-Up
21/10: Asteroid Occultation
16/09: First DR1 results
14/09: Pluto Stellar Occultation
15/06: Happy Birthday, DPAC!
10/06: 1000th run of the Initial Data Treatment system
04/05: Complementing Gaia observations of the densest sky regions
22/04: A window to Gaia - the focal plane
05/04: Hipparcos interactive data access tool
24/03: Gaia spots a sunspot
29/02: Gaia sees exploding stars next door
11/02: A new heart for the Gaia Object Generator
04/02: Searching for solar siblings with Gaia
28/01: Globular cluster colour-magnitude diagrams
21/01: Gaia resolving power estimated with Pluto and Charon
12/01: 100th First-Look Weekly Report
06/01: Gaia intersects a Perseid meteoroid
18/12: Tales of two clusters retold by Gaia
11/11: Lunar transit temperature plots
06/11: Gaia's sensors scan a lunar transit
03/11: Celebrity comet spotted among Gaia's stars
09/10: The SB2 stars as seen by Gaia's RVS
02/10: The colour of Gaia's eyes
24/09: Estimating distances from parallaxes
18/09: Gaia orbit reconstruction
31/07: Asteroids all around
17/07: Gaia satellite and amateur astronomers spot one in a billion star
03/07: Counting stars with Gaia
01/07: Avionics Model test bench arrives at ESOC
28/05: Short period/faint magnitude Cepheids in the Large Magellanic Cloud
19/05: Visualising Gaia Photometric Science Alerts
09/04: Gaia honours Einstein by observing his cross
02/04: 1 April - First Look Scientists play practical joke
05/03: RR Lyrae stars in the Large Magellanic Cloud as seen by Gaia
26/02: First Gaia BP/RP deblended spectra
19/02: 13 months of GBOT Gaia observations
12/02: Added Value Interface Portal for Gaia
04/02: Gaia's potential for the discovery of circumbinary planets
26/01: DIBs in three hot stars as seen by Gaia's RVS
15/01: The Tycho-Gaia Astrometric Solution
06/01: Close encounters of the stellar kind
12/12: Gaia detects microlensing event
05/12: Cat's Eye Nebula as seen by Gaia
01/12: BFOSC observation of Gaia at L2
24/11: Gaia spectra of six stars
13/11: Omega Centauri as seen by Gaia
02/10: RVS Data Processing
12/09: Gaia discovers first supernova
04/08: Gaia flag arrives at ESAC
29/07: Gaia handover
15/07: Eclipsing binaries
03/07: Asteroids at the "photo finish"
19/06: Calibration image III - Messier 51
05/06: First Gaia BP/RP and RVS spectra
02/06: Sky coverage of Gaia during commissioning
03/04: Gaia source detection
21/02: Sky-background false detections in the sky mapper
14/02: Gaia calibration images II
06/02: Gaia calibration image I
28/01: Gaia telescope light path
17/01: First star shines for Gaia
14/01: Radiation Campaign #4
06/01: Asteroid detection by Gaia
17/12: Gaia in the gantry
12/12: The sky in G magnitude
05/12: Pre-launch release of spectrophotometric standard stars
28/11: From one to one billion pixels
21/11: The Hipparcos all-sky map
15/10: Gaia Sunshield Deployment Test
08/10: Initial Gaia Source List
17/09: CU1 Operations Workshop
11/09: Apsis
26/08: Gaia arrival in French Guiana
20/08: Gaia cartoons
11/07: Model Soyuz Fregat video
01/07: Acoustic Testing
21/06: SOVT
03/06: CU4 meeting #15
04/04: DPCC (CNES) 
26/03: Gaia artist impression 
11/02: Gaia payload testing  
04/01: Space flyby with Gaia-like data
10/12: DPAC OR#2. Testing with Planck
05/11: Galaxy detection with Gaia
09/10: Plot of part of the GUMS-10 catalogue
23/07: "Gaia" meets at Gaia
29/06: The Sky as seen by Gaia
31/05: Panorama of BAM clean room
29/03: GREAT school results
12/03: Scanning-law movie
21/02: Astrometric microlensing and Gaia
03/02: BAM with PMTS
12/01: FPA with all the CCDs and WFSs
14/12: Deployable sunshield
10/11: Earth Trojan search
21/10: First Soyuz liftoff from the French Guiana
20/09: Fast 2D image reconstruction algorithm
05/09: RVS OMA
10/08: 3D distribution of the Gaia catalogue
13/07: Dynamical Attitude Model
22/06: Gaia's view of open clusters
27/05: Accuracy of the stellar transverse velocity
13/05: Vibration test of BAM mirrors
18/04: L. Lindegren, Dr. Honoris Causa of the Observatory of Paris
19/01: Detectability of stars close to Jupiter
05/01: Delivery of the WFS flight models
21/12: The 100th member of CU3
17/11: Nano-JASMINE and AGIS
27/10: Eclipsing binary light curves fitted with DPAC code
13/10: Gaia broad band photometry
28/09: Measuring stellar parameters and interstellar extinction
14/09: M1 mirror
Please note: Entries from the period 2003-2010 are available in this PDF document.