XMM-Newton Current Calibration Files

 
Back to Calibration Homepage

Introduction

To analyse the data collected with the instruments on-board XMM-Newton, together with the SAS you will need some calibration files (Current Calibration Files a.k.a. CCF).

Contrary to the usual way of producing and distributing Calibration Data versions after a certain period of time, the XMM-Newton CCF are updated dynamically and continuously. Announcements of single or multiple file updates are issued via the ccf mailing list. Successive CCF releases do not have version numbers. Once announced, CCF data files are available for immediate download from our project servers (see below on how to do that) and their respective Release Notes are published online.

The CCF Index File

Before processing your data with the SAS you must build a CCF Index File (CIF) of all the CCF appropriate for your observations. This is carried out with the SAS task cifbuild. One of the parameters of the task, analysisdate, can be used by experts who want to compare data processed with past calibration knowledge. A "normal" user, will leave this parameter at its default: the current date.

Alternately, this interactive web page, allows you to run our own local version of cifbuild. As a result, you may download via ftp the resulting CIF file, compare it with the one used by you or download all the CCF required to process your data.

The indexing algorithm used to build the CIF is described in the SAS on-line documentation for the task cifbuild or by typing in within any installation of SAS, the command sashelp --doc=cifbuild.

The CCF Repositories

All the XMM-Newton CCF ever made public, currently ~1400 files occupying ~3 GB, are available for download from our project servers. Although you may copy all these files, this would only be necessary if you wanted to process your data with the calibration knowledge at any given time in the XMM-Newton history, since many old CCF have been superseded by more accurate calibration. If this is not your purpose, we recommend you to download only a subset of all CCF, known as the Valid CCF Set, assembled and made available for download daily. The components of this set are all the CCF you need to process any XMM-Newton ODF at the current date. Currently, this set has a size of ~600 MB.

Mirroring The CCF Repositories

Depending on your needs, you can either mirror the full repository or the Valid CCF Set. For both we recommend to use rsync and only in case you can not use this tool for any reason, use plain ftp, that is always available.

rsync ensures that any CCF files deleted for any reason from our repositories will also be deleted from your local copy, therefore reducing the possibility that you use wrong CCF.

Using rsync

Assuming you store the CCF in /ccf, to keep it in sync with our repositories, please do as follows:

  1. To synchronise with the whole repository of CCF:

    rsync -v -a --delete --delete-after --force --include='*.CCF' --exclude='*/' xmm.esac.esa.int::XMM_CCF /ccf/

  2. To synchronise with the Valid CCF Set:

    rsync -v -a --delete --delete-after --force --include='*.CCF' --exclude='*/' xmm.esac.esa.int::XMM_VALID_CCF /ccf/

Before running these commands for the first time you may want to use the option --dry-run to see what the command would do without actually executing it.

Using plain ftp

You can always access our ftp server via anonymous user connection, as follows:

ftp xmm.esac.esa.int
Name (xmm.esac.esa.int:ccflib): anonymous
331 Guest login ok, send your complete e-mail address as password.
Password:your_user_name@valid_mail_address

You must enter a valid your_user_name@valid_mail_address mail address, otherwise our ftp server will reject the connection.

Once logged in, you should receive a welcome message similar to the following:

230-XMM-Newton SOC - European Space Agency (ESA)
230-Villafranca del Castillo, Madrid, Spain
230---------------------------------------------------------
230-Time: Thu Mar 10 10:13:21 2016.
230-
230-You are entering as anonymous from xxxx.xxxx.xxxx.xxxx.
230-
230-Server administered by ftp-admin@xmm.vilspa.esa.es
230-
230-Last update: July 11, 2002
230---------------------------------------------------------
230-
230 Guest login ok, access restrictions apply.
Remote system type is UNIX.
Using binary mode to transfer files.

Then, change directory to /pub/ccf and make a list of its contents as follows:

ftp> cd /pub/ccf
250 CWD command successful.
ftp> ls
227 Entering Passive Mode (193,147,152,102,121,88)
150 Opening ASCII mode data connection for /bin/ls.
total 1752
-rw-r--r--   1 ccflib      2115 Feb 18  2015 AAA_README.txt
lrwxrwxrwx   1 ccflib        30 Feb 18  2015 background -> constituents/extras/background
drwxrwxr-x   6 ccflib    110592 Feb  5 06:01 constituents
drwxr-xr-x   2 ccflib      4096 Aug 24  2002 doc
drwxrwxrwt   2 ccflib    634880 Mar  9 07:35 download
lrwxrwxrwx   1 ccflib        30 Jun  9  2014 esas_caldb -> constituents/extras/esas_caldb
lrwxrwxrwx   1 ccflib        34 Feb 18  2015 esas_caldb_old -> constituents/extras/esas_caldb_old
lrwxrwxrwx   1 ccflib        33 Feb 18  2015 filter_closed -> constituents/extras/filter_closed
drwxr-xr-x   8 ccflib      4096 Apr  6  2015 private
drwxr-xr-x   4 ccflib     77824 Mar 10 08:32 red_constituents
lrwxrwxrwx   1 ccflib        29 Feb 18  2015 responses -> constituents/extras/responses
drwxr-xr-x   3 ccflib      4096 Mar 10 09:53 to_be_deleted
drwxr-xr-x   2 ccflib     40960 Feb 23 11:09 valid_constituents
226 Transfer complete.

The directory constituents contains the historical repository of CCF. To get all these files, you may do it as follows:

ftp> prompt off
Interactive mode off
ftp> mget *.CCF

Similarly, to get only the Valid CCF Set, you can change directory to valid_constituents and invoke similar ftp commands.

Note that using these commands will download all specified CCF to the directory from where you started the ftp session.

Reporting Problems

Questions and problems regarding the CCF and how to download them should be addressed to the XMM-Newton Helpdesk.