XMM-Newton Current Calibration Files

 

Back to Calibration Homepage

Introduction

To analyse the data collected with the instruments on-board XMM-Newton, together with the SAS you will need some calibration files (Current Calibration Files a.k.a. CCF).

Contrary to the usual way of producing and distributing Calibration Data versions after a certain period of time, the XMM-Newton CCF are updated dynamically and continuously. Announcements of single or multiple file updates are issued via the ccf mailing list. Successive CCF releases do not have version numbers. Once announced, CCF data files are available for immediate download from our project servers (see below on how to do that) and their respective Release Notes are published online.

The CCF Index File

Before processing your data with the SAS you must build a CCF Index File (CIF) of all the CCF appropriate for your observations. This is carried out with the SAS task cifbuild. One of the parameters of the task, analysisdate, can be used by experts who want to compare data processed with past calibration knowledge. A "normal" user, will leave this parameter at its default: the current date.

Alternately, this interactive web page, allows you to run our own local version of cifbuild. As a result, you may download via ftp the resulting CIF file, compare it with the one used by you or download all the CCF required to process your data.

The indexing algorithm used to build the CIF is described in the SAS on-line documentation for the task cifbuild or by typing in within any installation of SAS, the command sashelp --doc=cifbuild.

The CCF Repositories

All the XMM-Newton CCF ever made public, currently ~1550 files occupying ~4.3 GB, are available for download from our project servers. Although you may copy all these files, this would only be necessary if you want to process your data with the calibration knowledge at any given time in the XMM-Newton history, since many old CCF have been superseded by more accurate calibration. If this is not your purpose, we recommend you to download only a subset of all CCF, known as the Valid CCF Set, assembled and made available for download daily. The components of this set are all the CCF you need to process any XMM-Newton ODF at the current date. Currently, this set has ~548 files with a size of ~2.9 GB.

Mirroring The CCF Repositories

Depending on your needs, you can either download the historical repository or the Valid CCF Set.

The rsync service normally used to mirror the XMM-Newton Current Calibration File repositories is temporarily unavailable.

As an interim solution, the XMM-Newton CCFs can be downloaded from the anonymous sftp server:

anonymous@ftp-bdt.cosmos.esa.int

Users should connect using an anonymous connection. No personal account is required. Users may connect to the SFTP server using standard SFTP client applications, such as FileZilla, or any other compatible SFTP client.

After connecting to the server, please navigate to the following directory:

XMM-Newton_public

This directory contains the two CCF areas:

pub
: Public CCF repository.

valid
: Valid CCF repository.

Users who maintain a local copy of the CCF repositories should download the required files from the corresponding directory. Please note that this temporary access method does not provide rsync functionality, so existing rsync-based mirroring scripts will not work until the service is restored.

Using lftp to mirror the CCF repositories

Users with lftp installed can automatically download and keep a local copy of the CCF repository synchronised with the SFTP server. The mirror command transfers only new or changed files on subsequent runs; the -c option enables resume of interrupted transfers.

Before running the command, replace /your/path/CCF with your local destination directory and /your/path/logs with a directory where the transfer log should be written. Ensure both directories exist and you have write permission.

To mirror the CCF repository:

    lftp -c "open -u anonymous:anonymous sftp://ftp-bdt.cosmos.esa.int:22; mirror -v -c /XMM-Newton_public/ccf/pub /your/path/CCF" > /your/path/logs/lftp_CCF_transfer.log 2>&1

To download only the Valid CCF Set, replace ccf/pub with ccf/valid in the remote path above.

The XMM-Newton SOC will update this page when the standard mirroring service becomes available again.

Reporting Problems

Questions and problems regarding the CCF and how to download them should be addressed to the XMM-Newton Helpdesk.