EPIC source finding in overlapping exposures thread: step-by-step case for an OBSERVATION in mosaic mode

Introduction

This thread explains how to apply EPIC source detection tasks to a set of overlapping EPIC observations. The steps described here are needed when users want to exploit the maximum detection sensitivity in areas of the sky where two or more EPIC observations overlap, like it is the case for EPIC Mosaic observations. Nevertheless, the procedure described here can be applied for the case where different independent observations overlap.

The example used here to illustrate the procedure uses a Jupiter observation taken in Mosaic mode (ODF 0200080701).

Expected Outcome

This thread produces a mosaic image of several XMM-Newton observations, or pointings within a given observations, together with the source list that results from running source detection algorithms over the overlapping fields.

SAS Tasks to be Used

Prerequisites

It is assumed that the processed eventlists, like the ones produced by the SAS tasks epproc and emproc, and corresponding attitude and summary files as well as the ccf.cif file are available for each observation (ODF).

Useful Links

Caveats

Last Reviewed: 25 May 2023, for SAS v21.0

Last Updated: 21 November 2014

 

Procedure

 

This threads can be followed with both, different observations of overlapping fields (see the Caveats below) and observations that contain different pointings, like it is the case of EPIC Mosaic mode observations. Two steps are needed in order to run the source detection algorithms in overlapping EPIC fields. In the first step, the task emosaic_prep separates processed EPIC calibrated event files (as output from epproc and emproc) taken in Mosaic mode into several pseudo-exposures corresponding to the different pointings of the mosaic observation. This step is also necessary when combining different observations of overlapping fields, since emosaic_prep also prepares the data and creates the necessary file structure needed by the task emosaicproc. In this later case, since only one pointing is available within the given observations, only one pseudo-exposure will be created per observation. In the second part of the process, emosaicproc performs coherently source detection on several exposures from the same or different observations, or pseudo-exposures as created by the task emosaic_prep.

Before proceeding with this thread, the following steps are mandatory. Please follow them carefully. SAS has been developed for a data model based on single pointing exposures, therefore, a SAS procedure dealing with different pointings is very sensible to directory changes, etc. 

  • Follow the SAS Startup thread and define the enviroment variables SAS_ODF and SAS_CCF. For this particular case, it is necessary to run the task odfingest with the options odfdir=<full_path_to_odf_directory> withodfdir=yes .
  • After running odfingest, the variable SAS_ODF must point to the *SUM.SAS file including the full path to the ODF directory.
  • Follow the Produce EPIC Event Lists thread to create EPIC-pn and EPIC-MOS event files.
  • If needed, produce GTI files and filtered event files for each EPIC camera following the EPIC Filtering for Background Flares thread. 
  • We assume in what follows that we start the further processing from a working directory containing the following event lists: MOS1evt.ds, MOS2evt.ds, PNevt.ds and the attitude file AttHk.ds. These files can either be the actual names of the files obtained or symbolic links to the actual files with full names, like, 0727_0200080701_EPN_S001_ImagingEvts.ds and 0727_0200080701_AttHk.ds.

Running source detection over an observation taken in Mosaic mode:

  • The first step in the process is to run the SAS task emosaic_prep. emosaic_prep will split the event file into several pseudo-exposures. In this case where the observation has been taken under the EPIC Mosaic mode, each pseudo-exposure will correspond to a different pointing of the mosaic observation. This thread assumes that the current working directory contains the event files for each EPIC camera.
     
    1. In this particular example, all EPIC cameras are used, however, any combination of two cameras will produce the same result. To run emosaic_prep it is mandatory to provide the names of the EPIC event files and attitude file.

         emosaic_prep pnevtfile=PNevt.ds mos1evtfile=MOS1evt.ds mos2evtfile=MOS2evt.ds \
           atthkfile=AttHk.ds 
       
    2. emosaic_prep will create the following directory structure under the working directory, where each subdirectory corresponds to each of the pointings contained in the observation,
        
       ./prep_mosaic_001/
 ./prep_mosaic_002/
 ./prep_mosaic_003/
 ./prep_mosaic_004/
    3. Check that the data contained in each of the directories is valid, ie. that the event lists have valid events, etc.  

    4. Each of the above subdirectory contains the following information. For example, prep_mosaic_001/ contains,
        
      PNevt.ds -> ../<ODF Identifier>_EPN_S001_ImagingEvts.ds
MOS1evt.ds -> ../<ODF Identifier>_EMOS1_S002_ImagingEvts.ds
MOS2evt.ds -> ../<ODF Identifier>_EMOS2_S003_ImagingEvts.ds
AttHk.ds -> ../<ODF Identifier>_AttHk.ds
<ODF Identifier>_SCX00000SUM.SAS -> ../<ODF Identifier>_SCX00000SUM.SAS
ccf.cif -> ../ccf.cif
gti_pointing_position.ds
PNevt_P001.ds
MOS1evt_P001.ds
MOS2evt_P001.ds


      links to the relevant files in the working directory, a file containing the GTI interval corresponding to pointing 001 (gti_pointing_position.ds), and the corresponding EPIC event files filtered in time to include pointing 001. The same applies to the other pointings.
       
  • The second step in the process is to run the SAS task emosaicproc.
     
    1. The task accepts any combination of instruments and pointings. If we use all the available instruments and pointings, the call to emosaicproc looks like this,

        emosaicproc \
          pnevlist='prep_mosaic_001/PNevt_P001.ds       prep_mosaic_002/PNevt_P002.ds \
                    prep_mosaic_003/PNevt_P003.ds       prep_mosaic_004/PNevt_P004.ds' \
          mos1evlist='prep_mosaic_001/MOS1evt_P001.ds       prep_mosaic_002/MOS1evt_P002.ds \
                    prep_mosaic_003/MOS1evt_P003.ds       prep_mosaic_004/MOS1evt_P004.ds' \
          mos2evlist='prep_mosaic_001/MOS2evt_P001.ds       prep_mosaic_002/MOS2evt_P002.ds \
                    prep_mosaic_003/MOS2evt_P003.ds       prep_mosaic_004/MOS2evt_P004.ds'

      by default, if not specified, the task will use three energy ranges:

       300-1000, 1000-7500 and 7500-12000 eV for EPIC-pn and
       200-1000, 1000-7500 and 7500-12000 eV for EPIC-MOS

      if different energy ranges want to be specified, the following parameters have to be provided in the call to emosaicproc,

        pnPImin='300 1000 7500'
        pnPImax='1000 7500 12000'
        mos1PImin='200 1000 7500'
        mos1PImax='1000 7500 12000'
        mos2PImin='200 1000 7500'
        mos2PImax='1000 7500 12000'

      where one or more energy ranges can be defined.

    2. The output of emosaicproc is stored in the subdirectory ./mosaic (./prep_mosaic for SAS version earlier than 12.0) under the working directory. Three main files are produced:

      mosaic_eboxlist.fits: This file is produced by the SAS task eboxdetect, hence the description of what can be found inside this table can be found in the documentation of the task. However, the source parameters are averaged (e.g. RATE, FLUX, HR) or summed (e.g. SCTS, EXP_MAP) over the mosaic pointings.

      mosaic_emllist.fits: This file is produced by the SAS task emldetect, hence the description of what can be found inside this table can be found in the documentation of the task. However, the source parameters are averaged (e.g. RATE, FLUX, HR) or summed (e.g. SCTS, EXP_MAP) over the mosaic pointings.

      Please be aware that for each ID_INSTR value (e.g. ID_INSTR=1, 2, 3 for EPIC-pn, EPIC-MOS1 and EPIC-MOS2, respectively) the SCTS column for ID_BAND=0 contains a non-null value provided that the detected source falls within all the pointings of this instrument (e.g. in an area where images overlap each other). However, HR values are calculated for each band even if the source does not fall within all the mosaic pointings of an instrument. The SCTS column for the summary row for each source (e.g. ID_BAND=0, ID_INST=0) contains the sum of counts over the mosaic pointings. Accordingly, this parameter is always filled with a non-null value for each detected source.

      mosaic_image.ds: image of the final mosaic image created by the SAS task emosaic.

      It is possible to display the detected sources on top of the mosaic image,

         srcdisplay imageset=./mosaic/mosaic_image.ds boxlistset=./mosaic/mosaic_eboxlist.fits

       

      Mosaic image of the combined observation, including all three EPIC cameras, with detected sources overlaid.

    3. For each pointing a series of products are also created which are stored inside the corresponding prep_mosaic_* directory. For example, images are produced for each one of the specified energy ranges and EPIC instrument. As shown in the figure below, it is then possible to produce a mosaic image, for display purposes only, by combining the images over the three energy ranges specified and three EPIC cameras,

         emosaic imagesets='./prep_mosaic_001/image_EPN_e0.fits ./prep_mosaic_001/image_EPN_e1.fits \
                         ./prep_mosaic_001/image_EPN_e2.fits ./prep_mosaic_001/image_EMOS1_e0.fits \
                         ./prep_mosaic_001/image_EMOS1_e1.fits ./prep_mosaic_001/image_EMOS1_e2.fits \
                         ./prep_mosaic_001/image_EMOS2_e0.fits ./prep_mosaic_001/image_EMOS2_e1.fits \
                         ./prep_mosaic_001/image_EMOS2_e2.fits' \
                 mosaicedset='./prep_mosaic_001/prep_mosaic_001.ds'

      Exposure maps (with and without vignetting, needed for mask generation), detection masks, and background maps are also created together with a source list (eboxlist_l.fits) (which is the result of running source detection combined for all cameras and energy bands simultaneously) for each individual pointing which can be used to display the sources overlaid in the mosaic image,

         srcdisplay imageset=./prep_mosaic_001/prep_mosaic_001.ds \
                 boxlistset=./prep_mosaic_001/eboxlist_l.fits

       
      From top left to bottom right, mosaic images of each one of the individual pointings (including the three EPIC cameras and three energy ranges), prep_mosaic_001, prep_mosaic_002, prep_mosaic_003, prep_mosaic_004 with detected sources overlaid.

 

Caveats

With the process explained here, it is also possible to carry out source detection on overlapping fields from different observations. Both SAS tasks described above have to be used in a similar manner, where emosaic_prep has to be run first on each individual observation to create the corresponding ./prep_mosaic_* directories, even if only one pointing per observation is present. In this case of overlapping fields from different observations, the parameter pseudoexpid should be used when calling emosaic_prep for each observation. For each call, pseudoexpid should take values that are spaced by at least the number of EPIC exposures in the preceding observation. The maximum value that pseudoexpid can have is 99. Once done, emosaicproc can be run by selecting any given combination of observations and cameras.

Note that the required computer memory is proportional to the number of input images and the size of each input image. For example, for a run with 36 input images, eboxdetect requires more than 1 Gbyte of memory and emldetect more than 700 Mbyte. This can be reduced by reducing the number of energy bands.