ACS File Names - HST
Files names and extensions used for ACS (J)
The file suffixes given to ACS data products are described in the table below and closely mimic the suffixes used by STIS. The initial input files to the calibration pipeline are the raw (RAW) files from Generic Conversion and the association (ASN) table, if applicable, for the complete observation set.
For CCD images, a temporary file, with the suffix BLV_TMP, is created once bias levels are subtracted and the overscan regions are trimmed. This file is renamed with the FLT extension after the standard calibrations (flat fielding, dark subtraction, etc.) are complete. The FLT files will serve as input for cosmic ray rejection (if required). For CR-SPLIT exposures, a temporary 'cr-combined image' (CRJ_TMP) is created and then renamed with the CRJ extension once basic calibrations are complete. Single MAMA images are given the FLT suffix once calibrations are complete. By definition, these images do not have an overscan region and are not affected by cosmic rays. The calibrated products of a REPEAT-OBS association will be several individually calibrated FLT exposures and a summed flat-fielded (but not cosmic-ray cleaned) SFL image.
|jxxxxxxxx_asn||_ASN||Association file for observation set||J8C032050_ASN|
|jxxxxxxxx_spt||_SPT||Telemetry and engineering data||J8C032XUQ_SPT|
|jxxxxxxxx_trl||_TRL||Trailer File with processing comments||J8C032XUQ_TRL|
|jxxxxxxxx_flt||_FLT||Calibrated, Flat fielded individual exposure||electrons||J8C032XUQ_FLT|
|jxxxxxxxx_crj||_CRJ||Calibrated, CR-rejected, Combined image||electrons||J8C032XUQ_CRJ|
|jxxxxxxxx_sfl||_SFL||Calibrated, Repeat-Obs, Combined image||electrons||J8C032XUQ_SFL|
|jxxxxxxxx_drz||_DRZ||Calibrated, Geometrically Corrected, Dither-Combined image||electrons/sec||J8C032050_DRZ|
Association tables are useful for keeping track of the complex set of relationships that can exist between exposures taken with ACS, especially with REPEAT-OBS, CR-SPLIT, and dithered exposures. Images taken at a given dither position may be additionally CR-SPLIT into multiple exposures. In these cases, associations are built to describe how each exposure relates to the desired final product. As a result, ACS associations will create one or more science products from the input exposures, unlike NICMOS or STIS associations. The relationships defined in the association table determine how far through the calibration pipeline the exposures are processed and when the calibrated exposures are combined into sub-products for further calibration. ACS data files are given the following definitions:
- An exposure is a single image, the "atomic unit" of HST data.
- A dataset is a collection of files having a common root name (first 9 characters).
- A sub-product is a dataset created by combining a subset of the exposures in an association.
- A product is a dataset created by combining sub-products of an association.
ACS association tables were designed to closely resemble the NICMOS association format, with three primary columns: MEMNAME, MEMTYPE, and MEMPRSNT. The column MEMNAME gives the name of each exposure making up the association and output product name(s). The column MEMTYPE specifies the role the file has in the association. A unique set of MEMTYPES specific to ACS were adopted to provide the support for multiple products.
Exposure types in ACS associations. The suffix "n" is appended to the MEMTYPE when multiple sets are present within a single association.
|EXP-CRJ||Input CR-SPLIT exposure (single set)|
|EXP-CRn||Input CR-SPLIT exposure for CR-combined image n (multiple sets)|
|PROD-CRJ||CR-combined output product (single set)|
|PROD-CRn||CR-combined output product n (multiple sets)|
|EXP-RPT||Input REPEAT-OBS exposure (single set)|
|EXP-RPn||Input REPEAT-OBS exposure for repeated image n (multiple sets)|
|PROD-RPT||REPEAT-OBS combined output product (single set)|
|PROD-RPn||REPEAT-OBS combined output product n (multiple sets)|
|EXP-DTH||Input dither exposure|
|PROD-DTH||Dither-combined output product|
A sample association table for a two-position dithered observation with CR-SPLIT=2 is presented in the table below. This example shows how both MEMNAME and MEMTYPE are used to associate input and output products. The MEMTYPE for each component of the first CR-SPLIT exposure, JxxxxxECM and JxxxxxEGM, are given the type EXP-CR1. The sub-product Jxxxxx011 is designated in the table with a MEMTYPE of PROD-CR1. The last digit of the product filename corresponds to the output product number in the MEMTYPE. A designation of zero for the last digit in the filename is reserved for the dither-combined product.
The column MEMPRSNT indicates whether a given file already exists. For example, if cosmic ray rejection has not yet been performed by CALACS, the PROD-CRn files will have a MEMPRSNT value of "no". The sample association table below shows the values of MEMPRSNT prior to CALACS processing.
Sample Association Table Jxxxxx010_ASN
Each task in the CALACS package creates messages during processing which describe the progress of the calibration and which are sent to STDOUT. In the pipeline processing for other HST instruments, trailer files were created by simply redirecting the STDOUT to a file. Because multiple output files can be processed in a single run of CALACS, creating trailer files presents a unique challenge. Each task within the CALACS package must decide which trailer file should be appended with comments and automatically open, populate, and close each trailer file.
CALACS will always overwrite information in trailer files from previous runs of CALACS while preserving any comments generated by Generic Conversion. This ensures that the trailer files accurately reflect the most recent processing performed. The string CALACSBEG will mark the first comment added to the trailer file. If a trailer file already exists, CALACS will search for this string to determine where to append processing comments. If it is not found, the string will be written to the end of the file and all comments will follow. Thus any comments from previous processing are overwritten and only the most current calibrations are recorded.
As each image is processed, an accompanying trailer file with the '.trl' ending will be created. Further processing will concatenate all trailer files associated with the output product into a single file. Additional messages will then be appended to this concatenated file. Thus, some information is duplicated across multiple trailer files, but for any product processed within the pipeline, the trailer file is ensured to contain processing comments from each input file.
Linking trailer files together can result in multiple occurrences of the CALACSBEG string. Only the first, however, determines where CALACS will begin overwriting comments if an observation is reprocessed.
The ACS calibration pipeline assembles data received from HST into datasets, applies the standard calibrations, and stores the uncalibrated datasets in the HST Data Archive. The structure of these data products is based on the STIS and NICMOS file format and consists of multi-extension FITS files which store science (SCI), data quality (DQ) and error (ERR) arrays.
ACS WFC data come from two CCD chips and are treated as separate observations, with a SCI, DQ and ERR array for each chip, and with both chips being stored in the same FITS file. The result is a FITS file with 6 data extensions plus a global header for a single WFC exposure. The WFC apertures are plotted with respect to the V2/V3 reference frame in Figure 2.2 in the Data Handbook and are oriented such that the x-axis is approximately to the right and the y-axis is approximately straight up. For pipeline data products, chip 2 is below chip 1 in y-pixel coordinates and was therefore defined as extension 1. Thus, the chip/extension notation is counterintuitive. To display the science image for chip 1, the user must specify the extension 'file.fits[sci,2]'. Similarly, the data quality and error array for chip 1 are specified as [dq,2] and [err,2].
A single HRC or SBC exposure comes from a single chip and has only 3 data extensions plus a global header. While the raw and calibrated WFC images contain 6 data extensions, the drizzled product will always contain 3 data extensions, no matter which detector was used. When WFC data is drizzled, both chips are included in a single FITS extension.
Figure 2.1 illustrates the basic format for storing ACS images. An uncalibrated (RAW) exposure contains a primary header plus, for each chip, a SCI extension (in 16-bit integer format), an empty ERR extension, and an empty DQ array. The calibrated product from CALACS contains a primary header plus, for each chip, a SCI extension (in 32-bit float format), an ERR extension (32-bit float), and a DQ extension (16-bit integer).
MultiDrizzle removes the effects of the geometric distortion and produces a drizzled (DRZ) product which has the multi-extension FITS file shown in Figure 2.1. The SCI extension contains the distortion-corrected data (32-bit float), the WHT extension contains the weight mask (32-bit float), and the CTX extension contains the context image (32-bit integer).