Files names and extensions used for FOS (Y)

ST-ECF names STSDAS suffix File content Example
Raw data files
yxxxxxxxx.d0f .d0h/.d0d Science data image Y0DV0206R.D0F
yxxxxxxxx.q0f .q0h/.q0d Science data quality Y0DV0206R.Q0F
yxxxxxxxx.shf .shh/.shd Standard header packet Y0DV0206R.SHF
yxxxxxxxx.d1f .d1h/.d1d Science trailer line NOT THERE Y0DV0206R.D1F
yxxxxxxxx.ulf .ulh/.uld Unique data log Y0DV0206R.ULF
yxxxxxxxx.x0f .x0h/.x0d Science header line Y0DV0206R.X0F
yxxxxxxxx.xqf .xqh/.xqd Science header line data quality Y0DV0206R.XQF
yxxxxxxxx.q1f .q1h/.q1d Science trailer line data quality NOT THERE  
Calibrated data files
yxxxxxxxx.c0 .c0h/.c0d Calibrated wavelengths Y0DV0206R.C0
yxxxxxxxx.c1 .c1h/.c1d Calibrated fluxes Y0DV0206R.C1
yxxxxxxxx.c2 .c2h/.c2d Propagated statistical error Y0DV0206R.C2
yxxxxxxxx.c3 .c3h/.c3d Special statistics Y0DV0206R.C3
yxxxxxxxx.c4 .c4h/.c4d Count rate Y0DV0206R.C4
yxxxxxxxx.c5 .c5h/.c5d Flat-fielded object spectra Y0DV0206R.C5
yxxxxxxxx.c6 .c6h/.c6d Flat-fielded sky spectra Y0DV0206R.C6
yxxxxxxxx.c7 .c7h/.c7d Background spectra Y0DV0206R.C7
yxxxxxxxx.c8 .c8h/.c8d Flat-fielded object minus smoothed sky spectra NOT THERE Y0DV0206R.C8
yxxxxxxxx.cq .cqh/.cqd Calibrated science data quality NOT THERE Y0DV0206R.CQ
yxxxxxxxx.trl .trl Trailer file .trl Y0DV0206R.TRL
Additional files
yxxxxxxxx.jif .jih/.jit Jitter files (.cmh, .cmj, .cmi) .jih/.jit NOT THERE Y0DV0206R.JIF
yxxxxxxxx.pdq .pdq PODPS Y0DV0206R.PDQ
yxxxxxxxx.poa .poa Post observation archive (ST-ECF products ) Y0DV0206R.POA

Uncalibrated (Raw) Data Files

The science files that are used as required input to calfos are listed in the above table. These files are described briefly below.

Science Data Files (.d0h/.d0d)

Science data files contain single-precision floating point values that represent the number of detected counts accumulated in each pixel. The number of data elements in the one-dimensional science data array depends on the observation mode; specifically, the number of diodes, the number of substeps, the number of Y steps, and the number of repeats (sometimes called slices or bins) used in the observation. The maximum number of data elements is 12288. The associated header file also provides information on the different steps to be performed during pipeline calibration processing, and the reference files and tables to be used in the calibration.

Science Header Line (.x0h/.x0d)

The science header line (SHL) file is a one dimensional array with a length equal to a line of the science data. It contains a partial copy of the unique data log.

Science Trailer Line (.d1h/.d1d)

The science trailer line (STL) file is also a one dimensional array containing the number of measurements rejected from the various combinations of x substeps, y steps, repeats, etc. The information in these files is used to compute the total effective exposure time per pixel which is later used to convert the counts into count rates.

Data Quality Files (.q1h/.q1d)

The science data files, science header line files, and the science trailer files have corresponding data quality files that contain the flags for bad or suspect data. These raw data quality files have quality flags as follows:

  • Good data has the data quality flag =1.
  • Raw data dropouts and filled raw data have the data quality flag =16.
  • Data failing a Reed-Solomon error check has the data quality flag =100.
  • Fill data have the data quality flag =800.

The data quality files are identified by the suffixes .q0h, .q0d, .xqh, .xqd, .q1h, and .q1d corresponding to the science data, science header, and science trailer files. Table 30.2 lists FOS data quality flags.

Standard Header Packet (.shh/.shd)

The standard header packet (SHP) contains the telemetry values for engineering data and some FOS-unique data. The engineering data include temperatures, currents, and voltages at various points in the instrument. The FOS-unique data varies depending on the onboard processing used for a given observation. The header packet also contains information used in the operation of the spacecraft, such as target name, position and velocity of the telescope, the right ascension and declination of the target, the sun, and the moon, and other proposal information used in the observation which was provided in Phase II of the proposal process.

Unique Data Log (.ulh/.uld)

The unique data log (UDL) contains the mechanism control blocks used to control the entrance aperture, entrance port, polarizer, and filter grating wheel assembly. This file also contains the discriminator level, disabled diode table, serial engineering data, instrument configuration, and exposure parameters.

Trailer File (.trl)

The trailer file contains many messages generated by the so-called "generic conversion" of the data from what was onboard the spacecraft into STSDAS images. These messages may contain information on missing or filled data packets. Informational messages produced by calfos during calibration are also stored in the trailer file. The trailer file is identified by the suffix .trl.

Calibrated Data Files

Several types of calibrated output files are produced by calfos. These are listed in Table 30.1. More extensive descriptions of each type of file are provided below.

Calibrated Wavelength Files (.c0h/.c0d)

These files contain single-precision floating point calibrated vacuum wavelengths corresponding to the center of each pixel of the science data. All FOS wavelength solutions assume the first pixel is pixel 0.

Calibrated Flux Files (.c1h/.c1d)

These files contain single-precision floating point calibrated fluxes (in ergs sec-1 cm-2 A-1) corresponding to each pixel of the science data.

Calibrated Statistical Error Files (.c2h/.c2d)

These files contain the statistical errors of the original data. These files are calibrated in lock-step with the science data files. Errors caused by sky and background subtraction, flatfields, and sensitivity are not included in the error estimates.

Calibrated Special Mode Data Files (.c3h/.c3d)

Data acquired in the rapid-readout, time-resolved, or spectropolarimetry modes require processing steps in addition to (or complementing) those used for standard ACCUM data. The calibrated data are then stored in special mode data files. For RAPID mode, the files contain the total flux, integrated over all pixels, and the associated statistical error for each readout. For TIME RESOLVED mode, the files contain the pixel-by-pixel average of all slices or bins, the difference between each slice or bin and the average, and the average propagated statistical errors. For POLARIMETRY mode, the file contains the Stokes I, Q, U, and V parameters, the linear and circular polarization, and the polarization position angle. The polarimetric quantities and the propagated errors are calculated for each of the separate pass directions, the combined pass direction data, and the combined pass direction corrected for interference and instrumental orientation.

Calibrated Data Quality Files (.cqh/.cqd)

The quality flags in these files flag the bad pixel values in the calibrated files. The quality flags from the raw data are updated and additional flags are added for problems detected in the calibration process. The data quality flags are defined in Table 30.2.

Intermediate Calibrated Output Data Files (.c4* - c8*)

At most, six sets of intermediate calibrated output files are produced depending on the observation mode.

Count Rate Object (and Sky) Spectra: The files containing the count rate spectra are corrected for overscanning, noise rejection, and lost signal caused by disabled diodes. These files are identified by the suffixes .c4h and .c4d.

Value Description
Category 1: Data not useful. Data values set to zero.
800 Data filled
700 Data filled due to GIM correction
400 Disabled channel
300 Severe saturation (uncertainty greater than 50%)
200 Inverse sensitivity invalid (λ < 1100 Å or λ > 7000 Å)
Category 2: Data Uncertain. Uncertainty not Indicated in Error Calc.
190 Large saturation correction (uncertainty greater than 20%)
170 Intermittent noisy channel
160 Intermittent dead channel
130 Moderate saturation correction (uncertainty greater than 5%)
120 Sky or background fixed or extrapolated
100 Reed-Solomon decoding error
Category 3: Data uncertain. Uncertainty in propagated error file.
50 Sampling less than 50% of nominal
  • Flatfielded Object Count Rate Spectrum: These files contain .c4 object-only data further corrected for paired pulses, detector background, flatfield structure and, if applicable, pipeline corrected for GIM effects. The flatfielded object spectrum files are identified by the suffixes .c5h and .c5d.
  • Flatfielded Sky Count Rate Spectrum: These files contain .c4 sky-only data further corrected for flatfield structure and, if applicable, also pipeline corrected for GIM effects. The flatfielded sky spectrum files are produced only if a sky observation was obtained. These files are identified by the suffixes .c6h and .c6d.
  • Background Count Rate Spectrum: The scaled background spectrum that has been subtracted from the .c4 product is identified by the suffixes .c7h and .c7d.
  • Flatfielded and sky-subtracted Object Count Rate Spectrum: If the sky is observed, then a smoothed sky-subtracted object spectrum prior to flux calibration is produced. The files containing the smoothed sky-subtracted object spectrum are identified by the suffixes .c8h and .c8d.

Trailer File (.trl)

We list the trailer file again as any log comments written by the pipeline calfos procedures are appended to spacecraft and generic conversion information already in the file. The trailer file is identified by the suffix .trl.