Calibrating the probes
Flat fields for Guide Probes
Whenever things change with the guide probes (FPA takedown, new camera installation, etc) new flat fields may be required. The following procedure describes how to make those. Broadly, there are three parts: obtaining suitable images for flat fields, creating the flat fields, and installing the flat fields.
Obtaining data for flat fields
During evening/morning twilight (or thin clouds with moonlight), suitable data can be taken to generate new flats.
In each filter (Clear, B, g, r, i) you need at least 20 "bright" images (counts ~40k max) and at least 20 "faint" images (counts ~10k max). Some day there might be an automated script to take this data.
Starting out, I'll assume it is twilight (or bright moonlight), the guider pipelines are running, and you've started images displaying. In the "GC1" and "GC2" tabs in the TCS GUI, make sure "Store" is checked, and that you are viewing the "Raw" extension of the images.
Starting with one of the filters, adjust the exposure time until a bright (~40k) flat is obtained. Allow at least 20 exposures to be taken. Adjust exposure time again until a faint (~10k) flat is obtained. Allow at least 20 exposures to be taken. Repeat with all filters and on both GC1 and GC2.
Creating flat fields from data
Step 0: check that the scripts/codes are up to date:
These three IRAF scripts live in the SVN under: trunk/scripting/raScripts/ and are installed at /opt/het/hetdex/bin/ (where they are inexplicably executable), so these are the paths which are used in the /home/mcs/astronomer/iraf/login.cl file.
Also, you should probably create a new directory and cd to it within IRAF, so you don't pollute the iraf directory. It would be good to name the directory with the date of the flat fields.
Step 1: determining which stored images should be used:
SJ's IRAF script getgcflats will collect data from a given night. From a relatively clean/empty directory in IRAF (ecl), run: getgcflats 20190414 (N.B., you will need to epar this task to set the UT date and UT time range when you took good data for flat fields, the default data ranges should be correct).
This script will look at all of the stored GC1 and GC2 images from that night and find those which have acceptable sky levels to be used in flat-fielding. An example of a good set of parameters is listed below, which runs for a night when I took appropriate exposures during morning twilight:
PACKAGE = clpackage TASK = getgcflats utdate = 20190414 UT date to get GC images (time_st= 11:53) data start time like HH:MM (time_st= 12:25) data stop time like HH:MM (fnt_min= 8000.) min median for faint flats (fnt_max= 23000.) max median for faint flats (brt_min= 32000.) min median for bright flats (brt_max= 44000.) max median for bright flats (good_mi= 12000.) min median for good flats (good_ma= 55000.) max median for good flats (minim = 10) minimum acceptable number of images (showf = yes) display all images for visual inspection (cleanup= yes) delete intermediate files (list1 = ) (mode = ql)
(NOTE: usually aim for fnt_min,fnt_max = 8000,10000; brt_min,brt_max = 45000,55000; but I didn't take perfect exposures for one filter here, so this works...)
It will display all of the images it finds in DS9 so you can verify there aren't any with guide-stars in them or anything else bad.
You may need to change the start/top times with epar - they are not queried when you run the script.
The output of this script are 2x5x2 = 20 lists (GC1/GC2, B/Clear/g/r/i, hi/lo) which will be used by the next script(s).
Step 2: create bad-pixel masks (BPMs):
SJ's other IRAF script gcbpm will be run to create BPMs for each filter on GC1 and GC2. You'll need to have DS9 running when you run it because it displays all of the masks. Typically using a threshold of 0.5 works well, but sometimes 0.1 gives nicer results. These are the things it does:
- combine all "bright" images (per GC, per filter)
- combine all "faint" image ()
- determine medians of central region of images
- make ratio image where ratio = (40k * bright_im / bright_med) / (10k * faint_im / faint_med)
- in this ratio image, pixels above the threshold (~0.5 is ok) are "good", and below go into the BPM
(Note that these BPMs are only important for normalizing the flat fields, not for the TCS bad pixel masks.)
PACKAGE = clpackage TASK = gcbpm (ratthres = 0.5) ratio threshold, probably 0.5 (cleanup = yes) delete hi/lo files and lists when finished (list1 = ) (mode = ql)
Step 3: combining and creating the flat fields:
SJ's other other IRAF script gcflat will be run on each "good" list and:
- subtracts a constant bias level from each "good" image (1k for GC1, 2k for GC2)
- masks each "good" image using the BPM (bad values to -999)
- combines all "good" images ignoring "bad" pixels
- normalizes the mode of the good pixels to unity
- re-names the flat field consistent with previous flats in use
PACKAGE = clpackage
TASK = gcflat
input = "@name.list" List of images to analyse (like @r.list)
utdate = "20201001"
(cleanup = yes)
(mode = ql)
These are the commands to create all flats for both guiders everything: (for the above example, using 20220813 data)
gcflat @gc1_g_good.list 20220813 gcflat @gc1_r_good.list 20220813 gcflat @gc1_i_good.list 20220813 gcflat @gc1_B_good.list 20220813 gcflat @gc1_Clear_good.list 20220813 gcflat @gc2_g_good.list 20220813 gcflat @gc2_r_good.list 20220813 gcflat @gc2_i_good.list 20220813 gcflat @gc2_B_good.list 20220813 gcflat @gc2_Clear_good.list 20220813
This will produce output files like: Guider1Camera_g`_20220813.fits Verify that these flats look good. Then you can delete all of the lists and BPMs if you want, or keep them for later.
For long-term organization, create a new directory in /home/mcs/astronomer/guider_flats to document when these flats were taken and who processed them. Move the fits files and image lists into that directory.
Installing new flat fields
The GC flat fields which are currently in use live in: /opt/het/hetdex/etc/configfiles/flats/ which is accessible to user "hetdex". You can't make permanent changes to these files, but you can replace the existing flats and then they will be used (until they do the next software installation).
- Copy the newly-produced flat fields to that location (with the automatically-generated file names, removing the date and "camera"), as user "hetdex".
- Restart PAS and the TCS GUI
- Turn on processing/cleaning and see if the results have improved (iterate on this procedure if necessary).
A better option is to commit these to the hetdex-pr repository under: trunk/pas/etc/flats/ and let Jim know to install that change.
Installing new masks
The current masks live here: /opt/het/hetdex/etc/configfiles/masks/ and should rarely need updating. These files are a list of points in x,y image coordinates (CW or CCW, but must be consistent) describing the boundary of the useful guider field of view. Change if necessary. Note that the BPMs you produced with gcbpm are FITS images of 0s and 1s, which is not the same format as TCS requires. (Furthermore, the BPMs from the flat-field process are optimized for normalization purposes, and TCS wants more inclusive masks which include all use-able pixels.)
Photometric zero points for Guide Probes
work in progress... ... calibrate these for good measurements of sky brightness and guide star flux (for transparency estimates)
