wiki:HetProcedures/RA/dailychecks

Version 13 (modified by stevenj, 7 months ago) (diff)

--

Here are some things that need to be done regularly for science operations.

Currently most of these are done by the Science Operation Manager, although some of that may be flexible in the future as our operational model evolves.


Daily checks

Before the 8:15am morning meeting:

  1. Check the NR Reader for last night
    • read PRs, OPS/night comments, respond/reply appropriately
    • also read the night RA's morning email summary
  1. Check the time accounting graph for last night
    • check that we have an activity entry at 18-deg evening twilight
    • check that there is no gap between stacking and the first target
    • check that all non-science activity entries have comments (e.g., idle/engineering)
    • check that all targets are consistent with the activity at that time (sci vs eng)
    • check for any un-used time/gaps and correct errors (or email RA if necessary)
    • eliminate "queue holes" that can be rolled into setup if OH remains <= 10min
    • add and increment targets to the -000 ENG program if appropriate to charge time for operational engineering activity entries
    • check for any other weird issues or obvious errors and take appropriate action
    • consider running nrcheck.py on zeus to catch any obvious issues
  1. Check the RA Night Report on nossy
    • check that all 'A/B' targets were incremented
    • check that no rejected targets were incremented
    • check for any crazy overheads
    • check that all setups with OH>10min have comments
    • check for any other weird issues or obvious errors
      • missing program IDs or other metadata?
      • acceptance/rejection errors or issues?
  1. Look at the gcplot output from last night
    • in /data1/archive/gcplot/gcplot_YYYYMMDD_targets.pdf
    • verify all targets were observed within their requested IQ/skysb/skytrans limits
    • verify all track choices seem sane
  1. Update (manually) exposure times on all OCD-extended-exposures
    • effective 21-2, all OCD observations are charged at the on-sky exposure time, not the nominal/requested
    • use htopx to edit the value of exp on all accepted VIRUS targets observed with OCD that are not 1080s (DEX, DFW, G09, NEP/TESLA, GAMA, SSA, etc)
  1. Check the Daily Site Status report and the Daily Telemetry Report via emails
    • look for anything crazy/weird
    • check that plot of pointing corrections looks sane

Join the 8:15am morning meeting whenever possible to contribute as needed to any PR-related discussions and field any operational questions about morning calibrations and that day's work plans.

After the "new data" emails go out at 10:30am CST / 11:30am CDT

  1. Check each of these programs' % Used at each priority here
    • For programs which include time shared/joined between multiple programs, log in to Hydra and access the PI-view for up-to-date totals (may need to add yourself as co-I to see it)
    • If a program has <0.2h remaining at any priority and still has targets active (i.e., <100% in the % Complete row), email the PI to let them know that allocation has been completed and they should defer their active targets.
      • Judgement can be used here w/r/t time balancing -- e.g., if Munich is lagging behind their partner share and a Munich program is poised to modestly exceed their allocation, I generally take no action.

Generally pay attention to OPS/night emails, address problems raised, answer questions, help improve operations and efficiency.


Weekly checks

  • review FCU lamp flux levels using LRS2 cals over the past week
  • evaluate latest week of ACQ stability test data
  • evaluate weekly HPF pedestal level graph
  • review per-RA priority and dTopt average statistics (currently broken)
  • approve timesheets in Workday and update timeoff spreadsheet
  • collect and summarize project updates in ongoing night staff project reports (maybe ~monthly?)
  • attend Tuesday software meeting and share updates/input on night staff projects or other science operations issues
  • Organize colloquium-viewing at AL lower level meeting room
    • Monday: send email reminder if there is a talk
    • Monday: confirm with Karen at AL about cookies (usually order 3-5)
    • Tuesday: come at ~3:20 to set up screen, projector, computer

bi-weekly: attend the bi-weekly telecons: give science operations report (edit document several days before meeting), answer questions, give feedback as required


Monthly checks

  • eng/gto program status updates
  • update partner time charged statistics, monitor for and anticipate any problems with distribution


Mid-trimester checks

  • monitor completion rates of all science programs, send updates to PIs
  • identify targets in the queue which are impossible/unlikely to be observed, work with PIs to improve feasibility if possible


Quarterly checks

  • solicit time-off requests, make schedule for OPS, nights, and day OPS
  • participate in HET Users Committee meetings (may require generating reports, responding to feature requests, etc)


Trimesterly tasks

  • create begin/end trimester reports
  • solicit Eng/GTO time requests, compile and send to HET Board
  • calculate time available each trimester and inform partners of their science time


Biannual tasks

  • give Science Operations report at the HET Board meetings in ~June and ~December


Annual tasks

  • complete annual personnel evaluations/appraisals