• TCS comms with VIRUS/LRS2
• TCS comms with HPF

If we have the situation where TCS information (STRUCTAZ*, etc) is not being communicated to LRS2/VIRUS (i.e., those values are -9999s in the FITS file headers), try the following.

• restart the VIRUS (or LRS2) server only - check if it is fixed
• restart both the instrument server and the TCS server

As of Nov 2021, Jason says that syscmd -V 'reset_tcs()' (as described below) cannot possibly help. Restarting the VIRUS or LRS2 server (not muxes or controllers) should be enough to restore TCS communications.

If the instruments are still not receiving TCS information, try running:

  syscmd -V 'reset_tcs()'

and

  syscmd -l 'reset_tcs()'

in the LRS2 and VIRUS servers. That may fix the problem.

This is the help statement (from JR):

camra::reset_tcs (10228)

Flags the socket connection to tcs for resetting. The reset occurs on the next subsequent failed ACK from tcs.

HPF TCS communications problems

HPF has a TCS telemetry relay running on its side to listen to the TCS events and encode those in HPF data file headers. Sometimes this connection is broken and causes serious problems with missing metadata. The solution we have been using (as recommended by Chad Bender in Feb 2019) is to restart HPF's TCS relay and the HPF's TCS client, as described below:

1. First, if necessary, restart the HET TCS server (if you suspect the problem originated there). The TO does this operation.

2. Second, restart the HPF TCS relay:
   • open a new terminal, log in to hpf (e.g., ssh hpf@hpf and run tcsrelay
   • confirm that you are seeing the correct screen - you should see messages like the text and screenshot below. (if window is not the right one use 'Ctrl-a n' or 'Ctrl-a p' to find it)

2023-05-11 20:21:54.181838 : Relayed 22160000 messages

• CTRL-C the script running there (this is a script, not a TIMS client). You will need to CTRL-C twice (It took 3x for me once -SJ) to actually kill it.
• Restart the script (up arrow or ./bashwrapperfor_HETTCSRebroadcast.sh)
• After that you can detach from this screen with ctrl-A D

3. Restart the TIMS HET Client once more to be safe. Two options:
   • During OPS, use the OPS web page to click "TCS Client Restart"
   • OR, on in the "Scripts" screen session, run: tims het shutdown, and then in the HPF HET Client screen session, run: timshet

It may also help to review our Screens Summary Page