I spent a week at another accelerator, this time at Michigan State University. Again, I provided data acquisition, computer, network, and data distribution support to our research group.
Our main data acquisition computer was left behind at Caltech to complete some tests while we were setting up at MSU, and I had been receiving enough reports of system crashes that I didn’t trust the machine. When it finally arrived, one day before our run began, I tried doing a clone to an external drive, when fsck noted corrupted filesystem data and began trying, endlessly, to repair the system. I spent the entire morning and a bit of the afternoon trying to clone the drive, and then I finally reinstalled Ubuntu and the software on the secondary SSD. I spent hours trying to locate and install 32 bit libraries and get rsync working again. I finally got it working by evening.
Everything went smoothly during the start of the run when we began noticing that binary streaming to disk was stopping. Investigation of error messages indicated malformed data, and I wrote a kludge to to keep the stream flowing. Rick quickly found one poorly formed packet in his data, so we have a clue where to pursue it.
Also, in reconstructing the OS and program, I didn’t completely restore some settings, and not all of my recordings were complete. I ended up going to other streamed data to reconstruct the stopped data, and I restored the recording settings.
Nigel kindly brought ice cream bars for snacks sometime during the run. I had coconut. (Second photo. The top photo is of our work area in the data room.)
That said, with all my work, the data flowed smoothly overall, and I’m pretty happy with my contribution to the run.