So. Hopefully FINAL update on this one. It turns out the fix we thought was the fix was not in fact fixy enough to be the fix.
In retrospect, I now believe that the the intermittent issues we had might be related to the addition of the IP security cameras on our campus and a lack of
VLAN capability with the existing switches. IP cameras as I understand them will vary the bandwidth used from idle states to triggering states. I know from my home setup using Zoneminder, I can configure them to output the full stream or setup a secondary stream with lower framerate and there are ways to modify the camera itself to initiate the alarm state vs. the server. So there's a potential for a lot of bandwidth to be used when the building is occupied vs. when there's no one around. Hence, I think that might be why it worked great in the evenings when most of the campus was quiet, vs
Sunday mornings when there were more bustling people.
Anyways, we punted. We invested in 300' of
conduit and 550' of fiber. We pulled the fiber in Thursday and earlier today we patched everything in and I can honestly say I've never been happier to see a little green light come up. So now our setup includes a broadcast room
switch that connects to the nearby booth via copper to a booth
switch and the far booth via fiber to another
switch. We put a hardware
router in between our main
switch and the
network, so now I have control over DHCP and whatever traffic passes over our
network to the larger
network.
Of course. Tomorrow is
Sunday. I might be posting this celebratory post prematurely, but when I left earlier this afternoon we had cameras and audio from all corners of our little world. Fingers crossed.