Digital Audio Drop Out

MRW Lights

Well-Known Member
Coming here with this one to see if anyone can come up with what we haven't... and for anyone not on various other Dante forums

We operate a broadcast facility as part of a large academic campus and we have several locations tied together to a central control center. One venue that operates Dante is fed via fiber to our central control, converted to madi and lastly madi coax to madi fiber for an SSL network for capture and broadcast. Both locations are synced with a GPS word clock. Somewhere, somehow, we are experiencing a drift that hits a limit and causes audio to drop for about 15 seconds and then recover. It happens randomly, usually back to back within a couple of minutes. Typically in a 2 hour show IF it happens, it happens twice and then not again until the next time.

signal chain
Yamaha/Dante network + 2 hops between locations, Focusrite Rednet D64R Dante>Madi, SSL Madi coax fiber converter, SSL Audio Router. The yamaha remote network gets word clock, the focusrite gets word clock, the router gets word clock. All of them scope within spec and we're working on a way to measure drift, but there really "shouldn't" be any. No audible artifacts in the audio like clock clicks or pops... can't for the life of me figure out why we sometimes, randomly, lose audio for about 15 seconds and then it comes back. Even weirder that when it happens it almost immediately happens again within 2-3 minutes and then could happen the same day or not again for months.... at least that we experience while actively monitoring. No errors show up in the system, latency on the dante side is fide, madi side works fine with 2 other remote locations with the same signal path...

Who has a better guess than me?
 
Are you seeing Dante clock leader changing when you view through log file?
 
Are you seeing Dante clock leader changing when you view through log file?

Supposedly it's not changing according to the remote location, but I did ask for their logs from the most recent incident, though I suspect they weren't running dante controller, so I've asked for a 24 hour log pull just to see if anything is happening on their end. Even if it is, theoretically their system is locked to our gps clock at their console end, it's clocked again when it gets to us as a boundary clock and every conversion point is clocked to the same clock. So... I would suspect any drift to be hardware failure, but at any given time it shouldn't be able to drift far enough to fall out of sync. "shouldn't" being my dying breath in this setup.
 
Supposedly it's not changing according to the remote location, but I did ask for their logs from the most recent incident, though I suspect they weren't running dante controller, so I've asked for a 24 hour log pull just to see if anything is happening on their end. Even if it is, theoretically their system is locked to our gps clock at their console end, it's clocked again when it gets to us as a boundary clock and every conversion point is clocked to the same clock. So... I would suspect any drift to be hardware failure, but at any given time it shouldn't be able to drift far enough to fall out of sync. "shouldn't" being my dying breath in this setup.

Where I was going - to me it doesn't sound like a loss of sync (initially at least) but that there is a compromise in the Dante leader clock that is causing Dante to elect a new leader. And that's what I'd look for in the logs, starting out anyway.

Good luck!
 
Where I was going - to me it doesn't sound like a loss of sync (initially at least) but that there is a compromise in the Dante leader clock that is causing Dante to elect a new leader. And that's what I'd look for in the logs, starting out anyway.

Good luck!

This is plausible, but if it's happening it would be at the remote location and I would expect them to hear the issue occur locally as well. It seems to only happen in the connection between the two buildings. I'm still trying to get logs from the other location... fingers crossed.
 
And if so, are *all the GPS clock masters* the same make, model, and FW release?

1 GPS Clock and all locations sync to that clock. I'm starting to gather more information that makes it seem more like a clock election issue at a specific remote location. We're working with an alternative at the moment and living to search the rabbit hole another day.
 
Solved... I HOPE!

By luck I was on the sound board when a drop out happened and got to watch / trace it in real time and have a timecode stamp to go back and review the signal path. We run an extensive enterprise broadcast network that is governed by a master GPS clock. "seeing" the issue happen in real time at the sound board, not being in our engineering shop or on another station, I was able to trace it all the way back to our clock. A quick call to a fellow engineer who is a service tech for that clock turned out a pesky reverse time jam where the clock can be set to auto sync to the satellite which if it's off far enough can cause the clock to restart. :doh:

So... suspicion has it that due to potential changes in orbit of the satellite and our specific geographic location we may have been falling out of sync with the satellite causing the occasional restart.

The temporary wait and see solution is that the system is in full manual and I'm waiting for an error notification... Long term is consistent manual sync, or finding a time to automate regular sync checks when we're "dark", which is almost never. So that should be fun...
 
Good catch. GPS satellites are in a medium altitude, non-geostationary orbit so they are constantly changing position. Depending on how much of the sky the antenna can see, there can be times when the receiver loses satellite lock, letting the clock oscillator free run. You might want to review the antenna placement for line of sight obstructions. GPS reception can also suffer from RF interference. Small, gas engines are notorious for RFI. At 1.5 GHz, coaxial cable loss needs to be kept to a minimum, use RG-8 or the bigger the better. Snow and ice on the antenna can cause signal loss, too.
 

Users who are viewing this thread

Back