A Troubleshooting Tale

microstar

Well-Known Member
Forgive the length of this post but I thought telling the whole story would be worthwhile.

On Monday the AV contractor and I wired in a new intercom power supply to a sound system installation at a local school and tested it. On Wednesday afternoon, the drama teacher sent me a series of text messages:
TEACHER'S TEXT #1: We much appreciated the repair, Unfortunately, the work done has caused something to go horribly wrong with our lighting board. Ever since the install, the lighting cycles on almost like they're on a chase sequence. I shut down, reloaded, checked cues, even BLACKED OUT the board, and it's still having lights on the first, second and third electric flashing on.
TEACHER'S TEXT #2: We have a show tomorrow. Any ideas what may have happened?
TEACHER'S TEXT #3: I should have informed you earlier, but we've had a two week turnover time between assemblies and shows each month. During spirit assembly, when music plays, lighting system flashes on and off uncontrollably.

MY RESPONSE: So this problem PRE-DATES the Monday intercom installation?

TEACHER'S TEXT #4: It only flashed previously when sound system was playing music. Now it does so all of the time.

MY RESPONSE: Please get approval for an emergency service call. I will need to come tonight after all the students are gone after your rehearsal.

A couple of hours pass and I get a call from the Head of Maintenance to make the service call and he will issue a PO.
I show up at 6:30PM as the last students are leaving. Stage and houselights are on and they are not using the sound system. Nothing is flashing/chasing so I ask the teacher what I could do to make it to happen. She says black out the houselights and set the board master to zero. I do that and wait....and wait.... and wait. Nothing happens.

A DMX to Electro Controls multiplex converter box is setting next to the light board. I had installed it in 2008 when I replaced the original Electro Controls console with a DMX board.
I turned it around to look at the front and the POWER and DATA LED indicators were alternately flashing at a furious pace, instead of the normal constantly on.
Fortunately I had brought along a replacement converter and its wall wart power supply, so substituted my converter and the POWER and DATA lights continued flashing.
Then substituted my wall wart and the flashing stopped, with both LED's constantly on. Then hooked the original converter back in, powered by my wall wart, and both LED's stayed constantly on. I then checked the voltage output of the wall wart without a load and it read correctly, with a smooth output.

Oddly enough I never did see the stage/houselights flash/chase during my time there. I stayed around for about 20 minutes to make sure everything was OK.
They apparently did their show the next day without incident and I've received no reports of further issues.
Later, I took apart the bad wall wart which was a linear type with a transformer, bridge rectifier, and capacitor. The capacitor was domed and tested bad. I soldered in a new capacitor, re-checked the output, and hooked it up to my DMX to EC converter. The LED's indicated properly. Moral of the story.... bad power supply capacitor... NOT the installation of the intercom power supply, NOT the sound system, and NOT the light board. Believe only a small portion of what the reported "causes" are!
 
Last edited:
That's called the "Cintuas" clause of repair, as in "Since Ya" did that repair, we have this new problem.
 
That's called the "Cintuas" clause of repair, as in "Since Ya" did that repair, we have this new problem.
Indeed. The genuine applicability varies - in auto service (presuming an honest shop), when something else fails after another thing was fixed, typically that failure was in process already. In wiring/electronics I tend to subscribe to the "last thing I touched" model... did I plug it in wrong? Did I swap 2 similar connectors? Am I off by a pin on the ribbon cable header? Not the case here.

I had some esoteric and some obvious hypotheses about the above...

{history lesson, old guy rambling, etc}

In a previous century I saw a conventional dimming system (analog? early DMX? converted a/dmx?) randomly flash various dimmers. Great until you're lighting a ballad. While the rest of the LX crew was looking up at the truss/stage and poking the dimmer rack with a stick, I looked down at the console and saw the channel status lights blinking, too. The LX crew speculated on all manner of problems starting at the dimmer rack, DMX boards, converters, stuff I didn't understand... and when it did it again I pointed out that the console appeared to either drive, or respond to, whatever was causing the problem. Turned out to be the console sending spurious commands. This intermittent problem existed for a couple of months until I just happened to be looking at the LX desk while nobody else was.
{/history lesson, old guy rambling, etc}
 
I am amazed at just how many problems are traced to a bad power supply ( or in one case a too long. under capacity cable causing a voltage drop)
 
This reminds me of my salad days, when anything wrong with the family computer was clearly a result of me playing video games on it.

The danger of course is sometimes weird stuff does weird things with other stuff. I'm reminded of a DMX USB dongle that would shut down the USB controller on a PC everytime the follow spot lamp was powered on.
 
I personally like the Sherlock Holmes school of diagnostic troubleshooting.

When you have eliminated the impossible, whatever remains, however improbable, must be the truth
 
On a tour some years ago, we were in a shed when severe weather hit. There was some intense lightning, and all departments powered down. Once the storm passed, we fired everything back up and we had suddenly developed an issue that was causing comm to buzz horribly and also caused some LED tubes to randomly flicker. We did what we could before the show, but we’re not able to pinpoint the cause. Next gig I began troubleshooting. I began with an inspection of the entire FOH snake, which appeared to be fine. I then began to methodically unplug things starting at dimmers which produced no clues. I then went out to FOH and began unplugging things out there, where I would eventually found the culprit. With a headset on, I unplugged the power to one of the flatscreen monitors for the lighting console and it all cleared up. Theory is we took a lightning strike and that PSU flipped out sending stray voltage through the DVI connection to the console which in turn ended up getting to the snake via the chassis of the console. Believe it or not, I had the same thing happen AGAIN on another tour the following year. I know it wasn’t the same PSU cuz I personally tossed the first one in the trash. They all thought I was joking when I suggested unplugging the external monitor on the desk, but they soon found out I serious, and all was well… 😎
 
I had a strobe fixture freak out and fault to ground. Diagnosis was easy, it went from the fixture, to the board I/O, through the board chassis, into my finger, through my belt pack, and found a good ground somewhere in the com system. Subsequently, the dimmer racks went into FULL DISCO PANIC of lights flashing on and off at full (quick breaker cycle fixed that)... and I lost feeling in my ring finger for a year. (ETC was able to expedite repairs in a fortnight, so the show was unaffected as this happened during tech). It's not so strange to lose feeling, it's when it comes back that's a bizarre experience, especially after a year. Never did find out why the board wasn't grounded.
 
Not a skill that anyone seems to teach anymore....

On a tour, opener drum tech told that he would get a poke from a studio color every time he passed by and brushed up against it, it sat on the main drum riser ds edge.

It did give a poke, a 120v poke, since that light was auto sensing, someone had wired the connector wrong and one leg of the 208v was on the ground, the light worked anyway as it thought it was 120v.

I asked him how long that was happening, 4 weeks he said.... 4 WEEKS....

I asked him why he didnt say anything earlier, his response....I dont know....

:wall:

Sean...
 
Not a skill that anyone seems to teach anymore....

On a tour, opener drum tech told that he would get a poke from a studio color every time he passed by and brushed up against it, it sat on the main drum riser ds edge.

It did give a poke, a 120v poke, since that light was auto sensing, someone had wired the connector wrong and one leg of the 208v was on the ground, the light worked anyway as it thought it was 120v.

I asked him how long that was happening, 4 weeks he said.... 4 WEEKS....

I asked him why he didnt say anything earlier, his response....I dont know....

:wall:

Sean...
"Doctor, you need to see my brother, he thinks he's a chicken!"

"Well how long has this been going on?"

"Six months."

"Six months? You should have brought him in sooner!"

"Yeah, but up until now we needed the eggs..."
 
Not a skill that anyone seems to teach anymore....

On a tour, opener drum tech told that he would get a poke from a studio color every time he passed by and brushed up against it, it sat on the main drum riser ds edge.

It did give a poke, a 120v poke, since that light was auto sensing, someone had wired the connector wrong and one leg of the 208v was on the ground, the light worked anyway as it thought it was 120v.

I asked him how long that was happening, 4 weeks he said.... 4 WEEKS....

I asked him why he didnt say anything earlier, his response....I dont know....

:wall:

Sean...
I ran into that with a powercon jumper that had the ground/neutral flipped on one side. Found out 40' up. Thankfully I was an inadequate conductor.
 
On a tour some years ago, we were in a shed when severe weather hit. There was some intense lightning, and all departments powered down. Once the storm passed, we fired everything back up and we had suddenly developed an issue that was causing comm to buzz horribly and also caused some LED tubes to randomly flicker. We did what we could before the show, but we’re not able to pinpoint the cause. Next gig I began troubleshooting. I began with an inspection of the entire FOH snake, which appeared to be fine. I then began to methodically unplug things starting at dimmers which produced no clues. I then went out to FOH and began unplugging things out there, where I would eventually found the culprit. With a headset on, I unplugged the power to one of the flatscreen monitors for the lighting console and it all cleared up. Theory is we took a lightning strike and that PSU flipped out sending stray voltage through the DVI connection to the console which in turn ended up getting to the snake via the chassis of the console. Believe it or not, I had the same thing happen AGAIN on another tour the following year. I know it wasn’t the same PSU cuz I personally tossed the first one in the trash. They all thought I was joking when I suggested unplugging the external monitor on the desk, but they soon found out I serious, and all was well… 😎
The interesting thing is that you can unplug everything and still have it damaged by a nearby strike. I repaired a number of new VCRs that were still in the box when lightning struck outside a store once. The EMP from the strike outside the store blew the head preamps. As each one was sold, it would come right back DOA when the customer opened it up and tried to use it. All had been on the same shelf in the stockroom next to the back wall.
 
Indeed. The genuine applicability varies - in auto service (presuming an honest shop), when something else fails after another thing was fixed, typically that failure was in process already.
That is, I have been told, the reason why the ticket from your mechanic will list anything else they noticed which *might* be an incipient failure, and the fact that you were advised, and told them not to "fix" it right now.
 

Users who are viewing this thread

Back