A lighting conundrum

[icewolf08]

So, I have had some interesting happenings over the past week of running the opening show of our season. I have had two HPL lamps in different fixtures go supernova explosively in the middle of performances. The interesting thing here is that both fixtures are on the same pipe with dedicated raceway. The raceway is actually split between two dimmer racks, though the fixtures are both on one. When the first one went, I didn't really think much of it because it happens every now and then. The second one went the next day and the damage to the lamp looked exactly the same as the previous. Unfortunately I didn't save the first lamp.

Now, on the same pipe I have two moving lights that use arc discharge lamps. Two nights ago one of those started to act up. The lamp just quit in the middle of the show and when I looked at the fixture after, it displayed a lamp error. Upon resetting the fixture, the lamp ignited and seemed to be fine. Last night we ran into the same issue and I reset the fixture in the middle of the show. The lamp re-ignited but still went out again. The lamp has 88 hours on it, it is basically brand new.

Today I decided to run some tests to diagnose the problem. I started by running though the cues in the show to see if there were any glitches or power variances that might be causing the issue. Everything worked fine. It seems that all the fans are running and everything seems normal. I have had the fixture on for a few hours now (as I write this) and have not been able to re-create the problem.

My thoughts would be to swap the lamp for one of the old ones (that never gave me any issues) for tonight's performance and see if the problem persists. If it doesn't then I suppose I would have ruled out a problem with the fixture, meaning I have a bum lamp. If the problem stays then I would assume that the issue lies with the ballast.

Thoughts? Other troubleshooting you might suggest?

EDIT: The MLs are Elation PowerSpot 700 CMYs. I have had them for four years and they have always worked nicely. I take good care of them. We even named them.

 

[jonliles]

They're on the same pipe, right? Anything mechanically (vibrations, ventilation, etc..) happening to the pipe to induce filament weakening/failure? Has something wonky happened to the HVAC to create localize thermal variances placing undo stresses on the lamp envelope?

 

[icewolf08]

They're on the same pipe, right? Anything mechanically (vibrations, ventilation, etc..) happening to the pipe to induce filament weakening/failure? Has something wonky happened to the HVAC to create localize thermal variances placing undo stresses on the lamp envelope?

Not that I can determine. We had some wonky things with HVAC during tech for this show, including an evening where we had no HVAC, but there were no issues then.

 

[Grog12]

First off it seems that you have 2 separate problems here.

1) HPL's going supernova.

2)Flakey moving lights.

I've seen the issue you're describing with the moving lights happen when an ISO/Opto was getting various amounts of voltage (long story) causing it to go out. Is the ML at the end of the DMX chain? Is there an overhead move that might be tugging on DMX? I'm gonna assume there isn't a weird control channel move written into the cues or you would have recreated the problem today. Heck even though the fan's going maybe you are getting an excess build up of heat....(wow I bet none of that is helpful the whole thing felt a little rambly while typing it).

As for the HPL's....dunno. Twice I'd chalk up to coincidence/ intern installing lamps touching them in the same place. 3rd time somethings up.

 

[icewolf08]

Well, none of the other DMX gear running on the same Opto is acting up. Yes, this particular unit is at the end of the chain. There is one piece of scenery that flies past this pipe, but it is a soft sheer drape that doesn't have much weight to it. Our cables clear it without issue.

I am now up to three hours running without issue today. I just wish that in this testing something would happen. It is hard to diagnose a problem that I can't recreate.

 

[derekleffew]

I'm guessing that, since you mentioned it, you suspect there may be a connection between the HPL lamps going super-nova and the ML showing a lamp error. Are you powering the ML via an R20/CC20, and is it in the same rack (and on the same phase) as the HPL lamps?

Were the HPLs at/near 100% when they went?

Post a good, hi-res picture of the ML lamp, so ship can tell us if it has an issue. I've had newer arc lamps than that have to be replaced for no apparent reason, but another lamp lasted indefinitely in the same ML.

 

[icewolf08]

Yes, the ML is powered via an R20 Module and it is not on the same phase as the HPLs that went nova. The rack (ETC Sensor+) CEM reads that the voltage on each phase is where it should be (120V). Will try to get photos.

 

[joeboo46]

I would say the HPL's were a coincidence, if a third goes in that manner I would be concerned. As far as the lamp in mover it could be possible that one of the thermal switches is nearing the end of it's life. The mover may only exhibit the problem in a certain position that the thermal switch (breaker/trip) doesn't like, making it very inconsistent. I have had this problem with many a mover before some days it will go for hours other days no strike at all. What type/brand of mover are you dealing with?

 

[zmb]

Any chance that you got a defective batch of HPLs?

 

[church]

I have experienced simultaneous multiple failures of lamps only once and it was eleven that bit the dust. All the lamps were on the same portable 24 channel dimmer rack which was running on a 120V/240V split supply. I was running one fixture per channel and had 23 channels full on at the time. The reason for the failure? All the lamps that blew had the complete filament seperated from the electrodes and still intact with clear quartz enevelopes.

The blown lamps were all on channels on the red side of the split. We traced the problem to loose neutral in the main electrical panel. We had run three shows with no problem and then wham! The strange thing was that the problem persisted for twenty minutes or so and then cleared. The only clues were the Phase indicators on the dimmer racks were flashing on and off and the sound system was buzzing intermittently.

When I saw the phase indicators flashing I metered both sides of the split and it looked okay. It was only when I unplugged all of the soca from the dimmer racks and plugged some work light on each side of the split via a non-dimmed split duplex on the back of the racks that normally feed colour changer PSUs and remetered the supply that we saw that we had 110V/130V which was our problem, as we plugged in extra work lamps on one side the voltage imbalanced worsened.

A very expensive day we informed the venue they had a neutral problem and their licenced electrician eventually found a broken neutral connection that was arcing. The only things run of this supply was their house sound system and portable lighting for venue clients.

It may be worth checking the neutrals hope it helps

 

[icewolf08]

Thanks for all the help so far guys. People asked for photos of the lamp so I have some here that I just took. These are big files just so you know.

Here is a shot of the whole lamp:


Left hand side seal:


Right hand side seal


Another view. Only thing of note in this view is maybe a little yellow discharge on the lamp, I don't know what that means.


I have decided to run tonight's show with the same lamp, if I encounter the same problem I will swap the lamp for tomorrow's performances and see if that makes a difference.

 

[shiben]

Im gonna go with ghost. Anyone died in your theatre? In all seriousness. One thing I do believe in...

 

[dhorn]

I have had problems with incandescent lamps failing and Movers having intermittent power issues due to wiring problems inside the wire raceway. How old are your raceways? Are the wires inside loose or bundled? Does each circuit have a dedicated neutral? Shared neutrals can lead to a host of problems, especially when dimmers and ballasted non-dim loads share a neutral. You may have to check for loose terminal screws at the J-box end of the raceway, the receptacle or the entire run from the dimmer racks. A loose screw termination can overheat under load and cause resistance problems or outright arcing.

Good luck tracking down your Gremlin's

 

[icewolf08]

So, Here is some new info. Last night the fixture/lamp ran the whole show without a hitch. This afternoon for the matinee the lamp went out again. So tonight I have switched to a new lamp and will see what happens. I figure that if the new lamp performs fine for the next few performances then I can chalk the problem up to a bad lamp. If we continue to have issues then I would assume that it is a problem with the fixture.

 

[MNicolai]

Not to shrug off the possibility of a loose neutral or other problems, you may want to try the old lamp, but while powering the fixture from a completely different power source. If this is during a show and you make it look nice, or when you're the only one around and you can just drag a cable into the middle of the stage.

If you can repro it every so many times per so many hours with the "bad lamp" fed from the R20, but can't get it to repro in that same amount of hours of use from a standard wall outlet, then that narrows it down to a bad fixture or lamp and puts your dimming systems and power feed in the clear. The exception would be if the main feed to that area of the building was bad with a loose neutral, but it is reasonably safe to presume the problem is not that evil or that problem would be manifesting itself into many more symptoms than just weird movers.

The amount of time it might take to repro the symptoms though might be enough to not make this one of your first troubleshooting options. Still, it's usually not that hard in a theatre to bring a power cable over from somewhere else and string it up to the batten and see if the problem shows up again in the next several shows, again using the old lamp.

 

[Grog12]

Alex is the problem happening at the same time everyshow?

 

[ship]

Fans might be operating, but that doesn’t mean they are operating at full speed.

You can tell a lot from a lamp, just wish many would look at it in prying it out of the base in perhaps a bad base... or at least look at it and gee, it's only been in use say 80 hours for a 750 hour lamp... question why the lamp is bad they are replacing.

The amount of crusty clouds I see on the 80 hour lamp co-axial side to the fill pinch says it’s probably being overheated. The silver crusties and blackening of them in burning in I see between fill pinch and a pole says the lamp probably isn’t operating at full voltage - though not as bad as I have seen though. It’s not really building up in the fill pinch hole either - this by way of your last time it operated not in failed condition or for other reasons such as boarder line voltage while you were operating it - yet without say voltage drop during full show level operation - clip lights, power brown outs and all.

Electrode buildup of bubbles doesn’t seem that bad in it’s still a functional lamp - not blackened and there is a buildup of bubbles on them which with more operation could clear up much of the floater silver crusties not already spent and deposited. You will still have a vapor cloud but less of one than with the spent more white crusties on the coaxial side. It’s now a bad lamp in functioning not efficient optically. In another hundred hours in the same fixture and under these condition and you will have a flourescent globe sized point source of light. by way of clouds and snow.

Want to save the lamp, put it into another fixture with a different ballast signature to it. The silver crusty problem should clear up but not the cloud/snow. This lamp can do no good any longer in this fixture and possibly is spent in any fixture due to the white snow and or clouds building which will block output. Too much light blocked and I don't recommend it's continued use.

Normally a bad HTI 700w/D4/75 or other mover lamp will go bad say in the first 20 hours and it is fairly rare. Not under my magnifying glass either to see if there might be a globe/pinch coning going on or a micro crack in letting the gas get out. Could be a slow leak of gas in the first few hundred hours, but I don’t see that possible given also the white co-axial crusty snow and cloud forming. Wiring seems good as with the lampholder so far as I can see. Still the snow on the globe... that normally comes from age and or over-heating. I see this a lot on modern fixtures - overheating problems. Could be effect of the fixture itself in operating normally but at high heat - the overheating I see but still a bit early for as much or marginally as much snow on the globe for this to be the cause.

That’s the lamp itself - judging on the lamp, I would expect it was both overheated and wasn’t getting proper voltage. Less bubbles on one electrode over another isn’t as much a problem in seeing - though one electrode seems drooping a bit in a bad placement type of thing. Not as bad as I see and once bench focused, shouldn't overall effect optics or lamp life. That’s in more bubbles to an electrode on one side over another, more a question of shut down cycle than anything else. As long as both electrodes are showing use and fresh bubbles not blackened or melted down at the tip, not an issue. I expect you can still measure a 4mm arc gap on these electrodes.

Mark from Osram could no doubt correct or confirm my observations above.

Here is my thoughts in citing the above neutral connection issue presented above as interesting and good concept. Were the lamp only showing silver crusties - and more extensive than what I see and light vapor clouds forming at this age, I would suspect the fixture. Instead I suspect the power train to the fixture in it not operating at it’s correct voltage. I don’t think this a fixture problem.

This power train voltage problem could be anything from a plug on the fixture that has loose terminals to 1/4" of insulation stripped off a conductor where say ½" was needed and the plug’s terminal clamp don’t properly engauge the conductor given a bunch of insulator also clamped into the terminal. Seen that a lot - people just don’t read instructions any longer.

(Gee thanks on the 100' stage pin cable - now Stage to Edison adaptor that you did the easy in shoving the furrels and pre-stripped wire into the terminals. Don’t help in neither proper strain relief clamp no matter how much you twist the conductors to fit, or them ferrules getting stuck within the connector. Just cut the cable at some point in not doing me a favor as it will be cut anyway once it returns. Same type of thing with someone used to say a silver/yellow Cooper/Eagle home owner grade plug in stripping only 1/4" of insulation off the conductors and on a industrial plug - that not enough insulation stripped. See it a lot = this not properly terminated plug.)

From the improperly stipped plug, to the house problem of a lug nut someone forgot to tighten somewhere.... I believe you have a voltage issue here. Difficult given metal tends to expand with heat, but perhaps in the second part of the show or with intermission, there was some cooling and given the before this pre-heat it is more shown in the second act.

On the HPL lamps... hard to say in it could be related given the same phase of power, or could be other types of bad lamp - less likely or the last that installed the replacement touched the globe. Good lampholders I would assume. Explosive failure could be voltage shock or touching the lamp even for a dimmer that would normally smooth out such a shock to the lamp. Ah’ that flicker in the dark. Short of looking at the shards of glass in cause of failure and base, it’s hard to tell the cause. That sparking in the system is very suspect though given the same pipe, but probably not related.

I don’t think your mover has a problem though you could swap it out with another in verifying this. Do the easy first - check if insulation about conductors is within the connector terminals, and if those terminals on the plug are tight. Easy first check.

After that, the rest of the electrical power supply system. I don’t think the mover light lamp is a bad lot of it and it’s even if bad trying really hard to continue functioning. The fixture itself seems to be overheating it by way of cooling and supplying proper voltage to it. I think it working properly just not getting enough voltage. Something between plug and supply side is a problem.

 

[DavidNorth]

Alex,

I like Jhorn's reply and I would look at circuit wiring in your strips, if for no other reason but to rule them out. A couple of calls in the last several years point me to this area.

If your circuit strips are older, it is possible that they have been wired with common [or ganged] neutrals. I've seen where several lamps blow consistently on a few circuits and I've also talked to some studios supplied with quite a bit older strips that were confirmed to have common neutrals.

Here's what happens....

Let's take three fixtures, each on their own leg of power from dimmers and that have equal resistances at full level. In this case, the current draw on the three power legs is equal and even if don't have a neutral, everything is ballanced and lit. Now, if you move the loads or change some of the levels, your current draw on some phases goes up and on others goes down. The by-product is that voltage across the lamps change as well and then they start to blow or get dim....but only when the neutral is compromised.

Now think about having an intermittant connection because the overloaded neutral has been getting warm for many years, loosened its screw, or maybe has developed a large amount of carbon reducing its contact surface. Same problem has occurred but not predictably. Sounding familiar?

So if you have X number of distinct circuits on your strip and you have less than 2X wires [plus grounds] in the wiring feeding your strips, likely this could be the issue. The neutral wire may be charred, burned in half or is not making a solid connection. You could also have a neutral broken in the SO drop or even a lift line for the pipe slowly sawing through the circuit conduit in the grid [yes, have a great pic of that one].

Yes, you can also have this occur on the feed to the rack, but in that case you typically have problems all over the rig.

Let us know what you find,

David

 

[icewolf08]

Here is an update on what is going on.

I have been unable to reproduce the issue outside of show conditions. Unless someone has a suggestion on how to measure the voltage at the lamp I am not sure how to do that with the fixture running. We didn't have a show on Sunday so I will see what happens tonight with a brand new lamp. As the general consensus seems to be a lamp issue, this seems to be the logical step. If there are not more issues, then problem solved. If the issues continue then the next step will be to move to an alternate power source.

It does seem odd to me that a power issue would crop up well into the run of the show as opposed to during tech when we are doing more work with levels and such. Also seems odd that they would only affect one fixture... I have two identical fixtures on the same pipe powered off the same R20 module (different circuits).

I haven't checked how the raceways are wired, though I can do that.

Alex is the problem happening at the same time everyshow?

No, it didn't.

Ship, you had asked (in PM) about the orientation of the fill pinch with regard to the reflector. The manual for the fixture says that the pinch should point towards the reflector, which is how I have installed the lamps.

 

[STEVETERRY]

Alex,

I like Jhorn's reply and I would look at circuit wiring in your strips, if for no other reason but to rule them out. A couple of calls in the last several years point me to this area.

If your circuit strips are older, it is possible that they have been wired with common [or ganged] neutrals. I've seen where several lamps blow consistently on a few circuits and I've also talked to some studios supplied with quite a bit older strips that were confirmed to have common neutrals.

Here's what happens....

Let's take three fixtures, each on their own leg of power from dimmers and that have equal resistances at full level. In this case, the current draw on the three power legs is equal and even if don't have a neutral, everything is ballanced and lit. Now, if you move the loads or change some of the levels, your current draw on some phases goes up and on others goes down. The by-product is that voltage across the lamps change as well and then they start to blow or get dim....but only when the neutral is compromised.

Now think about having an intermittant connection because the overloaded neutral has been getting warm for many years, loosened its screw, or maybe has developed a large amount of carbon reducing its contact surface. Same problem has occurred but not predictably. Sounding familiar?

So if you have X number of distinct circuits on your strip and you have less than 2X wires [plus grounds] in the wiring feeding your strips, likely this could be the issue. The neutral wire may be charred, burned in half or is not making a solid connection. You could also have a neutral broken in the SO drop or even a lift line for the pipe slowly sawing through the circuit conduit in the grid [yes, have a great pic of that one].

Yes, you can also have this occur on the feed to the rack, but in that case you typically have problems all over the rig.

Let us know what you find,

David

I like this as a possible source of the problem.

ST