LAN Problems with Crestron Unit

spiwak2005

Active Member
Joined
Nov 21, 2004
Location
Utica, NY
So after spending nearly 2 hours on the phone with Crestron support (which is great btw), I discovered that our LAN connection in the booth was dead. That was ultimately the end of Crestron being able to help get our DMPS3 unit working again. Of course our Network guy was off, and my network knowledge is minimal at best. My questions are of a network troubleshooting nature, and just trying to figure out what is causing the problem so I can talk to our IT people rationally. Here goes with a long-winded, frustrated post...

Background: College recital hall/lecture hall. 250 seats. I'm the Media Services Specialist for the college -essentially the occasional TD of this space, as well as a ton of other classroom and event support around campus. Sound & projection upgraded in 2015 (before I was there) with Crestron 3 Series processor in the booth, HDBaseT to lectern and projector. Touch screens in booth and at lectern, controlling video switching and 5 mics, audio through London Soundweb processors and breakout boxes. Plus "Attended Mode" shifting audio to Soundcraft Si Performer board in booth.

We've had some power issues with extended outages and even more extended shutdowns of the building's network switches. All the Crestron endpoints are assigned static IP's on the network so I assumed this is what was initially causing failure in getting the Crestron to control the projector, audio, etc. After all the power/network issues were resolved, I fired the system back up and it seemed to work fine. In my infinite wisdom, rather than just leave it alone I HAD to do a firmware upgrade on the Crestron. I was able to connect via IP and saw all the endpoints - HDBT transmitters, receivers, audio processors and touch panels. The upgrade appeared to be working...then lock up, crash, bang, boom. Nothing works anymore - touch panels won't run the program that controls the system so can't turn on projector, can't turn on a mic, can't switch audio to the board, because it's all locked up on the processor. Re-run the upgrade and it will only connect via USB. Run it and it still says no upgrade installed, run it again, etc. I never made the connection that it simply wasn't talking to it's endpoints across the LAN.

I'm just not savvy enough in network speak and network setup, I leave that to the college's IT guys. But I know enough to be dangerous. Did I somehow cause the LAN connection at the wall ports to stop working? I found the switch in the basement and did notice a good dozen or so ports showing no activity. But the rest of the switch looked fine. Barring the possibility that the switch is bad (I have no testing tools), is there any chance that the network just decided to start blocking those ports? There are 4 wall jacks in the booth and two on stage - the onstage ones work fine when connected to my laptop. At first all 4 wall jacks in the booth worked, then only 1 of them worked for a little while. Then all 4 were dead.

I might get a network guy to come down to the theater on Wednesday. And he might be able to at least look at his monitoring software on Tuesday afternoon if I'm lucky. What should I tell him to look at with regards to the Crestron? He has a file with all the MAC addresses and originally assigned IP's yet says he has no idea what sort of setup was ever installed in there. (sigh) This is a seldom used (to the extent of it's abilities) space for anything other than large class lectures a few times a week, and mostly "tech-less" (unattended) classical music rehearsals/performance. Yet I have to figure this all out quickly due to a couple high profile, "hands-on" events coming up soon. Any guess what could be causing the problem to ease my mind until I can get one of our internal staff members to actually help?
 
Last edited:

RonHebbard

Well-Known Member
Premium Member
Joined
Jun 12, 2004
Location
Waterdown, ON, CA
So after spending nearly 2 hours on the phone with Crestron support (which is great btw), I discovered that our LAN connection in the booth was dead. That was ultimately the end of Crestron being able to help get our DMPS3 unit working again. Of course our Network guy was off, and my network knowledge is minimal at best. My questions are of a network troubleshooting nature, and just trying to figure out what is causing the problem so I can talk to our IT people rationally. Here goes with a long-winded, frustrated post...

Background: College recital hall/lecture hall. 250 seats. I'm the Media Services Specialist for the college -essentially the occasional TD of this space, as well as a ton of other classroom and event support around campus. Sound & projection upgraded in 2015 (before I was there) with Crestron 3 Series processor in the booth, HDBaseT to lectern and projector. Touch screens in booth and at lectern, controlling video switching and 5 mics, audio through London Soundweb processors and breakout boxes. Plus "Attended Mode" shifting audio to Soundcraft Si Performer board in booth.

We've had some power issues with extended outages and even more extended shutdowns of the building's network switches. All the Crestron endpoints are assigned static IP's on the network so I assumed this is what was initially causing failure in getting the Crestron to control the projector, audio, etc. After all the power/network issues were resolved, I fired the system back up and it seemed to work fine. In my infinite wisdom, rather than just leave it alone I HAD to do a firmware upgrade on the Crestron. I was able to connect via IP and saw all the endpoints - HDBT transmitters, receivers, audio processors and touch panels. The upgrade appeared to be working...then lock up, crash, bang, boom. Nothing works anymore - touch panels won't run the program that controls the system so can't turn on projector, can't turn on a mic, can't switch audio to the board, because it's all locked up on the processor. Re-run the upgrade and it will only connect via USB. Run it and it still says no upgrade installed, run it again, etc. I never made the connection that it simply wasn't talking to it's endpoints across the LAN.

I'm just not savvy enough in network speak and network setup, I leave that to the college's IT guys. But I know enough to be dangerous. Did I somehow cause the LAN connection at the wall ports to stop working? I found the switch in the basement and did notice a good dozen or so ports showing no activity. But the rest of the switch looked fine. Barring the possibility that the switch is bad (I have no testing tools), is there any chance that the network just decided to start blocking those ports? There are 4 wall jacks in the booth and two on stage - the onstage ones work fine when connected to my laptop. At first all 4 wall jacks in the booth worked, then only 1 of them worked for a little while. Then all 4 were dead.

I might get a network guy to come down to the theater on Wednesday. And he might be able to at least look at his monitoring software on Tuesday afternoon if I'm lucky. What should I tell him to look at with regards to the Crestron? He has a file with all the MAC addresses and originally assigned IP's yet says he has no idea what sort of setup was ever installed in there. (sigh) This is a seldom used (to the extent of it's abilities) space for anything other than large class lectures a few times a week, and mostly "tech-less" (unattended) classical music rehearsals/performance. Yet I have to figure this all out quickly due to a couple high profile, "hands-on" events coming up soon. Any guess what could be causing the problem to ease my mind until I can get one of our internal staff members to actually help?
Calling @MNicolai
Toodleoo!
Ron Hebbard
 
  • Like
Reactions: spiwak2005

MNicolai

Well-Known Member
Premium Member
Fight Leukemia
Joined
Mar 30, 2008
Location
Sarasota, FL
Confirm your assumptions one at a time.

Are the ports on the wall all patched to the network switch?
Is the network switch a campus switch that's likely to have more security enabled, or an isolated network that was likely provided/configured by the AV contractor and would probably have minimal security enabled?
After the upgrade, does the processor have the same IP address and subnet that it had previously?

The processor could've ended up in DHCP mode where it's waiting to be told by a DHCP server what IP it should have. I would remove the processor from the network, connect it your laptop with a small 5-port WiFi router that has DHCP turned on. You should either be able to connect directly to it at the IP address it had before you upgraded or it should land in the same IP subnet as your laptop if it's in DHCP mode. Complete the upgrade, get the processor stable again, set it to the correct static IP, and then reintroduce it to the installed network and see what happens.
 

spiwak2005

Active Member
Joined
Nov 21, 2004
Location
Utica, NY
Confirm your assumptions one at a time.

Are the ports on the wall all patched to the network switch? YES
Is the network switch a campus switch that's likely to have more security enabled, or an isolated network that was likely provided/configured by the AV contractor and would probably have minimal security enabled? CAMPUS SWITCH
After the upgrade, does the processor have the same IP address and subnet that it had previously? Still showing DHCP off and the same IP & subnet as it was.

The processor could've ended up in DHCP mode where it's waiting to be told by a DHCP server what IP it should have. I would remove the processor from the network, connect it your laptop with a small 5-port WiFi router that has DHCP turned on. You should either be able to connect directly to it at the IP address it had before you upgraded or it should land in the same IP subnet as your laptop if it's in DHCP mode. Complete the upgrade, get the processor stable again, set it to the correct static IP, and then reintroduce it to the installed network and see what happens.
Responses to your questions above in bold. The campus network is connected to a POE switch in the rack. I will switch the processor to DHCP on and see what happens while I wait for a response from the Net Admin. Thanks for your help thus far.
 

spiwak2005

Active Member
Joined
Nov 21, 2004
Location
Utica, NY
Does the processor look like it actually took the update? From your description, I'm not clear if the processor is stable but just not communicating with your endpoints or if the processor is bricked.
Crestron DMPS engineer said it was showing the correct firmware version. Touch panel engineer noted that the panel had defaulted to a local IP address (169.264...) so put the responsibility back on our local network.
 

MNicolai

Well-Known Member
Premium Member
Fight Leukemia
Joined
Mar 30, 2008
Location
Sarasota, FL
Seems odd that you would have a static IP on the panel and it would flip into DHCP mode because of a firmware update on the processor.

Usually the only way you get 169.254.xxx.xxx is if the panel doesn't know what IP it's supposed to be and has to got hunting, and has no DHCP server to assign it an address. I suppose if you're using the control subnet you could be seeing something unusual like that, but most systems I see don't bother trying to use the control subnet because of how wishy washy Crestron is about what is and is not supported by it.

Regardless, if you put a 5-port POE switch at the network rack and plug just the DMPS and touch panel into it, if both devices are configured for correctly for static IP's you should be able to get them to communicate. It is possible that you would need to go into Toolbox and update the hostname/IP tables to re-associate the touch panel to the DMPS program file, but I wouldn't start messing with that unless other options have been exhausted.
 

Users who are viewing this thread