The above Ad will no longer appear after you Sign Up for Free!

DMX control program: It's time.

Discussion in 'Lighting and Electrics' started by jumpjet, Mar 28, 2006.

  1. jumpjet

    jumpjet Active Member

    Messages:
    133
    Likes Received:
    0
    Location:
    Chicago Suburbs
    So, the Enttec USB to DMX dongle has been out for over a year now, and still, I haven't found a decent program that takes advantage of it.

    Unless anyone has any suggestions of an ongoing project I can help with, I will write my own. There are many features I find lacking with almost every board, and especially with those that are free.

    With a free, open-source console that runs on windows or linux, any school or NPO could be up and running for under $100 bucks if they could get their hands on a computer.

    What I am looking for from this community is a list of features that you would like to see. My intention is to make it modular, so that you can run it in various modes depending on your needs. Please list both features that you have seen other places, and those you haven't.


    Features that I want:
    Runs on Windows and Linux.
    Compatible with the Enttec USB-DMX dongle
    Unlimited channels
    Unlimited dimmers (limited by the number of USB-DMX converters)

    Simple mode and expert mode (Simple mode being several buttons that have preset scenes that would be used for band concerts, simple presentations, etc.)
    Module for 2-scene preset
    Module for recorded cues
    Module for submaster pages
    Listening from DMX devices for feedback.
    Audio triggering of cues/chase
    Tap rate for cues/chase
    A better macro language (variables, for/next and do/while loops)
    Show/scene timer (for comparison of what was shorter/longer between runs)
    Record lockout
    Ability to record show separate from settings, on multiple removable media options (Floppy, Hard drive, usb key)
    undo function
    Moving lights capability (similar to Hog, but there are some issues I have with that as well)
    Tracking or non-tracking
    USB output to control other devices/computer(for instance, to trigger a powerpoint slide in a cue)
    Control via keyboard, mouse, joystick, trackball.
    Ability to add new DMX controlled items via downloadable libraries
    Intelligent control for odd DMX items like fog, hoists, etc.(Hoists would show speed, feet decended/remaining. Fog could show volume, chiller fan speed, etc.)

    I'm not sure what kind of data is reported back from dimmer racks, but it would be handy to be able to parse that data as well. I believe sensor+ connect allows you to retrieve data through a cat-5 cable...

    It seems that the true power of a program comes many times from plugins that are written by the community. I would like this to be included as a feature as well.



    Extra goodies
    -------------
    Video Input
    Metering of audio mix
    Timecode smpte
    Remote control via ethernet
    Expandability to remote stations including video output, go/keypad, Remote lockout from master.

    It would be interesting to experiment with an algorithm that based on where lights are, and how they have to be oriented to hit the same spot on stage, the computer could determine what the other lights have to do to hit the same spot. This would eliminate the need to record positions, merely tell the instruments where on stage you need light.
     
  2. jumpjet

    jumpjet Active Member

    Messages:
    133
    Likes Received:
    0
    Location:
    Chicago Suburbs
    One other thing I would like to implement would be a skinning feature that would allow you to have various themes that emulate popular consoles in look and function.

    My concern is where this might impact copyright infringement.

    All thoughts are appreciated.
     
  3. soundman

    soundman Well-Known Member

    Messages:
    1,084
    Likes Received:
    113
    Location:
    Nashville TN
    Have you looked at magic q by chamsys? the link is http://www.chamsys.co.uk/ It is what I use when I get a chance. My experince is limited to a few hours in real world stuff but I have played around with it offline with capture quite a bit.
     
  4. koncept

    koncept Active Member

    Messages:
    590
    Likes Received:
    3
    Location:
    .
    keep me informed, i would be very interested in the final and would be glad to help with the programming if it is in a language i am semi familar with.
     
  5. jfitzpat

    jfitzpat Member

    Messages:
    86
    Likes Received:
    1
    We certainly share your sentiment. Entry level for us is <$300 (though that does include DMX in, for tactile input from existing controllers and a SMPTE generator/reader) and that can still be a struggle for some schools.

    Speaking as a commercial vendor I think that the two reasons you aren't seeing a lot of killer products for the Enttec are:

    a) Implementing the functionality of a major console is a much bigger project than it first seems - especially good support for intelligent fixtures

    b) The Enttec dongle itself

    Project size is neither here nor there, but man-years do go into some controllers. That is hard to do by one person as a hobby and, once the project relies on a loose collaborative group, you get the extra overhead of working by consensus. This can make a big project take awhile.

    As far as the dongle, the real issue is the division of labor. The dongle is cheap, but uses a serial/USB chip. This makes it appear as a 250 Kbit UART com port under, say, Windows. This both creates some unnecessary end-to-end data delays and it also puts some additional CPU overhead on a non RTOS platform.

    The dongle is, I believe, also half-duplex, so while it might be possible to support RDM with some additional resistors, it does not lend itself well to simultaneous control output and data input.

    At least one hobbiest has created his own variation of the dongle which is an in and an out. We still could not use it because of our latency, total channel density, and data update rates (ex. we wanted a higher than usual data rate to support smoother control of video/media servers) but, like the Enttec, a small system could certainly be built on it.

    He started (I think) in '04 and has been trying to get a collective effort going. He has a 'wiki' for specifications (with links to his overall project) here.

    You might consider supporting CITP, the open protocol from Capture (www.capturesweden.com). If use the selection protocol and accept DMX in both directions (like we do), you get a very integrated user experience. You can then use the 'focus' tool to point and click on the stage.

    Of course, Capture is a commercial product. But CITP is open and other vendors have talked about supporting it. Also, you could start a seperate open source visualization effort and use CITP (one benefit is that it allows you to run on either the same machine or seperate machines - doing good visualization can be very CPU intensive so it is sometimes nice to have the jobs split).

    Good Luck!
    -jjf
     
    Last edited by a moderator: Mar 29, 2006
  6. jumpjet

    jumpjet Active Member

    Messages:
    133
    Likes Received:
    0
    Location:
    Chicago Suburbs
    Thanks for the input guys!

    jjf, I have a few questions for you. The issues that you refer to with the dongle: Does that refer to the open source dongle, or the USBPRO version?

    I'd like to undertake the project without significant limitations, like the speed.
    What speed is acceptable for the video/media servers. I believe that the pro version supports up to 40 Hz.

    On the website, it says that the pro version has:
    # 1 Input & 1 Output connector (there is only one DMX port though)
    # RDM (talkback) compatible
    Will talk at any comm's speed, compatible with any system for which the FTDI chip has drivers (Windows, OSX, Linux..)


    Hrm....
     
  7. jfitzpat

    jfitzpat Member

    Messages:
    86
    Likes Received:
    1
    With DMX systems there are really three speeds to consider:

    Packet Rate
    Data Update Rate
    End to End Latency

    The first speed is limited by the standard and varies based on the number of channels. The connection is a 250 Kbit UART stream and each channel takes 11 bits (1 start, 8 data, 2 stop). So each channel takes 44 uS (0.000044 seconds) to transmit. But not all 512 channels need to be sent. So, if you send 32 channels, the data is transmited in about 1.4 mS (0.001408 seconds). If you send 512 channels, it takes about 22.5 mS (0.022528 seconds).

    In addition to the channel bytes there a serial 'break', a required 'mark-after-break', and a one byte start code (typically zero). So each DMX packet requires another 140 uS (0.000140 seconds) of transmission time. So, adding this in to our transmission times we get a max packet rate for 32 channels of about 646 times per second and a max packet rate for 512 channels of about 44 Hz.

    Good controllers *always* send DMX packets at the max possible rate for the number of channels used because DMX packets *HAVE NO ERROR CHECKING*.

    Bit errors are pretty much unavoidable, so always transmitting at the fastest possible packet rate cuts down the amount of time dimmers and fixtures are responding to bad data. In other words, transmitting faster does not effect the incidence rate of bit errors, but it does diminish how long bit errors are displayed on stage. If a 0 becomes a 128 (one bit error), and the update rate is hundreds of times per second, a dimmer flash will not be normally seen. If you are only sending packets 12-15 times per second, a dimmer flash will typically be visible to the audience.

    It is hard for Enttec based products to send at the max speed for 512 channels, let alone 128 or 256 primarily because packets and framing are done in application space on a non-RTOS system. This is probably not a big deal for a small rig, especially if it is running a fairly spastic dance floor. But it is a real concern with larger rigs (more cabling and electrical noise to generate errors) and situations where the audience is focused on a specific point.

    Data Update Rate is how often a controller generates new data for the DMX packets being sent. Think about it in terms of a 3 second dimmer fade. If you calculate new data twice a second, a fade from full on to off would take take 6 'steps' in our fade - no illusion of a continuous fade to the human eye. If you generated new data 15 times a second, the fade would occur in 45 steps and to the casual user would look fairly smooth, since about 12 Hz is the average human perception rate.

    However, most good desks for intelligent fixtures use data update rates in the 20-25 Hz range. This is because there is no standard for motor speed channels, so instead of adjusting an instrument's own speed control, motors are left at 'full speed' and the consoles generate more steps to generate smooth moves at various speeds (humans seem more velocity perceptive than intensity perceptive).

    However, unlike a motorized head, media servers tend to move images, etc. 'instantly' (no mechanical dampening). So we selected an update rate of 30 Hz to accomodate the faster 29.97 video frame rate here in the US. This makes console generated movement smoother in generated video (ex. flying a logo around a video screen).

    The big problem most Enttec based products have isn't that the data update rate is too slow (though the ones we have looked at all seem to be pretty slow), but that the update rate varies considerably based on workload. Ideally, generated movement and fades should not get 'jerkier' because a user opened another Window, etc. Again, a small rig, 8 bit fixtures instead of 16 bit pan/tilt, less scrutiny, etc. the less this matters.

    End to End latency is the interval of time from when the user inputs something and the output is reflected on the DMX stream. Data packet rate (described above) plays a role, since you can't get new data to a fixture until the old data is done being sent, so the Enttec products already take a bit of a hit. But they also get a hit from the additional overhead of using, say, the Windows COMM stack. It isn't that the extra buffering and copying of data take that much time (though every little bit adds up), but that the que system makes the latency 'dither'. In other words, if a board is a bit slow - but consistantly slow, a good operator will compensate without even really thinking about it. But if response time varies, a good operator will sense the responsiveness as being poor even if the average latency is relatively low.

    Again, how-good-is-good-enough varies. We have a user who splits laser tables off to seperate universes (fewer channels, higher DMX packet rate) when they want to 'play beams' live in a large space. That particular operator believes that there is a qualitive difference in cutting the 1/44th second 'dither' inherent in a 512 channel DMX stream to a 1/600th second (or whatever) dither when it comes to his live beams in an arena setting.

    None of this means that you can't do a decent controller with the Enttec adapters. Just that there are reasons why the 'clone' products based on it have more channel, fixture, and etc. limits than the originals.

    Good Luck,
    -jjf
     
  8. Pie4Weebl

    Pie4Weebl Well-Known Member Fight Leukemia

    Messages:
    2,438
    Likes Received:
    339
    Location:
    New York City
    wow this sounds pretty cool, make sure to keep us posted on what progress you make.
     
  9. jfitzpat

    jfitzpat Member

    Messages:
    86
    Likes Received:
    1
    Well, we've been shipping for a couple of years... ;)

    -jjf
     
  10. jumpjet

    jumpjet Active Member

    Messages:
    133
    Likes Received:
    0
    Location:
    Chicago Suburbs
    You raise some great points!

    First, the issue of the packet rate. If I understand you correctly, this is an issue inherent with the operating system. I wonder if increasing the priority of the application would help at all in windows or linux based environments.

    Another possible solution would be to get an open-source RTOS that is compatible with i386 processors. This way, it could be crated up with the program on a boot disk. No installations, no muss, no fuss.

    The data update rate might be ok. On the Enttec website, they quote the "refresh rate" which I'm going out on a limb here and hoping is the data update rate. They say on the pro version it is 40hz. I wonder how much the rate varies based on workload, and how much it can be minimized.

    I can't speak to end-to-end latency, and can only hope that it is minimal...

    I also can't plan for too long consistantly without coding SOMETHING. So without further ado, the very first preliminary barebones of this project.
     

    Attached Files:

  11. Radman

    Radman Well-Known Member

    Messages:
    1,219
    Likes Received:
    7
    Location:
    Franklin, TN
    So, just curious as a programmer myself, what language is that, C++?
     
  12. jfitzpat

    jfitzpat Member

    Messages:
    86
    Likes Received:
    1
    No, it is the design of the dongle. Virtually all modern OS's manage dramatically more data IO, but the Enttec shows up like a comm port to the application.

    It would be better to replace the driver and manage communication buffering differently in kernel space. This would offer some improvement in performance and CPU utilization, while making applications cleaner and easier to write.

    Better still would be to what most upper tier manufacturers do and move the buffer and low level protocol management to the dongle itself. This not only dramatically improves packet refresh rate (especially at smaller channel sizes, where the timer granularity for pre-emptive tasking on Windows, Linux, and OS X are proportionally huge), it significantly lowers CPU usage and dramatically increases total channel capacity.

    It is important to remember that the Enttec approach also makes very poor use of USB 1.1 bulk transfer bandwidth. If a dongle is just a tad smarter, you don't need to waste your host computer and your USB link resending all the same zeros over and over. But, of course, that is why our hardware is $100 more to the consumer (assuming they don't need the included SMPTE in/out). Still, a better design shouldn't be $100 for hobbiests, since there is no overhead (distribution channel, etc.) and it would open up much higher channel densities. For example, we have no difficulty running 16 universes via USB 1.1 from a Windows machine (we actually have run 32, but capped it because of limitations on some USB chip sets, similiar to why we cap ArtNet universes because of limitations in some NIC adapters).

    Yes, you can get better performance out of the Enttec approach with elevated thread priorities. That is how we helped a vendor we work with a lot get Enttec to work as a DMX in for their app - raised priority for an input thread dropped lost packets from chronic to occassional. Still, enough bad packets occured that a validator of sorts had to be run on each one (another waste of CPU time). Again, it would be significantly better to manage this at the Kernel level when the data arrives from the USB stack. For example, since the Enttec driver does not 'understand' DMX, it simply discards data that is not read by the application fast enough. A smarter driver could parse the data and keep a 512 image of the current (last) input levels for the app to interogate. The concept could be expanded so that even short transitions could always be caught.

    In this sort of app you might need to raise priority anyway, but you don't want to do so lightly. For example, under OSX and Windows, once you elevate to 'high' it can be difficult to terminate an errant app (since the task managers are at the same priority). This can turn any app error into a full reboot instead of a terminate-and-restart, and the minutes could be important in a show situation.

    But you would lose a lot of benefits from going to a PC in the first place. For example, running Windows Media Player or WinAmp to play music for your show, or running other apps (like a visualizer) on the same platform. You would probably even lose network support (for things like ArtNet and CITP) since you'd have to be responsible for hardware specific network drivers.

    And, at the end, you'd have the same inefficient comm arch. limitations as the original OS. Again, you'd get more mileage from writing a new driver, and more still from doing a better dongle.

    I've yet to see an Enttec based app with a data update rate faster than 12-17 Hz, even though most are one universe (or less). We do 30Hz under Windows (16 universes out (always max data packet rate), 2000+ moving fixtures, all doing movement generation in 32 overlapping fades, and one universe in), so it is certainly possible. But, based on my experiences trying to help someone get one universe into a PC using the Enttec arch., I'd say your work is cut out for you.

    Good Luck!
    -jjf
     
  13. jumpjet

    jumpjet Active Member

    Messages:
    133
    Likes Received:
    0
    Location:
    Chicago Suburbs
    Si, Ryan, it is in C++.

    So in the last few days I have done an extensive amount of research and come to some conclusions.

    1.) Jfitz REALLY knows what he is talking about! :D

    2.) Freestyler is a fairly comprehensive free solution for the windows gui candy users, although it seems quite counter-intuitive.

    3.) The Enttec opendmx usb dongle has limits that make me unhappy.

    For those reasons I have refined the focus of the project for now. I am going start with a console (text) based offering for windows and linux that can handle most of the features I mentioned earlier without major hiccups.

    From there, I will either try to help light jockey turn into something I would want to use without plucking my eyes out, or begin tailoring development of this program to the more GUI side.

    Jjf, again, I really appreciate all your feedback, especially since I am suggesting a program that is in essence in direct competition to what you guys are sellin' over there.

    If it's ok by you, I'd still like to pick your brain on occasion for this.

    Koncept, I'll let you know, as I'm sure I'll need some help once I get things a little more framed out.
     
    Last edited: Mar 30, 2006
  14. jfitzpat

    jfitzpat Member

    Messages:
    86
    Likes Received:
    1
    FWIW, Freestyler is essentially a downgrade clone of Martin's Light Jockey. Ease of use has never been LJ's strength. But, despite it's complexity, I think it is a pretty interesting program (I was sorry to see a primary mover-and-shaker behind the program and Martin part ways).

    My biggest criticism of FreeStyler would be stability. I have a lot more patience for hobbiest applications that may crash, but at least try some very new and innovative features. But if a hobbiest is largely cloning I'd like to see something a little more stable. It may be no big deal that, say, clicking on an icon name can crash FreeStyler (after all, it isn't like the user paid for it), but poor reliability gives the overall market (which I think is really just getting started) a bad name.

    Think about it this way, FreeStyler is 4 years and counting. Ben Suffolk's project started on '04. It takes time and effort to develop a good controller and it isn't as if I won't be working on my own new things during your development cycle. Besides, like I said, the one thing I don't think the world needs is another crappy PC based lighting controller.

    So, pick away.

    -jjf
     

Share This Page