Lighting and Sound America – March 2006Technical Focus/Automation
Wireless Remote Control on Stage
By James David Smith
Over the past few years, wireless control has successfully made its way into secure networking and factory automation. Now, it is poised to make inroads in theatrical motion control.
In 1989, while investigating solutions to the wireless needs of a Phantom of the Opera tour, I discovered a fascinating technology for the remote control of cranes on construction sites and locomotives in train yards. These systems did not use spread-spectrum radio, which was not yet available for civilian applications. Instead, they used traditional, narrow-band RF, configured on unique frequencies that were licensed to each user on an annual basis. Reliability was exemplary. But these systems were costly and suffered two limitations that would keep them eternally out of the theatre: they were slow, with limited resolution, and their licenses were limited to specific geographic locations. If you used these systems for lighting, you would see the steps – often as few as eight or 16 levels. And forget about touring – you couldn’t move this stuff five miles without violating your FCC license.
Meanwhile, at the other end of the spectrum (pun intended) were affordable hobby radio-control systems for model airplanes, boats, and cars. Though inexpensive, these have never delivered the reliability and repeatability that a theatrical production demands. Even so, many people have attempted to adapt and customize hobby RC for theatrical use – with mixed and sometimes dangerous results.
Today, thanks to widely available spread-sprectrum digital radio, wireless is being used successfully for three kinds of applications in theatre: general networking, lighting control, and motion control. Generally, lighting systems require the highest resolution and fastest response. Motion control requires a higher level of reliability, with fail-safes and redundancies designed to prevent unintentional motion and ensure the remote device can be reliably stopped at any time. Commercial wireless networking provides high bandwidth, but with varying and sometimes lengthy latency delays.
What about a system that provides speed and resolution and high reliability and low latency? Well-designed, dedicated theatrical RF systems can do just that. There are several FCC-approved frequency bands that can be used across North America, making it viable for touring. A small portion of the 2.4GHz industrial/scientific/medical (ISM) band is useable virtually worldwide (although not the entire 2.4Ghz band, as some believe).
For motion control, wireless must be fail-safe. If the radio link is broken, the device must safely stop. That device could be a 500lb. theatrical set piece heading toward the edge of the stage. Even if it’s a 5lb. mobile rubber chicken, you don’t want it diving into a cello player.
What can go wrong? PlentyA well-designed remote control system will be impervious to foreseeable faults. And there are lots of them:
Radio frequency jamming is the wireless equivalent of severing a cable. Operating in a congested or noisy radio band can exacerbate this kind of problem. No matter how well a radio is designed, however, it can be jammed. This occurs when one signal is so much stronger than another that a detector cannot find the weaker signal. Radio signals will be stronger if the source is closer, is designed to be more powerful, or has a high-gain and/or more focussed antenna. In the 2.4GHz band you’ll find 802.11 wireless networking, Blue Tooth, cordless phones, and numerous proprietary wireless systems. You’ll also find the largest and most insidious jammers: microwave ovens. They produce a lot of energy right around 2.4GHz, enough to cook food in seconds. Meanwhile, wireless systems are designed to emit very little energy, counting on exquisitely sensitive receivers. Herein lies the problem: all microwave ovens leak, at least a little bit. It is generally agreed that the levels of radiation coming from your microwave are, from a human health standpoint, harmless--but not to your 2.4Ghz wireless systems.
Imagine yourself driving a huge locomotive through a train yard by wireless remote, on the way to a service bay. It’s passing by the employee lunchroom, where a microwave oven is in use. The train stops. Not acceptable.
There are other challenges:
Almost any component failure will cause the communication link to become intermittent or stop entirely.
If the radio stops working and the last instruction was “go straight ahead at 100%”, there could be trouble. Safe receivers must require constant contact to remain active.
If two simple radio systems are operating in proximity to one another, and one is sending the message to go, while the other is sending the message to stop, the result at a receiver may be unpredictable. It might respond to either message, or interpret the garbled jumble of the two as a third message. Thus, digital encoding with error detection and correction are a must.
Even when the data protocol is robust and fail-safe, microprocessors and microcontrollers may behave oddly when subjected to static discharges, power surges, brownouts, electromagnetic interference (EMI) or radio frequency interference (RFI). And, like any electronic component, they can simply fail without explanation, with outputs in any state.
Microprocessors can even fail in a partially functional state (most often due to software problems). Take a processor that is no longer running its main program correctly to read input data, but continues to run an interrupt that transmits old data over and over and over. That packet could be telling a motor to run, and the receiver could accept it as valid data.
The most likely components to fail in a power driver are the ones that carry high current, because they are under the most stress. Most semiconductors will fail as a short circuit, continuously delivering power (they melt internally into a blob of conductive metal). This means that a probable fault may result in uncontrolled full power being delivered to a motor.
The primary objectives when designing a remote control motion system are:
1. Ensure that any failure will disconnect power and safely stop anything that is moving.
2. Wherever possible, monitoring and intervention systems should not be software based.
3. Electronic hardware must be protected from static, surges, interference, etc.
4. Data protocols must be resistant to errors and interference.
5. End-user performance requirements must be met. The system must be fast, precise, and reliable, delivering a pleasing result show after show after show.
Safe radio is possible
A well-designed wireless motion control system will have a very low rate of failure, and any foreseeable failure will result in the target device safely stopping. In general, wireless must work as reliably and robustly as wired systems do. And we all know wired systems do fail on occasion; dirty connectors and controls, physical damage, and poor maintenance will take their toll, no matter what the technology. With wireless, there are fewer damaged cables, but more damaged (or worn-out) batteries.
How do you ensure a faulty system will always stop? Fail-safes, performance monitoring, and automatic intervention.
A fail-safe, in simplest terms, is anything from a hardware dead-man switch to software cyclical-redundancy-checking (CRC). It refers to any means that prevents or appropriately responds to a predictable mode of failure.
If you can, design props so they will only operate when someone nearby is holding a switch closed. When open, this switch disconnects power from the motors. In theatre, however, this is impossible a lot of the time; in my own work, the Phantom gondola and the sleigh in the Nutcracker both require driving into the wings unoccupied.
The next best place to put a dead-man switch – a foot-pedal or momentary switch that must be held down – is with the operator. But how can you be sure that the release of that switch at the transmitter will cause the motor at the receiver to stop? What if the radio isn’t working to send the status of that switch to the mobile unit? The answer is independent performance monitoring; my favorite implementation is a heartbeat system. A pulsing signal is fed into one of the controller inputs. This dynamic signal must make its way through the wireless system and arrive intact at the receiver to engage a main-contactor relay.
The pulse is generated as early as possible in the system: if the operator is using a handheld controller, the pulse will be generated within that same unit. The dead-man switch engages this signal, sending the pulse only when the switch is closed. The pulse is decoded at the other end of the wireless system, preferably in parallel with the motor drivers. Every stage of the system must work successfully to communicate the constantly changing pulse, from source to destination. If the heartbeat is not changing, or is not at the correct frequency or amplitude, the main contactor opens and no power can reach the motors.
A heartbeat is used, rather than a steady-state signal, to ensure that the entire system – from beginning to end, wired and wireless – is quickly and appropriately responding to changing signals. Various faults could result in a signal being stuck on or off, but everything must be functioning properly for a changing signal to get all the way from input to output and be within specification. The quality of the heartbeat decoder is significant: it must not be easily fooled by 60Hz or 120Hz noise, pulse-width-modulation signals in motor drivers, or other predictable electrical interference or pulsing systems. Ideally, both ends of the heartbeat system will be constructed with simple and reliable electronic hardware that is not microcontroller- or software-based. Simple hardware is far more robust and isn’t subject to buggy code.
The wireless system delivering motor outputs must be the same one delivering the heartbeat. There must be no possibility of the heartbeat getting through when the motor control signal is not, and vice-versa. Ideally, all system inputs should be contained within a single data packet that includes device ID and checksum. This is where we lean on the power of microcontrollers.
If your wireless remote control prop is operated by a person at all times, a heartbeat-controlled main-contactor provides ample protection. When something goes wrong, the operator disengages the heartbeat and the system halts. When the main contactor relay opens, there is no power to move the system. If there is a possibility of coasting, the piece should be equipped with electric brakes that lock when power is removed.
But this is not quite enough if the piece is fully automated, with no human operator directly responsible at all times. In this case, there still remains a very real possibility of a semiconductor failure that causes full power to be applied to the drive motor. The heartbeat may be present, along with a control signal telling the motor to stop, but the motor continues to run.
This can only be prevented with another more local implementation of independent monitoring and intervention. A circuit must compare the control signals going to the power driver and the power coming out of the driver, and open the main-contactor relay if there is a discrepancy. If there is power when there is not supposed to be, the system must shut down using a mechanical interrupter (the main contactor relay).
In the case of crane and locomotive controls, all of these techniques are employed at all times: the digitally encoded signal is robust; a dynamic check signal is monitored (the heartbeat); power driver outputs are monitored. Any fault removes power to the motors.
To be safe, theatrical wireless remote control must employ these same techniques, while also delivering shorter latencies and higher resolutions than demanded by industrial remote control systems. Wireless lamps must come on at the same time as wired ones, and they must dim with the same smooth responsiveness. Motors must operate safely and reliably. The radio link must be resistant to interference. Multiple systems must be able to co-exist without affecting one another.
Does this sound like a tall order, especially on a budget? You may be surprised to hear that safe and affordable theatrical technology for remote control is on the market. So the next time you face a design problem that could be done with radio, have no fear – it can be done beautifully and safely.
James David Smith is the president and product designer for RC4 Wireless Dimming. He can be reached at jsmith@jamesdavidsmith.com or on the Remote Control Lights and Motion user group at http://groups.yahoo.com/group/ theatrewireless/ [now superceded by http://theatrewireless.freeforums.org/]. He will be presenting the Wireless Remote Control seminar at the USITT Conference and Stage Expo in Louisville KY on March 29, 2006.
Sidebar:
Remote Control at The Met Opera and Shaw Festival
By James David Smith and Ian Phillips
January 2006
The current Metropolitan Opera production of Wagner’s Lohengrin, directed and with set design by Robert Wilson, was first seen in the 1997-98 season; it is in this year’s repertory, as well. In the production, two set pieces, King Heinrich’s throne and Lohengrin’s Swan, required movements that could not readily be accomplished with a wired automation system. Wire guidance could be used much of the time, but there could be instances where the piece would have to stray from the guide to maneuver around an unexpected obstacle.
The solution: Randy Sautner, an electrician at the Met, investigated commercially available systems and decided that nothing on the market at the time of the original 1997 production was suitable. He developed a home-grown solution based on Microchip PIC microcontrollers and custom electronics. For wireless communication, he used Abacom radio modules, which are easily interfaced to microcontrollers.
The custom firmware of the system, written by Randy, provides redundancy and error-checking, while delivering the necessary resolution for fine control of speed and direction. Data packets include checksum-based error detection to avoid false triggering. A main relay allows power to be cut in the event of a transistor failure.
This was a pioneering system that broke away from the dangerously unpredictable performance of hobby airplane systems, at considerably lower cost than commercial remote control systems of the time.
As specified, units can follow a wire guide, but can be switched to
manual control on the fly. Normally, the wire-guided path is followed,
but occasionally an actor or set piece end up in the guidance path and
must be accommodated. Equipment included a Microchip PIC16C71 processor,
Abacom TXM-418 UHF radio module, Rosstron, Inc. motors and drives, Mangescraft
relays, and Powersonic batteries. Also involved in the project were technical
director Joseph Clark and master electrician Joseph Gracey. Lohengrin
will play several performances at the Met in April and May.
Bring on the swimming pool
For a production of the Cole Porter musical High Society at the Shaw Festival’s coming summer season, a Greek pavilion must move autonomously from far upstage left to downstage center. The front stairs will extend to become a swimming pool (sans water) for two scenes. The second time the pool comes out, it will close only part way. In other scenes, the cast will use the stairs to reach the pavilion deck, which is more than 18” above the stage. A cable and dog in the stage can be used to move the unit diagonally, but the pool has to ride with the complete unit. A second smaller unit will move downstage, tracking left to right.
The scene shop has decided to use a heavy-duty glide system, similar to a drawer glide, to mount the pool to the pavilion. A 24V DC wheelchair motor, driving a chain and shaft system, drives the pool. The motor, drive mechanism, and two 12V DC batteries are operated wirelessly. The smaller unit is operated and controlled the same way. Safety, particularly implementation of the wireless E-Stop, was a primary concern.
Shaw’s wired NISCON RAYNOK automation system features individual contactors for each drive, with an overriding E-Stop control. With the assistance of the manufacturer, interfacing to the system was fairly simple. Ian Phillips, electrician for the project, wanted the wireless automation to behave exactly as if it was wired, so he needed to get E-Stop, forward, reverse, and brake signals to the remote piece. These lines are wired from the RAYNOK to Shaw’s RC4 radio transmitter, which then make their way to the Pavilion by radio. Motor travel time is controlled by the RAYNOK software, allowing the unit to be stopped in a mid position. Full extension and retraction is detected and limited via a rotary limit switch.
Shaw’s RC4 wireless dimming system is regularly used for DMX-controlled lighting. The transmitter also has 12 patchable analog inputs, which can be configured as CV dim or non-dim inputs. This enables two different departments to use the wireless system (good board of directors’ argument here).
At the RC4 four-channel receiver/dimmer, it’s just a matter of connecting interface relays to the motor controller board. The motor controller Ian chose has many accessory inputs, but he used only the forward and reverse.
To ensure the highest level of safety, he added an RC4-MSS motor safety system, which encodes the E-Stop line using the heartbeat method and ensures the remote system will be shut down if a serious fault occurs anywhere in the control chain.
As with any high-power battery system, fusing is important. This system uses two voltages: 24V DC for the motor, controller, and motor brake relay, and 12V DC for the receiver, E-Stop, and control relays. It cannot be stressed strongly enough that careful attention must be paid to fusing! Always insert fuses as close to the battery as possible and use the lowest appropriate rating. The relays are interlocked, so that a forward and reverse command can not take place at the same time. The relays are also in series with the rotary limit switch for the out and in positions. The onboard accel, decel, and speed potentiometers take care of the power curve, ensuring a smooth movement. Auxiliary air brakes get their own RC4 wireless dimmer channel.
The MSS heartbeat resolved many concerns. Most DC dimmers use a type of transistor called a MOSFET. If you blow one up, odds are that it fuses “closed” and stays on no matter what you do. With the heartbeat system, if the dimmer is locked on, the pulse is no longer present and the output is shut off. The same is true if the dimmer is stuck off. The only way to get that E-Stop relay closed is for the heartbeat pulse to be constantly working in both the on and off states.
Your next question might be “what if radio noise triggers the dimmers?” Ian says, “In my experience, with the equipment we use, this doesn’t happen. But if it did, the noise would have to look pretty much exactly like a legitimate heartbeat signal, and it would have to sustain the illusion for quite some time. Frankly, it’s not going to happen. RF hardware and software are at a point now where any quality system can deal with extraneous noise. The worst-case scenario should always be a shutdown, never an unexpected on level.”
Shaw’s two pieces of wireless automation are working perfectly!
Ian Phillips is assistant head of electrics, the Shaw Festival. He will be a presenter at the USITT Stage Expo seminar on Wireless Remote Control in Louisville KY, March 2006 and can be reached at ian.phillips@sympatico.ca.