Thursday, 23 August 2018

Flowers for Marcel - the end of my intelligent locomotive experiment

This weekend I finally abandoned my attempt to build an intelligent, "internet of things" locomotive for my garden railway. It was a very disappointing decision, because it so nearly worked. It worked on the bench, but the realities of a garden railway with dirty track, poor connections and so on meant that it was never reliable outside.

I'd chosen the most reliable and simplest of all my LGB locomotives, the Corpet-Loubet 0-6-0. This was the first one I ever bought, back in 1999, and had given perfect, trouble-free service ever since, using a Zimo decoder for DCC.

Objectives


The objectives for the project were:
  1. 100% compatibility with DCC - the loco should work on my DCC-enabled layout exactly like any other
  2. tight feedback control over speed, so the selected speed would be maintained regardless of load or layout voltage variations
  3. "non stop" operation to keep the loco moving over dirty track, dead spots and so on, using a substantial super-capacitor
  4. constant status reporting over WiFi of several things including motor current, track voltage and distance travelled
  5. in addition to DCC control, the ability to control speed, accessories, CV settings and everything else over WiFi

Design


I selected the Particle Photon as the microprocessor. It is about the size of a USB memory stick, yet has a powerful ARM processor running at 125 MHz and, built-in WiFi. My earlier experiments with adding WiFi to the Arduino had been a painful failure, so this was extremely important. Another important advantage is that the libraries are a superset of the Arduino. As it turns out, however, it isn't really suited to this job - more on that later.

To meet the first requirement, I needed an implementation of a DCC decoder. Fortunately there is a very nice one out there, NmraDcc. It's written for the Arduino, but needed only one small change to run on the Photon - because the Photon does not support hardware timer interrupts. You write functions to handle speed changes, accessory operations and so on, and it calls them when the corresponding DCC commands are received.

For a computer to be useful, it needs to communicate with the world around it. I needed the following inputs:
  1. Track voltage (analog)
  2. Internal power supply voltage (analog)
  3. Supercap voltage (analog)
  4. Motor current (analog)
  5. Motor back-EMF, used to measure speed and hence keep it constant (analog)
  6. Axle position sensor, used to measure absolute train position (digital)
  7. DCC signal, via opto-coupler from track voltage (digital)
  8. Motor control signal feedback (digital)
  9. Motor over-current (digital)
  10. Accessory over-current (digital)
and the following outputs, all digital:
  1. Control signal to motor control FET, operated via built-in pulse width modulation (PWM)
  2. Control to motor reversing relay
  3. Accessory outputs, as many as possible (limited by number of available GPIO pins)
The first thing to get right was the power supply. The non-stop feature is built using two 30F 2.7V supercaps in series, giving a 5.4V 15F capacitor. This is kept charged by a tiny buck regulator from Ebay, adjusted to give a constant 5.3V output. It has built-in current limiting to 1A. When the loco is placed on the track it takes about a minute for the cap to charge. A 3A boost regulator, also from Ebay, then turns the 5.3V back into 16V to run the motor. An arrangement of hefty Schottky diodes (chosen because of the lower voltage drop) normally takes current direct from the track, but as soon as the track voltage drops below 16V, the supercap provides the power. This arrangement is also used on my LGB track cleaner, where it works perfectly.

The power supply to the microprocessor comes via a second Ebay buck regulator, that drops the supply voltage to 12V for the accessories (lights etc), then a 7805 linear regulator that provides a stable 5V supply. At least, it is supposed to, though that turned out to be one of the weak spots of the design.

The motor is controlled via a power Mosfet, an IFR9540 that can happily switch 20 amps. Reversing uses a relay. The conventional way to control a motor these days is via an H-bridge (e.g. an L298) but I couldn't see how to get the back EMF this way, so I went for a relay instead. (I did think of a way later but by then I had built the board).

The analog inputs are first scaled by a couple of resistors, so they will never be more than the 3.3V input range of the CPU, then connected to it via a 4K7 resistor. This ought to protect the CPU, but experience showed that it doesn't.

The digital outputs are buffered via 4K7 resistors to an ULN2804 8-way Darlington. The accessory outputs, of which there are 6, are taken to a terminal block.

Software


That just leaves the software. There are several components to this:
  1. DCC decoder, using NmraDcc
  2. Basic motor and accessory control, turning the intended speed into a PWM ratio to control the motor voltage, direction into the sense of the reversing relay, and setting the accessory outputs
  3. Feedback-based motor control, reading the actual speed and adjusting the motor voltage so it matches the intended speed
  4. WiFi interface, sending and receiving messages.
  5. Generating status messages
  6. Interpreting commands received by WiFi
I invented a log format, which is sent once per second and includes not only the items to be monitored but also various internal variables that show how the motor feedback calculations are working. These messages are sent to an IP multicast address, so the loco does not need to be configured for where to send them. They contain the IP address of the loco, which can be used for sending commands back to it. A simple Python program listens to the messages and logs them, and allows commands to be typed and sent to the loco.

Speed Control


The hard part of the software is the feedback based speed control. The concept is simple enough: measure the actual motor speed, and adjust the motor control so that it matches the desired speed as set by the user. It seems simple, but control systems are always a compromise between agility, i.e. responding quickly to changed circumstances, and stability, which most importantly means not oscillating. In this case there's no point if it takes say ten seconds to respond, since the changed circumstances - like going round a tight curve - will likely have gone away.

Motor speed is measured by back EMF, the voltage that all electric motors generate in the opposite direction to the supply. Since the motor is controlled by turning the power on and off, it's straightforward to measure the voltage while no power is applied. A complication is that it is not at all linear with the motor speed, so the software needs to understand the actual relationship.

I also implemented direct speed compensation in response to supply voltage variation. As the supply voltage drops (due for example to dirty track) the motor feed is directly increased in proportion. This sounds like a good idea, but it does lead to problems as described below.

In practice I never found a set of control parameters that really worked. Anything that gave enough agility also led, some of the time, to control oscillation, meaning that the loco sped up and slowed down as it moved along at a constant speed setting.

Photon - the Good, the Bad and the Ugly


The Photon seemed the perfect part to use for the CPU in this project. It has a lot of good points, but in the end the bad points overcome them:
  • it takes a long time to start. By default, it not only has to find and log on to the WiFi network, but also set up a connection to its cloud-based server. This can take up to 10 seconds. In the situation where it is rebooting because it has briefly lost power, this is completely hopeless. It's possible to program it to avoid the second part, but it still takes a couple of seconds before it starts running its program again. The PIC processors typically used on DCC decoders are running code within milliseconds, meaning that the outage passes unnoticed.
  • it's impossible to run fine grained (microsecond) timers. For some reason to do with the cloud server, again, the finest timer resolution you can get is one millisecond. It would have been good to control the output FET directly in software, but that's impossible. Instead you have to use the on-board PWM, which significantly complicates the software.
  • it's electrically very fragile. Even a momentary signal over 5V applied to its inputs, even via a 4K7 resistor - meaning the current is limited to a milliamp - instantly destroys the whole chip.
  • it loses its configuration pretty easily - the WiFi credentials and worse, the credentials needed to contact the cloud server. Of the five Photons that gave their lives to this project (see previous point), one in particular lost it every time the software crashed. Reinstalling it is a painful process, requiring a physical USB connection to a computer and a whole series of arcane commands.
  • it's very difficult to debug. This is normal for a small embedded processor or Arduino, but the Photon is trying to be one step above this. There's no interactive debugger support, and no way to do "debug with print statements" either. Programs bigger than you could run on an Arduino are just impossible to develop as a result.
The Photon is no doubt a good part to use if there is a rock-solid power supply, it's interfaced only to gentle things using the same stable supply, and the program is pretty simple. But none of those applied in this case.

Practical Experience



The Photon board, carefully trimmed to fit - just! - in the available space.
The CPU is to the right, with the power controller to the left.
I spent a long time - several months, on and off - trying to make this work. Getting all the necessary electronics onto a board that would fit inside the loco was quite a challenge. It looks as though there is loads of room, until you try squeezing all the components in. There is also a fairly severe height limitation, especially at the sides in the water tanks. It would be easy enough using tiny surface mount parts, but they don't lend themselves to prototyping. Instead I used 0.1" stripboard, and a lot of wires.

The first Photon was a victim even before the board was built, when I was testing the basic circuit ideas using plug-in breadboards. Two wires touched and pfff! - the end of the first Photon. Another one died when I foolishly ran the board without it being firmly screwed down in the locomotive, and short-circuited the traces underneath it on some tool on the bench. That did a lot of damage, and not just the CPU. Two more just mysteriously died, for no obvious reason.

The feed-forward in the speed control turns out to have an unfortunate effect on dirty track. As the voltage drops, the loco tries to pull more current, the voltage drops further, and so on. If the initial voltage drop is large enough - i.e. the track is dirty enough - the track voltage drops so low that the loco stops. In the end it's better just to accept the loco slowing down.

There was a similar problem with the non-stop circuit, which is trying to keep its supercap fully charged even if the track voltage is lower than normal. That also leads to the same kind of evil feedback loop.

The biggest problem, though, was with the power supply to the CPU. On the bench, everything worked perfectly. But out on the garden track, no matter how carefully I isolated it, no matter how many smoothing caps I introduced, the CPU would unpredictably reset itself. This wouldn't matter, except that (see above) it then took several seconds before the train would start to move again, if at all.

Lessons Learned


The main thing I learned from this is not to do it again. The benefit is really not worth it. That said, if I did do it again I would;
  • use a CPU board that is more of a microprocessor rather than a "cloud of internet of things".
  • enforce total galvanic isolation of the CPU from all of the ugly electrical side of things. There are chips available that do this even for analog signals.
  • build a power supply for the CPU which is galvanically isolated - there are some neat chips that do this - and protected in every possible way from any kind of electrical ugliness: supercaps to keep it running for tens of seconds, beefy zeners to protect against over-voltage spikes, and so on.
My plan for now is to re-purpose the board I built. It's still useful to have real-time monitoring of the electrical conditions around the track. I plan to put it into one of Marcel's carriages, just as a passive WiFi reporter of track voltage.

The End of the Experiment



Marcel's new simplified electronics - just a Zimo decoder and
a tiny board to make the 9V for the carriage llighting.
I might have persevered for longer, but summer time came round, and Marcel's two French coaches looked lonely sitting in their siding. I really wanted to get him back on the layout and running. It took me just an hour to construct a carrier board for his old Zimo decoder, and a tiny Ebay buck converter to run the lights. Very soon he was chugging happily round the layout with his coaches in tow. He seems extremely happy, and probably very glad to get off my workbench and back outdoors.



Marcel on the layout with his short train of two French coaches








No comments: