Friday 31 July 2020

Building Cisco's Japan Development Center

Definitely the most interesting thing I did in my time at Cisco was to start an engineering team in Japan. How that came about is a story in itself.

My job at Cisco had nothing to do with Japan. I ran the router software group, IOS, which was a seriously full time job. It comprised about 500 people, mostly at the corporate HQ in San Jose, California, but plenty spread around the world including the UK, where I was initially hired, India, France, and several locations in the US. My move to the US coincided with a total, absolute hiring freeze after the crash of 2000. Senior management understood that we needed to grow the group, though, so every time some remote acquisition turned out to be surplus to requirements, we would acquire bits of the team. I had people in Colorado, North Carolina, up north in Sonoma County, and sundry individual contributors working from wherever they happened to have been hired.

Senior management was expected take on various odd tasks that had nothing to do with the day job. One such assignment that came my way was giving the opening keynote speech at the company's customer conference (Cisco Live) in Japan, in 2003.

Back in the 1980s, international standards work had taken me to Japan several times. During the 1990s my wife went there often for the same reason, and I would sometimes tag along. But this was my first visit since before I'd joined Cisco, in 1999. There were things I'd forgotten since my previous visit, like when the airport bus arrived at a hotel and the staff ran to meet it, then stood and bowed as it pulled away. You get used to this, but after a five year interval it surprised me again.

I was greeted very courteously by the head of marketing in Japan, who has since become a very good friend. It's assumed in Japan that foreigners will need their hands held at all times. It takes a lot to convince them that you can safely use the metro and railway system without getting lost. I think it would be a serious loss of face to mislay a visiting Vice President, so even if they believe you they are reluctant to let you try it. Consequently, I had been met at my hotel - the New Otani - and accompanied to the conference location.

I had a carefully prepared presentation - it had never occurred to me to ask for help or any kind of corporate guidance. But it was only in the hour before I gave it that I learned it was "the" keynote for the conference. I made a few hasty changes and it seemed to work OK. I spoke in English but there was simultaneous translation to Japanese. I asked the head translator, a very distinguished Japanese guy in his 50s, how fast I should speak. "About a quarter as fast as your CEO" was his answer. In fact it was very easy to pace myself. Every single person in the audience of hundreds was listening to the translation on headphones. There was enough sound leakage that I could tell when the translator had stopped, and start the next sentence.

That trip led to a couple more to meet Japanese customers, and that in turn led to a fairly surreal activity. We were trying to convince one of the big Japanese operators to switch to Cisco for their core network. Part of this was a technical collaboration around mobile networking for which I was the corporate sponsor. Every three months we would have a meeting, mostly in or around Tokyo though sometimes in the US, with half a dozen people from each side. Their technical team would present what they wanted to do, and how, and our technical team would respond. The two teams totally disagreed, but it didn't matter. At the end the customer's VP and I would both give little speeches saying how impressed we were with the spirit of cooperation and the progress that had been made. And then three months later we would have exactly the same meeting, with exactly the same presentations, and exactly the same speeches. We got some truly excellent Japanese meals out of it.

It was all worth it though. After several years of this, and long after I had left Cisco, we won the business, worth hundreds of millions of dollars.

You'll gather from this that I loved Japan - and still do - and was very happy to have good reasons to go there as often as possible. I got to know the country manager - now sadly no longer with us - quite well. Our only disagreement was over how I should address him. He wanted me to do it the American way, using his first name. In the Japanese culture first names are used only by childhood friends and immediate family, and even then not always. I just couldn't bring myself to do it, and always addressed him in the Japanese way as Kurosawa-san. If we had met in the US, it would have been different.

Over dinner on one trip he told me that it was his dream to have a corporate engineering activity in Japan. It was a constant ding against American suppliers that they did no R&D in the country, and he wanted to counter that. I thought it was a great idea, but when I presented it to my management back in California they practically laughed in my face. It would be expensive, we wouldn't be able to find or hire the right people, it made no sense, and so on. So that was that.

But Kurosawa-san was resourceful, and at some point, knowing that he had the practical support he needed from me, he managed to convince the CEO, John Chambers. That changed everything. Suddenly I was told to make it happen.

The biggest challenge was to find someone to run it. The key to any remote team like this is to find someone who understands the corporate thinking, and who also understands the local culture. Generally this is impossible, which is why so many remote teams fail miserably. I had a very lucky inspiration. One of the UK team which I'd inherited when I joined Cisco was bored and looking for something new. A Norwegian called Ole, he still had some Viking blood, one of life's adventurers looking for the next Big Thing. It helped a lot that he already knew some of the Cisco Japan team. He was signed up almost before I'd finished asking him.

The other big challenge was to assemble the nucleus of the team. But that turned out to be much easier than I'd expected. In 2004 Cisco's prestige was high and people were keen to be part of its product development team. There were people already working for Cisco Japan, who had taken jobs in support for want of anything better, who were happy to move into engineering. Through personal contacts we found engineers at Japanese companies who were happy to make the change. One of the new team was Japanese but working for Cisco in California. Quickly I had a nucleus who could be trusted to grow the team - although, in the end, it never did grow.

We had to put the team somewhere. Initially we borrowed some space from the country sales operation, in Shinjuku, but the hope was that eventually it would reach a hundred or more people. For that it made sense to think about a location outside Tokyo, which led to our "fact finding" trip to Kanazawa in Ishikawa prefecture, and our amazing lunch with the prefectural governor that I've written about before.

Everything came together in spring 2005. We had a team, we had an office, and we had someone to run it all. I went to Tokyo for three weeks to get it all started, and luckily my wife was able to come with me. Rather than stay in a hotel, we rented a very pleasant apartment in the Aoyama district of Tokyo. It's the closest I have ever come to living in Japan. We shopped for food in the local supermarkets, an interesting experience for my wife who neither speaks Japanese nor can read any of the characters. Most things can be identified from pictures on the labels, but she needed my help to distinguish salt from sugar and flour. We had a wonderful time there, one of the most memorable trips of my life.

For the next year or so, I visited Japan every three months. It led to some complicated itineraries, since I generally combined them with a visit to the team I still had in the UK. Cisco had rented an apartment in Tameike for Ole and his wife, absolutely vast by Japanese standards, a ten minute walk from the New Otani where I stayed on every trip. Each room was bigger than a typical Tokyo apartment. I spent many memorable evenings there, though the next mornings were sometimes a bit hazy. Apart from the team itself, he'd built a "support network" in Japan who helped him and all of us with every aspect of things.

Since I was in Japan so often, I got to know the country sales team well too. I visited several important Japanese customers as "the man from HQ". I would sit there in total incomprehension as the "fireside chat" meeting ran its course, but apparently just my presence made a big difference.

It was important for the team to know their colleagues in California, and we arranged for them all to visit at the same time. The trip happened to coincide with Halloween, and we arranged a fancy-dress party at home. One of the team's hobby was traditional Japanese kimono, and she had brought with her a complete outfit. She looked amazing, delicate and beautiful in the Japanese tradition, and definitely took first prize by popular acclaim.

I left Cisco about a year later, and Ole decided to return to Europe at the end of his two year contract. A local manager was hired. But by then Cisco had lurched into much more aggressive expense control, and the planned expansion never happened. The country manager retired, a victim of corporate politics - the destiny of all who reach the senior ranks of Cisco. With no sponsors left, the group lingered on for a surprisingly long time, but in the end a bean-counter somewhere spotted it and its destiny was sealed. Some of the engineers returned to non-engineering roles, some moved to the US, and some left the company.

The country manager, who I got to know well, once said to me "When you make a friend in Japan, you make a friend for life." And it's true. Even now, fifteen years later, I still have good friends there who I see whenever I get a chance to visit.

Saturday 4 July 2020

The Garden Railway: Trouble with Turnouts

The design and construction quality of LGB equipment is astoundingly good. You can leave the trains outside in all weathers with no damage or deterioration, whether from rain, snow, intense heat, or UV radiation. The locomotives will pull heavy, friction infested trains all day long without complaint. If anything does break, even the tiniest moulding is available as a spare part, albeit at a high cost. The track too is tough as anything - you can step on it without damage, and electrically it works far better than you could reasonably expect.

But nothing is perfect. The one place where their attention to quality seems to have lapsed is the pointwork, those necessary but fiddly places where trains get a choice of direction. They're eye-wateringly expensive - roughly $100 for a new, electrically operated turnout. But with LGB you just have to get used to that. The problem is, they just aren't that well designed. There's a sort of pervasive optimism, a feeling of "it'll be alright on the night", that applies to every aspect of the design: electrical, mechanical and trackholding.

My garden railway currently has a total of 17 LGB turnouts, all electrically operated via my NCE DCC controller. All the ones on the main running lines are 16xxx medium radius. There is a yard with the small, 600mm radius 12xxx turnouts, mostly bought new 20 years ago. The others are a mix of new, at various times over the last 10 years, and some eBay bargains, of which the oldest was probably 40 years old.

Keeping them all in good working order, so the trains run over them smoothly without derailing, jerking, or just coming to a halt, requires constant attention.

Electrical Problems

All of the electrical side of LGB stock suffers from a degree of design optimism. There are simple rubbing contacts everywhere, for example between the pickups, motors and other electrical connections. The wires are made of brass, which slowly forms an insulating oxide layer on the surface, so intermittent electrical problems slowly arise as the trains gets older, especially when they live outdoors.

The slider pickups on the locomotives are a case in point. The idea is excellent, but the connection from the slider to the rest of the electrics depends on a fragile spring, wound with wire barely thicker than a human hair. If ever there is a short circuit in the engine, the spring heats up to the point where it loses its temper - which is to say it stops being a spring, so the pickup stops working. It's possible, but very fiddly, to replace the springs, and to make it more complicated a different part is needed depending on the particular locomotive.

This pervasive electrical optimism really strikes hard on the points. The outer running rails are solid brass, connected to the adjacent track by heavy, springy fishplates. No problem there. But the connection to the switch rails - the ones that move - is made by very primitive sliding contacts under the rail. This works way better than it deserves to when the track is new, but as it ages the contacts and the rail itself oxidize, and the force holding it all together weakens. The net result is that trains hesitate or flat-out stop as they are going over the points.

It doesn't help that there is a lot of dead track. The place where the two rails cross - the "crossing" or "frog" depending on your train-speak dialect - would ideally be connected to one rail or the other depending on the point setting. This is difficult to arrange, and LGB didn't try. These sections are made of insulating plastic, meaning that one wheel, at least, stands no chance of picking up power. Four wheel locomotives, like "Shiny", our Wismar railbus, are especially vulnerable - the more wheels the better.

The diverging rails are connected invisibly, under the sleepers, by metal strips that are spot-welded to the running rails. They also are a bit optimistic. On several of my older points these welds have failed, leaving a lengthy piece of rail with no connection.

Underside of turnout showing
soldered connecting wires
The solution to all these problems is to make soldered connections between the various pieces of rail. This is a bit daunting since large-section brass rails conduct heat away from the joint area very effectively, and there is an obvious risk of melting the plastic base. To deal with the second problem first - the plastic used for the bases is pretty resilient to soldering. It softens, but doesn't melt, when you heat the rails up. It will melt and burn instantly if you touch it with the hot iron, though.

I've been pretty successful soldering fine wires to the underside of the rails. My technique is:
  • start by cleaning the metal very thoroughly, with a fibreglass "scratch brush", until it is gleaming
  • then cover the joint area in non-corrosive resin flux
  • I use a 50W temperature controlled iron, set to its highest temperature of 425°C, with a substantial chisel-shaped bit about 7mm across - providing plenty of reserve heat
  • hold the iron flat against the rail, holding it as far as possible from the plastic, and hold the solder against the iron - when melted it acts as a heat transfer fluid
  • now hold the iron in place until the rail is hot enough to form a proper joint with the solder. It's easy to see this because the blob of liquid solder suddenly spreads out on the metal
  • now add the wire, then hold it in place with a screwdriver or similar until the solder solidifies again. This will take a while - up to 30 seconds - because of the heat retained by the rail
  • Don't touch anything! - the rail stays painfully burning hot for a long time afterwards.

Mechanical Problems

In real life, track is held on the sleepers by some kind of spike driven into the wood, which either directly holds the rail, in US practice, or holds a metal plate which in turn presses on the base of the rail, in Europe. (It's different for serious railways, with high speeds and heavy trains, but light and narrow gauge railways work like this). The LGB track provides a good visual impression of this, but it really isn't very strong. The rails are held in place by tiny flaps of soft plastic, less than a millimetre thick. It takes very little to twist and break them.

On normal track this isn't really a problem. The sleepers all support each other, so they aren't subject to high stresses. And even if one does break, the rail is still supported by those around it. Points are a different story though. For example, the very first sleeper, closest to the moving switch rails, is the only one supporting the point motor and the first few inches of rail. It can easily get broken, and when it does, the vertical relation between the fixed rail and the moving one is lost. Trains fall off the track as a result.

It's impossible to repair the track base. What I have found effective is to glue the rail in place on the damaged sleepers, using the remains of the simulated spike. The plastic is something soft and difficult to stick to, but I have found a two-part epoxy that works well, Loctite EA9340. I originally bought it to make some repairs in the kitchen, where prolonged exposure to steam softened regular hardware-store epoxy, but it seems perfect for this too. Another advantage is that it dries to a murky dark green, making it pretty much invisible on the track.

The technique is simple. First get everything as clean as possible. Clean the rail with a fibreglass brush, and swab everything with alcohol. Then mix up some epoxy and make it into a blob around the base of the rail, so it looks like part of the sleeper. If several sleepers are damaged on the same point, do it for all of them.

Sometimes you can't blame LGB. One of my points was hit by a heavy steel ball, from playing French bowls (petanque) in the garden. The rail was badly twisted both horizontally and vertically, and many rail fastenings were broken. After I dismantled it and straightened the rail out, the epoxy worked perfectly to hold the rails in place. The repaired point is back on the layout, and trains pass it without problems.

Trackholding Problems

In Victorian times facing points - ones where the train has a choice of which way it goes - were regarded with horror. Railway designers went to great lengths to avoid them on main lines wherever possible. Where they were unavoidable, they always had facing point locks, which held the switch rails firmly in place while a train passed over them. They were interlocked with the signals, so it was impossible to clear a train to pass over the points unless the locks were in place.

Sadly our LGB points don't have these devices. They are held in place rather feebly by the magnets in the point motors. It's quite common to have a tiny gap between the fixed and moving rails - a fraction of a millimetre, but enough to cause problems. If a flange rides over the sharp end of the rail it can move the rail under it, opening the point and dropping into the gap on the wrong side. The rest of the train inevitably derails when this happens.

I haven't found a really good solution to this. Some point motors work better than others. I had one point that would consistently cause derailments. It was an old one, from eBay, with an older design of point motor. Replacing the latter with a newer motor held the rail in place much more firmly, and solved the problem.

The ideal, in the absence of an actual lock, would be a really firm over-centre spring mechanism, but I can't see an easy way to do this. In any case the force produced by the point motor probably wouldn't be enough to overcome it.

LGB four-wheel carriages and trucks have pivoting axles, to simplify going round the tight 600mm radius curves. Normally these are held at the correct angle by the traction on the coupling, but that doesn't work if the train is being pushed. And sometimes they get stiff. So they will occasionally end up trying to go through a point when the wheels aren't aligned correctly with the track. This makes the above problem a lot worse. It causes another problem, too.

In real life, points have check rails, or guard rails, which ensure the wheels go the right way through the "crossing" or "frog", where the two rails cross. The check rail presses against the back of the wheel and stops it slipping into the wrong, diverging flangeway.

Unfortunately the check rails on LGB points are mostly decorative. They are way too far from the rails to be really effective. Mostly this doesn't seem to matter, but on the three-way point they are not only too far away, but not in the places they need to be. There are so many problems with this item that it deserves an article to itself.