Cisco

(I wrote this over three years ago, when Cisco first started to show serious signs of losing the plot. They seemed to make a miraculous recovery, but now things look grimmer than ever, and my conclusion remains as valid now as it was then).

----------------------------

Cisco's recent woes will be no surprise to those who know the company from the inside. The only surprise is how long it has taken. I was surprised, though, by the naivety of some of the analysts' remarks, and in particular the apparently universal view that all Cisco needs to do is get rid of a few dubious acquisitions like Flip, for everything to be restored to its rosy past. One I read said, "The fall in Cisco's gross margin from 67% to 50% is due to the lower margin on consumer products. Disposing of these is a necessary step to restoring margins." Or something like that anyway.

The simplest of arithmetic shows that this cannot be true. Linksys is by far the largest part of the consumer portfolio, but it is still well under 5% of revenue. The others are just noise. Even if all these business ran at zero margin, the total effect on overall margin would be a couple of percentage points. So it can't be that.

Cisco has been trapped in an awkward spot for a long time, and the consumer business is just one aspect of a well-motivated attempt to get out of it. When you have a near-100% market share, there is just nowhere else to go. Well over half of large enterprises in the US still furnish their entire network infrastructure more-or-less unquestioningly from Cisco. Practically all of Cisco's actual profit comes from the switches and routers that are to be found in nearly every wiring closet and data center in the western world. Everything else is funded out of this apparently inexhaustible gravy train. Growing this to a $40B business has been an amazing achievement, but it's not easy to find another one like it, and this is no longer a rapidly growing sector as it was a decade ago. Hence the moves to the consumer, the data center and others.

But nothing lasts for ever. The core functions of enterprise networks, exemplified more than anything by the Catalyst 6500 family, are already a commodity. For well under $500, Broadcom or Marvell will sell you a chip that does everything you actually need from the Cisco product. They'll even give you complete designs, ready to be sent to a board shop. The days are long gone when Cisco's added value lay in the extraordinary breadth of IOS's support for obscure proprietary protocols. Networks now are all-IP - as indeed John Chambers has been saying. It's only inertia, combined with quite a lot of Fear, Uncertainty and Doubt, that makes people stick with Cisco's high-margin products. And just to twist the knife, the maturity of WiFi means that the enterprise is - finally - moving away from the all-wired model that puts multiple ethernet jacks in every office and cube. That alone has the potential to demolish Cisco's profitability. Worse, competition from HP, Huawei, and white-box vendors - none of whom have Cisco's cost structure or grossly inflated overheads - is driving down the margin.

Let me expand a little on the "grossly inflated overheads". Cisco's development organization functions like 18th century Europe or India - lots of principalities with headstrong leadership, who (mostly) do not compete with each other for territory, but have no intention of sharing anything either. That's why every Cisco product family has its own system architecture, its own ASIC development, and - increasingly - its own software. (Time was when everything ran IOS, which was reasonably common across all platforms, but now IOS has no future relevance for most of them. Only the name, IOS-something-or-other, remains common to what are in fact totally different software systems). The good thing about this strategy is that it avoids the "one egg, one basket" problem that for example brought down DEC in the 90s. The bad thing is that it is hugely expensive, and tends to get worse and worse - it's rare that divergence returns naturally to convergent paths.

This has been a deliberate strategy on Chambers' part. The last time Cisco had any real cohesive technical strategy was under Ed Kozel, in the late 90s. Since then the CTO has been paid to not interfere in the operation of the individual principalities. Charlie Giancarlo seemed to be looking for some kind of unification during his brief reign, and for his pains - ahem - decided to seek new opportunities. Since then there hasn't even been the pretence of any overall leadership of product development, just a "council" whose role is to rubber-stamp whatever each group was doing anyway.

So what can Cisco do? The core business is moving to a commodity, and shrinking anyway. Numerous attempts to move out of this area have met, at best, moderate success (by Cisco standards that is - even the failures make more revenue than many mid-sized companies, but they aren't profitable). What does Cisco really have? The answer, of course, is a vast and mostly loyal customer base. Cisco as a vendor and integrator still has huge strengths - in particular, its (entirely justified) reputation for digging customers out of the dirt no matter how they got there. It's still true, in the enterprise at least - as it once was for IBM - that "nobody gets fired for buying from Cisco".

So Cisco needs to make the same transformation that IBM did 20 years ago under Lou Gerstner, from being primarily a product company to being primarily a service company. And, in the process, to realise that practically all of its product development is context, not core. That will be very tough to accept - it means questioning everything, even the core switch business, not just closing down Flip. It will take a while to happen, and the risk in getting from here to there is huge.

On Airline Seats

Having just spent 11 hours on an airline seat - with more to come - including several hours when I was pretending to be asleep but wasn't, I couldn't help reflecting on them. The seat in question was KLM Business Class. KLM is in the majority of long-haul airlines who give you a seat which is flat but not horizontal, on a 60-inch pitch. Only a few airlines - BA and Virgin among them - give you a horizontal bed.

Overall, KLM's seat was pretty good, as sleeping on a hillside goes. They've done a better job than Lufthansa, who use what seems to be the same seat design - Lufty gives you absolutely nowhere to put your book, or your glasses while you sleep. KLM's version is full of handy cubby-holes. There is just one weird thing. The socket for the headset is at the very back of a cubby-hole under the armrest, and the plug is a complicated triangular thing that has to be orientated correctly. I ended up on my knees in front of the seat trying to get my headset plugged in. Then I accidentally pulled it out, and since I couldn't repeat my act of worship without disturbing my sleeping neighbour, I resigned myself to no more sound. Eventually I did get it back in, but I can't imagine how everyone else deals with it. Surely it wouldn't have been hard to put it somewhere more accessible, like on the front of the seat?

BA was (as far as I know) the first with the lay-flat seat, in about 2001. The design has evolved since, but it's still basically the same. It gives you a full six-foot horizontal bed, with the additional advantage of complete isolation from your neighbour. The upstairs window seats have complete privacy, and a handy locker you can reach without standing up - and of course a magnificent view of whatever you happen to be flying over, such as this picture of Greenland. It is about as close to perfection as an airline seat can get. My wife likes her favourite seat, 64K, so much that she refuses first-class upgrades so she can keep it.

Air France's first attempt at a flat seat was a disaster. Their Mark I version was at quite a steep angle, so you constantly slid downhill. Every ten minutes or so you'd wake up, as your torso slid down into the space under the seat in front. To make it worse, the footrest was only a couple of inches wide, barely enough even for your ankles. So when you woke up you would first scrunch your ankles into the footrest, then wriggle yourself back up the seat, the thin metal digging painfully into the bones of your feet. That was definitely not a relaxing flight. I'm told they've improved it since, though I haven't had a chance to try it. (I will in a couple of weeks, so maybe I'll write a paxrep then).

Virgin deserves an honourable mention - full marks, and maybe some bonus points too, for trying. But like an over-eager child they tried too hard and ended up spoiling the effect. Their seat is truly flat, rather than made up of a footrest, swab and back. The mechanism involved is truly impressive. Rather than just reclining until eventually it goes flat, like the others, the seat rears up like the Loch Ness Monster then suddenly collapses forwards, so you sleep on the reverse side of what you sit on. Unfortunately it's also rather hard, so not as comfortable as it ought to be. The biggest problem, though, is that the mechanism is so complex that you can't operate it yourself - you have to get the crew to do it. If, like me, you like to switch back and forth between prone and sitting - especially on day flights - you're out of luck. Once it's bedtime, it's bedtime until they serve breakfast. Though I did once have a fantastic flight from London to Tokyo which was so empty that I had a bed and a seat, on opposite sides of the aisle.

It's easy to tell whether your airline has horizontal or flat-but-not-horizontal seats, before you fly. Just go to seatguru.com and look at the pitch. If it's much less than 72 inches, the seat can't possibly be flat. (The exception is Singapore, who make the seat wider so your feet stick into your forward neighbour's armrest. Sadly, my only recent flights on Singapore were in First, so I can't comment on how well this works).

Meanwhile, the US airlines still haven't got to grips with even the flat seat, 10 years after BA pioneered it. Their websites will tell you they are "introducing it on selected routes", but I know nobody who has managed to fly on one of these selected routes. They don't include obvious ones like across the Pacific, that's for sure. I think maybe United is still stuck in the same timewarp as when its stewardesses were below pension age.

Usine à Gaz (Gasworks)

The French expression for something incomprehensbly complicated is "une usine à gaz" - meaning "a gasworks". Looking at the picture you can see why - rusty pipes going in every which direction, exposed equipment all over the place.

Years ago I had the dubious privilege of working for the most foul-mouthed Frenchman I've ever met. He was also completely irrational and irascibile. He would jump off the deep end over at the slightest - but utterly unpredictable - opportunity.

But he did have his amusing moments. Once he was explaining to me a conversation he had had with the CEO of Gaz de France, the French nationalised gas company, when he worked for IBM. Their IT infrastructure was a complete mess. Or, as he told me, "Je lui ai dit, votre systeme est une usine a gaz" - "your system is a gasworks".

To anyone else, anyone else at all, this would have been a perfectly reasonable remark. But to the head of the French gas company - he probably thinks a gasworks is a very fine thing. I don't know how he responded - if he knew the guy in question reasonably well, he probably rolled his eyes and ignored him, as we all did. Or maybe he said, "You're quite right - and what's wrong with a gasworks?".

Anyway I wish I'd been there.

Wirenuts!

If you've ever done any household electrical work in the US - like installing a new power outlet - you'll be familiar with wirenuts, those orange (usually) conical gizmos that are used to join two wires together. I'll cut straight to my question: why!? I've never come across something which not only works so badly, but has such potential to burn a house down.

The theory is simple enough - twist together two wires, then screw the wirenut onto the wires. Sharp edges engage with wires, holding them together, providing an insulated over, and ensuring a decent electrical connection. If only that was the reality.

Today I had to fix the pool light in my swimming pool, which just stopped working after being installed only a few months ago. When I opened up one of the electrical boxes, I found that the installer had used wirenuts that were way too small. The wires had come untwisted. Result, no light. Annoying, but not too serious - although for someone who couldn't fix it themselves, it would have involved a callout and a cost of $150 or more.

The former owner of my house made various minor changes to the electrical wiring, installing extra sockets, switches and the like. Of course he used wirenuts for all of them. He hadn't tightened them properly, resulting in poor contact, and eventually lots of heat which (with the greatest good luck) did not cause anything more serious - though I did find scorch marks. I've got rid of all his handiwork now, of course.

If you've only ever dealt with electricity in the US, I can hear you thinking, so what other way could you do it? In Europe, wirenuts are unknown. Instead, terminal strips are used - plastic strips holding little brass blocks with two screws in. You put a wire in each end of one place, and tighten the screw down making a reliable and permanent contact. They don't take up any more space, and they don't set fire to your house. Yet in the US they are completely unavailable - even major trade suppliers like Digikey don't have them.

I'm sure our former owner could have found a way to abuse these in an incendiary fashion. But still I don't understand why wirenuts have such a hold.

boost intrusive - wonderful!

I wrote a while ago about my disappointment with boost serialize, so it's only fair to redress the balance and say how delighted I am with boost intrusive. The STL list, map and set classes are extremely handy, but to those of us old enough to have cut our teeth on assembler, there's something rather unsatisfactory about all that behind-the-scenes memory allocation going on.

Back in the good old days (like in VMS or RSX), pretty much every data structure began with a pair of forward/backward pointers which could be used to link it into a list. All the key lists in the system (active processes, paged memory allocations, devices, ...) were held like this, with a listhead having the same structure. Insertion and removal are dead easy, and the relatively small size of the systems meant that linear searches weren't really a performance problem.

When I first started writing C++, I wrote a generic linked-list class like this, and an AVL tree class as well. I reused them for a while but then started using STL. But there was always this nagging feeling about those little memory blocks being manipulated behind the curtain.

Recently I've been working on something where memory usage is a problem. I thought those days were behind us, but this one (correlating information about network flows gathered through Netflow) was using up all the memory (8 gbytes) on the machine. Something had to be done. I put all the data structures on a diet, figured out how to get rid of stale information, and so on. But those 32 bytes being wasted for every list entry suddenly seemed important. I'd seen boost intrusive in passing but it had always looked a bit complicated. Time to read it seriously... and discover just how simple it all is.

Those old doubly-linked list structures from RSX... easy:

class my_structure : boost::intrusive::list_base_hook<>
{...

and then:

typedef boost::intrusive::list<my_structure> my_structure_list;

And that's it! You can have sets, too, with lots of nice facilities for lookups, and space/performance tradeoffs, and many other things. For example, I like to keep sets of things indexed by something in the structure. I have a much-reused class for doing this, built on std::set. But lookups require the construction of a whole data structure just to carry the key. I wrote a hack to make this reasonably efficient, but I don't like it.

They thought of that. The advanced lookup and insert functions just let you lookup using the key, without constructing an instance of the structure. The necessary C++ in the structure definition looks a little odd first time, but once you're used to it, you'd never do it any other way.

I'm gradually migrating all my recent code to use boost intrusive. It is just so much better than the STL classes.

Yeah for boost! More thoughts about it here.

The French and American Medical Systems

While I was in Paris the other day I needed to see a doctor. This reminded me why I like the French medical system so much, and why I dislike the US system so much.

A typical visit to the doctor in the US goes something like this.

1. Make an appointment. If you're lucky, you can generally get one the same week.
2. At the due time, drive to medical centre and find a parking space. The medical centres are the size of shopping malls with similar parking problems.
3. Enter the vast, overwhelming medical centre, and try to find the right reception desk, out of hundreds.
4. Give your file number to the receptionist. Don't bother with your name, the computer won't know what to do with it.
5. Wait. Probably 10-20 minutes.
6. Nurse #1 shows up and leads you through a maze of corridors to a tiny room, where she tells you wait.
7. Wait another 20 minutes.
8. Nurse #2 shows up and takes your blood pressure, temperature, height, inside leg measurement and a dozen other things utterly unrelated to the reason for your visit. You also explain your symptoms to the nurse. Heaven forfend that an actual doctor should have to waste his valuable time listening to a patient.
9. She leads you to another room, where you wait another 20 minutes.
10. Doctor shows up. He maybe asks a question or two, gives his diagnosis, and disappears. I don't believe I've ever spent more than 60 seconds in the room with the doctor. His time is just too precious to waste with patients!
    
11. Wait some more.
    
12. Nurse #n shows up with a prescription, if you're going to get one. Though actually my experience is that they will do all they can to avoid giving you any medication.
13. Finally you're done. Nurse #n leads you back through the maze to the reception desk, where you pay $50 or so "co-pay". Your insurer will pay most of the cost, which for a simple consultation like this is typically $200-300.

At the end of all this, you've spent an hour or more, seen a doctor for 30 seconds, and in all probability don't have a correct diagnosis or medication, though you are considerably poorer.

Here's how it typically works in France.

1. Make an appointment, usually the same day. In this particular case, I called, spoke to the doctor himself, who said, "Can you be here in ten minutes?".
   
2. Go to doctor's office, which is typically an apartment in a stylish building (in Paris) or a little rented office with parking just in front of the door, outside Paris. Not a terrifyingly huge medical centre.
   
3. The doctor opens the door for you - not a receptionist or some one from a vast nursing staff, but the actual doctor.
4. You sit in the doctor's office, explain your symptoms, and he asks lots of questions, examines you if necessary. He treats you like an intelligent person rather than as a numbered and rather stupid case file. You spend maybe 10-15 minutes with him.
5. He gives you a prescription - which he writes with his very own fountain pen. You pay, typically $30-50 for a consultation like this.
6. You're done. You've spent 15 minutes with the doctor, and little or no time waiting. You haven't had to deal with an army of nurses. You have a diagnosis and, if appropriate, suitable medication.

The US is a wonderful country, and I appreciate it more and more as I live there. But the medical system is something I'm afraid I'll never get used to or really even understand. What do doctors do when they're not seeing patients, i.e. 99% of the time? The whole system has one primary objective, which is to pour money into the insurance companies. Yet Americans - especially poor ones who are the most disadvantaged by the system - will fight to the death to prevent it changing, as the "Obamacare" debate and rhetoric shows.

Living Life Backwards on the Romance Car Express

Ever since the very first time I visited Japan, nearly three decades ago, I've wanted to travel on the Odakyu Line's Romance Car Express. They have these really cool trains with the driver's cab 747-like on the roof, so passengers in the first car have a panoramic view of the track ahead. And how could you resist the name? I've even contemplated travelling the length of the line just for the sake of it. On a recent trip I finally had a reason. We were travelling to Hakone, for a stay at a traditional Japanese inn (ryokan) and to do some autumn leaf spotting, and the Romance Car is the ideal way to get there, in fact the only direct service from Tokyo.

So we went out to Shinjuku station, supposedly the biggest station in the world if Tokyo hasn't overtaken it by now. Tickets are only available from machines, and while I suppose I could have figured it out in the end, I was very relieved when a helpful member of staff showed up and guided me through the whole thing, including buying the special all-inclusive Hakone Tour ticket. This kind of extraordinary personal service only happens in Japan. I chose our seats in the very end part of the Romance Car, and off we went to catch our train.

All passenger trains in Japan have the locomotive built in, from the Shinkansen down to the humblest one-car rural train. That means they're symmetrical - the front is the same as the back. And in the case of the Romance Car, that means there is an observation car at the back as well as at the front. And guess where our seats were... the booking system doesn't warn you that your view will be of the track receding behind you.

The view of course is exactly the same. But - which I hadn't expected - there's something very surreal about seeing the world zooming away backwards behind you. After a while it begins to feel like your whole life is running backwards. The journey only lasts about 90 minutes. If it lasted for several hours, I'm sure you'd start to get younger, eventually becoming the little boy in short trousers whose Dad used to hold him on the bridge parapet to watch the trains go by.

The most surprising thing is the level (grade) crossings, of which there are a great many. They open within seconds of the train passing - whereas they close much earlier, for safety. So as soon as you pass, the barriers open and - Japan being a crowded place - cars, bicycles and people flood across the road.

On the way back we bought regular tickets for a car in the middle of the train. We weren't sure we could take the Living Life Backwards experience again.