Cisco - and the end of an era

This week has seen the close of the final chapter of my involvement with Cisco. That may sound strange, considering that I left the company eight years ago. But on Monday, they suddenly closed down the team in the UK that I built 15 years ago. They were probably the best software team in the whole company - and if that sounds like an overblown claim, I'll justify it later. So it's ironic that this happens as Cisco says they need to invest massively in software and hire thousands of people. But these things happen in large companies.

I joined Cisco in 1999, to run their almost-new UK development team. It was about 25 people then, and within a year we'd increased it to nearly 100. Hiring in Silicon Valley was crazy and the company was desperate to get engineers on board. An anomaly in the way the European and US HR systems worked together (or didn't) meant I hired all those people with (as I recall) just three requisitions.

My own recruitment process had been strange. The guy who started the UK team had decided to retire, and was looking for a replacement. He got my name from a couple of former colleagues. The discussion went something like, "I need someone to take over the team, and X and Y said you're the right person for the job. When can you start?" It wasn't quite that simple, though, because a bunch of people at HQ in San Jose had to approve me as well. So I flew over and met them, and they all said "[UK manager] says we should hire you, what will it take to get you on board?" Well, one person out of the half a dozen did actually ask me a few interview-style questions. And I met the VP of the group - my boss's boss - but he'd been fired by the time I started, a few weeks later. As I discovered later, that was an indication of the way things work at Cisco.

I knew about half of my new team already. The network software development community in the UK is small, and several of us worked together at DEC's network team, that had dissipated a few years earlier. Hiring consisted of sitting down with the team every couple of weeks and running through the list of people from our past lives who we should call. Given the prestige associated with Cisco back then, most of the ones we wanted ended up joining us. Actually I think they all did.

Half the team was just outside London, about a mile north of Heathrow. From my office window I could see, and even hear, Concorde taking off every morning at 11. The other half was in Edinburgh, where they had an office as unlike the boring Cisco norm as could be - a converted whisky warehouse down by the former docks in Leith. Directly opposite was the imposing modern montrosity of the Scottish Office. Around the corner was the Malmaison Hotel, where I stayed every time I visited. The converted seaman's hostel housed not only a very classy and comfortable hotel (just writing about it brings to mind its comfortingly luxurious rooms with their huge, soft white-sheeted beds), but also an excellent restaurant.

Leith wasn't always a classy place - 50 years ago you definitely would not go there at night. Even now it hovers between being an 'in' place and hideous 1960s tower blocks. But if you need to forget, the Malt Whisky Society is a short walk from the office - and we had a corporate membership. You could sip cask-strength (50%) spirit at very reasonable prices. I do mean sip, at this strength it's almost undrinkable, and the next day's hangover can be a problem too.

For the first couple of years, Cisco was rich. Who could forget the weekend in Zermatt, staying at a Leading Hotel of the World, spouses invited too, the agenda carefully planned so that most of the day could be spent on the slopes. (And during the rare moments of work, my introduction to MPLS Multicast, a topic of immensely arcane complexity). For two years we held our office Christmas party at Gleneagles, a spectacular 1930s Scottish hotel famed for its golf course and its gothic opulence.

I said earlier that this was the best software team in Cisco. We'd carefully selected its members from the finest of all the people we'd worked with before. I was the 29th person there, and within a year we'd grown it to 80 people. Not everyone in a team that size can truly be an A+ player, whatever Google would have you believe. But we had way more than our fair share. In San Jose, Cisco was suffering terrible attrition. The week I joined, my boss was too busy to speak to me because two leading lights - famous names in the industry - had quit on the same day, to go to classic startups and make their fortune. But in the UK, Cisco was the best show in town. During the 15 years of the group's life, not a single one of the A+ players left - for the simple reason that there was nowhere else for them to go.

Starting a remote group is hard. Its founder, my predecessor, had scraped together whatever unloved, unwanted project remnants he could get. Yet within a couple of years we had flagship projects, like Cisco's IPv6 implementation and the high-speed packet forwarding path, CEF, simply because all the people in San Jose who knew these topics had left for greener pastures.

It wasn't all roses, though. Cisco IOS (never iOS!) was the heart of all its most successful and profitable products, and the maintenance burden was massive. At one point we calculated that we were maintaining about 750 distinct versions of the code. At the same time there were two huge (and as it turned out misguided and ultimately unsuccessful) infrastructure projects going on, which each required major disruptive change throughout the entire code base. At my very last all-hands meeting before I left Cisco, one developer summarised it neatly in a so-called question: "We spend nearly all our time fixing bugs and committing them into dozens of subtly different branches, and the rare actual development we do is forced upon us by projects that are nothing to do with us. We never get to do anything for the benefit of the stuff we actually own. Isn't something seriously wrong?" Well, I was their VP, I could hardly agree with them. Not openly anyway, though it was exactly what I thought too.

But I'm getting ahead of myself. About 18 months after I joined, there was a huge shakeup in senior management in San Jose (not an uncommon event). I was asked to move to the US and take on leadership of the whole of IOS (well, the most interesting parts anyway). It was the job of my dreams. It would have been impossible to refuse, even though it meant giving up our much-loved house in France, a house we'd never expected to move away from. (During all the time I was working for Cisco in the UK I was commuting every Monday and Friday from the south of France, staying in my flat in London during the week. Sometimes tough, but not without its advantages).

Our traumatic move from Valbonne to California took place in June 2001. For four years I did what I'd been told was wanted, injecting leadership into the group, encouraging good ideas, defending my team (nearly 500 people) from the bureaucratic madness of a giant company, trying to juggle the conflicting demands of a dozen hardware fiefdoms. We made some pretty cool stuff happen. Then it slowly dawned on me that actually this was the last thing anyone wanted me to do. What they really wanted the IOS team to do was fix bugs and keep its mouth shut. There were some serious egos in Cisco (still are, though mostly not the same ones) and they were the ones in charge. It didn't help that we'd had our own management turmoil. The protective, supportive and incredibly smart manager who'd brought me to the US had been replaced by a complete idiot whose preoccupation (and sole talent) was brown-nosing all he could, and trying to keep his job. The only good thing he ever did for Cisco - but it was really good - was to take a senior position at one of their major competitors, where he single-handedly demolished their entire core software base.

It was very far from an enjoyable time. My last year at Cisco was the most miserable of my professional life. Finally I took a job at a startup, and started to enjoy going to work again. At Cisco, I felt that the 20% of my job that was enjoyable made up for the 80% that wasn't, an endless succession of political meetings where nothing was ever decided. In the startup world, I enjoy my job 99% of the time.

When I moved to the US, the UK team remained part of my bigger group. I visited them three or four times a year. I also started a development team in Tokyo, but that's definitely another story.

It's now been longer since I left Cisco, than the time I spent working there. The bitterness I felt when I left has long faded. At DEC I had the good fortune to work, briefly, for one of the giants of the computer industry, Alan Kotok. He explained very well: "Everything that's not at corporate HQ is attached by a giant rubber band, held in place by a fragile little pin. At the slightest disturbance the pin will shake loose and the band will twang back to HQ." And that, sadly, is what happened to my UK team that had been so carefully built and nurtured. It was a huge shock to learn that they were being jettisoned, even though they could have been of such huge value to the company as it seeks its troubled and uncertain pathway in the future of networking.