Thoughts from upside down

Flying the T-6

2012-01-26T20:21:00.000-08:00

One of the many good things about Attitude Aviation is the variety of exotic aircraft that you can fly there. You can fly a Pitts and even, with enough time, patience and money, get to the point where they'll let you solo it or take friends for hair-raising aerobatic rides. They also have a Waco, which is a huge, lumbering - but aerobatic - biplane of pre-WW2 design, and a Marchetti, which is a nice touring airplane that just happens also to be fully aerobatic, including inverted. Not to mention the L-39 which allows you, at eye-watering expense, to fly a jet.

And recently, they finally got their T-6 (also known as a Texan, or in the UK a Harvard) back on the line. This is another WW2 design, classified as an "advanced trainer". In those days, a 19-year old would start flying a Stearman, solo after a handful of hours, and after getting to barely double figures would find himself behind the stick of a 550 HP monster, the size of a combat fighter. A few tens of hours in that and he'd be in a Spitfire, Hurricane or P-51, in combat for real. And maybe he'd come back, too... or maybe not.

Attitude originally had the T-6 on the line a few years ago. I flew it a couple of times, partly to prepare myself for flying the P-51 - but that's another story. Soon afterwards it had a landing mishap. The saga of the repair work and repaint that followed makes for a very long story - but finally, two years later, she showed up again at Livermore airport. I felt I needed to do something to celebrate getting my commercial pilot's license, so the timing was perfect.

The first thing about the T-6 - compared to the normal small planes we fly - is that it is seriously big. The low-mounted wing is at chest height, and the canopy stands about ten feet above the ground. Getting into it involves first clambering up on to the wing, then hoisting yourself into the cockpit, standing initially on the seat before you carefully lower yourself into place. There are two beams for your feet, with rudder pedals at the ends, and underneath that a huge void. Whatever you do, don't drop something in there! All the controls are massive. It feels truly indestructible. After all, they were made to take tremendous abuse from very inexperienced pilots.

It's also been said that they were specifically designed to embody every known defect of all the combat aircraft of the time. So no matter what you eventually found yourself flying, it would seem tame compared to the T-6. One thing to do the first time you fly a new type - especially if it's as different as this - is to get familiar with all the controls. This means sitting in the cockpit, manual in hand, making sure you know where everything is and what everything does. Maybe you even move some of the controls. But whatever you do, don't move the landing gear control. Because this aircraft has no protection against that - move it to the "retracted" position while on the ground, and it will obligingly tuck the wheels up under itself, doing a lot of expensive damage in the process.

Finally, you feel ready to go flying (with an instructor - this is not a plane that Attitude will let you solo). The first thing to do is start the engine. It's not like a car, this is not a trivial matter. First of all, the prop must be manually turned through a couple of revolutions. Big radial engines like this are notorious for getting puddles of oil in the lower cylinders. Oil being incompressible, if the engine starts like this, the cylinder will be wrecked. This would be highly embarrassing, as well as expensive.

That done, you can climb back into the cockpit, and do battle with one of the most frustrating pieces of equipment on the aircraft. Like most piston engines, this one needs priming - using a manual pump to squirt raw gasoline into the inlet manifold. But the pump is incredibly stiff, and once you're strapped in it's at arms-length. I confess to having been completely unable to unlock the pump, needing someone else to climb up and do it for me. Though not expensive, this is definitely embarrassing - a bit like not being able to release the old-fashioned parking brake on your first driving lesson.

Finally, four shots of primer, and the primer has been closed and locked again. The starter is, uniquely, operated by a pedal between the rudder pedals. As soon as the engine fires, you must get back smartish onto the pedals, to be sure the brakes are fully applied. At the same time, you have to get the engine actually running. Radial engines start one cylinder at a time, or so it seems. The first kick must be carefully nurtured, the throttle advanced very slowly, as (hopefully) another cylinder starts to fire, and another, to the accompaniment of loud bangs and the occasional flame from the exhaust. Watching someone else start a radial engine (and listening to it!) is a delightful experience. Doing it yourself is fraught, with the knowledge that if you fail you'll have to start over, including the dreaded primer.

Finally the engine is running smoothly, all cylinders firing. Now it's time to taxi. As you release the brakes and start to move, you realise just how huge this thing is. Luckily the brakes are conventional - not heel brakes, or the strange composite arrangement with a bicycle brake lever on the stick like on the L-39. But it can't be too easy. In a taildragger you want to keep the stick fully back all the time when on the ground (let's ignore quartering tailwinds for now). That locks the tailwheel within a narrow range of motion. So if you want to make a tight turn, for example to reverse direction in the runup area, you must release the tailwheel lock by moving the stick forward. But for takeoff, it's vital to have it locked. And of course sometimes it doesn't just lock itself - you have to wiggle the aircraft to and fro, until everything is right.

So, you've been through the (long) pre-flight checklist, got the oil nice and warm, taxied into position, and you're ready to take off. This is the frightening bit - first because you have no idea how it will behave when you apply power, and second because once you've taken off, you will, sooner or later, have to land again. (A very experienced Pitts pilot once told me that it took him several hundred flights before his first thought on becoming airborne stopped being, "Oh, sh*t, now I have to land it!").

The T-6 is actually fairly docile as it rolls down the runway. It doesn't have any vicious habits. It does take some heavy footwork to keep it rolling straight, especially as you bring the nose up. And, unlike the Pitts, it rolls a long way - although it has 550 HP, it weighs a lot too. Finally it reaches rotation speed, and with a pull on the stick you're airborne.

We flew over to our usual aerobatic practise area, which gave some time to get a feel for the aircraft. The stick is quite heavy but easy to get used to. It doesn't have a lot of adverse yaw (unlike the Citabria, for example) so only modest rudder inputs are required. And it makes that wonderful radial noise as you putter along at a modest power setting and 130 mph or so. You just know that everywhere you go, people are looking up and thinking, "What's that?"

We did a few of the usual exercises with a new type, starting with turns and then steep turns. They're easy to fly, the forces, while heavy, are consistent and well balanced. Then we went on to simple aerobatics - loops and rolls. These, too, went easily enough. It takes a lot of force on the rudder to keep a roll straight, and because the roll rate is not very high, the loop finishes with the nose pointing distinctly downwards. Loops go nicely enough, diving for 180 mph then pulling round at 3.5G or so.

Once in flight, the T-6 flies fairly unremarkably. You forget its size, since it doesn't really matter up there. The panel is typical of its era - there's no six-pack, just instruments and switches dotted about at random. The landing light switches, for example, are on the left of the main panel - while all other lights are on their own panel on the right side of the cockpit. (I confess that I didn't find them in flight when I wanted them). The fuel gauges are even worse. They're purely mechanical, with sight gauges down in the void under the floor on either side of the cockpit. Before strapping in I did manage to locate them, but I found reading them in flight to be impossible. The gear lock confirmation is similar - you have to look through two tiny windows in the upper surface of the wing, to see the yellow lock pins. I said I could see them, and at the time I convinced myself that I did, but to be honest it's pretty questionable.

We did a couple of stalls, which were also unremarkable. Then it was time to go and land - with a wonderful view of Mount Diablo in the setting sun. I had an unpleasant experience landing a much smaller vintage taildragger a couple of years ago, so I get a bit apprehensive at this stage. With the T-6, the plan was to wheel-land it. Smaller taildraggers are normally landed on all three wheels at (much) the same time, but with bigger ones it's common to land with the fuselage level, just on the main gear. This means that the wings are still flying and will happily bounce back into the air if the tail drops, increasing the angle of attack. So you have to "stick" the airplane by pushing the stick forward at touchdown. Timing and finesse are everything - too late, and you bounce. Too early, and you thunk painfully hard into the runway - and then bounce. Too much, and you can ding the prop as the nose drops towards the runway.

As it turns out, the T-6 is a pleasure to land. Maintaining a steady 95 mph on final, just gently lowering the main gear onto runway then pushing the stick forward give a gentle touchdown. It's much easier than a wheel-landing in a Citabria!

The flaps on the T-6 are odd in a couple of ways. First, they're "split flaps" - all of the action happens under the trailing edges. From the cockpit the wings look solid, and when you move the flap handle, nothing visibly moves. All the action is underneath. In consequence, it's easy to forget to retract them after landing. And if you forget them until after shutdown, you have a problem. They are operated by an engine-driven hydraulic system, and once the engine has stopped, the only way to retract them is with a manual pump somewhere in the bowels of the aircraft. Guess how I know.

We did a couple of touch-and-goes, which went rather well, then we returned to base. My one real landing didn't go so well - I dropped in hard enough to elicit comment, rather than the smooth touchdown I'd managed earlier. And I'm afraid to say I was a bit heavy-footed on the rudder. I think I've got so used to the Pitts, which is very controllable and reactive on the rollout and will forgive a surprising amount of clumsiness, that the much more leisurely response of the T-6 caught me unprepared.

So, finally, it was back to Attitude, and engine shutdown. The sun was just setting as we landed, so the usual post-flight photo opportunity gave some nice sunset-type shots.

And some time, I really will have to spend some time just getting comfortable with landings. Apart from that the T-6 is a pussy cat, albeit a very big, heavy (and expensive) one.

Solar Power - so far, a mixed success

2012-01-24T15:52:00.000-08:00

One of the unexpected benefits of losing my job was that my wife, who had taken over paying the household bills, took a close look at the power bills and had conniptions. She decided right there to get a solar power system installed.

This was quite an undertaking - the roof was getting old so we decided to have that redone at the same time, which in turn involved overhauling the solar heating panels for the pool which were there. The couple of weeks before Christmas were extremely noisy and dirty as big, strong men jumped up and down on the roof.

Now, the installation is complete - well, we've paid all the bills anyway. PG&E came by yesterday and installed the new meter, so now we can sell them electricity, That's good. But sadly, it isn't actually working completely, and it's not clear whether the problems can be fixed. That's not so good.

The system consists of 26 separate panels mounted at a slight angle on the roof, facing slightly east of south. Why they didn't make them face true south, I have no idea. We went for the highest-tech system on offer, using high grade panels from SunPower, each with its own inverter. This allows each panel to function separately and optimise its use of the available light, rather than wiring them in series so the output is always set by the weakest. This also allows us to get accurate real-time reporting of the power generation. At least, it should.

Our first problem is some trees on a neighboring property. In the winter (i.e. now), they start to shade the panels from about 11.30. From the pretty graphs the system produces, I can see that we're losing about a third of the production due to those trees. Generally I like trees, and I don't particularly want them cut down, even if the neighbor would agree. What is really annoying, though, is that if the panels had been placed differently on the roof, the shading would not have been as bad. This is something that the survey should have caught, and didn't. Once the sun is higher in the sky, I guess from some time in March, this won't be a problem. But still, it is probably going to lose us 10% or so of the total annual production.

The second problem is with the "fancy technology" part of the system - specifically, the Envoy box that monitors the panels and sends the data off to a company called Enphase. This is a Kleenex-box sized gadget, which communicates with the panels over the power line. This makes perfect sense rather than running a second set of cables. Or at least, it would make sense if it worked.

Initially this box was in my office, next to the router which it also needs to be connected to - by a cable, since for some reason they didn't use WiFi. That didn't work because the data-over-power couldn't get a good enough signal. So they moved it to the garage, close to the breaker box. That meant installing a second, completely separate, powerline bridge to carry the data.

I've always been suspicious about data-over-power. The idea of mixing all those tiny, sensitive little data bits with all the noise, inductive loads and general mess of electric power just doesn't seem like it should work - try connecting an oscilloscope to your power line and you'll see what I mean. I suppose I should feel reassured that it doesn't work. It's just that I'd rather it did.

Two of the panels simply don't show up on the Envoy. When I queried this, it turns out that the data signal to the panels isn't strong enough. "You'll have to move it to an outlet closer to the panel," the installer said. Well, but there isn't one. I tried anyway just moving it to another outlet. (The thing takes up two outlets, incidentally, and must not use any kind of an adapter or extension cord). But this time, the powerline bridge didn't work. And given that, there was no way to find out whether the communication to the panels was any better or not.

For now, we're stuck. The controller can only see 25 out of the 26 panels which are installed, at great expense, on the roof. It isn't clear to me whether the 26th panel is actually able to generate any power under these circumstances - since it's the one that receives the worst shade, maybe it doesn't matter much!

Even with the system as it is, there's a certain satisfaction to paying less to PG&E each month. But it would be even better if it worked properly, considering what we paid for it.

Instrument Flying by Helicopter

2012-01-10T14:17:00.000-08:00

My latest flying training project is to work towards my commercial helicopter license (CPL-H). Having, finally, got my airplane commercial a couple of months ago, it seems like an obvious thing to do. So my plan over the next year or so is to gradually "tick all the boxes" for the experience requirements for the CPL-H, as described in FAR 61.129(c).

One of these is to get 5 hours of "actual or simulated instrument flying". Since our Robinson 44 is not allowed to fly in actual conditions (i.e. in clouds), this means flying "under the hood", using a gadget which restricts the pilot's view to just the instruments. This is how most instrument training and currency is done, even in airplanes, since you can't really count on clouds to be there when you need them, especially in our part of the world. Yesterday I completed my second helicopter flight under the hood, amounting to a magnificent total of 3.4 hours so far.

In a plane, with its high glareshield, there are lots of effective ways to limit the pilot's view. I use something called "Foggles" which are like industrial eye protection goggles with most of the lens covered in a translucent film. Only the bottom part, corresponding roughly to the reading part of bifocal glasses, is left clear. As long as the pilot isn't actually trying to cheat, these do the job very well. In fact, it's surprisingly easy not to cheat, assuming your goal is to be trained rather than just to pass a checkride. (In the latter case, it isn't really worth trying to cheat because the examiner will notice immediately). I remember one flight, on my own in actual cloudy conditions, where I popped out of the cloud and it took me a while even to realise that I didn't have to rely on instruments.

In the helicopter, it's harder to do an adequate job of restricting vision, because the visibility is so much better. The only thing that works - and then not very well - is the decidedly steampunk Francis hood, looking like something a submarine commander should be wearing. Even so it's impossible not to see ground through the lower part of the canopy - you just have to try not to notice it.

Flying the helicopter under the hood is much harder than a plane. For a start, you can't take your hands off the controls. A plane will cheerfully fly for tens of seconds, hands off. You sometimes see a plane referred to as a good "instrument platform", meaning that it is naturally stable. My own Cessna TR182 falls into this category. The heli, on the other hand, always requires a hand on the cyclic (the up/down/sideways control corresponding to the stick or yoke in a plane). Normally it's your right hand, but you can switch hands if you need to, for example to twiddle the Garmin 430 in the usual R44 instalation. Even that is tricky - it's difficult to fly as smoothly with the "wrong" hand (I guess left-handed helicopter pilots may find the opposite, but probably not - it's all a question of habit). And you have to be constantly prepared to get a hand on the collective really fast if the engine stops, to enter autorotation in the couple of seconds you have before the aircraft simply drops out of the sky.

Flying, and especially instrument flying, involves a lot of fiddling round with bits of paper - charts, approach plates and so on. It is very difficult to refold a flimsy IFR en-route chart with just one hand. For this reason, IFR-approved helicopters have either a crew of two, or an autopilot (or both of course). But helicopter autopilots are eye-wateringly expensive - little change out of $100,000 - and anyway not available for training aircraft like the R44. So, even during an instrument checkride, the instructor (or examiner) can be used as an "autopilot".

I've done a lot of airplane instrument flying, both under the hood and in real clouds. Much of it has been partial panel, i.e. without a functioning attitude indicator, which generally mysteriously "fails" at the very start of my instrument currency flights. So much so, that I've got used to flying without using it very much - the turn coordinator and the altimeter work just as well. That absolutely does not work in the heli, at least not at first. It is so much more sensitive that the AI absolutely has to be the primary reference all the time, just like it says in the IFR manuals. It took me a little while at the start of my first flight under the hood to realise this. Once I did, my flying became a lot more stable. In an airplane, quite a casual instrument scan works fine. But in the heli you really have to be moving very quickly between all the instruments. It takes only a small distraction to suddenly find yourself in a 30 degree bank or a couple of hundred feet off altitude. This is especially a problem when tuning radios or fiddling with the GPS - you have to scan back and forth between the AI and whatever you're twiddling, making the latter just an extra part of the scan. It's hard.

Once cruise flight is mastered, or at least under control, the next thing to tackle is approaches and holds. In principle an approach is just flying, but there's a lot more to keep track of - altitudes and headings change, and there's radio work too. A general problem with instrument flying is what happens when you get overloaded. You can go very quickly from feeling comfortable to being on the edge of complete panic, not because anything has gone badly wrong but just because there is so much to keep track of. Then small things start to go wrong - altitude and heading deviations, maybe a bit of an extreme attitude - and that piles on the workload too. It takes self-discipline to take a few deep breaths and get back to focusing on basic flying, get straight and level again, go around if you have to. For some reason the hood adds to stress too - flying in actual clouds, without the hood, seems less daunting. This may be just a very basic animal reaction to not having complete vision.

The handful of approaches I've flown have worked out pretty well, all things considered. My first helicopter ILS was the long, long descent into Moffett (KNUQ). Although it's charted as a series of step-down fixes, in practice you can fly the whole thing down the glideslope, all the way from HOOKS - 20 miles out and 5500 feet above the field. It's probably the longest ILS descent in the world, 15 minutes or so of following the needles. Fixed-wing, I flew it once with everything covered up except the VOR head - everything done by making tiny corrective inputs as the needles start to drift. It's a great exercise, but I'm certainly not ready to do it in the heli just yet!

What next? I only need one more flight like this to "tick the box" for my CPL-H. A helicopter instrument rating is pretty useless, since it's unlikely that I'll ever fly one that's equipped for IFR. But there again, it might be fun to do after the CPL-H - amazingly, it only requires another 10 hours of hood time to meet the legal requirements.

Algorithm Design: Efficient LDPC Encoding (Part 4: Optimization)

2011-12-11T16:23:00.000-08:00

In Part 3, I described the implementation of the algorithms for reducing an LDPC code to an encodable form. At that point, all the algorithms were the most efficient possible. The only remaining performance gains would come from improving the code. This is rarely worth spending too much time on, but in this case the overall performance is completely dominated by two inner loops. One iterates through a sparse representation of a row, adding it to a dense row. The other iterates along the elements of the dense representation, adding them. Halving the time spent in both of these loops - just a few instructions each - will halve the execution time of the whole algorithm. So it's worth taking a close look.

Let's start with the add-sparse-to-dense loop. The original code used conventional STL iterators to scan through the elements of the sparse row, then for each element, converted it to the offset-and-mask combination for the particular bit number, and applied it using an xor operation. It's the obvious way. But each sparse row is added to a dense row tens of thousands of times, so it's worth considering whether any part of this operation can be amortized.

The final solution was to pre-calculate a vector containing the offset-and-mask for each entry. The latter was actually represented as a class, called "bitref". In the source code, this results in a vector<bitref>, which is iterated through in the usual way. The compiler is nevertheless clever enough to inline all this and reduce the inner loop to just four machine instructions: two to extract the offset and mask, one to perform the xor operation, and one to advance to the next entry. Not bad. Performance was improved substantially, reducing the time for phase 2 of the algorithm by a factor of about three.

There remains the overhead of the loop. Given the tiny size of the content, the two instructions of loop overhead, and their impact on pipelining, are worth worrying about. In assembler, the obvious way to unroll the loop would be to lay out sequential instructions corresponding to its maximum size, then jump into these at the appropriate point corresponding to the loop size (i.e. the number of entries in the vector). This is difficult to do in C++ though.

In the end I came up with something similar, but with a distinct unrolled loop for each possible size. This was done as my first tiny adventure in C++ template metaprogramming, and is described here. The compiler is smart enough to translate the switch statement based on vector size into an indexed jump, which then executes the entire loop in a straight line. That gave me about another 5% improvement.

Having got the inner loop as tight as possible, it was time to think about the next layer of the loop. Gcc does a good job of inlining functions when it's the right thing to do, but examination of the assembler output (-S option) showed that it was not inlining a couple of critical functions here. I played around with the compiler parameters that control inlining for a while and things got a little better, but I could just not convince it to inline one critical function. Of course the "nuclear option" of making it a macro always exists, but I really wanted to avoid that. I tried the "flatten" function attribute for the outer loop, which tells the compiler to inline absolutely everything, but after the compiler had run for half an hour or so I stopped it. I think it got put off by all the calls to boost::format that I use in my debug log macros.

Eventually, I found a minor rearrangement of functions that got everything inlined. That gave me another 5% or so performance improvement.

That dealt with the inner loop of phase 2, adding sparse rows to dense rows. In phase 3, the inner loop is adding dense rows to dense rows. Unrolling this loop was easier since it is always over the whole length of a dense row - over 64K bits, or 2K operations. There's nothing to be gained by completely unrolling such a loop. Instead I changed the code to do it in "gulps" of 16 entries at a time, then used a normal loop to deal with the remainder at the end. I also rearranged things here so that the call to the inner loop was fully inlined.

And that is about as far as things can be taken. The original C++ code took about 400 seconds for a column-weight 3, 32K data bit code. The final code takes under 7 seconds. I never ran a column-weight 5 code to completion with the original code - it would certainly have taken thousands of seconds, maybe much more. But now, it runs in about 45 seconds.

Of course there's a price to pay for all this. One of the first principles of writing maintainable systems is never to keep the same information in more than one way. This code violates that all over the place - for example, the sparse and dense representations of rows. But without this kind of approach, the code would be unusable anyway, so its maintainability wouldn't matter much. It has certainly been one of the most interesting bits of programming I've undertaken in a long timer.

Algorithm Design: Efficient LDPC Encoding (Part 3: Implementation)

2011-12-08T10:15:00.000-08:00

In Part 2 I described the algorithms that need to be implemented in order to transform the base matrix of an LDPC code into a reduced form that can be used by a practical encoder. As I mentioned in Part 1, we originally built a very straightforward Python implementation, where the matrix was represented literally as a bunch of rows of 0s (mostly) and 1s (rarely). Extrapolating from its performance with toy-sized codes, it would have taken months or years to reduce a life-sized (>32K bits) code. We needed something a bit faster, like a few seconds, so I set out on a C++ implementation. C++ is naturally 20-50 times faster than Python, but it would take a lot more than that.

The first, obvious, step was to change the representation to a sparse array, where only the 1 values are held explicitly. The Python code spent most of its searching arrays of 0s trying to find the occasional 1, adding a further O(n) to its execution time.

During the first phase, all the work consists of swapping rows and columns. To support this efficiently, the sparse array consists of "bitnodes" representing a 1. They are linked into lists both for the row and for the column, and contain pointers back to each of these. This means that when rows are swapped, the columns get to find out about it with no further work, and vice versa. The implementation makes extensive use of the Boost intrusive library, about which I've already eulogized. In the original implementation, the row and column lists were held in order, though I ended up rethinking this later. Here is the structure of a bitnode:

   class bitnode
    {
    private:
        typedef bi::set_member_hook<bi::link_mode<bi::auto_unlink> > row_hook_t;
        typedef bi::list_member_hook<bi::link_mode<bi::auto_unlink> > col_hook_t;
        row_hook_t row_hook;
        col_hook_t col_hook;
        row_t *my_row;
        col_t *my_col;
    public:
        // member functions follow
   .
     .
};

Note the use of the intrusive member hooks, which allow the same structure to be linked into several lists (or sets). The backpointers to the row and column allow the row and column numbers to be tracked as they are swapped, which would not be the case if they were held explicitly.

This basic implementation worked well for codes with a column weight of 3, taking about 300 seconds to transform a 32K bit code. For a column weight of 5, though, which results in a much larger gap, it was unusable.

A little instrumentation showed that all the time was spent adding rows together. In the set-based implementation of sparse rows, every addition involved either the creation or the deletion of a node in a tree, a relatively expensive operation. The solution was to switch to a dense representation for the gap rows only. So, just before starting phase 2 (elimination of the ones in the F region of the matrix), the gap rows are converted to a dense representation, with one bit per possible position. This is simple enough in theory but took a lot of reworking of other structures, such as the columns. It was worth it, though: the time dropped to around 60 seconds for the column weight 3 codes, and to around 300 seconds for the column weight 5 ones.

Adding a sparse row to a dense row means walking the bitnodes in the sparse row and xor'ing the corresponding bit. Adding a dense row is just a tight loop xor'ing the 32-bit words together, an O(n) operation. These two inner loops are the key to performance - we'll come back to them later.

As always, when you speed up one part, you find another bottleneck. In this case it was phase 1 again. The best column to swap when making the diagonal was selected by simply scanning them linearly, which is obviously expensive. The solution was to keep a constantly-sorted list of the best one - actually a priority queue, implemented yet again as a boost intrusive set. However this changes constantly - when a row has been incorporated into the lower triangle, the columns it contains now have one less 1 in the region of interest. Increasing the gap also affects it. Fortunately, the row structure makes it easy to update just the columns that are directly affected, which is O(b), and then to correct their position in the list. Hence the total operation each time is O(b log(n)) which is much better than than O(n) as previously.

For a column weight of 3, this made phase 1 practically disappear as a performance concern, as I expected. But for a column weight of 5, it was still taking the majority of the time - which I didn't expect. Further analysis showed that keeping the columns in order was very expensive. Every time a row was moved to the gap, every column had to be re-sorted. On further thought, there is only one time when it helps for a column to be sorted, which is when it is being processed as the diagonal element. So just sorting it there, once per column, would work just as well and remove an O(n) element from the algorithm With this change, phase 1 moved down into the noise - for a column weight 3 code, it is about 4% of the total time.

At this point there are no further fundamental improvements to be made - the order of work to be done for each phase cannot be reduced. Further improvement can only come by coding optimizations, which will be discussed in Part 4.

Algorithm Design: Efficient LDPC Encoding (Part 2: Algorithms)

2011-12-06T15:52:00.000-08:00

In Part 1, I described the problem we are trying to solve, taking a sparse matrix and solving the corresponding system of simultaneous equations (around 33000 of them) so that we can build an efficient hardware encoder for Low Density Parity Check (LDPC) codes.

Efficient encoding requires that the original sparse matrix be transformed such that all the encoder has to do is calculate a number of parity checks. Most of these are very sparse, so they can use shared hardware. A small proportion (about 3% in a typical code) are dense, i.e. they have about the same number as 1s and 0s, and so cannot share hardware.

The resulting transformed matrix is called the "reduced" matrix, and when it is complete it has the following form:

+------------------------+--------+------------------------+
| D | E | F |
+------------------------+--------+------------------------+
| A | B | C |
+------------------------+--------+------------------------+

Rows in the D/E/F part are called the "gap" in the literature. Initially the reduced matrix is set to be identical to the base matrix, and the gap is empty. In a matrix representing a system of simultaneous equations, such as this, rows can be swapped without changing the meaning, as can columns. Also, rows can be added together (although columns cannot be). In binary addition, 1+1=0. We use these facts to rearrange the matrix into reduced form, by the following steps.

1. Transform part C into "lower triangular" form (LTF), in which everything above the main diagonal is zero. This can be done by swapping rows and columns. At each step, we look for a column that has just a single entry above the current diagonal row, then swap it with the current diagonal column. Finding a suitable column is the key to performance at this step.

2. Sometimes, we can't find such a column. This is how the gap gets created. We choose a column with the smallest number of such entries and swap that. Then we exchange rows so that the populated rows move into the gap area.

3. When this part is finished, C is in lower triangular form, but the gap is not. The next task is to complete the task for the gap, by emptying F altogether and getting E into lower triangular form. So far, all rows are still sparse, since no row or column has been changed apart from ordering.

4. For each row in F, we eliminate all bits using Gaussian elimination. Starting with the rightmost one bit, we add the corresponding row from C which has this as its rightmost bit (i.e. on the diagonal). We repeat this, moving leftward, until the F part of each row has been emptied. In the process, the rest of the row becomes dense, with on average as many 1s as 0s.

5. We now have F empty, and we need to transform E into lower triangular form. We do this by Gaussian elimination again, this time using rows from the gap. We start from the bottom and work up, creating the diagonal as we go, so that we don't put back bits that we have already eliminated.

6. Now we're done. E and C between them have a neat diagonal line with nothing above it. F is empty. A and B are still sparse, but D and the lower triangle of E are dense. All the bits in columns in A and D are data bits. The check bits, in the remainder of the matrix are generated from these.

Let's take a look at the performance of each of these steps. First we need to define some terms:

b: the number of 1 bits in a single row in the base matrix. This is small, and independent of the size of the code. It's also referred to as the row weight. We refer to the number of 1s in a single column as the column weight.

g: the number of rows in the gap region (D/E/F). Although this is much smaller than the number of data bits, it is directly linked to it. For a column weight of 3, it is about 3.3% of it. Hence anything which is O(g) is also O(n), though with a much smaller actual value.

n: the total number of rows.

The task falls into three phases:

-- Phase 1: rearrangement of rows and columns to create the C region. This has to be done once for each row (less the gap rows), and each time, we have to select the best available row. If we simply scan the rows looking for the best one, this will be O(n), making the overall task O(n²). We'll explain later how this can be made O(n log(n)). In addition we have to create the gap. Rippling a row up into the gap is O(n), and has to be done for each gap row, so the total task is O(n*g). In principle this is O(n²), but because g is so much smaller than n, with suitable design it can be kept small, comparable with the O(n log(n)) time of the row rearrangement.

-- Phase 2: eliminating all the 1s in the F region. There are g rows to deal with, and once the process starts they quickly become dense. Hence O(n*g) row additions are required, where one (the gap row) is dense, and the other, coming from the C region, is sparse. The amount of work per addition is O(b), making the whole task O(n*g*b).

-- Phase 3: eliminating the upper half of the triangle in the E region. There are g rows, and O(g) bits to be eliminated in each row, so there are O(g²) additions. Since these involve adding dense rows to each other, the amount of work per addition is O(n), making the whole task O(n*g²) - or in other words, O(n³). For small codes, this phase is dominated by phase 2, but as the code size increases it starts to dominate the total time. This is especially true if larger row or column weights are used, since the gap becomes proportionately larger (about 10% of the total rows for a column weight of 5).

These are the fundamental limits of the algorithm - no matter how clever the design, the three phases will have complexity O(n log(n)), O(n*g*b) and O(n*g²) respectively. The trick of a good implementation is to achieve these limits, and to minimize the actual values in each case. Part 3 discusses how this was done.

Algorithm Design: Efficient LDPC Encoding (Part 1: Background)

2011-12-06T14:17:00.000-08:00

I've been working lately on a system design which requires, among other things, highly effective and efficient error-correcting codes (ECC). We've decided to use a Low Density Parity Check (LDPC) code. These are currently considered to be the best "soft" ECCs, i.e. where there is information about the reliability of each received bit as well as its putative value. The story behind LDPCs is interesting: they were invented by Robert Gallager in his PhD thesis in 1960, but they were way beyond contemporary computing power. It didn't help that when he wrote the definitive textbook on ECCs in 1966, he didn't mention them! So they languished, forgotten, until a decade ago. By then TurboCodes had been independently invented. They also provided a means for "near Shannon limit coding", i.e. extracting as much data from a noisy signal as theoretically possible.

LDPCs have two properties which led to the problem I needed to solve. First, there is no formula that provides the best code for a given set of constraints (block size and code rate). You can use the same general scheme to build ten different codes, the details being decided by a random number generator, and some will be significantly better than others. That means that to find the code you want to use in practice, you need to generate a whole bunch of them and try them out over a large number of messages and error densities.

That leads to the second problem. An LDPC starts out as a very sparse matrix, describing a large number of parity checks each of which covers a small number of bits - hence the name. We want to have 32768 bits of user data, and a reasonable configuration is to have each bit covered by three checks. If we use a half-rate code (same number of data bits and check bits) then each check covers six bits. So we have a matrix where each row is 64K bits long and has just six 1 bits.

The matrix doesn't say anything about which bits are data and which are check bits, only that a valid codeword has to satisfy all the checks. So given 32K data bits, the way to generate the corresponding 32K check bits is to solve the 32K simultaneous equations that the sparse matrix implicitly describes. Easy!

Well, no, not easy at all. The practical use of LDPCs requires a transformation of the matrix into something that normal hardware or software can encode in a linear and reasonable amount of time. Solving the equations directly is an O(n³) problem, i.e. the time required increases with the cube of the number of unknowns. So we have to some preprocessing on the matrix to get it into a form that the hardware can work with. There's an excellent paper by Qi and Goertz describing how to go about this. The algorithm it describes is, not surprisingly, also O(n³). This needs to be run for every trial code, and we would like to try hundreds of them.

Our first attempt at coding the algorithm was written in Python, using the obvious data representation, i.e. a big matrix containing mostly 0s and a few 1s. It was written so we could understand the algorithms and piece together a complete system, rather than for performance. On a "toy" code of a few hundred bits, it took a couple of minutes to run. On slightly larger codes - nowhere near the size we need for our system - it took most of the day. By extrapolation, to generate a life-size code would have taken months or years.

Clearly, we needed an implementation more focused on performance - not just code optimization, but selecting algorithms to minimize the time at step of the algorithm. And that is where it begins to get interesting. More on that in Part 2.

IRP rediscovered - first steps in Template Metaprogramming

2011-12-01T05:58:00.000-08:00

One of the nice things about the PDP-11 assembler was its powerful macro features. Not only could you do basic text substitution, you could create loops using the REPT directive, for a fixed number of iterations, or IRP, which iterated over a list of arguments. It was especially good for setting up data structures, which nowadays would be viewed as a rather crude application specific language (ASL). (Before I start getting hate-mail, yes, I know this was originally from the PDP-10).

For whatever reason, the designers of C eschewed all this and just went for simple text substitution. Every now and then I have a bout of nostalgia for the PDP-11 assembler, especially when trying to build elaborate descriptive data structures. Of course there's always M4 but the learning curve is huge. Actually I'm a long way down the forgetting curve for M4, a long while back I built a very elaborate set of macros for tracking register usage and many other things for some MIPS assembler that I wrote. But it was a long time ago.

Then just the other day I really needed the old REPT directive. I've been working on a very interesting algorithm design problem, for reducing low-density parity check codes (LDPC) to a form where they can be encoded by practical hardware. The innermost loops of this algorithm are extremely performance critical - by nature this is an O(n^3) problem (i.e. the complexity increases with the cube of the size of the code). For a realistic sized code of say 32K data bits, the innermost part of the algorithm gets executed several billion times. Normally I'm content to let the compiler worry about the details of code optimization - today's compilers (gcc and MSVC) do a wonderful job. But in this case, saving a single instruction could cut seconds off the execution time, so it was worth digging a bit deeper.

Of course the first part of optimization is to use the right algorithms and data structures. I'd already done all that, cutting the execution time by a factor of thousands compared to our initial, simple implementation. Now I was looking to shave off another factor of two by paying attention to the details.

One such detail was to unfold the critical inner loops, replacing them by linear sequences of instructions with no tests or jumps. After some careful crafting of data structures, the loops were extremely tight, less than ten instructions. One of the loops has a large repeat count, so it was easy just to do it in gulps of 16 at a time. At that level the loop overhead is negligible, and when the remaining number is less than 16, the last few can be done one at a time.

The other loop was trickier though. The number of iterations is small, in the range 6-20, so the whole loop has to be done at once. A quick experiment showed that gcc implements a switch statement using a jump table, so it would be quick to dispatch to the right unrolled loop. But how to generate the code without tediously repeating the same statements over and over?

That was when I thought of using metaprogramming, i.e. programs that run at compile time rather than at execution. The idea is to declare a template class, parameterized by an integer that tells it how many instances you want. The resulting code looks like this:

template<int I> struct repeat
{
    void apply(vector<operation> &ops, vector<operand> &v)
    {
        ops[I-1].do(v);
        repeat<I-1>().apply(br, v);
    }
};

template<> void repeat<0>::apply(vector<operation> &ops, vector<operand> &v) { };

The details of what's being done aren't too important here. "op" is a vector of operations, which says what to do and which operand vector element to apply it to. We want to make sure that each operation in the vector is applied.

The "apply" function first does the operation corresponding to its parameter, then recursively invokes the class with a parameter of one less. But how to get the recursion to stop? This is where the specialized function declaration comes in. The compiler will always choose an explicit specialization over the generic definition, so when the parameter reaches zero, this empty function is selected and the recursion stops.

The code that uses the class looks like this:

switch (ops.size()) {
case 6:
    repeat<6>().apply(ops,v);
    break;
.
.
.
case 20:
    repeat<20>().apply(ops,v);
    break;
default:
    for (auto opi=ops.begin(); opi!=ops.end(); ++opi) {
         opi->do(v);
    }
    break;
}

I happen to know that the vector size will normally be in the range 6-20. The default is there so the code will work, albeit less efficiently, if it isn't. If you really had no idea of the limits, you would first deal with chunks of say 16 at a time, then finish off the remainder using the above technique.

It looks as though this will produce horrific code, with the recursion and everything else. If you compile without optimization, for debugging, indeed it does, with a deep nest of function calls, each with its own call, entry and exit sequences. But if you turn on full optimization, gcc produces exacly what you would if you hand coded - just the exact set of instructions required to implement each iteration of the loop. (I imagine MSVC would too, though I haven't tried it). You'll notice that the "repeat" object is instantiated, but since it has no content, this doesn't actually do anything.

To the real experts in metaprogramming (all dozen of them), this is child's play. But for the casual visitor to the topic, like myself, it's a neat technique that can save a lot of tedious and error-prone repitition. As I expected, unrolling this innermost of inner loops saved about 5% of the execution time, which is a useful contribution to my overall target of 50%.

Favourite Restaurants #4: Kaiten Sushi, Shinbashi, Japan

2011-12-01T01:21:00.000-08:00

When I first started travelling to Japan, I would generally stay at the Shiba Park Hotel. My business there was at the Japanese national standards body, whose offices were just across the street from the Tokyo Tower and a short and pleasant walk through the Shiba Park itself from the hotel.

In the evening a longer walk - fifteen minutes or so, one subway stop - led to the Shinbashi area. This is a maze of tiny side streets, packed with minuscule restaurants that fill with salarymen (the Japanese word for middle-class office workers) at lunchtime. After work they're back, for a beer or two with their colleagues, and a plate of noodles or sushi before setting out on their long commute to the distant suburbs. It was in one of these, many years ago, that a friend who was learning Japanese managed to order a plate of chicken sashimi (yes, just raw chicken) with a bowl of what tasted like rotten strawberry jam.

Close to the main square at Shinbashi Station, the one with the steam locomotive in it, is a kaiten sushi restaurant. Kaiten - written 回転 in Japanese - just means "turning round". You've probably been to one - instead of ordering from a waiter, you have a conveyor belt in front of you covered in little dishes of sushi. You take whatever you want, and at the end they figure out your bill by counting the plates. This system depends absolutely on having a very high turnover. It takes only a short while, maybe 15 minutes, for the fish to start to dry out and look distinctly unappetising. Kaiten sushi tends to be a lunchtime thing, when there are big crowds in a short time.

An additional benefit of course is that you don't need to be able to speak the language. Assuming you can recognise the things you like - or don't mind taking a risk - you just pick things out as they pass.

A further sophistication of the same idea is to replace the conveyor belt by a canal with little boats carrying the plates of sushi. This was an American invention - Isobune Sushi in San Francisco's Japantown claims to have invented it, though for all I know so does every other boat sushi restaurant in the country. To my great frustration, I have never been able to work out what makes the boats move round the canal.

But back to Shinbashi. We first went to the Kaiten sushi on our first trip together to Japan (though we'd both travelled to Japan before). It's a very unassuming place, full of salarymen during the week and shoppers at the weekend. It's important to go when it's busiest, before about 1.30 - as I explained before. Sometimes that means a bit of a wait, then you squeeze onto two tiny stools (if there are two of you of course - though it's very common for people to eat their on their own), squashed between the other diners. Service is minimal, though courteous and attentive anyway since this is Japan. Every three places or so there's a hot water tap, a pile of cups and a box of teabags (o-cha - green tea - of course), along with a chopstick dispenser, napkins, soy sauce and pickled ginger. You just take what you need and wait for your favourite sushi to roll by. If you want beer or sake, you have to order that.

In the middle of the island, three or four sushi chefs toil continuously, replenishing the dishes. If you watch them carefully you can see what they are making, usually in batches of half a dozen or so dishes, and if it's something you're waiting for, you can prepare to grab it quick. The normal protocol is just to take things from the belt, but if you want something that isn't there or is a bit special, you can ask one of the chefs and they'll make it for you.

When you've had enough, you just stand up and walk to the door. The cashier shouts to the other staff, who counts your dishes and shouts back the price, you pay - usually in cash - and that's it. There's nor formality to it and of course, in Japan, no tipping.

For some reason we really took to this place. Every time we go to Japan we manage to squeeze in a visit. It hasn't changed in the 20+ years we've been going there, although I guess the staff must have moved on. Each time we dread that it will have closed - so many of our favourite spots in Tokyo have closed and been replaced by office buildings, like the "Rubbery Pancakes" breakfast spot next to the Shiba Park. But, so far, it has still been there every time.

Dedicated Wallpaper Screen - everyone should have one!

2011-09-08T18:09:00.000-07:00

There are all sorts of reasons why a person might need cheering up. Luckily, there are also all sorts of ways to cheer yourself up. For example, there are books which make me laugh out loud no matter how down I'm feeling - which really does make the black cloud go away, at least for a while.

One which I can really recommend is to collect all of your favorite photos, and have a second screen on your desk which shows you a randomly changing selection of them. They don't have to be pictures that you've taken yourself, of course, but there's something especially cheering about seeing places you've been, cool things you've done, happy times you've had...

It occurred to me a while ago that you could easily hook up a screen like this, to a superannuated computer. Gee, you could even build a little desktop gadget with a screen and a microprocessor... then of course before I could do anything about it, Philips brought one out. They were rapidly followed by a bunch of no-name Chinese products with an extra twist - they contained a virus that infected your computer as soon as you connected them. Neat trick. I had an actual Philips one, which was malware-free but not explosion-free. One day I noticed that it was a funny shape, and when I took it apart the internal lithium-ion battery had exploded, with enough force to bend the solid metal frame. It has never been quite the same since, and anyway there's no longer room for it on my desk.

At the same time I bought the picture frame, I started assembling my favourite pictures - the same ones as in my Flickr account, with the addition of some family pictures that I don't particularly want to share with the world at large. I found a wallpaper changer that worked well for me, and whenever my computer was idle, they'd scroll by in front of me. It was great, but there's one problem: if I'm not actually using the machine, I'm probably not looking at it. And if I am using it, of course, the wallpaper is invisible under the clutter of a dozen windows.

Fast forward to my new Linux system. At the same time I bought it, I also bought a new monitor. There was nothing wrong with the old one, except a broken stand elegantly patched up with cable ties, but the new one has more pixels, and bigger is always better (n'est-ce pas?). So, I suddenly had a spare large format (1600 x 1200) monitor, and thanks to a superhuman feat of tidying, a space for it on my desk.

It took me a while to get round to making this all work, though. There are people who have had multi-monitor systems for years, but I've always preferred to have a single large one - hence the present 1920 x 1200 display.

It seemed obvious to me that I would need a second graphics card, so off I went to Fry's and bought a low-end one, a Realtek HD5450. I've got used to how well Linux deals with new hardware, so I just plugged it in and expected it to work. My optimism was misplaced, however. I'm running a recent version of Linux (11.04, Natty) and all the bits weren't in the right place, fixed with a bit of googling. No matter what I did, though, the system only used the built-in graphics on the motherboard.

A bit more googling showed me how to change the BIOS setup so it would use the new card - but then it would only use the new card. There was no way to get it to use both of them at once.

There was a seriously heart-stopping moment in the middle of all this. There was one BIOS setting that resulted in a psychedelic display as the system booted - a patchwork of constantly changing colors. Eventually the system came up normally - but how could I change the BIOS settings again? It seemed for a short while as if the only thing to do would be to buy a new motherboard! Luckily, unplugging the new card magically made everything work again.

In the end the solution was simple. You can run two monitors off the same graphics card, just plugging them into two different sockets. And I didn't even need the new card - thank you Fry's for an extremely liberal return policy.

It still wasn't completely over, though. Convincing Linux to run both displays wasn't at all obvious. Not only did I have to tell both Gnome (the desktop system) and the monitor drivers, but I had to do it in the right order - otherwise one undid the changes to the other. The X-windows configuration had to be changed manually, using the magic command "sudo dpkg-reconfigure xserver-xorg". Somehow this reads your mind, figures out what you're trying to do, then generates the corresponding xorg.conf file.

Finally I had it all working. Still not there, though. The "wallpaper" monitor is to the left on my desk. But Gnome wants to put everything on the left screen, unless you've explicitly moved it to the other one. There's a box you can click to make a different screen primary, but it only has a limited effect. Luckily, most X apps remember where you last put them, so it's just a question of "training" the apps you use most. Every now and then, though, I click on something, and can't understand why nothing has happened. Then I look at the wallpaper screen, and it's over there. So I call it, "here Fido", over it comes, and another app has got it figured. (Well, actually I drag it with the mouse).

And what a constant pleasure it is, to see all these pictures of things I've done and places I've been. While I've been typing this, I've had...

the tiny landing strip in the remotest part of Baja California, whwere we went whale spotting
a beautiful, moss-clad waterfall in the Oirase Gorge in northern Japan
a still-life composition of freshly caught fish in the market in Cap Breton, France
several shots of the Golden Gate Bridge, taken from different aircraft at different times, including the ones taken while flying under it in the heli
a visit to Potsdam, Germany in 1986, when it was still East Germany, with just a single smelly Trabant visible in the whole of a huge plaza

...and of course lots more. The screen in the picture at the top is showing a public footbath at Sakurajima in the very southern tip of Japan. What a visual feast!

Memorable Meals #1: The Governor's Lunch, K, Japan

2011-09-05T20:06:00.000-07:00

A few years ago my then-employer decided to open a development centre in Japan, and asked me to take care of making it happen. As a confirmed lover of Japan, I was delighted to do it. It was initially going to be quite small - and as it turned out, it has stayed that way - so the initial office was in central Tokyo, in one of the sales offices. But at the time, there was talk of expanding to something much bigger, maybe hundreds of engineers, and of opening a second office later on, outside Tokyo. This led to an invitation to one of the Japanese provincial towns, where the prefecture had established an advanced research centre for computer science. I'll be discreet about the actual place, just like in Japanese (and Victorian) novels, and call it K.

As a consequence, I was invited with our Japanese country manager to visit the town and its research centre. Also on the agenda was lunch with the Governor of the prefecture (roughly the equivalent of a US state).

There was a lot involved in setting up our operation, and I was in Japan for three weeks. Luckily we were able to work things so my wife came over with me, and we rented a very nice apartment in the Aoyama district of Tokyo. That's a story for another time, but meant that we both went along to K. We took the train, starting with the Shinkansen (bullet train) line from Tokyo out towards Niigata on the Japan Sea coast. It was February - Tokyo was cold, around freezing. The train trundles through the Kanto plain for about an hour then suddenly plunges into an enormous tunnel, over 20km long.

When it came out, we were quite unexpectedly in a true winter wonderland, with huge banks of snow beside the tracks and enormous snowflakes falling gently to ground. We changed to another train, which followed the valley for a while, then plunged into another giant tunnel. At the other end it was still snowing and we thought we must be high in the mountains still - until we saw the sea. Something I know now - but didn't then - is that the Japan Sea coast gets huge amounts of snow - tens of metres are common, even at sea level. The journey continued along the coast, past small fishing towns and villages, still in falling snow. I love travelling by train in Japan, and this was perfect. By the time we arrived at K it was dark.

The next morning, the day of the lunch with the Governor, was fine, though cold. Isabelle went out shopping and sightseeing - the town has a famous park dating back to the samurai era. It was really bitterly cold and there was snow everywhere.

The lunch was very nearly a disaster before it even started. The country manager was horrified to see me on my own. "Where is your wife?" he asked, shocked. It turned out that she was expected at the lunch too - which was a surprise to me, since in Japan business is entirely conducted between men. Women in the professional workplace are treated as honorary men, but families remain unknown even to colleagues who have worked together for decades. Luckily I managed to track her down - thank goodness we both had rented Japanese cellphones. We snatched her up in the main shopping street, in a kidnap scene from a bad movie.

The restaurant could have been anything from the outside, but once inside we realised that it was an extraordinary place. It had been there literally for centuries, since the days of samurai warlords. It's the kind of place that foreigners just never see, that you see on Japanese soaps when the political bosses get together to fix something behind the scenes. Being Japan, there is absolutely nothing ostentatious or showy about it, everything is in the details.

We took our seats around the table - or rather, non-seats. The Japanese tradition is to sit cross-legged on woven grass mats, or tatami. However even the Japanese find this uncomfortable, and increasingly you find an invisible hole under the table where you can put your feet, as you sit conventionally on the edge of the tatami. In this case, it was even heated to keep our feet warm. Each place had a menu card, and ours had been translated into English. The polite conversation began. It was difficult - we had a translator, but it's difficult to be spontaneous when every remark has to be translated. In addition to the Governor, there was also the head of the research institute that we would visit in the afternoon.

I came close to making a big mistake. There'd been a program on the television the previous night about the railway that used to run to a nearby rural town, very nostalgic with shots of old people coming home from the market, interviews with schoolchildren trying to make a museum out of the station. By way of trying to make relevant conversation, I mentioned it. What I couldn't know was that another nearby long rural line was about to close, no doubt the reason the program had been shown. Rural railways are a very emotional topic in Japan - they were being built until relatively recently, in fact this one only opened in 1964, and the Governor had heard more than enough about the topic lately. The language barrier came to our aid as he defended the decision to close the line.

Every dish was exquisite, served with charm and elegance. They were all delicious. At a refined meal like this, there are numerous dishes, all served separately and cleared before the next one arrives. That's unusual in Japan, where it's more common to bring most dishes at the same time, and the notion of a western-style course is much more fluid.

Well, there was one dish that caused us some difficulty. The Japanese name is "konowata", pickled entrails of sea slug. Remarkably, you can reuse the sea slug afterwards - its entrails grow back again, a useful evolutionary trait as it turns out. Luckily for us, it is astoundingly expensive, about $50/kg, which means we only got a tiny amount of it. It was the centrepiece of its course, but could be readily swallowed without touching the sides.

There were 15 dishes in total, each served on a special plate or dish which no doubt has some traditional significance. Beer and sake were served throughout, though we drank very little considering what was in store for the afternoon. Finally the meal came to an end and, after the usual polite formalities and much bowing, we went out into the snow.

It's unlikely I'll ever experience another meal quite like that one. The evening meal, in a hotel with the heads of some local computer companies, was utterly unremarkable. The next day we returned to chilly Tokyo, by plane this time, but the memories of the Governor's Lunch will be with us forever.

Boost: a retrospective (part 4) - the Curious

2011-08-31T17:24:00.000-07:00

In part 2 and part 3 I talked about the best and the worst (in my opinion, naturally) of Boost. Here are some interesting things which don't fall into either of those categories.

Boost Units

Does it make you uneasy to use the same type - say float or double - to represent a whole bunch of things which are fundamentally different, like length, time, volume? Or related but measured differently, like millimeters, feet and miles? It has always made me vaguely uncomfortable, and of course it has led to some spectacular disasters (not mine!). But doing something about it would be a lot of work. Defining, say, a millimeter class would be easy, but handling all the legitimate operations involving more than one unit would just bury you.

Enter Boost Units, which has a completely generic understanding of all these things. All of the meta-arithmetic, like knowing that distance divided by time gives speed, is done at compile time using some very heavyweight template metaprogramming. But you don't need to know about that. You just declare d, t and v as furlongs, fortnights and furlongs_per_fortnight respectively, and dividing d by t gives you v. Simple. Define t2 as seconds and assign it to t2, and seconds will automagically be converted to fortnights (slightly more than a million to one - so one microfortnight is conveniently close to a second, a fact used in one obscure corner of DEC's VMS operating system).

I put this in the "curious" category, rather than the "good", only because I've never had a chance to use it myself, being a systems kind of a person rather than say a mechanical engineer. But if I ever get round to rewriting my robotics code in C++, I will certainly use it.

Shared Pointer

Memory leaks are the bane of C programming, along with buffer overflow. They can be largely avoided in C++ by using auto_ptr to represent ownership of a structure. But this breaks down if there is not a single owner, for example if an object needs to be passed on to another function and then forgotten. It's just about guaranteed that a program that works this way will have leaks, even if they only occur in obscure error conditions.

Reference counts are a partial solution, but they just replace one problem with another since now everyone has to be disciplined about adjusting them. And of course they-re intrusive - the object has to have a reference count, and know to delete itself when the count drops to zero.

boost::shared_ptr tries to provide a solution to this, by keeping a behind-the-scenes reference count object. On the face of it, it looks perfect. If you are dealing with all-new code, and you keep solid discipline about never using a regular C-style pointer to the objects, maybe it even is perfect. I've used it for managing buffer pools.

I put this in the "curious" category because of what happens if you have to deal with a less structured environment. You can extract the raw pointer easily enough, to pass to a function that expects it. As long as that function never expects to take ownership, that's fine. Above all it must never delete the object, obviously. But there's a more subtle problem. If you have code which uses a mixture of raw pointers and shared_ptr's, there's a risk of creating a second shared_ptr from a raw pointer. And that is catastrophic, because now there are two reference counts, and whichever one goes to zero first will delete the object, leaving the other with a dangling reference and, microseconds or days later, a mysterious segfault. Guess how I know.

Proponents of the class would obviously argue that this is something you should simply never do, that you should have the discipline to avoid. But if you had perfect discipline, you wouldn't need the class in the first place - you could just remember at all times who controls the object, and be sure they delete it if they need to. So really all it has done is replace one way to shoot yourself in the foot with another.

Really the only solution to this is to keep the reference count in the object. Boost provides a class called intrusive_ptr which supports this, but I find the approach kind of backwards. I preferred to write my own base class for the referenced object. More on that in another post.

Sentries

The "sentry" is a programming paradigm for making sure that you undo everything you do, extending the "resource acquisition is initialisation" paradigm. The "do" part is done in the constructor of a sentry object, the "undo" part in its destructor. This ensures that the "undo" will always happen, even in the face of exceptions, return or break statements and so on. The classic example is locking, and indeed boost::thread provides a mutex::scoped_lock class which does exactly this.

But there are many other use cases, and the details of the do/undo operation vary quite a bit. For example, it's common in C to have a function that sets an attribute value, returning the previous value. The undo operation is to call the same function, with the saved value.

It's easy to write a sentry class for some particular case, like the mutex lock. It's not hard to write a generic sentry for a particular kind of do/undo - and indeed I have written a bunch of these.

But it seems to me that what would be ideal would be a generic sentry template class, that would figure out from the template arguments what kind of do/undo it is dealing with. This is beyond my own template metaprogramming skills, or at least beyond the learning investment I'm willing to make. But it does seem odd that it isn't part of Boost.

Lambda

There are often times where it would be convenient to have a small, anonymous function - for example, the ordering function passed to a sort operator. Java and Python both provide ways to do this, which in computer science is called a "lambda function". The new version of the language, C++0x, also supports this.

But until that's available, C++ requires you to explicitly define a function, generally nowhere near the place where it's used. This just makes code harder to read and maintain.

boost::lambda is an ingenious attempt at solving the problem, pushing template metaprogramming to its utmost limits. The basic idea is to define a placeholder for a parameter. Then, simply using the placeholder implicitly declares a lambda function. Conventionally, the placeholders are "_1", "_2", etc. Simply writing "_1*2" generates a function that returns twice its argument - regardless of the type of the argument you supply later, as long as it supports multiplication of course. For trivial functions like this, Lambda works very nicely. (Although boost::bind also uses this syntax, and inexplicably, the two trip over each other. There's a workaround, by #defining an alternative syntax for lambda. But it's odd that Boost let this slip by).

Unfortunately, C++ doesn't provide a clean syntactic way to do a lot of things that ought to be very natural, like calling overloaded functions. So, although the authors have put a huge effort into trying to make language features work, in the end Lambda is more of a curiosity than a general purpose facility. I've used it to construct arbitrary combinations of filter functions based on user-supplied criteria, for which it did the job nicely and much more simply than any alternative I could think of. But you need to find the right application.

Worst Ever Dining Experiences #4: South Kensington, London

2011-08-30T17:26:00.000-07:00

Before I bought my flat in London, we often used to stay at a boutique hotel in South Kensington called Number 16. It was a converted row of houses, all terribly English. Eventually it priced itself out of what we thought was reasonable, considering the small if pretty rooms, and we swapped our allegiance for the Royal Garden. There are leafy squares to the south of Old Brompton Road, and plenty of restaurants and useful shops within walking distance. South Kensington station is close by, and the South Ken Museums are a 10 minute walk away. All in all, a nice area.

We'd just arrived from somewhere. We were tired, and didn't want to hike across London. Somewhere, we saw something favourable about an Italian place, in one of the side streets by the tube station.

It seemed fine, typical of London neighbourhood Italian restaurants. I can't remember what we ate, probably something involving veal or pasta or both. What I do remember is the wine we ordered, a bottle of Chianti Classico - a reliable standby with Italian food. When the bottle came, it was a Chianti but not a Classico. This is more than just a matter of a name - the "Classico" suffix represents a 50% or more increase in value and in quality. But it tatsed fine, and we weren't in a mood to make a fuss, so we drank it with our meal.

When the bill came, I noticed a line that said "Chianti Classico". I mentioned to the waiter that this wasn't what we'd had. The bottle was still there on the table for him to see. His reaction was a surprise, to say the least. He started screaming at us, accusing us of goodness knows what. I suppose he thought we were trying to get a cheap meal. We weren't of course, but we don't like paying for things we didn't get.

This went on for a while, and no doubt I yelled back at him, until eventually the owner came by. By this time I was certainly in no mood to pay for the "Classico" we hadn't had, and I told him so. He came straight out and accused me of trying not to pay. Eventually, getting fed up with whole scene, I suggested that he call the police.

Suddenly everything changed. He became as nice as anything. "I give you dinner for nothing," he said, "Next time you in London, you come here, I give you wonderful meal, best wine." Clearly, a visit from the police was not at all his idea of how the evening should end. He no doubt had a kitchen full of illegal immigrants, and probably quite a few health code violations. Restaurants are an ideal way to launder illegal money, too (as are nail parlours, but don't ask how I know that). Who knows what else he was afraid of.

So everything was amicable, and to profuse apologies we left. As you can imagine, we were quite bemused on our short walk back to the hotel.

The place was still there for quite a while afterwards, though it has gone now. Evidently the police took a while to catch up with it. Needless to say, we never did claim our free meal.

Boost: a retrospective (part 3) - the Bad and the Ugly

2011-08-25T19:22:00.000-07:00

In part 2 I talked about my favorite elements of the Boost libraries. Boost is wonderful, but even so there are things that are not so good. These, the ones which (in my opinion) are best avoided, form the subject of this post.

Serialization

I wrote a while ago about my frustration with this library. It seemed the perfect solution to a data pickling need I had, until I discovered that it can't cope with polymorphism. It claims to, but it randomly crashes deeply nested in incomprehensible function calls if you try. There may have been a solution, but life is just too short to figure it out. The reason for all this is that its authors decided to invent their very own subclassing scheme, completely orthogonal to the one that C++ uses. They may have had their reasons, but it's a complex subject and clearly they missed something.

Asio

If you've ever needed to do low-level socket I/O, you've probably been tempted to write an object wrapper around the C function calls and data structures. You may even have taken a look at Boost to see if they have already done this. In which case, you'll find that they have. I've certainly been down this path, and discovered Boost Asio at the end of it.

You will next discover that Asio is extremely complex, with all kinds of interacting classes that you have to be aware of and create. I spent a day or so trying to get my head around it, finally getting to the point where I felt safe putting fingers to keyboard. Then I discovered that despite all that complexity, it couldn't do what I needed. This was nothing fancy, just listen on a port, and create a thread to handle each TCP session as it arrives. Turns out Asio has a race condition - by design - which can result in missed connections. Some searching showed that there's a workaround for this, but it's complex and requires even more delving into its complexities - and isn't without its own problems anyway.

I had a long meeting to attend, so I figured I'd print the documentation and peruse it during the meeting. Over 800 pages later, my meeting had finished anyway, but the printer still hadn't. At this point, I decided that anything which takes 800 pages to describe it - for such a relatively simple function, this isn't Mathematica after all (1465 pages) - just can't be worth the learning curve.

I wrote my own family of socket classes. It actually took me less time to write and debug than it did to print the Asio documentation, never mind read it! I've been very happily using them ever since. Probably, you will do the same, but if you'd like to use mine, you're welcome. You can find them here.

The Build System

Everyone knows Make. It's convoluted, nearly incomprehensible, and a syntactic nightmare, but everyone has used it and can bodge their way out of a tight corner if they need to.

But why use something everyone knows, when you can invent something unique of your own? Sadly, this is the path that Boost took. They have their own unique build system called Bjam. I'm sure it's very elegant compared to Make - it would take a huge effort not to be - but it's still very complex, and poorly documented too. In fairness, it does (mostly) "just work" if you need to build Boost from sources. But if for whatever reason you do need to get under the covers, woe betide you.

I discovered this when I needed to cross-build Boost for our embedded processor. This is always tricky because of the config stage, where the build system looks to see what capabilities the system has, where things are located and so on. For a cross-build, of course, you can't auto-discover this just by poking around at the system you're running on. That part went OK, though. However editing the build files to pick up the right cross-compiler, cross-linker and so on, was just impossible. I found quite a bit about it on the web, but never quite enough to make it work.

Fortunately, our hardware ran a complete Linux system and with a little fiddling we could just build it native on our box. But if you can't do this - and most embedded systems can't - then you can forget using Boost. Which is a shame.

Boost: a retrospective (part 2) - the Good

2011-08-24T11:58:00.000-07:00

In part 1, I explained how I came to regard Boost as an essential part of C++ programming. There are some parts of Boost that I can be pretty sure I'll use in any decent sized program, to the point where I have a generic header file that pulls them all in and makes them accessible without even needing to ask.

Regex

Before Boost came along, you couldn't really use regular expressions in C/C++. Which is a great pity, because they are just incredibly useful especially if you have to deal with any kind of human input. There is a Gnu regex package, but it is GPL, so unusable in anything you plan to sell or keep to yourself, and it has a determinedly C-flavor interface which means you'd have to write a C++ wrapper round it anyhow. My Winlife32 program had to parse human-style text input without regex, and what an incredible pain the neck that was!

Lexical Cast

Can you remember how to use atoi, itoa, atof, and all the other zillion variants along the same lines? No, neither can I - and actually quite a lot that you'd expect to find, don't even exist. lexical_cast to the rescue! To convert a string to any type - including your own types as long as they define a stream >> operator - just write lexical_cast<type>(str). If it can't be converted, you get an exception which you can use to trigger an appropriate error message.

In the other direction, lexical_cast<string>(value) will convert anything that has a stream << operator to a string. Simple, but indispensible.

Function / Bind

I already talked about these in Part 1. I can't imagine trying to write code without them now. Although they are proscribed by Google's internal coding standard, because they "would encourage functional style programming". I have no idea why that is supposed to be bad!

Format

Printf is incredibly useful, but fraught with issues viewed from a 2011 C++ perspective. It's not type-safe, it's very fragile and can cause your program to just roll over and die. And of course there's no question of dealing with user-defined types. Boost Format is used pretty much exactly like printf, except it fixes all of these problems and more. For example:

cout << format("unit %d temp = %.2f deg C") % index % get_temp(index);

will do exactly what you'd expect (note the neat reuse of the % operator, similar to Python by the way). But actually, you don't need the "d" of "%d", because it knows it's dealing with an int, and will format it accordingly. Replace the int with a type of your own, having a stream << operator, and it will work too.

There's a lot more to it, if you want to use it - numerous extra formatting options, positional and named arguments.

Foreach

The STL containers, and their Boost extensions, are incredibly useful. But iterating through their contents is so painful, syntactically. After the hundredth time you've typed something like:

for (vector<int>::const_iterator i=vec.begin(); i!=vec.end(); ++i)

you are really just about ready to scream. Boost Foreach to the rescue! You can replace all this with:

foreach (int i, vec)

(OK, I've cheated a little, my generic header file #defines "foreach" as "BOOST_FOREACH" just to make the code prettier). Notice that i is just an int, not an iterator, so you don't need to use '*' or '->' with it. This is especially neat for containers of pointers, which get very awkward. (Boost has another solution for those, too, the Pointer Container library - though I've always found it doesn't quite do what I need).

The new C++0x standard makes the problem go away, since it has a built-in container iteration syntax, as well as the auto type declarator. But Foreach will have saved a decade or so of tedious and ugly typing.

Intrusive

I've eulogised elsewhere about this. Suffice it here to say that you get all the convenience of the STL container types, without the behind the scenes manipulation of little extra memory blocks, and their associated run-time cost. For anyone whose system-programming teeth were cut in C or assembler (or Bliss!), this is just so much nicer.

Date_Time

Who hasn't struggled with all the complexities of date and time? Input, output, arithmetic - they're all a nightmare. Nearly all of these problems go away with Date_Time. Date and time arithmetic is simple, comparisons are simple. Unfortunately input and output are heavily tied into the C++ locale system, which is basically incomprehensible. It's much easier to write your own parsing and output code than it is to figure out how to make locales work. But that's a nit because for everything else, these classes are indispensible.

Others

Just because I haven't mentioned them, doesn't mean I don't like the other bits of Boost (though see the forthcoming part on the Bad and the Ugly). Special mention should go to Thread and Python, which are both indispensible if you want to use threads or interact with Python, respectively. Each of them takes a fairly ugly C interface and wraps an elegant C++ interface around it. Python makes it trivial to combine Python and C++ code, or support Python scripting within a C++ app.

Part 3: the Bad and the Ugly

Boost: a retrospective (part 1)

2011-08-23T17:11:00.000-07:00

My love affair with Boost started with my first, self-appointed programming task at Anagran, the fan controller for our box. I wanted a table of functions, corresponding to each of the temperature sensors. Some of these were parameterless, corresponding to unique items, while others were indexed by interface card number. I wanted to be able to put a "partly cooked" function object in the table, with the interface number frozen but other parameters to be supplied through the ultimate call. This is called a "partial function application" or "partial closure" in computer science.

STL provides C++ with some glimmerings of functional programming, with "memfun", "bind1st" and so on. It seemed like it ought to be possible to write something appropriate, but making it usefully generalized also seemed like a lot of work. Surely someone must have done this already!

Searching for it led me to Boost, "one of the most highly regarded and expertly designed C++ library projects in the world" as they modestly say at the top of the front page. It is however true. It's a huge collection of highly-generalized classes and functions for doing an amazingly large number of extremely useful things. It's an open-source project whose authors, while not anonymous, keep a very low profile. I can only assume they love a challenge (and have a lot of spare time), because they do some extremely tricky things, under the covers. But for the user, they're mostly very straightforward to use.

So over the last five years, I've discovered more and more that can be done with Boost. Although I've called this a "retrospective", I'm not planning to stop using it.

Boost makes extensive use of "template metaprogramming", which is a kind of compile-time computing. When C++ templates were invented, the idea was to allow simple compile-time parameterization of classes and functions, for example so you could write a "minimum" function to return the lowest of its arguments regardless of whether they were int, float, double or some user-defined class. As the concept evolved, it became possible to make very complex choices at compile time. In fact, you can write just about any program to produce its output directly from the compiler, without ever even running it, if you try hard enough. It's hard to get your head around, but fortunately you don't need to.

Function and Bind

These were the first Boost packages I discovered. Function defines a general, templatized function class. So you can define a variable as "function<int(foo*)>" and assign to it any suitable function. In particular, assign a member function of the foo class and all the right things will happen.

The Function class is useful, but it is the Bind class that really transforms things. You can take any function, bind some or all of the parameters to specific values, and leave the others (if any) to be supplied by a subsequent call to the bound object. This is exactly what I was looking for in my fan controller. For example, suppose you have a function "int foo::get_temperature<(double)>". Then you can write:

  function<int(double)> fn = 
    bind(&foo::get_temperature, my_foo, _1);

to store a function which will apply its argument to the "my_foo" instance of foo, which you use for example as:

  printf("temperature at %f is %d\n", v, fn(v));

(Of course you shouldn't be using printf, you should be using boost::format, but that comes later). The "_1" is a placeholder, whose meaning is "take the first parameter of the final call, and put it here". Bind takes care of types, making sure that the actual parameter is (in this case) a double, or something that can be converted to it. If you want to, you can even apply bind to previously bound functions - though you might want to ask yourself why you're doing it.

This is absolutely perfect, for example, for callback functions that need to keep hold of some context. In C you do it using void* arguments, which is unsafe and generally wretched. This can be avoided in C++ by defining a special-purpose class, but that requires the caller to know about it, which ties everybody's shoelaces together more than is healthy.

The only problem with function/bind - which is true of any code that makes heavy use of templates - is that compiler errors become incredibly verbose and just about useless. A single mistake, such as getting a parameter type wrong, results in pages of messages, none of which gives you the slightest clue as to what you actually did wrong. The first time you compile a new chunk of code that makes extensive use of bind, you will typically get thousands of lines of errors, corresponding to just a handful of typos and the like. The trick is, to find the message line that gives you the actual source line - which is buried in there somewhere - then just go stare at the line until you figure out for yourself what you did wrong. The rest of the messages can be summarized as "you did something wrong on this line".

Part 2: The Good (things I just wouldn't live without)
Part 3: The Bad and the Ugly

VirtualBox - virtually complete: part 1

2011-08-23T12:39:00.000-07:00

My new Linux machine is almost complete now. It has been running for a couple of months. Since I lost my laptop along with the company I worked for, and haven't seen a reason to buy another one yet, the Linux machine has become my main computing platform for just about everything.

However there are some things which can only be done on Windows. One is iTunes - I only use it as a backup for my iPhone, but for that it is indispensable. Another is the package that updates the navigation data for the plane. Then there is expensive software I bought for Windows and don't plan to buy again - Mathematica, CorelDraw and so on. Not to mention my HP printer/scanner which has no Linux driver. So I've had two machines sitting next to each other, with a KVM switch to go back and forth when needed.

Of course the solution to this is obvious - virtual machines. I've been idly looking at VirtualBox, one of the open-source VM solutions, for a while. It took something odd to make me dig a bit deeper - the discovery that our electricity supplier (PG&E) has a web page where you can see your hourly consumption. It's fascinating to study, and one thing it showed is that my two desktop computers were accounting for about 60% of the house's background electricity usage. So, turn one of them off, instant 30% energy saving. (If only I could do something equally miraculous for the pool, which accounts for over half of our electricity).

I started by creating a second Linux system. That seemed less scary, though as it turned out it was harder than Windows. It's simple enough - tell VBox to create a virtual machine, then boot it with a Ubuntu CD in the drive and it pretty much goes all by itself. My plan for this VM was to use it as a web server for stuff I want to host at home, rather than through my web provider. It all went well, until I tried to set up a "shared folder".

In its pure form, the VM concept means that the VM runs in complete isolation in a bubble inside the real operating system. This is fine until you do actually want to share stuff. You can do it over the net, using FTP or whatever, but even that isn't "out of the box". By default, Vbox creates VMs using internal NAT addresses, so there is no way to access the VM from anywhere else, including the host. That can be fixed with a couple of clicks, selecting "Bridged Adapter" in the "Attachedto" drop-down, instead of "NAT". But still, it's a clunky way to do things.

So you can also create a "shared folder". This is just a regular directory on the host system, but it looks like a remote filesystem to the guest. It's easy enough to set up, but I just could not get it to work. I successfully mounted it just once. After that, attempts to mount it always failed with "no such device" (or some such). Even deleting it and creating a new one didn't work.

Finally I discovered that, because I'd ticked the "auto-mount" box, it was being automatically mounted in /media. Well, duh, I suppose.

With that done, my Linux VM was usable. Fortunately, because it turns out in my main (host) system, Firefox had autoupdated itself to V6 - and the Flash player doesn't work an more. No more stock charts from Google, no more news video clips from the BBC. This seems to be a problem that a handful of people have run into, with no solution. So for now, I just use an older version of Firefox running inside the Linux VM whenever I need Flash.

The next step was to install a Windows VM. But this is already too long, so that will be for another time

IAS - the PDP-11 Interactive Applications System

2011-08-13T17:38:00.000-07:00

(This originally appeared on my web page).

DEC's approach to operating systems for the PDP-11 was anything but disciplined. New ones got invented every time some engineer or marketing person blinked. In the early days, there was a real-time kernel called RSX-11A, designed for memory-resident applications in what we now call embedded processors. Features got added to this rapidly - code bloat is nothing new. By the time it got to RSX-11D it had a complete disk-based file system, a program development environment, and support for every peripheral in the Small Computer Handbook (and there were plenty of them - peripherals on the PDP-11 obeyed the same strategic imperatives as operating systems - see above). At this time, a bright young engineer called Dave Cutler decided that enough was enough, and set out to create a small system that would do the same, which he called RSX-11M.

Meanwhile, the PDP-11 also had a timesharing system very loosely based on TOPS-10, called RSTS/E. A senior engineering manager, newly installed in Geneva, decided that it would be a smart move to develop a system that could do both real-time and timesharing, based on RSX-11D. It was specifically targetted at the planned PDP-11/70, which was a kind of super-11. Since he had newly moved to Europe (his name was David Stone, by the way) he gave the project to the European Software Engineering group that he had just invented in Reading, England. This was about the time that I joined DEC, and after a few misadventures I found myself assigned to the project.

The system was to be called IAS, which if I remember rightly stood for "Interactive Applications System". It added to the RSX-11D kernel a clever timesharing scheduler, a bunch of security features, and a new command language. These were the days of MCR, a command language which makes even the Unix shell look lucid. (To delete a file you typed "PIP file/D" for example). The then-boss of software decided we needed a Digital Command Language, which of course later become a feature of VMS, but IAS was the guinea-pig. In fact, all DCL commands were translated into the corresponding MCR and the fired off to the appropriate utility. The command interpreter that did this was thrown together in great haste, and remains to this day the nastiest piece of software I have ever encountered.

I had tremendous fun on IAS. Like V1 of any system, it lacked just about every feature anyone wanted, and they all had to be added for V2. It says something for the team that in fact they mostly got there in V3, and they mostly worked. The team by the way consisted of about six people - that's probably about the same number that Microsoft has doing quality control on the stupid paperclip in Office 97. My first job was to write the driver for the latest new disk, the RK06. It had about the same capacity as a couple of floppies; four of them would fit in a six-foot cabinet. I was duly sent on the course for writing device drivers, but on the first morning I finished reading the manual and by the end I had coded the driver. It did various nifty things that nobody had done before on the 11, like overlapped seeks, and ended up becoming the basis for all future IAS and RSX-11D drivers although I remained unhappy with a lot of it.

My next job was to write a new terminal driver. Despite what I said about the command interpreter, the old terminal driver was pretty special too. Support for new features and new hardware had been thrown in over several years, and it was impossible to figure out how it worked. One story that sticks in my mind: I changed it to suppress nulls on formatted output (because of a misfeature in the command interpreter). Thereafter, if you typed rapidly, it would drop the first character of every other line. I never figured out why, I just removed the fix.

The terminal driver was one of the most enjoyable bits of work I ever did. It was all table driven, and in fact was object-oriented 15 years before it became fashionable. Thus adding a new device just meant writing some standard routines and plugging them into the tables. It seems obvious now, but it was pretty revolutionary at the time! I cooperated with the guy who was writing the driver for the new VMS system, so when I invented "read with prompt" (mainly to make the output on hardcopy terminals look prettier) this found its way into VMS, and ten years later was used in a way that I certainly never thought of to double the performance of All-in-1.

All of this found its way into V2. But by then, Cutler had decided that RSX-11M was going to take over the world. Since he was at the heart of things in Maynard, and we were 3000 miles away in Reading, it was pretty easy for him to get the sales and support people to listen to him. IAS did get some very loyal customers, including Boeing and the US Navy, who stuck with it long after Digital had tried to kill it.

In fact, as the cost of developing and maintaining software soared, Digital did try to rein in the PDP-11 situation. As a result, we had to combine IAS and RSX-11D into a "unified product strategy". (I seem to have spent a lot of time over the years taking products that were never meant to be the same thing, and making it look as though they were).

IAS had many features that RSX-11M didn't, such as a proper timesharing scheduler. This led to RSX-11M+, which was RSX-11M with a bunch of features intended to match IAS. This was really a stretch for 11M, in complete conflict with the "size is the goal" philosophy which Cutler had made into a rubber stamp that appeared on all of the early 11M design documents. Nevertheless, M+ had the visibility in Maynard and IAS didn't, and got the development funding. This meant that several new features first appeared in M+ and then had to be retrofitted into IAS.

One of these was "PLAS" (I forget what it was supposed to stand for), which gave programs access to the memory management features and was a kind of do-it-yourself virtual memory. It fell to me to implement the linker support for this. Now the linker was Cutler's first ever piece of software at DEC. It was very clever; doing memory layout for a machine as constrained as this could never be easy. In addition, it supported PSECTS which were modeled on what the IBM/360 did for memory management, and of course found their way into VMS as well. Thus a memory section could be shared between overlays, or not, and could be marked for code or for data, and could be overlaid to support Fortran COMMON or not, and so on. There were about seven different PSECT attributes, and so over a hundred different ways that memory allocation could be handled. And overlays - who remembers those? The linker had a write-only language called ODL (Overlay Description Language), which allowed you to set up hugely complicated overlay structures. As the PDP-11 address space (64 kbytes!) become more and more of a constraint, ever-fancier overlay techniques were invented, and since the linker had to handle them all its own overlay structure was the most complex of the lot.

But by this time the writing was on the wall for the PDP-11 as a general-purpose machine. The VAX and VMS had been a huge success and all serious investment went on them, rightly so. Personally I moved on after we released V3, in about 1979, but IAS retained an engineering group for a few more years. I think DEC continued to support it, for the benefit of the handful of big customers (like the US Navy) who were still using it, up until 1988 or so.

The PDP-11 spawned several great operating systems, the most famous nowadays being Unix. But IAS had something the others never did, a unique ability to support both timesharing and real-time applications at the same time. A big 11/70 - which is to say a megabyte of memory and a few tens of megabytes of disk - could give decent timesharing support to 20 or 30 users, and there were people who ran it at the limit of 64 users and seemed happy with it. Try telling that to the youth of today!

Python and Tkinter: wonderful

2011-08-09T14:12:00.000-07:00

Having time on my hands at the moment, and no work commitments - that's another story though - I decided to start taking a new look at the robotics stuff I was playing with a year or so ago.

I'd written nearly all of the code to make a six-legged robot - a hexapod - walk with various different gaits and postures - the so-called inverse kinematics. It was in straight C, since I intended it run on a little embedded CPU which had no support for C++, nor floating point for that matter. And I'd built a development environment, including visualisation for the leg movements, using Visual Studio.

Things have moved on since then, though. For one thing I've pretty much switched to Linux for my computing environment. For another, the Roboard has really become the obvious onboard computer - it is now available with Linux, and it offers a full-function x86 including floating point, in a tiny size that will fit in my fairly small hexapod. And since it supports full GCC, I can write the code in C++. The C code is just so cluttered - I can't for the life of me imagine why anyone would prefer to code in C. It's full of irrelevant details that make it hard to read and even harder to get it to work. So, a rewrite is called for.

The only problem, is the GUI that I'd painfully created using the Visual Studio tools. Painful because there are 6 legs, and each has numerous parameters and state variables. I'd created the dialog box from hell. Every tiny change meant nudging numerous components around to get it to look right. What a pain. But it was done, for now anyway.

That was when I thought about Tkinter, which I've never used before. I've become a huge fan of Python in the last couple of years, using it for anything where performance is not a big deal. I also wrote a very powerful Python-based scripting system for my now-defunct employer, using Boost Python. So using Python and Tkinter for the GUI was kind of an obvious thing to do.

Somewhere in the mists of history I acquired Python and Tkinter Programming, which I think is the definitive book on the topic. I skimmed that, and with frequent help from Google - especially this site - started putting my new GUI together.

What a pleasure! Tkinter automatically takes care of making a reasonable layout, given some general guidance through the pack and grid functions. You no longer have to think about the minutiae of positioning, or spend ages getting boxes to line up with each other. I just couldn't help putting together a bit of infrastructure for collections of config variables, so they are now super-easy - just a list of names and default values and Python and Tkinter take care of everything.

In total it has probably taken me about 6 hours to get everything together - but that included learning Tkinter from scratch and writing quite a bit of infrastructure. And now I have everything I need to control my inverse kinematics, and have an animated visualisation of what it's doing.

I'll never do GUIs any other way now. Tkinter is wonderful!

Favourite restaurants #3: Pizza Cresci, Cannes

2011-08-06T16:46:00.000-07:00

My sister used to buy the Daily Sketch, a now long-forgotten English newspaper, on her way to work every day. This was a long time ago - she married and moved out when I was 11. When she came home I would seize it and read the cartoon on the back page, Peanuts. Among the many incomprehensible cultural references, to a child growing up in England in the 1950s, was the occasional mention of "pizza pie". Pizza was pretty much unknown in England back then - probably there were Italian restaurants in London that served it, but those were hardly the kind of places we could afford to go to. It would be a good few years before I'd find out what it meant.

Now, you can probably get pizza in every country in the world. Really it's amazing how quickly it has spread. Of course it was already commonplace in the US back then, which was why Charlie Brown took it for granted. I've eaten pizza in just about every country I've visited, there are times when you just need a break from the local food no matter how much you like it - as in Japan - and certainly if you don't, as in Korea.

Pizza's introduction to England was courtesy of Pizza Express, a London chain (originally) that made them before your very eyes, and made a very tasty pizza too. They even published a pizza cookbook, which worked surprisingly well considering that a domestic oven doesn't get anywhere near hot enough. Though my own introduction to pizza was at a local restaurant when I worked in Reading, Mama Mia - long since closed I'm afraid.

When we lived in France, we would make the pilgrimage every summer right across the south to the beach town of Hossegor - site of another favourite restaurant. It was a long drive - 8 or 9 hours, especially before the autoroute was finished and you had to dice with death on the three-lane stretch between Salon and Arles. By the time we got home we were exhausted and hungry. We would pile out of the car, leaving it packed to the gills with bags and often cases of wine that we'd stopped off for at Buzet, and cram into Isabelle's tiny Abarth to drive down to Cannes to eat.

Tradition had it that we always went to the same place, Pizza Cresci on the waterfront. Just the location is the stuff of dreams - right across the street from the harbour, packed with millionaires' yachts. Oh, and right next to the Municipal Police, hence easily recognised by the illegally-parked police cars, as you can see in the picture at the top. You might expect that in such a touristy location, the food would be mediocre. You couldn't be more wrong!

Pizza Cresci has, quite simply, the very best pizza I've ever tasted, anywhere in the world. I've been to the original pizza restaurant in Naples, and to some of the most famous ones in the US. They've all been good, but none has been quite as good as Cresci. My special favourite is their Pepperoni. They use a thin crust, crisp around the edges but deliciously soaked in melted cheese and oil in the centre. With a sprinkling of hot oil... just sinfully moist and delicious. Isabelle's favourite is something quite unique, an aubergine (eggplant) pizza, very thin slices of aubergine, a little cheese, and the same yummy thin base.

Of course we went there at other times too - if we were tired and just couldn't be bothered with eating at home, it was so easy. And it's huge (by French standards anyway), so even when it's packed at the height of the tourist season, you never have to wait long. But since moving to California, it's a wee bit less convenient and we hadn't been there for a long time. Then this spring, we visited Sorrento and Naples, then spent the weekend in Nice. Fresh from Napoli, the self-appointed capital of pizza, we decided to have lunch there. It was as wonderful as in our memories! The pepperoni pizza was delicious, the aubergine too (so I'm told), and as always with view of the Cannes waterfront.

Forget all the famous many-starred restaurants in Cannes, head straight for Cresci. It's the place to eat!

Worst Ever Dining Experiences #3: The Fat Duck, England

2011-08-06T13:21:00.000-07:00

The Fat Duck is to food as Damien Hurst is to art. Neither is either, by any reasonable definition. Rather, they're an attempt to see how far you can lead people down the path of the ridiculous, if you constantly reassure them how sophisticated they are. Not that different from the story of the Emperor's New Clothes. In both cases they have been highly successful: the Fat Duck managed to fool the Michelin inspectors into giving it three stars, which is truly astounding.

The original concept of a restaurant was that it was a place you could go to get a decent meal. Surprisingly, it was originally not a generic term, but the name of a specific establishment in Paris, the word coming from the French verb restaurer, to restore or replenish. Since 1765, when it was first coined, the meaning has evolved somewhat. If you want a good meal you naturally think of going to a restaurant. But at the same time some of the top restaurants of the world have evolved beyond simply giving you a good meal, to giving you a unique eating experience. You don't go to Troisgros or the French Laundry just because you're hungry. (Or maybe some people do - there was a distant friend of the family, wealthy, who lived in Roanne and who supposedly ate at Troisgros every night. Why not - he could afford it and it was better than eating the French equivalent of beans-on-toast at home).

Though there are still top restaurants whose focus remains just a good nosh - the Savoy Grill in London, for example, which serves excellent but basically unsophisticated food, that Desperate Dan would feel at home with.

There's no question that the Fat Duck provides a unique culinary experience. It's just that it has strayed so far from any notion of food that you can't really call it "eating", except in the raw physiological sense that you do put something in your mouth, chew it and swallow it. Though I'm not sure "eating" applies to things that aren't food, except maybe in the sense "the dog ate my homework".

Our visit there was before it had acquired the fame it has today, about ten years ago. No Michelin inspector had yet been bamboozled into giving it three stars. We were with another couple, making four of us. There were several amuse-gueules - tiny teaspoon-sized concoctions, each more bizarre than the previous one. I remember some kind of purple jelly thing. None of them had much in the way of flavour, I suppose the idea is to look extraordinary - which they did.

But it is the main course that I'll never forget. It was supposed to be the greatest item on the menu - after all, if you're going there, go for the best. Raw pigeon. Yep, pigeon breast, raw. Marinated in something that had changed it a bit, but fundamentally, a raw piece of pigeon. Isabelle took one tiny mouthful and left the rest, in disgust. Foolishly, I persevered with it. It was edible, though a bit chewy, and with little taste. I really can't imagine, with hindsight, why I ate it. It wasn't enjoyable, it wasn't interesting, and as it turned out it was exceedingly unwise.

I don't remember the details of the rest of the meal, except that were even more weird amuse-gueules. Eventually we left and drove for twenty minutes back to our hotel. I just about had time to run from the lobby to the nearest toilet, where I was violently ill. There's a reason why raw pigeon isn't a common element in the human diet, and I'd just discovered it. Fortunately, I'd recovered by the next morning - after a thoroughly unpleasant night - and was able to take our flight home.

I guess the inspector who awarded the three stars must have chosen something else, or maybe has developed an immunity to fowl-borne gastric infections. Though the Fat Duck did have an extended bout of poisoning its customers a couple of years back, for reasons that have never been very clear. Amazingly, that has done nothing to harm its reputation. The local Indian restaurant would have been shut down (we're talking hundreds of customers here, not just one or two, over a period of months). But when your reputation is about shocking people rather than feeding them, maybe it doesn't matter.

Favourite Restaurants: #2, Memories of India, London

2011-07-31T12:10:00.000-07:00

After I bought my flat (apartment) in London, in 1997, for several years I was spending most weekdays on my own in London, returning home to France for the weekends. Cooking for one is no fun, and nor is eating alone in fancy restaurants. So friendly neighbourhood restaurants, where I could go with a book and eat a pleasant meal quietly in a corner, were highly sought after.

England is of course full of Indian restaurants. I doubt that there's a town that doesn't have one. In London there are literally thousands. The menu is pretty much the same in all of them, and with luckily rare exceptions the food is good too. But of course some are better than others. When I explored the ones in the area round my flat, I quickly found one that was head-and-shoulders above the others. Memories of India is an absolutely typical London Indian restaurant, that just happens to serve exceptionally good Indian cuisine.

When I lived just across the street, I would go there at least once a week, and when my willpower was feeble (often), more than that. Settling down at a corner table with a good book, a few poppadoms, a beer or two, a Shah Gostaba and a few side dishes was just as good as sitting down at home to a meal - with the added distraction of people-watching. This is a popular tourist area, with many mid-priced hotels and a handful of big ones. Every few minutes a couple or small group would pause outside the window to study the menu. And each time, the owner would rush outside to give them his sales spiel. About half the time it worked - to their good fortune, since they could easily have ended up in the nowhere-near-so-good place a few doors down. The other customers were a complete mixture, everything from noisy groups of caricatural American tourists to earnest young couples from the provinces, identified by their accents such as Geordies (from the Newcastle area) or from Northern Ireland.

"Eeh bah gum lass, even London 'as Indian restaurants, just like back 'oom in Bradford", they'd murmur softly to each other, holding hands discreetly across the table on their once-in-a-lifetime-treat outing to London. Or so I imagined, anyway.

Watching the owner makes you realise just what a treadmill it is to run a restaurant. Every single night he's there, keeping an eye on the place, trying to attract customers, greeting the regulars. The place can run without him - once when I went there he was on holiday. But still, there's not much of a break.

It's ten years now since I moved to California and let my flat. But when I stay in London it's usually in this area, often at the Royal Garden round the corner on Kensington High Street. And in that case, a meal at Memories of India is mandatory. Amazingly, even after ten years the owner still remembers me on the one or two occasions a year when I visit.

After I moved to the US I harboured a little fantasy of opening an Indian restaurant locally, just so I could call it "Memories of Memories of India". Surprisingly, here in Silicon Valley where half the population seems to be Indian, there are very few good (as good as Memories of India) Indian restaurants. So Memories of Memories of India, or M2I for short, would I'm sure be a great success, and soon I could open another branch - called, of course, Memories of Memories of Memories of India, or M3I. Eventually I could be the owner of a nationwide chain of excellent local Indian restaurant, its size limited only by the breadth of shopfronts to accommodate the ever-increasing names.

Well, it was a nice idea. My bubble was burst when I discovered that there are already several restaurants in the US called Memories of India. In fact, there's at least one more in London, too. But not as good as mine, I'm sure.

Moving my Linux system to an SSD

2011-07-22T11:19:00.000-07:00

Phew, done it! I decided to put a solid-state disk (SSD) on my new Linux system, to get some experience of using them. I expected to run into a few obstacles, but as usual with Linux I also expected to figure them out with a judiciously worded Google query. (Open-source would not work without Google).

It wasn't that easy. My goal was to move my existing system. A re-install from scratch would have been easier but then I'd have to try and remember all the hundred or so packages I've installed since, remember how I configured things like Apache and MySQL...

With Windows of course this is impossible. Thanks to that brilliant invention (not), the registry, you have to reinstall from scratch every time, then spend hours locating the CDs and babysitting the installation process. Been there, done that, and it's one of the main things I dislike about Windows.

I googled around and found a few descriptions of what to do, and indeed it seemed easy. So I physically installed my brand new 64GB SSD, which worked fine, and copied everything on my 1TB hard drive (not that much) to it. The of course I just needed to make it into a bootable system... and that is where the fuin started.

Ubuntu uses a boot system called Grub. I'd found clear instructions how to run that. Unfortunately for me, I just upgraded to Ubuntu 11.04 ("NattyNarwhal"), which uses Grub2, not Grub. And of course, they are completely incompatible - they don't seem to have a single command in common. (This seems to be a fairly widespread disease in the open-source world, deciding to reinvent some well-known piece of the system and make it gratuitously different. The most egregious example is Python 3, though bjam, the build system for Boost, runs a close second).

And I just could not find how to make Grub2 make changes to anything other than the currently running system. I could install it on the SSD, but trying to boot from it was a failure. Ubuntu finds file systems by their unique ID (UUID), and the SSD system was looking for the old hard drive's UUID. If both drives were plugged in, the SSD would boot Grub which then cheerfully booted from the hard drive. But if not... not so good.

I tried creating a "Super Grub 2" boot CD, hoping I could use this to make the changes. This however deserves some kind of award for unhelpful software. There is no "help" command and as far as I can tell no documentation either. So that was out.

By now I'd spent (wasted) two whole evenings on this project. Only my intense natural reluctance to give up on things kept me going, since I didn't actually need this to work. Then I tried a slightly reworded Google query and found this article. If you're an experienced Linux sysadmin you're probably thinking, "What an idiot, everybody knows that's how you do it". But I didn't know...

The key to success, once the files have been copied and /etc/fstab edited, is the following:

sudo -s
for f in sys dev proc ; do mount --bind /$f /ssd/$f ; done
chroot /ssd
grub-install /dev/sda
update-grub

The magic here lies in the chroot command, which is what gets Grub to work on a different drive from the current one. Now I know about it, I'm sure I'll find all sorts of other places to use it.

That done, the SSD booted perfectly by itself, and with a bit of juggling of SATA connections, I got it to boot with the hard drive as well. Problem solved.

By the time I finished this, I was ready for a glass of wine and dinner. So I can't yet say whether the SSD has indeed given me the expected boost in system performance. But at least I've got it working.

The Joy of Country Codes

2011-07-20T17:05:00.000-07:00

The creation of South Sudan as a distinct national entity implies a huge number of practical problems. Little things like creating a viable government, a functional police force and army, a national infrastructure, and all the rest. For the sake of the people there, who have already suffered much more than enough, I hope that it works, although one newspaper article has already referred to it as a "pre-failed state".

Among all these things, one that has to happen is the creation of country codes which are used for all kinds of things. In particular, there is the two-letter code that appears at the end of a URL (like '.uk'), and the international dialling code. Alphabetic country codes are designated by a standard called ISO 3166, administered by the International Organisation for Standardisation, or ISO for short - the acronym carefully chosen since it is actually not an acronym at all, standing for neither the official French nor English names of the organisation. International dialling codes are designated in a standard called E.164, administered by the International Telecommunications Union (ITU).

The ITU grants an E.164 code to a country within 30 days of it being recognised by the United Nations. These codes are mostly two digits long (e.g. 44, the United Kingdom) or three digits (e.g. 351, Portugal). There are two exceptions: 1 and 7. 1 isn't actually a country code at all, since it designates the geographical area of North America including Canada, the USA, and various Caribbean islands. 7 used to be the USSR. Here be politics. It may seem simple enough to assign codes to countries, but actually it's a political and diplomatic minefield. When the codes were assigned, the Cold War was at its height and the USSR could not accept that the US could have a single digit (even though it didn't) and they didn't. Or so I suppose - I wasn't there, at the meeting of CCITT (former name of ITU-T) Study Group 2 where all this must have been hammered out. So now 7 is Russia, except that it's also used by Kazakhstan, and despite the fact that these two countries between them probably have fewer telephone numbers than some countries with three-digit codes.

Reading lists isn't everybody's cup of tea, but they often contain little gems of curiosity. Anyone who's had to call Taiwan knows that its country code is 886. But you won't find this in E.164. Rather, what you'll find is the entry for China, 86, with a footnote saying "within country code 86, 866 is used to designate the Chinese Province of Taiwan." But don't try dialling 866, because it won't work. Looking further, you'll find an entry for 886 with the text, "886 is reserved for assignment by the United States". In other words, China was not prepared to permit any reference to Taiwan as an independent country (their long standing position) but wasn't going to stand in the way of a practical arrangement that would let everybody get on with things.

Two-letter country codes, which are by far the best known, are in fairly short supply. In theory there are 676 (26*26) of them, but they try to have some mnemonic significance, and for well over 200 countries and other territories this makes it difficult to fit everything. For some reason, 'M' is a very common initial, and nearly all of the m's are used up. When I wanted to add Molvania (along with Elbonia and the Wallis and Grommet Islands) to the list of countries in our in-house database app, the best I could do was MJ. (Incidentally the same is true for state names in the US - M is the most-used first letter of the two-letter state codes). But luckily for South Sudan, even though 'S' is also a common initial, SS is still free. Perfect, n'est-ce pas?

Well, except that "ss" has, ahem, unfortunate connotations. Opinions differ as to whether the ISO Secretariat will allow this or not. And if not, all the other likely possibilities are already taken. About the best that I can think of would be to use a possible French version of the country name, "Sudan Meridional" (a word for "southerly" in French, along with "austral"). That would give SM. Of course that has connotations too, and very favourable they could be too for the fledgling country. Think how much money tiny Tuvalu has made from the happenstance of getting TV. There must surely be any number of kinky websites that would pay for as SM domain name... or there again, maybe the ISO Secretariat wouldn't be too keen on that either.

ISO 3166 is full interesting little tidbits. You know, of course, that FR is France. Well almost. Actually, it's "France including Clipperton Island". Where is that? I hear you ask. You mean you didn't know that it's a (usually) unpopulated atoll in the middle of the Pacific, 1000 kilometres from the nearest other land (Mexico)? And why on earth is it considered to be part of France? Thus begins a lengthy distraction on Wikipedia. Oh, Clipperton Island also has its very own code, CP, although according to Google there is not a single site that uses it.

And you know the code for my country, the United Kingdom of Great Britain and Northern Island (to give it its full title)... everyone knows that it is UK. Well, except it isn't. Actually it's GB. For some reason when they starting assigning URLs they decided to use UK instead. So UK is covered under a handful of oddball codes (along with CP incidentally) with the text "Reserved on request of the United Kingdom lest UK be used for any other country".

ISO 3166 has its share of odd diplomatic compromises and oddities, too. I bet you didn't know that BO is "Bolivia, Plurinational State of".

And you thought all this stuff was easy.

(Update, 16th August: they did get SS, read about it here. And the dialing code is +211.)

Favourite Restaurants: #1, Bar Oceanic, Hossegor

2011-07-06T08:10:00.000-07:00

To me, a "favourite" restaurant is one that I go to every time I'm in the vicinity - a place that's part of my personal tradition. When I think of the place, I think of the restaurant. The food has to be excellent, of course, but it's about more than just that, it's about a feeling.
This week I'm in Hossegor, so it seems fitting to choose Bar Océanic as the first one. Hossegor has been a family tradition for over 20 years now, and a lot longer than that for my wife, who used to come here for her childhood holidays. We'll gloss over exactly when that was, but the house where I'm sitting typing this came into the family 21 years ago. The location is perfect, a two minute walk from the beach, a ten minute walk from the town, and close to the beachfront restaurants.

Hossegor is most famous now as a surf place. The annual world championship has an event here every summer, taking advantage of the giant Atlantic waves as they crash into the beach. Or not - there can be some really impressive surf here, but it's the exception rather than the rule. We've watched the championships take place in a millpond-flat sea, the surfers desperately seeking even the smallest wave.

Before the surf, Hossegor was - and still is - a family town for holidays. It has wonderful beaches, part of the vast expanse of of perfect, clean sand which runs from the Spanish border for a couple of hundred kilometres nearly to Bordeaux. It's the combination of families and surfers that makes Hossegor unique - it's a very lively place, with bars open until the small hours and beach parties, mixed with grandparents and their progeny and everything in between.

The beachfront, a two minute walk from our house, has a bunch of restaurants of different styles, from Dick's Sand Bar (not advised if over 25) to restaurants catering rather more to the family crowd. And of these, our long-term favourite is le Bar Océanic. Their menu is a range of south western French staples. And the king of such dishes is confit de canard, duck cooked slowly in its own fat for hours and hours until it becomes tender yet moist. Yum, especially when it's grilled at the last minute to make the skin crispy, then served with fried potatoes, either French fries or sliced thinly then fried slowly in more duck fat. Needless to say, this is on the menu. It may not be quite literally true to say it's the only main course I've ever eaten there, but it's close. Served with a bottle (or two or three, depending on the number of people) of excellent local wine, it's just irresistible.

The really surprising thing about le Bar Océanic, though, isn't so much the food as the proprietor. Or more specifically, his astounding memory. We've been regulars in Hossegor for 20 years, and I think we ate there the very first time we came, but that means just once or twice every year. And there was a period when we didn't come here in the summer, for about six years, just a couple of out-of-season visits in that whole period.

Yet every time we go there, he not only remembers us but even our favourite dishes. Can you imagine? You have thousands of customers who you see at most a couple of times a year, and you remember their menu choices individually? I just don't know how he does it.

When we moved to the US, there was a period of about three years before I managed to get back to Hossegor. The day I arrived, I was walking past le Bar Océanic. The proprietor happened to be standing outside. "Bonjour Monsieur," he said, as if I had just been in there the night before.

The Porkolab: all of our agents are busy helping other customers

2011-06-29T16:23:00.000-07:00

You've all called the "customer service" number for some faceless monolith like AT&T or Comcast and heard, over and over, "all our agents are busy helping other customers". The useful content - the denotative significance, as semanticists would say (though it might take them a couple of goes, especially if they've already started on the semiotic cocktails) - is "so you're just going to have to hang on for another hour or two". But what else is it saying? Why did they choose this rather than just stopping at "all our our agents are busy right now"?

The answer is of course that it is subtle psychological manipulation. If they just said "all our agents are busy", you could reasonably get cross after the third full cycle of hold music, asking yourself why they don't hire more agents. But because they say "helping other customers", there's an instinctive reaction - even amongst cynical uber-rationalist grouches such as myself - that this wouldn't quite be cricket. The subtext is "the other people they're helping are just as important as you are, you have no right to expect that you should be served any quicker". Clever, huh? Instead of being mad at Comcast for being such cheapskates and not hiring enough people, you feel guilty because you were putting yourself above your fellow man.

I'm sure they pay psychologists a lot of money to come up with this. It's amazing how easy it is to manipulate people's feelings without them realising - there are various recent books about this, though offhand I can't remember any of their titles.

Another good one is "your call is important to us". I don't actually care whether my call is important to them - I just care that it's important to me, because otherwise I would long since have given up listening to cheesy electronic renditions of classical music. But it really does work - even though you know it's a blatant lie, and the only reason your call has the slightest importance is because of the remote chance that you might transfer your custom elsewhere, it still takes a little of the edge off your impatience and anger.

Then there's "please listen carefully, because our menu has changed". Without knowing you personally, how can they possibly know whether it has changed since the last time you were obliged to take your cardiac health in your hands and call them? Nearly always, you can short-circuit all of the "press 1 to pay your bill, press 2 to order a new service, press 872344100000019461111 to complain about your existing service" by just banging away on the 0 button - guess how I know. But the subtext here is that not listening carefully would be on a par with conspicuously keeping your iPod earphones on during the cabin safety announcement. Not only would you be imperilling your own safety, but you would also be acting irresponsibly to society at large. You would be a Bad Person.

You may wonder how they decide exactly what subtle implicit psychological lies they decide to use. Do they have batteries of simulated users, with blood pressure monitors and positron scanners? Probably. I was inspired to write this partly by having to deal with Comcast, but also by a piece of business spam I got today (you know the kind of thing, very professional-looking emails inviting you to vitally important conferences about the latest developments in HR). It was from someone calling themselves "Jayme Porkolab". In British English, a "porkie" is rhyming slang for a lie (via "porky pie"). I'm sure the "porkolab" is just the place where they try out all these messages.

"Hey, we're getting a lot of complaints from Islamabad about people being rude to the call centre operators. I think we need some new decoy messages."

"OK, let's get on to the Porkolab and see what they can come up with."

Comcast - such awful customer service that words fail me

2011-06-28T17:39:00.000-07:00

So yesterday, while I was eating breakfast, my Internet service went down. The lights on the cable modem weren't blinking, and indeed nothing worked. It was time to go the office anyway, so I figured they would have fixed it by the time I got home.

But they hadn't. Still nothing worked. After the usual "press 1 for this" hell, and a ten minute wait, I finally got to talk to someone. "Oh yes," he said, "we discovered a signal leak back in February, and you hadn't made a service appointment, so we turned off the service."

Hang on... "Did you tell me about the problem?" "Oh, we probably left something on the door."

"And what about reminders, like on each of the five bills I've paid in the meantime, or an email?" "Yeah, that would probably be a good idea, but we don't do that."

"OK, so now I've made a service appointment, can you reconnect the service?" "Oh no, we have to wait for the technician to inspect it".

So... they may or may not have told me, for sure I never saw anything. Then they sit on their hands for five months, and then cut off the service without warning. And this passes for their idea of customer service!

The service guy did come on time, though he called the wrong number to warn me he was on his way! And guess what... his conclusion was that a different service guy needed to come, maybe in a day or two, maybe not. And no, of course they can't restore the service even though it was working fine for five months with whatever fault exists.

Comcast is outrageously expensive, the only thing that stops me switching right away to AT&T is that I know for sure that their customer service is even worse. Sigh.

Cross Country to Alturas

2011-06-26T23:59:00.000-07:00

About a year ago I finally decided to get serious about my Commercial Pilot's License (CPL). Not that I have any ambitions for a second career as a pilot, but with well over 1000 hours of flying under my belt it just seemed like an appropriate thing to do.

Getting a CPL involves a checkride with an examiner, a written test, and a bunch of experience requirements, such as a minimum of 250 hours of flying. Most of the training goes towards the checkride. For single-engine planes, this requires a bunch of new things which don't form part of the Private Pilot checkride, specifically some not-quite-aerobatic maneuvers: chandelles, lazy-eights, eights-on-pylons.These are actually pretty hard to fly to the required standard, but after struggling with them for a while I can begin to see some point to them: they definitely do make for a finer touch in aircraft handling.

One of the experience requirements is a long cross-country flight, at least 300 miles round trip with the furthest point at least 250 miles away. Oh, those are nautical miles, of 6080 feet or about 15% more than the everyday statute mile. Why flying uses nautical rather than statute miles - and the corresponding unit of speed, knots (nautical miles per hour) rather than mph - is lost in the mists of history. And not all the time, by the way - some things, such as visibility requirements, are expressed in statute miles. More on that later.

There are a bunch of other requirements associated with this flight, such that, although I have many flights of this length, none of them qualifies. There have to be at least three landings, it has to be day VFR, and it has to be solo - nobody else on board, not even a non-pilot passenger. I suppose this is intended as a test of flight planning and navigation skills. The FAA, or the part that deals with "airman certification" anyway, doesn't seem to have heard of GPS, and doesn't know that nowadays all you need to do is enter the code for the destination airport and follow the resulting magenta line on the screen. Well, almost - it's as well to take things like mountains and restricted airspace into consideration, and to make sure you have enough fuel. So anyway... in the preparation for my checkride, I needed to make such a flight.

There are lots of places I could go. The LA area is over 250 miles, so a round trip to say Santa Monica would qualify, or to southern Oregon, or to somewhere in Nevada like Elko. I've done all of those, though. It would be nice to do something I've never done before. Looking at my wall-map of California, with pins for the airports I've been to, showed one big pin-free expanse, the north-east of the state. There a very few towns and it's not even on the way to anywhere. Alturas, the north-eastern most town in California, is 251 miles from Palo Alto, so just qualifies.

A straight line from Palo Alto to Alturas passes overhead Livermore, east of Mount Diablo, overhead Sacramento and Grass Valley, then across 8-9000 foot mountains to Quincy. The mountains are gradually replaced by high plains at around 4000' with the occasional pointy story-book volcanic peak. You can't fly in quite a straight line because of the TFR for the pilotless aircraft testing centre at Chico, so I flew a slight dogleg to the south which also kept me out of the big military training areas in the region. The easy way to do this is to find a waypoint in about the right place for the desired route. My route ended up as KPAO-SUNOL-HAGAN (the waypoint in question)-KAAT (Alturas).

The flight northwards was uneventful but beautiful. Crossing the mountains is always a bit scary - there really aren't too many good options if the engine stops. Luckily it didn't. I flew just to the south of Quincy airport. That's a scary place too - I went there one Saturday when I was on my own. It's the only airport I know that has mountains in the traffic pattern! You can't fly left traffic at approach speed, if you tried you'd quite literally hit a mountain.

Past Quincy, there was really almost nothing. I picked out Susanville to my right. There were just vast, green grassy plains, a few lakes, the very occasional small settlement and a few roads. For nearly a hundred miles I didn't pass within ten miles of a town. The last part of the flight, over the high plains, was very bumpy, too much for the autopilot, so I hand flew. Finally Alturas came into sight. It's a tiny place and the runways are narrow, so it took a while to find the airport. Needless to say I was the only person flying.

It was with great relief that I pulled up to the fuel pump and stopped. It had been an enjoyable flight but it's a long time to sit in the same position. The airport was completely deserted - a sign gave the number to call for fuel, and it took about 15 minutes for the fueller - who is also the county building inspector - to show up. Later the airport manager showed up too. I ended up spending nearly an hour on the ground, which allowed me to witness the arrival of another airplane. It's a quiet place, and beautiful in a Swiss, mountainous kind of way.

It would have been nice to walk into the town, which is only about a mile away, but it would all have been closed by then and anyway I wanted to get back before dark. Alturas would make a nice destination for a quiet weekend, except that unfortunately there is no car rental. Without a car there is really not much you could do, and as far as I can tell the nearest are at Klamath Falls, over 60 miles away.

So after saying my farewells - it's hard to imagine I'll ever go back, although it's a pretty place - I set off on the next leg. I'd decided to hop over the mountain to Cedarville. It's only about 30 miles but the Warner Mountains are in the way, ascending to 9000 feet, so the whole flight is spent first climbing then descending. Cedarville is an even tinier place, population 514 - the kind of place you'd retreat to when you just couldn't stand the bustle and crowds of Alturas (pop 2827). It's in the centre of a long narrow valley mainly occupied by three large alkali dry lakes, imaginatively called the Lower, Middle and Upper Alkali Lakes. Unsurprisingly, the airport was deserted, the narrow runway further limited by high weeds at the edges. I turned straight around and set off on my next leg, to Susanville.

I chose to fly down the eastern side of the mountains, just for a change of view. At the end of June there was still plenty of snow above 8000 feet. Once out of the valley I quite literally saw no human presence until I was close to Susanville, a hundred miles south.

Susanville has a huge prison, like so many small, remote California towns. California of course has the highest per capita prison population in the world, higher than China or Soviet Russia, thanks to some seriously misguided right-wing politics and the even more misguided "war on drugs". The state is bankrupt and can't afford to run even the most basic services, but is still not willing to reduce its prison population. But that's not really a topic for this post.

The prison did however make a convenient visual aiming point. When I landed there were still a few people around the airport, and a very friendly dog. But I didn't linger - I was getting hungry as well as tired.

The rest of the flight was uneventful - back across the slightly scary mountains, then over Sacramento, to the left of Mount Diablo, and so home. The landing was spectacularly beautiful - by good luck I landed exactly as the sun set, disappearing under the horizon as I descended on short final.

When not flying, I'm working on my written - actually a computer based multiple-choice test. Fortunately, all the possible questions are published by the FAA, and assembled into a book called Gleim, which also has (one hopes) the correct answers and the reasons behind them. The majority of the questions are perfectly reasonable, having to do with things like the airspace rules and how to read charts. But others are ghosts of a former age. There is not a single question about the use of GPS, even though probably 90% or more of long flights are made using it. But there are dozens of questions on a nearly-obsolete device called the "Automatic Direction Finder", a miracle of 1940s vacuum-tube electronics. The operational side of the FAA is busy closing down the corresponding radio beacons as fast as it can. New planes (sold in the US anyway - the rest of the world is a different story) haven't had ADF equipment in them for well over a decade. Mine had one when it was new, in 1980, but I had it removed when it developed a fault about five years ago.

Not only is the ADF obsolete, but the questions are about ways of using it that nobody has ever done in flight, complicated trigonometric exercises to do with angles of intercept. Flying a plane looks easy enough but it actually takes a lot of concentration. The idea of doing complicated mental math at the same time is as absurd as it would be to do the crossword. (Airline pilots do of course, but they have at least one extra pilot and three autopilots which they are obliged to use in cruise. And they are not constantly looking for reasonable places to land if the engine stops, since they have several of them).

My pilot friend Bill thinks that a lot of the regulations, and the questions that go with them, are just a thinly-veiled intelligence test. For example, there are three definitions of "night" (sunset, civil twilight which is about 30 minutes later, and one hour after sunset). There are many things you must or must not do at night, but which definition applies to which activity is pretty much random and just something you have to learn off by heart. The same applies to the various sources of aviation weather information. These exist both on the ground (via phone and now the Internet) and in the air, from various different places. Each one provides slightly different permutations of all the available information, without rhyme or reason. There are of course numerous questions on these. And, as you probably guessed, there are no questions at all either on Internet data sources, or on in-flight satellite based weather - which between them probably account for 90% of what pilots actually use nowadays.

The written will be my next step, in another few weeks - whatever one may think of it, it just has to be done. I don't expect it to be as enjoyable as my cross country flight to Alturas. More pictures from the trip here.

Worst-ever Dining Experiences: #2, Mouleydier, France

2011-06-23T17:12:00.000-07:00

One of the nice things about eating out in France is that pretty much any restaurant you go to will serve an enjoyable meal, corresponding to the price you pay and sometimes even a lot better. They have to - the French take their food very seriously, and a restaurant that disappoints its customers will see very little repeat business. It follows that tourist traps are an exception, since they don't really depend on repeats. But even in Paris I can only think of one meal where I've been disappointed - often we've just dropped into a restaurant round the corner from where we're staying because it looks nice, and been delighted with the food. Of course it could be because Paris just has such a fantastic ambience anyway - but I think there's more to it than that.

The same is true outside Paris. Even when I first used to visit France, before my French was up to very much, I remember some spectacularly good dining experiences. We'll gloss over the restaurant in a small town in Brittany that was - I swear - closed for lunch, preferring to remember a hotel in the same town where we had an assiette de fruits de mer that took all evening to eat, there was just so much wonderful stuff.

But of course there are always exceptions. This one happened a very long time ago when my daughter - now in her late 20s and a mother herself - was just a bump. We went on a last-time-as-just-the-two-of-us holiday to the Dordogne, renting a cottage outside the town of Bergerac for a couple of weeks. It was a wonderful holiday, and filled with equally wonderful meals in the picturesque, ancient towns that litter the area. I've never had a chance to go back to the area, but I loved it.

Some way into the holiday, buoyed by the confidence of having eaten so well in restaurants chosen fairly randomly, we decided to eat at a place we'd noticed in the nearby village of Mouleydier. It looked nice enough, nestled in a corner near the station. Probably it was called Cafe de la Gare - it seems to have disappeared now, for which we should be thankful.

We should have been suspicious straight away. Instead of the customary buzz and crowded tables, the place was deserted. It felt chilly, despite the pleasant June weather. It was nothing fancy, with cheap checked table cloths, but in itself that doesn't mean a thing. The Cafe de Commerce in Geneva - an old favourite from when I used to visit there regularly - was just like that. Their speciality was delicious lake trout. The portions seemed a bit small, but when you'd finished the first one, they brought you another.

And worse, we were the only people there, though another group did come in later. But that was it. The service was surly, the waitress (most likely the proprietress) was sloppily dressed and didn't even seem especially clean. She certainly didn't seem to be taking any pride in anything she did. I've mercifully forgotten the main course, though I do remember that neither of us found it very appetising.

What does remain in my memory is the cheese board. France is a country that famously has hundreds of cheeses, all of them delicious even if some are a bit of an acquired taste. My first ever business trip to France took me to Annecy, where a restaurant chosen completely at random served an extraordinary cheese called Explorateur. It's about 99% fat (or so it seems), extraordinarily creamy and with a delicious nutty taste. Sadly, it's essentially unobtainable in the US - on the rare occasions when you do see it, it's invariably past its best. Creamy cheeses like that neither last long nor travel well. But in France it can be truly ecstatic.

The Mouleydier restaurant didn't have Explorateur, nor any other of the hundreds of delicious French cheeses. They had a Brie of some kind, dried up to a crusty yellow on the outside and definitely inedible. They had a couple of dried up goat cheeses. Soft, creamy goat cheeses with just a faint nutty taste are one thing. Well-matured ones are quite another, and most definitely an acquired taste. The theory, so I'm told, is that just a single small piece on the tip of a knife will taste so strong that it will seem like you ate a whole cheese. Back in the times when the art of eating was to make a few vegetables and a scrap of meat seem like a feast, this mattered a lot. But well-matured and then dried out is never going to be a good combination.

The final cheese, and the only one that could conceivably be eaten, was some kind of Tôme, a fairly hard cheese eaten like Cheddar. I can still remember that there was a black wax rind, which had at least partly protected it from drying out completely. We dared to take a piece of this each, but even after cutting away the driest of the exterior, it was old and dry and not up to much.

By the time we left, we were once again the only people there - the group of three or four guys who had come in after us had evidently had enough of a good thing, and already left.

The following night we went back to the excellent restaurant in the centre of Beaumont, a town a few kilometres away. Mouleydier remains the only place in France where I've been offered - in a proper restaurant at least - a truly awful meal. (The story of the fast food cafe in Entrevaux will have to wait for another time).

Worst-ever Dining Experiences: #1, Janos, Tucson - Pretentious to die for

2011-06-22T14:54:00.000-07:00

I chose the title carefully... not necessarily worst-ever meals, but considering the whole experience (including in some cases the extended period of discomfort afterwards).

So... we visited Tucson, Arizona. Our main reason was to go to an Indian Rodeo in Sells on the Tohone O'Odham reservation, which was quite an experience. But that's not important right now. On our first night, we asked the hotel to recommend a restaurant. We had trouble finding it, because it was nowhere near the town centre, but in the hills quite a way to the north in a fancy golf resort hotel kind of place.

There's one advantage to the location... Tucson is quite a pleasant small-ish town, but one of the main trunk rail lines goes literally through the middle of it, with level crossings every few hundred yards. This means that several times per hour, a mile-long freight train passes through, horn blaring, not just audible but loud through the whole city. All night long. Amazingly, you get used to it.

Anyway, it was obvious as soon as we arrived that this was a very pretentious place. It just had that air about it. We both get easily irritated by places whose pretentiousness greatly exceeds their quality, and this place had it stamped all over. We took our huge, pretentious seats, in the pretentiously decorated dining room, led by the pretentious hostess.

As we suspected, the meal was so-so. Nothing wrong with it, but nothing special either. For half the price and a tenth the pretentiousness, we would probably have quite enjoyed it. And of course the wines were over-priced. A dead giveaway for a restaurant that takes itself much too seriously is that the wines are priced at about double what you'd reasonably expect. I can only suppose that people who like pretentious places want to pay more for wine, I guess it makes them feel superior or something.

What elevates this place from merely annoying to the short, distinguished list of The Best of the Worst, though, was the total contrast between the airs and graces they gave themselves, and the actual quality of the service. Our waiter was a kid, maybe 19, who had clearly received little or no training. Probably he'd learned on the job at Denny's or some such, then managed to talk his way into a job at the restaurant. The service would have been perfectly acceptable, if we'd been at Denny's and paying Denny's prices. He had no idea how to serve and kept getting things wrong.

One thing which drives my wife absolutely nuts is when she's still eating and the server says "You still workin' on that?" It would annoy her even in Denny's, but frankly eating a meal in Denny's is more work than pleasure so it may make sense. At a fancy restaurant, even a legitimately fancy one, it is just so inappropriate. Maybe a Jeeves-like "Is Madam still profiting from the enjoyment of her repast?" would be better, but simply observing that she has not put her knife and fork down beside each other, and is still chewing, and saying nothing at all, would be even better. You can guess, of course, that our Denny's waiter came by not just once but several times to ask, "You still workin' on that?".

Finally I ordered dessert, I can't really imagine why since by then we were thoroughly fed up with the whole place. The waiter served it, but didn't bring anything to eat it with. After waiting a while - I wasn't in a hurry anyway - I finally helped myself from the next table. Nobody even noticed.

We didn't leave a tip. That doesn't happen often. We were just relieved to escape the overwhelming pretentiousness combined with total ineptitude.

The next day was delightfully different. We had lunch at a diner in the middle of nowhere on the way to Sells, for less than the tip would have been the night before. And in the evening we went to a very unpretentious restaurant near the university - it pains me that I can't seem to find it now, and it probably isn't there any more. The tables were small, the waitress was friendly, and the food was excellent. What a contrast!

Jeppesen Customer Service (not!)

2011-06-21T20:28:00.000-07:00

One of the first things I did when I bought my plane was to install a then state-of-the-art GPS navigator, a Garmin GNS530. This complemented the smaller GNS430 which was already there, albeit lacking some necessary approvals. The data for these comes on little data cartridges, slightly bigger than an SD card and completely proprietary to Garmin. I signed up with Jeppesen, the universal and only provider of aviation navigation service, to send me updated cartridges every month. Until about a year ago, this worked perfectly. I would take the new cards to my plane, swapping them for the old ones which I then sent back to Jepp. Once a year I sent them a check for quite a lot of money, and everything was flawless.

A couple of years ago, Jepp was acquired by Boeing. I presume that the new owner sent in the usual bean-counters who proceeded to look for "economies", of course at the expense of the customer, whose only role after all is to pay the bills.

The first sign that things weren't right came with last year's bill. I've always had a substantial discount for having two GPSs in the plane, which makes it only about 25% more than a single one. This had been forgotten, and the bill was for twice the single-unit price. I called, listening to hold music for about half-an-hour, interrupted occasionally by the usual pathetic excuse about "increased call volume" - the latter caused, of course, by the fact that they had messed up their complete order management system. Finally I spoke to someone who apologised, took my credit card number, and told me it was fixed.

I got various bills and reminders, which I ignored, until eventually the supply of new data cartridges dried up. I called again. This time I got someone very unhelpful who accused me of not paying my bill. Luckily I'd kept notes of the previous conversation, and eventually all was restored.

That was last year. This year they had a new trick, aimed no doubt at saving a few more dimes. They have suspended the data cartridge service altogether, replacing it with an online service where you update your own cartridges. They sent me a USB thingy to plug the cartridges into, and told me to call them for more information. It actually took several calls to get the service set up. Tonight, I finally tried to download into my cartridges. One worked fine. But then, I only had the right to a single cartridge! So we're back to where we were before.

I'm sure that all the calls to try and straighten things out have already wiped out the savings from not sending me the data cards. So I have to spend more time, not to mention hours on the phone to Jeppesen, and it costs them more money. If there was an alternative, I'd already be there - the disadvantage of monopolies.

But I'm sure that somewhere there's an accountant who's jolly pleased with himself.

Telephony, the definitive work

2011-06-04T16:42:00.000-07:00

One of the treasures of my youth was the Harold Hill public library, which I should really write about some time. For some unknowable reason, their collection included the two-volume definitive work on the UK telephone system, Atkinson's Telephony, published in 1949. I've always been fascinated by complex electro-mechanical devices, and I borrowed these books dozens of times. Of course that may be why I'm fascinated by them.

Telephones are run by computers now, but back then everything was done with relays and stepping switches. A telephone exchange was a vast hall filled with them. The noise was just unimaginable as the switches followed the subscribers' dial pulses (telephone users have always been called subscribers) then rocketted off on their own to look for the next free switch, later clattering back to their rest position at the end of a call.

The automatic telephone system was invented, improbably, by an undertaker called Strowger. The story has it that the telephone operator in his small town just happened to be the wife of his rival. In those pre-automation times, people would call the operator and say "connect me to the undertaker". You can guess who got the business. The telephone dial and the clever two-dimensional stepping switch that interpreted it were his invention, and the system was forever after called the "Strowger System".

Manual telephone exchanges were still very much present when I was a teenager. I had a friend who lived in the next town, Brentwood, which was one of the very last places to be converted to automatic operation, in the 1970s. Even manual exchanges involved some complex electro-mechanical gadgetry, though, to detect when the user picked up their phone, make it ring and so on. Atkinson deals at length with this, and the problems of interconnecting manual and automatic systems. Operators are always referred to as "she" - not sexist, in itself anyway, just a reflection of the fact that all telephone operators were women back then. And not outsourced to Bangalore either - every telephone exchange, even automatic ones, had a hall full of operator positions to handle long-distance calls and other odd cases.

About half of the first volume deals with the basics - cable design, network organisation, and of course the relay. A relay is just an electro-mechanical switch, a bunch of contacts operated by a coil when electricity is applied. They're still used extensively today - a car has dozens of them - but in the telephone system there were millions of them, implementing complex logic functions that today would be a few dozen lines of C code. The basic idea is simple enough but the realities of the telephone system called for great subtlety in their design. There were relays that were quick to engage but slow to release, and vice versa, relays specially designed to respond quickly to dial pulses - all done by varying the windings, the magnetic arrangements and the contacts. Their design is a lost art now.

The rest of the first volume deals with the circuits and arrangements for manual exchanges. The real excitement is in the second volume, the circuits for automatic exchanges. Just how do you build an electro-mechanical telephone exchange? The basic concept seems simple - the pulses generated by a telephone dial operate Strowger switches, until all digits are dialled and the final switch - called a final selector because it works very differently from all the others - has made a connection to the called number. But actually making it all work, dealing with all the real-life complications, was a phenomenal achievement. It's not surprising that the first ever computers - built to crack German codes during World War II - were designed by engineers from the Post Office Research Laboratory. They were the only people to have designed circuits of this level of logical complexity.

It's worth commenting on the difference in the scale of the phone system in those days. The book uses the city of Derby as an example. Its population then was over 100,000 people. The telephone exchange had a maximum capacity of 4000 lines, or one for every 25 people. In 1949, ordinary people never had their own phone. Businesses did, and a small minority of better off people would have a phone at home. For ordinary people working- and middle-class people, there were phone boxes. (The book includes a lengthy section on the classic British phone box design, including its weight - being made largely of cast iron, it weighed 13 cwt, or about 660 kg). My parents didn't have a phone at home until about 1965.

There are lengthy chapters on the design of small rural exchanges, partly I suppose because they were small enough that the book could describe them in complete detail - each of the hundreds of selectors and relays. There are complete circuit diagrams, and text to explain the function of every last relay contact and selector. It is the best technical design documentation I have ever seen, bar none.

The smallest of the rural exchanges, designed to occupy a small brick building somewhere on the outskirts of a village, had just 100 lines. It replaced an operator in the back of the post office working a manual plug-and-socket switchboard, disproportionately expensive and only available during the day time. At night, the doctor would be plugged straight through to the main circuit, and everyone else would just have to wait until morning. There were always telegrams for anything urgent that might arise. (My wife tells the story of her father using just such a rural manual exchange, fuming as he waited for the operator, grumbling "She's knitting, I just know it, she's finishing her row before she answers". And he was probably right).

Sadly, the book was never updated to discuss the nec plus ultra of electromechanical telephony, Subscriber Trunk Dialling (STD). When it was written, trunk calls - anything further than the next town - were handled manually. You would dial 0 for the operator, and say in your best BBC voice, "Derby 6543 please". If there was a circuit available you'd be connected. More likely, she would call you back later when a circuit became free.

The first STD installation was in 1958, and the first public long-distance call was made by the Queen, from Bristol. It came to my home town in 1964, I can still remember the presentation in the local Post Office. It made no practical difference to us, nor to most people - the furthest we ever called was my aunt in London. Although calls were automatic, they were still ferociously expensive.

The implementation of STD relied on a magnificent piece of equipment called a register/translator, or its "marketing" name of GRACE. Each unit was a box about two feet square, packed full of relays and miniature uniselectors. As the subscriber dialled, each digit was remembered on a mechanical switch. As soon as the first three digits had been collected - the STD code, or as we would now call it, the area code - it could start translating them into the actual routing code which would switch the call through the trunk network to its destination. Once that was done, the remainder of the digits would be replayed, one by one, from the switches that stored them. Sadly there seems to be no detailed technical description of these systems still available. I was told by someone who joined the Post Office in the seventies that the schematics, spread out, occupied the whole floor of a large office. It was truly a tour de force of the available technology at the time.

It was thanks to my curiosity about the design of trunk network that a friend and I became "phreaks" in the 1960s, long before we knew the term existed. But that is a story for another time. For now, back to reading Atkinson... there are a couple of details of the UAX14 exchange design that I'm not clear about...

Jottings from Japan

2011-05-27T19:52:00.000-07:00

(I wrote this back in November, on my way back from Japan, but for some reason never got round to finishing and posting it). More pictures here.

My second trip to Japan this year, for the usual reason: trying to spend some time with my wife as she zooms around the world, bouncing off of various cities such as (in this case) Belfast, Seoul and Paris, in that order. I'm burning up my collection of airmiles and so (poor me) the only way I could get the dates I wanted for this trip was First Class on Singapore Airlines. Summary: not bad! Unfortunately on the LA-Tokyo route, they aren't yet running the Airbus 380 - I would have loved to have tried that in First. But the 747 wasn't bad either. Though there are things they do which, in my opinion, create unnecessary stress for the passenger- such as having to choose between Dom Perignon and Krug.

I tried both, alternating every time I had a top-up. The Dom, I can report, was much lighter than the Krug - surprising since compared to lesser champagnes Dom has a lot of body. But it was almost spring-like compared to the Krug's biscuity heaviness. Both were pretty good though, as was the Griotte Chambertin I had with the main course. My only disappointment with the flight was entirely my own fault - I left a half-finished book, which I was thoroughly enjoying, on the plane.

I stayed, as usual, at the New Otani. This is a wonderful hotel, in my opinion the most agreeable in Tokyo. It's the only hotel in the world to have its own private 16th century samurai garden, which guests (and indeed anyone else) can wander around freely, or admire over breakfast from the restaurant. The story is simple enough: in 1964 Japan hosted the Olympic Games, but didn't have a large, modern, western-style hotel. Mr Otani, a successful industrialist whose statue can be found tucked away in a corner of the samurai garden, was asked to help. But where to find the land in super-crowded Tokyo? In addition to the Imperial Palace, Tokyo has a second palace used for visiting heads of state and such. What could be simpler than to use some of its vast grounds for the hotel? And somehow, Mr Otani managed to get the part containing the samurai garden. Incidentally the New Otani appears in the James Bond movie You Only Live Twice as the headquarters of the evil empire - construction was complete but the hotel had not yet opened when the film was made in 1964.

The first day I worked, visiting our Japanese reseller, a typical Japanese "fireside chat" type of meeting. Then in the evening I traveled down to Haneda, Tokyo's domestic airport, to meet my wife. Well, it used to be Tokyo's domestic airport. The Japanese government has finally opened it to international flights, as an alternative to the super-inconvenient Narita. Now there's a vast new international terminal, complete with its own subway station. Quite a change from a couple of years ago when I flew to Korea from Haneda - the terminal was a tin shed somewhere on the outskirts of the airport, with only an infrequent shuttle bus service.

The next day we took the train to Hakone, a journey I've already described here. But first we went to Shimbashi for sushi, to the same kaiten sushi restaurant we've been visiting now for nearly 30 years! It's a low-price operation for the salarymen in the area, but because the turnover is so high, the fish is wonderfully fresh.

Hakone was wonderful, we were incredibly lucky with the weather - the previous day had been overcast and damp, today was bright sunshine. We stayed at a traditional Japnaese ryokan, selected through the wonderful service of japaneseguesthouses.com. The bath was at just the perfect temperature, which means just bearable, the kaiseki dinner was excellent. And in the morning, looking closely at the prints which lined all the corridors, we realised that they were original Hiroshige - which means worth a small fortune, tucked away inconspicuously in a modest inn.

Hakone can be visited as a day-trip from Tokyo, and everything is organised for just that. The train ticket includes all of the various means of transportation required - a bus along a twisty mountain road to Lake Ashi (Ashinoko), a quick walk to find the ancient cedar avenue which was once the main road from Tokyo to Kyoto, then a lake steamer disguised - improbably - as a pirate ship, complete with swashbuckling pirate captain, photographer in tow. In the 25-minute journey the pirate captain walks around the whole boat, and just before you dock he returns with the pictures for sale.

One reason why Hakone is so popular is the fantastic views of Fuji-san, first from the lake and later from the cable-car that crosses the mountain. There's a change in the middle, at one of the surreal, hellish, sulphur-laden volcanic springs that are found only in Japan. But the journey isn't over yet - at the bottom of the cable car is a funicular, and then a tiny train that twists and turns around corners that even a car would struggle with, back to the mainline train station. Along the way we visited the Hakone Art Museum. There were two things to see there: the wonderful autumn foliage on the maple trees that fill the garden, and the throngs of photographers, all with the very latest cameras and extraordinary lenses, taking pictures of the wonderful autumn foliage.

On Saturday, we had the great good fortune to go along to a Japanese family event, the annual sushi party of the company where I used to work. It's difficult for westerners to get to meet Japanese families, and we're really very lucky to have such good friends who will invite us to something like this. Everyone was there, colleagues, spouses and their children. The food was truly amazing, and went on forever, and the company was wonderful too. I'm incredibly lucky to have such good friends in Japan.

I spent Sunday on my own - my wife had to return to another meeting in Europe. I went to Ueno Park and visited the Science Museum, recalling my first ever visit to Japan in 1982. It has changed a lot - for one thing it has doubled in size, and it is much more modern and visitor-friendly. Back then it was just a lot of dusty display cases. But one thing that hasn't changed is that there is absolutely no allowance whatsoever for non-Japanese visitors. Apart from a few signs in the hallways, nothing is labeled in English. Now - unlike then - I can just about read the signs, though it takes a long while. But it is reasonable enough when you consider how few foreign tourists there are in Japan, apart from Kyoto. A while back we went to the Museum of Modern Art, in another corner of Ueno Park, and
we saw just one other foreign visitor there. Of course it's self perpetuating - Japan apparently does want to attract foreign tourists but for as long as you need to be able to speak and read some Japanese to survive, it just isn't going to happen.

Finally on Monday it was time to return. As usual I took the bus out to the airport. At the first-class check-in desk the lady was very concerned - never a good sign, especially when you're using an airmiles ticket. There are just so many things to go wrong. She walked away, talked to colleagues with a worried look on her face, made a phone call. I feared the worst, beginning to envisage a hasty one-way ticket purchase and the uncomfortable return journey in economy. Finally she turned back to me.

"Mr Harper... did you leave a book on the aircraft when you arrived?" Well yes, I did.

"I'm very sorry, the book is in our office, would you mind waiting a few minutes while my colleague collects it?"

This could only happen in Japan! Once I left a valuable (to me) notebook on Swissair (yes, it was a long time ago), in Business Class. When I called, they said, "Oh, we never keep anything, it will have been thrown away." But Singapore Airlines - in Japan - even kept a $5 paperback for me, unasked.

It got better, too. As I was waiting in the emigration line, after security, someone came running up to me, apologising - I'd left my laptop behind at security... and they had to run after me to return it! If I'd done that at Heathrow it would already have been on E-Bay!

The return journey on Singapore in First was as good as you'd expect - there's really nothing to be said.

My Very First Linux System

2011-05-20T14:22:00.000-07:00

Ubuntu rocks. For more, read on...

About a year ago, I converted my home desktop machine from Windows XP to Windows 7, hoping to improve its dire performance problems. That was not a success. Not only was the Windows 7 performance if anything worse than XP, but a number of other things stopped working as well. Based on my experience I cannot imagine why anyone would ever choose to run W7 instead of XP. W7 is full of poorly conceived "security" misfeatures, making it painful to use and even harder to manage. For example, it's impossible to save an image from a webpage to local storage. And I never did get the Cisco VPN to work.

I've been thinking for a while that it would be a good idea to have a Linux server at home. I've switched just about all my non-embedded stuff at work from Windows and Visual Studio to run on Linux systems. VS is actually quite decent but things need to end up on Linux anyway and keeping them dual-platform is too much trouble.

When I first used Unix, Linux hadn't even been thought of. I equipped my network architecture team at DEC with workstations in about 1988. Of course at DEC we didn't have the System Whose Name Must Remain Unspoken, we had Ultrix - which was just BSD Unix. And Linux is both our development and our production environment at work. Running Linux as a user is no issue, but I was worried about the sys admin aspect of the thing. My perception of Linux was of constantly having to build things from source, of irrenconcilable version conflicts and unfindable drivers and bits of operating system. So I kept putting it off.

Then a friend told me about a system they had just bought. Six-core 3.2 GHz AMD CPU, 8 GBytes of memory... all for under $500. That sounded too good to be true, but I went to the Tiger Direct web site and there it was. The temptation was too strong. It's difficult to buy a PC-type system without getting - and paying for - Windows, whether you want it or not. NewEgg for example won't sell you a barebones system like this.

A few days later it all showed up in several large packages. I've always bought assembled systems before, but this one came as a collection of parts - case, power supply, motherboard, CPU, disk, memory, and a few other bits and pieces. I was apprehensive about putting it all together but in fact it was easy. The only trickly part was getting the cheap-and-nasty metal plate around all the motherboard conneectors to fit propely. Functionally speaking, this isn't strictly necessary, but it would look funny without it.

Actually the system came without memory, because it was backordered, and also I'd ordered the wrong part for the CD/DVD drive as well. So it took a couple of trips to Fry's to get everything required. I'd also bought an Ubuntu book which conveniently had a CD in the back, avoiding the problem of downloading the distro and burning a CD - for which I'm not sure I have the software.

And then came the miraculous part. I put the CD in the drive and booted the machine. Ubuntu came up, and with a couple of clicks started to install itself. Nothing could ever have been easier! Once my new system was running, there were numerous things I wanted to install. For every single one of them, a single command and a short wait downloaded it, installed it and got it running. No scratching round trying to find ancient CDs, no complicated licence agreements, no complex configuration screens. And better yet, no Windows registry! Everything just installed and run.

I'd never used Gnome or any other Linux GUI before. Up to now, all my Linux usage has been from the command line - which I rather like, actually. I had low expectations of Gnome but I couldn't have been more wrong. It does everything Windows does, and without all the irritating bogus security misfeatures. I was especially pleasantly surprised by OpenOffice, which can read and edit all my Word, Excel and Powerpoint files. And it's free.

There was only one thing that gave me problems. No surprise, it was the interconnection with my Windows system which, for now anyway, I have kept in place. There is a piece of software, Samba, that allows a Linux system to provide a Windows file server. Unfortunately it has to accomodate all of the superfluous complexity and options of Windows, so configuring it is a nightmare and I just couldn't get it to work. Finally a friend pointed me to SWAT which straightened it all out for me. A key item is that the accounts and passwords for Samba have absolutely nothing to do with any other accounts and passwords on the system. Connection in the other direction, accessing Windows files from my Linux system, is still distinctly shaky. It kind of works, sometimes, but keeps running into all those security misfeatures that made me dislike W7 so much in the first place. I have yet to get it so the Linux machine can write files on Windows, even using the adminsitrator account.

One thing I wanted to do was open up the machine for very limited outside access, so I can retrieve my files from it. This was amazingly easy - install Apache and it just works. Then a little configuration on my Linksys router, five minutes to create an account with dyndns.org, and now I can (with a password of course) access my files from anywhere.

I love my new Linux machine. Next step: move Windows to be a VM under Linux. More on that later...

Cisco's Pearl Harbor

2011-05-17T17:34:00.000-07:00

Even 70 years later, the Japanese raid on Pearl Harbor is an emotional topic for many Americans. But viewed dispassionately (maybe easier for me since I'm not an American, by birth anyway), it was a strategic masterstroke and a flawless piece of execution. Japan, a country with practically no natural resources, had created a vast Pacific empire. The US, reacting rather tardily, decided to slow them down by imposing an oil embargo which would be catastrophic for Japan's ambitions. What to do? The concentration of the US Pacific fleet at Pearl Harbor created a unique opportunity to destroy at a single stroke America's ability to enforce the embargo. It was not an easy thing to do - Hawaii is way out of land-based range for the aircraft of the time - but the raid was cleverly planned and rehearsed. Nobody can deny that it worked. Contrary to a common belief nowadays, it was not a "kami kaze" raid - the Japanese pilots and aircraft nearly all returned safely, ready to fly another day.

So, clever strategy, excellent execution. What could go wrong? Well, we all know what went wrong. Not surprisingly the US was seriously irritated (as we Brits would put it) by this devastating attack from a country that was not even officially at war - owing to the one execution failure, the incompetence of the Japanese ambassador. And over the following four years, and despite the complete destruction of the Pacific fleet, America fought back and of course eventually wiped the Japanese empire off the map.

I don't know the answer to the question, "X is to strategy as strategy is to tactics". I don't think we have a word for that - the kind of cosmic level above strategy - so let's call it "meta-strategy". While Pearl Harbor was strategically practically the only choice for Japan at the time, at the meta-strategic level it overlooked the fact that the US has such vast resources, both human and natural, that it simply cannot be beaten militarily. Until Pearl Harbor, Japan's empire was an annoyance to the US, a strategic threat, but not a reason for military action. Afterwards, the US had no choice but to react in the most devastating possible way - and was able to do so.

What does this have to do with Cisco? By the mid-2000s, it had the enviable position of total dominance in its market, but had nowhere else to go. Hence the famous misadventure into the consumer domain. But there was something much more adjacent to its existing empire in the enterprise - the computing side of the data center. There were a few truly visionary people inside the company who could see this and tried to lead them there. There were a few false starts, but finally the acquisition of Mario Mazzola's Nuevo Systems spin-in/out in 2006 gave them a credible product to address this market. Since then Cisco has turned it into a $1B product line.

What could be wrong with that? Brilliant strategy to get the company out of a tight corner, excellent execution. Now maybe you can see the parallel with Pearl Harbor. Because shipping a product to compete directly with the traditional computing providers was a direct attack on them, the commercial equivalent of bombing their Pacific fleet into shrapnel. Worse - and this is where the analogy breaks down - IBM and HP were not just competitors, but also major sales channels for Cisco.

HP has taken this very badly. They couldn't just walk way from their Cisco partnership, but it's clear that a decision was made at the highest level that Cisco was now the enemy. Five years later we are seeing the result of this, even to the extent of hiring away Cisco's top techincal architects - not to mention acquiring 3Com. Curiously, it was HP who destroyed my previous large-company employer, DEC, after Ken Olsen's disastrous "snake oil" speech and outright rejection of Unix. Looks like they'll get Cisco, too.

As for IBM, whatever nice things may have been said in public, they never got over being driven out of the networking business by Cisco in 1999. There has never been much love there, and Cisco's move into their very own territory was just one step too far.

I don't know what Japan could have done if it hadn't attacked Pearl Harbor, and I don't know what "adjacency" Cisco could credibly have attacked if it hadn't been the data center. But the meta-strategic mistake was the same in both cases. The US will come and get you if you annoy them enough - just ask Osama bin Laden. And so will IBM and HP.

Python, readline and insanity (mine)

2011-05-10T16:17:00.000-07:00

A bit of a technical post here. For the last week or two I've been writing some financial modelling software in Python, to see whether it's really possible to make serious money from a rather clever scheme that a friend of mine came up with. When I came to put the command line interface (CLI) together, I wanted something with all of the usual Unix CLI stuff - recall, editing, completion, help. The standard way to do this for C/C++ code is using the GNU readline library, which is what the Linux shell uses for example.

I recently got round to doing this for another piece of software, written in C++. There, though, I couldn't use readline. Readline is licensed under GPL, which means, essentially, that if you use it in your code, everything you write legally enters the public domain. Luckily there is a non-GPL near-equivalent, called editline. This provides all of the useful functionality of readline - which actually is about 5%, like most software 95% of what it can do is unknown and probably incomprehensible to 95% of the people who use it (just like Word or Excel).

I had no such constraint with my Python code. Python provides a module for interfacing to readline which makes the basics - command recall and editing - incredibly simple to use. Simply importing the module changes the behavior of the console input function raw_input to provide all this. Nothing could be simpler.

But I wanted more. Specifically, I wanted tab completion and also Cisco-like help, where typing '?' prompts with what to enter next. In C, completion is done using a callback, and you can define different completions for different functions - in particular, you can have one for tab and another for '?'. But the Python mapping provides only a single callback. More on that later.

The implementation looked simple. One function call (set_completer) to register the callback function, another (parse_and_bind) to tell readline how to handle the special characters. I found a good example and pretty much copied it. That's where the insanity comes in. The example, cut and pasted into a file, worked perfectly. My code didn't. Hitting tab did nothing at all. Putting diagnostic code into the completion handler produced noting. Even deliberate errors - like using unassigned variables - did nothing. My code simply wasn't being called. There followed a couple of increasingly desperate hours of cutting and pasting, copying code from the working example to the non-working code and vice versa... all to no avail. The example worked, my code didn't - even when they were identical!

I'd set aside an hour or so to get all this stuff done, and by now I was into the third hour and still getting nowhere. I was just a little frustrated. Finally, by application of the Sherlock Holmes Principle ("once you have eliminated the impossible, whatever remains, no matter how improbable, must be the truth") I realised what was happening.

The Python interpreter makes heavy use of exceptions, both for intended cases (e.g. key not in a dictionary) and for programing errors (e.g. referencing an undefined variable). Normally, an uncaught exception - from an error - ripples right out to the outermost function, where the interpreter prints a stack dump. You know not only that something bad happened, but where. Things get different when C code gets in the way. Python passes the exception down into the C code, but what happens to it there depends entirely on the C code. And in the case of the readline library, the C code just swallows the exception and says nothing. So if there's a bug in the completion handler, it just fails silently.

Once I realised this, the solution was obvious - put a catch-all handler in the outermost completion handler function. That tells what the error is, though it doesn't give a stack dump. Once I did that, I could see the errors and quickly had things running.

There was another problem though, which I alluded to earlier. Writing in C, you can have a distinct callback for tab and '?'. In Python, there's no way to do this, and no other way to find out which character was typed since it doesn't get put anywhere. There's simply no way to know which it was. I did find a patch to the Python readline module which made this possible, but I don't want to deal with a special version of the code. So my compromise is to treat them exactly the same. If either occurs as the first character of a field, it's treated as '?', and generates a helpful message telling you what's expected. Once you've typed anything else, what you get is completion.

So, the moral of this is, always put a catch-all exception handler in the outermost Python callback function, and life will be good.

It has a button to do WHAT?

2011-05-08T21:42:00.000-07:00

I recently set up a new computer at home to run Linux. That's another story, soon to be told. (It's an excellent idea, by the way). In the process of finding room for it, running cables and so on, I accidentally pulled on a cable and sent a bunch of stuff tumbling off the back of my bookcase. There it hung by its cables for several days. Since everything seemed to work, I didn't pay much attention.

Today I needed to print something. No printer. Hmm. In sorting out the mess of USB cables for the new mouse and keyboard, I'd unplugged it. Getting to the cable required untangling all the stuff which fell off a few days ago. No problem - just the cable modem and the WiFi router, and their associated cables. I restored everything to its place, and even tidied up a bit, feeling extremely virtuous.

It wasn't for a while that I noticed I no longer had an Internet connection. A quick glance at the cable modem showed that its usual array of blinking lights had been replaced by just one steady orange light. I looked around and everything seemed to be cabled correctly, so I power cycled the modem. The lights gradually lit up, then started blinking - success, I thought. But then suddenly they all went out, back to the status quo ante. Hmm, again.

I tried all the usual things, power cycling, unplugging, checking software. All the machines could still talk to each other, just not to the outside world. I even checked the television, in case there was a problem with the cable itself. It was fine.

At this point I was getting desperate. Of course, it could just happen that Comcast had an outage at the exact time that I sorted out my little mess. Nothing left but to give them a call - not something I was looking forward to, 20 minutes listening to hold music, get cut off, another 20 minutes, then an operator in Islamabad trying to pretend he's in the US... ouch, especially on a Sunday afternoon. Plus which, the only way I know to get the number is... from their website. Which I can't reach.

So I took another look around. On top of the black plastic modem I noticed a tiny black plastic button. Well, who knows, it might do something. I pressed it and instantly - kaboom! - all the lights started blinking and all was well with the world again. Out of curiosity I pressed it again - no more blinking lights. I looked closer. The one orange light that stayed on is marked "standby" and... yes, so is the button, almost illegibly.

So, the modem has a button you can press which stops it working. It does it so thoroughly that it stays not working, even if you power cycle it. Obviously, I must have pressed the button as I was retrieving and rearranging my boxes. But... WHY? Why put a button on a piece of consumer gear, intended to be used by technically unsophisticated people, whose sole purpose is to stop it working in a mysterious and mystifying way? Apart from the inconvenience, it must generate quite a lot of support calls, from people like me who press it accidentally then can't get anything to work. Support calls cost money, even when you outsource them to Bangladesh.

Somewhere there is someone who thought this was a good idea, such a good idea that they convinced Comcast that it was worth the cost of all those support calls and the risk of a bunch of unhappy customers. I would love to meet him and have him (well, it's almost certainly a him, statistically speaking) tell me exactly what he was thinking.

Culture in the Laundry Room

2011-05-05T22:48:00.000-07:00

Like many Europeans, one of our minor surprises when we moved to the US was the laundry equipment. The top-loading washing machine with its vicious agitator swirling back and forth was not only three times bigger than anything we'd seen before, it also tore clothes apart three times as quickly. Maybe that's why casual clothing caught on so quickly in the US - anything with any kind of structure is rapidly reduced to a formless mess by the traditional top-loader. Another surprise is that the washer and dryer are considered to be part of the house, and are left in place by the previous owner - in Europe anything which isn't physically part of the structure moves with the owner.

Anyway, it quickly became urgent for us to replace the machines with something less destructive to our apparel. We bought a nice, civilized Bosch front-loading washer and matching dryer, and our clothes lasted a lot longer. It's true that when you find a dropped sock on the floor five seconds after starting the washer, a top-loader is handy. But having our clothes last for more than three washes made up for it.

All good things come to an end though. Our wash cycles started lasting longer and longer, until the machine was taking more than two hours to wash a few shirts. The service guy came, ran a bunch of diagnostics, and declared that there was noting wrong with the machine, but took $200 anyway. Then it started to smell bad. The service guy came again, took another $200, and declared that there was still nothing wrong with the machine. We certainly weren't going to pay him again, especially when a quick browse showed that we could buy a new machine for not much more money than we had already paid him.

A big surprise was the choice of brands: LG and Samsung. Pretty much nothing else is available. Bosch shows up on about page 10 of the web catalog, along with the traditional American brands like GE. Soon, we had a brand-new, non-smelly Samsung washing machine. Made for the American market, it is about twice as big as its predecessor, which is actually quite handy since it means you can stuff more washing into it.

One thing we weren't expecting was its cultural contribution to our household. We're used to things going beep - I've lost count of how many things beep. Most things beep when they've finished, like the microwave and the toaster. The fridge never really finishes, so it beeps if you leave the door open. I've never heard the garbage disposal beep, but there's probably some occasion when it does. So how is a poor appliance to compete in this melee of beeps? Well, Samsung has an answer. Whenever the washer wants attention - when it has finished, when it has started, and various other odd times as well - it plays the opening bars from Schubert's Trout Quintet.

This is sheer genius. Whoever would have thought of a washing machine that plays uplifting classical music? It really raises the bar for all of these appliances. The watery theme is perfect for a washer, though Handel's Water Music would be even better. The knife sharpener (if we had one - I prefer an old fashioned stone) should naturally play Katachurian's Sabre Dance when the knife is truly sharp. Haydn's Clock Symphony is a clear choice for the timer, and maybe his Miracle Symphony for the microwave.

But there could be more, much more. Twenty years ago nobody imagined there was a fortune to made in downloadable ringtones. Soon, all these appliances will be WiFi enabled, and you'll control your toaster from a web browser. Then, they can all have menus of tones for different things, and it can be monetized - a neologism which I'd hate if it wasn't so descriptively handy. For only $2.99, a choice of ten different tones for your bagel maker. Hmm, maybe I should patent it. But since I don't have the stomach for that (my last attempt at a personal patent got definitively rejected after I'd spent a serious amount of money on it), just consider this to be prior art.

Airline Seats Part 2, also Sorrento

2011-04-19T17:56:00.000-07:00

As promised, here's an update on airline seats to cover AF business class. To my surprise, my flight from Paris to San Francisco was a 747, not the Airbus 340 I was expecting. It looked as though it was seriously overdue for a refit, so maybe the 340s are different. This is the first time I've flown AF long haul in five years. The good news: they no longer have the tiny and exceedingly uncomfortable footrest. The not-so-good news: they don't have a footrest at all. Every reclining seat I've ever flown on, even long before lay-flat, has had a fold-out footrest at the bottom of the extending leg-rest. Not this one though. It really wasn't very comfortable for sitting reading. I wonder why they did it this way.

The entertainment system was seriously out of date, a flashback to the 1990s. No video-on-demand, just a handful of movies running at fixed times. No audio-on-demand either. And everything looked kind of old and tired. It was a surprise, because the KLM 777 I flew on to Europe was much better - and it's the same company now.

On the other hand, the food and wine were excellent. This is where AF really excel, even in economy. The champagne was excellent, and they had an excellent Crozes-Hermitage as well to accompany the lamb main course I chose. AF serves a second meal just before arriving, a proper cooked meal and not the greasy sandwiches that BA serves.

The reason I travelled was to join my wife in Sorrento, Italy where she was attending a conference. And then we stopped in Nice on the way back as well. Sorrento was just wonderful. I think this must be the perfect time of year to go there. The weather was perfect, warm but not too hot. And the traffic was perfectly reasonable. We got round to Positano, Amalfi and Ravello all in one day, and it only took us just over an hour to drive all the way back from Ravello. In summer the narrow, twisting coast road must be a nightmare. The SITA long-distance buses make it more exciting - the drivers operate on a "take no prisoners" basis and assume that hooting just before a bend will ensure that there is no traffic just around it. We did get into one formidable tangle, caused by a driver who wouldn't go ahead through an easy gap and caused a deadlock with a bus, not helped by people pulling out to pass the blocked traffic then getting blocked themselves. The summer must be continuously like this. Ouch.

The best restaurant we found in Sorrento was L'Antica Trattoria. We did eat at the "best restaurant in town", Il Bucco. It was a good meal, but disappointing considering the price. And we ate "authentic" pizza in Naples. Our first attempt was a serious disappointment, a tourist rip-off place on a main square when the place we'd selected from the guide turned out to be a real dive. But the second attempt, Pizzeria Brandi - one of the oldest in the city - was a great success. Though still not as good, in our opinion, as Pizza Cresci on the waterfront in Cannes (and also in Nice).

The picture is the view from our hotel room, at dusk - a perfect view over the Bay of Naples to Vesuvius and to Naples itself. Many more pictures on Flickr.

Cisco

2011-04-09T04:08:00.000-07:00

Cisco's recent woes will be no surprise to those who know the company from the inside. The only surprise is how long it has taken. I was surprised, though, by the naivety of some of the analysts' remarks, and in particular the apparently universal view that all Cisco needs to do is get rid of a few dubious acquisitions like Flip, for everything to be restored to its rosy past. One I read said, "The fall in Cisco's gross margin from 67% to 50% is due to the lower margin on consumer products. Disposing of these is a necessary step to restoring margins." Or something like that anyway.

The simplest of arithmetic shows that this cannot be true. Linksys is by far the largest part of the consumer portfolio, but it is still well under 5% of revenue. The others are just noise. Even if all these business ran at zero margin, the total effect on overall margin would be a couple of percentage points. So it can't be that.

Cisco has been trapped in an awkward spot for a long time, and the consumer business is just one aspect of a well-motivated attempt to get out of it. When you have a near-100% market share, there is just nowhere else to go. Well over half of large enterprises in the US still furnish their entire network infrastructure more-or-less unquestioningly from Cisco. Practically all of Cisco's actual profit comes from the switches and routers that are to be found in nearly every wiring closet and data center in the western world. Everything else is funded out of this apparently inexhaustible gravy train. Growing this to a $40B business has been an amazing achievement, but it's not easy to find another one like it, and this is no longer a rapidly growing sector as it was a decade ago. Hence the moves to the consumer, the data center and others.

But nothing lasts for ever. The core functions of enterprise networks, exemplified more than anything by the Catalyst 6500 family, are already a commodity. For well under $500, Broadcom or Marvell will sell you a chip that does everything you actually need from the Cisco product. They'll even give you complete designs, ready to be sent to a board shop. The days are long gone when Cisco's added value lay in the extraordinary breadth of IOS's support for obscure proprietary protocols. Networks now are all-IP - as indeed John Chambers has been saying. It's only inertia, combined with quite a lot of Fear, Uncertainty and Doubt, that makes people stick with Cisco's high-margin products. And just to twist the knife, the maturity of WiFi means that the enterprise is - finally - moving away from the all-wired model that puts multiple ethernet jacks in every office and cube. That alone has the potential to demolish Cisco's profitability. Worse, competition from HP, Huawei, and white-box vendors - none of whom have Cisco's cost structure or grossly inflated overheads - is driving down the margin.

Let me expand a little on the "grossly inflated overheads". Cisco's development organization functions like 18th century Europe or India - lots of principalities with headstrong leadership, who (mostly) do not compete with each other for territory, but have no intention of sharing anything either. That's why every Cisco product family has its own system architecture, its own ASIC development, and - increasingly - its own software. (Time was when everything ran IOS, which was reasonably common across all platforms, but now IOS has no future relevance for most of them. Only the name, IOS-something-or-other, remains common to what are in fact totally different software systems). The good thing about this strategy is that it avoids the "one egg, one basket" problem that for example brought down DEC in the 90s. The bad thing is that it is hugely expensive, and tends to get worse and worse - it's rare that divergence returns naturally to convergent paths.

This has been a deliberate strategy on Chambers' part. The last time Cisco had any real cohesive technical strategy was under Ed Kozel, in the late 90s. Since then the CTO has been paid to not interfere in the operation of the individual principalities. Charlie Giancarlo seemed to be looking for some kind of unification during his brief reign, and for his pains - ahem - decided to seek new opportunities. Since then there hasn't even been the pretence of any overall leadership of product development, just a "council" whose role is to rubber-stamp whatever each group was doing anyway.

So what can Cisco do? The core business is moving to a commodity, and shrinking anyway. Numerous attempts to move out of this area have met, at best, moderate success (by Cisco standards that is - even the failures make more revenue than many mid-sized companies, but they aren't profitable). What does Cisco really have? The answer, of course, is a vast and mostly loyal customer base. Cisco as a vendor and integrator still has huge strengths - in particular, its (entirely justified) reputation for digging customers out of the dirt no matter how they got there. It's still true, in the enterprise at least - as it once was for IBM - that "nobody gets fired for buying from Cisco".

So Cisco needs to make the same transformation that IBM did 20 years ago under Lou Gerstner, from being primarily a product company to being primarily a service company. And, in the process, to realise that practically all of its product development is context, not core. That will be very tough to accept - it means questioning everything, even the core switch business, not just closing down Flip. It will take a while to happen, and the risk in getting from here to there is huge.

On Airline Seats

2011-04-09T01:48:00.000-07:00

Having just spent 11 hours on an airline seat - with more to come - including several hours when I was pretending to be asleep but wasn't, I couldn't help reflecting on them. The seat in question was KLM Business Class. KLM is in the majority of long-haul airlines who give you a seat which is flat but not horizontal, on a 60-inch pitch. Only a few airlines - BA and Virgin among them - give you a horizontal bed.

Overall, KLM's seat was pretty good, as sleeping on a hillside goes. They've done a better job than Lufthansa, who use what seems to be the same seat design - Lufty gives you absolutely nowhere to put your book, or your glasses while you sleep. KLM's version is full of handy cubby-holes. There is just one weird thing. The socket for the headset is at the very back of a cubby-hole under the armrest, and the plug is a complicated triangular thing that has to be orientated correctly. I ended up on my knees in front of the seat trying to get my headset plugged in. Then I accidentally pulled it out, and since I couldn't repeat my act of worship without disturbing my sleeping neighbour, I resigned myself to no more sound. Eventually I did get it back in, but I can't imagine how everyone else deals with it. Surely it wouldn't have been hard to put it somewhere more accessible, like on the front of the seat?

BA was (as far as I know) the first with the lay-flat seat, in about 2001. The design has evolved since, but it's still basically the same. It gives you a full six-foot horizontal bed, with the additional advantage of complete isolation from your neighbour. The upstairs window seats have complete privacy, and a handy locker you can reach without standing up - and of course a magnificent view of whatever you happen to be flying over, such as this picture of Greenland. It is about as close to perfection as an airline seat can get. My wife likes her favourite seat, 64K, so much that she refuses first-class upgrades so she can keep it.

Air France's first attempt at a flat seat was a disaster. Their Mark I version was at quite a steep angle, so you constantly slid downhill. Every ten minutes or so you'd wake up, as your torso slid down into the space under the seat in front. To make it worse, the footrest was only a couple of inches wide, barely enough even for your ankles. So when you woke up you would first scrunch your ankles into the footrest, then wriggle yourself back up the seat, the thin metal digging painfully into the bones of your feet. That was definitely not a relaxing flight. I'm told they've improved it since, though I haven't had a chance to try it. (I will in a couple of weeks, so maybe I'll write a paxrep then).

Virgin deserves an honourable mention - full marks, and maybe some bonus points too, for trying. But like an over-eager child they tried too hard and ended up spoiling the effect. Their seat is truly flat, rather than made up of a footrest, swab and back. The mechanism involved is truly impressive. Rather than just reclining until eventually it goes flat, like the others, the seat rears up like the Loch Ness Monster then suddenly collapses forwards, so you sleep on the reverse side of what you sit on. Unfortunately it's also rather hard, so not as comfortable as it ought to be. The biggest problem, though, is that the mechanism is so complex that you can't operate it yourself - you have to get the crew to do it. If, like me, you like to switch back and forth between prone and sitting - especially on day flights - you're out of luck. Once it's bedtime, it's bedtime until they serve breakfast. Though I did once have a fantastic flight from London to Tokyo which was so empty that I had a bed and a seat, on opposite sides of the aisle.

It's easy to tell whether your airline has horizontal or flat-but-not-horizontal seats, before you fly. Just go to seatguru.com and look at the pitch. If it's much less than 72 inches, the seat can't possibly be flat. (The exception is Singapore, who make the seat wider so your feet stick into your forward neighbour's armrest. Sadly, my only recent flights on Singapore were in First, so I can't comment on how well this works).

Meanwhile, the US airlines still haven't got to grips with even the flat seat, 10 years after BA pioneered it. Their websites will tell you they are "introducing it on selected routes", but I know nobody who has managed to fly on one of these selected routes. They don't include obvious ones like across the Pacific, that's for sure. I think maybe United is still stuck in the same timewarp as when its stewardesses were below pension age.

Usine à Gaz (Gasworks)

2011-04-06T14:36:00.001-07:00

The French expression for something incomprehensbly complicated is "une usine à gaz" - meaning "a gasworks". Looking at the picture you can see why - rusty pipes going in every which direction, exposed equipment all over the place.

Years ago I had the dubious privilege of working for the most foul-mouthed Frenchman I've ever met. He was also completely irrational and irascibile. He would jump off the deep end over at the slightest - but utterly unpredictable - opportunity.

But he did have his amusing moments. Once he was explaining to me a conversation he had had with the CEO of Gaz de France, the French nationalised gas company, when he worked for IBM. Their IT infrastructure was a complete mess. Or, as he told me, "Je lui ai dit, votre systeme est une usine a gaz" - "your system is a gasworks".

To anyone else, anyone else at all, this would have been a perfectly reasonable remark. But to the head of the French gas company - he probably thinks a gasworks is a very fine thing. I don't know how he responded - if he knew the guy in question reasonably well, he probably rolled his eyes and ignored him, as we all did. Or maybe he said, "You're quite right - and what's wrong with a gasworks?".

Anyway I wish I'd been there.

Wirenuts!

2011-04-02T17:29:00.001-07:00

If you've ever done any household electrical work in the US - like installing a new power outlet - you'll be familiar with wirenuts, those orange (usually) conical gizmos that are used to join two wires together. I'll cut straight to my question: why!? I've never come across something which not only works so badly, but has such potential to burn a house down.

The theory is simple enough - twist together two wires, then screw the wirenut onto the wires. Sharp edges engage with wires, holding them together, providing an insulated over, and ensuring a decent electrical connection. If only that was the reality.

Today I had to fix the pool light in my swimming pool, which just stopped working after being installed only a few months ago. When I opened up one of the electrical boxes, I found that the installer had used wirenuts that were way too small. The wires had come untwisted. Result, no light. Annoying, but not too serious - although for someone who couldn't fix it themselves, it would have involved a callout and a cost of $150 or more.

The former owner of my house made various minor changes to the electrical wiring, installing extra sockets, switches and the like. Of course he used wirenuts for all of them. He hadn't tightened them properly, resulting in poor contact, and eventually lots of heat which (with the greatest good luck) did not cause anything more serious - though I did find scorch marks. I've got rid of all his handiwork now, of course.

If you've only ever dealt with electricity in the US, I can hear you thinking, so what other way could you do it? In Europe, wirenuts are unknown. Instead, terminal strips are used - plastic strips holding little brass blocks with two screws in. You put a wire in each end of one place, and tighten the screw down making a reliable and permanent contact. They don't take up any more space, and they don't set fire to your house. Yet in the US they are completely unavailable - even major trade suppliers like Digikey don't have them.

I'm sure our former owner could have found a way to abuse these in an incendiary fashion. But still I don't understand why wirenuts have such a hold.

boost intrusive - wonderful!

2011-03-14T23:16:00.000-07:00

I wrote a while ago about my disappointment with boost serialize, so it's only fair to redress the balance and say how delighted I am with boost intrusive. The STL list, map and set classes are extremely handy, but to those of us old enough to have cut our teeth on assembler, there's something rather unsatisfactory about all that behind-the-scenes memory allocation going on.

Back in the good old days (like in VMS or RSX), pretty much every data structure began with a pair of forward/backward pointers which could be used to link it into a list. All the key lists in the system (active processes, paged memory allocations, devices, ...) were held like this, with a listhead having the same structure. Insertion and removal are dead easy, and the relatively small size of the systems meant that linear searches weren't really a performance problem.

When I first started writing C++, I wrote a generic linked-list class like this, and an AVL tree class as well. I reused them for a while but then started using STL. But there was always this nagging feeling about those little memory blocks being manipulated behind the curtain.

Recently I've been working on something where memory usage is a problem. I thought those days were behind us, but this one (correlating information about network flows gathered through Netflow) was using up all the memory (8 gbytes) on the machine. Something had to be done. I put all the data structures on a diet, figured out how to get rid of stale information, and so on. But those 32 bytes being wasted for every list entry suddenly seemed important. I'd seen boost intrusive in passing but it had always looked a bit complicated. Time to read it seriously... and discover just how simple it all is.

Those old doubly-linked list structures from RSX... easy:

class my_structure : boost::intrusive::list_base_hook<>
{...

and then:

typedef boost::intrusive::list<my_structure> my_structure_list;

And that's it! You can have sets, too, with lots of nice facilities for lookups, and space/performance tradeoffs, and many other things. For example, I like to keep sets of things indexed by something in the structure. I have a much-reused class for doing this, built on std::set. But lookups require the construction of a whole data structure just to carry the key. I wrote a hack to make this reasonably efficient, but I don't like it.

They thought of that. The advanced lookup and insert functions just let you lookup using the key, without constructing an instance of the structure. The necessary C++ in the structure definition looks a little odd first time, but once you're used to it, you'd never do it any other way.

I'm gradually migrating all my recent code to use boost intrusive. It is just so much better than the STL classes.

Yeah for boost! More thoughts about it here.

The French and American Medical Systems

2011-02-19T00:43:00.001-08:00

While I was in Paris the other day I needed to see a doctor. This reminded me why I like the French medical system so much, and why I dislike the US system so much.

A typical visit to the doctor in the US goes something like this.

Make an appointment. If you're lucky, you can generally get one the same week.
At the due time, drive to medical centre and find a parking space. The medical centres are the size of shopping malls with similar parking problems.
Enter the vast, overwhelming medical centre, and try to find the right reception desk, out of hundreds.
Give your file number to the receptionist. Don't bother with your name, the computer won't know what to do with it.
Wait. Probably 10-20 minutes.
Nurse #1 shows up and leads you through a maze of corridors to a tiny room, where she tells you wait.
Wait another 20 minutes.
Nurse #2 shows up and takes your blood pressure, temperature, height, inside leg measurement and a dozen other things utterly unrelated to the reason for your visit. You also explain your symptoms to the nurse. Heaven forfend that an actual doctor should have to waste his valuable time listening to a patient.
She leads you to another room, where you wait another 20 minutes.
Doctor shows up. He maybe asks a question or two, gives his diagnosis, and disappears. I don't believe I've ever spent more than 60 seconds in the room with the doctor. His time is just too precious to waste with patients!
Wait some more.
Nurse #n shows up with a prescription, if you're going to get one. Though actually my experience is that they will do all they can to avoid giving you any medication.
Finally you're done. Nurse #n leads you back through the maze to the reception desk, where you pay $50 or so "co-pay". Your insurer will pay most of the cost, which for a simple consultation like this is typically $200-300.

At the end of all this, you've spent an hour or more, seen a doctor for 30 seconds, and in all probability don't have a correct diagnosis or medication, though you are considerably poorer.

Here's how it typically works in France.

Make an appointment, usually the same day. In this particular case, I called, spoke to the doctor himself, who said, "Can you be here in ten minutes?".
Go to doctor's office, which is typically an apartment in a stylish building (in Paris) or a little rented office with parking just in front of the door, outside Paris. Not a terrifyingly huge medical centre.
The doctor opens the door for you - not a receptionist or some one from a vast nursing staff, but the actual doctor.
You sit in the doctor's office, explain your symptoms, and he asks lots of questions, examines you if necessary. He treats you like an intelligent person rather than as a numbered and rather stupid case file. You spend maybe 10-15 minutes with him.
He gives you a prescription - which he writes with his very own fountain pen. You pay, typically $30-50 for a consultation like this.
You're done. You've spent 15 minutes with the doctor, and little or no time waiting. You haven't had to deal with an army of nurses. You have a diagnosis and, if appropriate, suitable medication.

The US is a wonderful country, and I appreciate it more and more as I live there. But the medical system is something I'm afraid I'll never get used to or really even understand. What do doctors do when they're not seeing patients, i.e. 99% of the time? The whole system has one primary objective, which is to pour money into the insurance companies. Yet Americans - especially poor ones who are the most disadvantaged by the system - will fight to the death to prevent it changing, as the "Obamacare" debate and rhetoric shows.

Living Life Backwards on the Romance Car Express

2010-11-21T01:01:00.000-08:00

Ever since the very first time I visited Japan, nearly three decades ago, I've wanted to travel on the Odakyu Line's Romance Car Express. They have these really cool trains with the driver's cab 747-like on the roof, so passengers in the first car have a panoramic view of the track ahead. And how could you resist the name? I've even contemplated travelling the length of the line just for the sake of it. On a recent trip I finally had a reason. We were travelling to Hakone, for a stay at a traditional Japanese inn (ryokan) and to do some autumn leaf spotting, and the Romance Car is the ideal way to get there, in fact the only direct service from Tokyo.

So we went out to Shinjuku station, supposedly the biggest station in the world if Tokyo hasn't overtaken it by now. Tickets are only available from machines, and while I suppose I could have figured it out in the end, I was very relieved when a helpful member of staff showed up and guided me through the whole thing, including buying the special all-inclusive Hakone Tour ticket. This kind of extraordinary personal service only happens in Japan. I chose our seats in the very end part of the Romance Car, and off we went to catch our train.

All passenger trains in Japan have the locomotive built in, from the Shinkansen down to the humblest one-car rural train. That means they're symmetrical - the front is the same as the back. And in the case of the Romance Car, that means there is an observation car at the back as well as at the front. And guess where our seats were... the booking system doesn't warn you that your view will be of the track receding behind you.

The view of course is exactly the same. But - which I hadn't expected - there's something very surreal about seeing the world zooming away backwards behind you. After a while it begins to feel like your whole life is running backwards. The journey only lasts about 90 minutes. If it lasted for several hours, I'm sure you'd start to get younger, eventually becoming the little boy in short trousers whose Dad used to hold him on the bridge parapet to watch the trains go by.

The most surprising thing is the level (grade) crossings, of which there are a great many. They open within seconds of the train passing - whereas they close much earlier, for safety. So as soon as you pass, the barriers open and - Japan being a crowded place - cars, bicycles and people flood across the road.

On the way back we bought regular tickets for a car in the middle of the train. We weren't sure we could take the Living Life Backwards experience again.

Business Meetings in Japan

2010-11-16T21:51:00.001-08:00

If you're used to doing business in the US, business meetings between Japanese companies seem very strange at first. In the US, a meeting generally lasts several hours and is planned well in advance. The discussion is focussed, with an agenda and a deliberate progression towards a conclusion. At the end, though there are generally no formal minutes, there are action items and agreements. You know what you've achieved, whether it was a lot or a little.

When I first started working closely with my Japanese colleagues, I just couldn't understand what was happening. Meetings, even at a high level, can happen at short notice. They're rarely longer than an hour. And at the end, as a Westerner you have little idea what has been discussed, and no idea at all what has been concluded. It's just like a cosy chat amongst old friends. Very often, it's not at a formal conference table, but sitting in armchairs around a low table. For this reason I call this meeting style the "Fireside Chat".

An OL ("office lady", a kind of general administrative assistant, invariably female and fairly young) brings in drinks - hot tea in winter, cold barley tea in summer. The head people from the respective teams start a conversation, others join in occasionally, showing due respect. At the end one of them looks at their watch and says, "ah, I think it's time to finish" (actually what they say is a lot less explicit than that). Everyone stands up and files out to the elevator, performs the ritual elevator bow, and the visitors depart. "What was that all about?" you're left thinking to yourself.

Since business does get done in Japan, just as effectively as anywhere else, you can't help wondering how this works. Of course the well-known Japanese dislike of conflict plays a part. But there are as many reasons for business partners to disagree in Japan as there are anywhere else - price, specification, availability, all that stuff - yet they evidently do get resolved somehow.

In the end I realised that geography has a lot to do with it. In the US, such meetings typically involve a plane journey, often an overnight stay. They're expensive and it's important to get the most out of them. In Japan, most large companies have their offices in Tokyo. It's rare that it takes more than a half-hour taxi or subway ride to get there. So meetings are cheap, and you can afford to do them quite often. They don't need to be so focussed, it's OK to let things kind of evolve naturally over several of these apparently rather loose discussions. That coupled with the natural Japanese way of doing things just leads to this completely different style.

Of course, as with everything in Japan, it may be that I've missed the point completely.

boost serialize - not such a good idea!

2010-11-05T13:57:00.000-07:00

I'm a big fan of the boost libraries for C++. Mostly, they are a huge productivity gain - things like regex, function and bind can reduce the work involved in a complex program by half or more. So when I needed to pickle a very large (gigabytes) and complex data structure between restarts, boost was the obvious place to look.

Boost serialize certainly looked like the answer. Just add a few lines to each class saying what you what to save and restore, then with a single function call, you can pickle the whole structure and later reload it. It takes care of loops, diamonds and all the other things that happen in real-life data structures. And if you don't like the default way of saving something, it's easy to write your own. How good can it get!?

Well, that's the theory, and it could have been the practice too. But it isn't. For some reason known only to themselves, serialize's authors decided that the C++ inheritance mechanism wasn't for them. They invented their own completely parallel mechanism for dealing with polymorphism and subclasses. The effect is that trying to save my structure results in an exception thrown from somewhere in an enormous depth of function calls. I persevered, and tracked down what was happening. For some reason - and it's just impossible to plough through all the code and figure out the details - it silently ignores some of the calls to the subclass registration process. After several days of trying to figure out the details - on a live application because that's the only place the data can be collected - I have finally given up. I'll live without this capability in my program.

Unfortunately this is a common problem with boost. It seems to be de rigeur to invent new ways of doing things even though they are more complex and don't work especially well. The build system is another case in point - instead of using make, known and hated by generations of programmers, they invented their own, bjam, which is unknown and incomprehensible. If it works for you, great. If not, forget it.

I'll carry on using boost, but you do have to be selective. Unfortunately.

More thoughts on Boost here.

Divisumma, the Star!

2010-09-12T15:29:00.000-07:00

A few weeks ago I got an email fom someone saying, in essence, "Can I come round and get some pictures of your Divisumma please?"

I was surprised, especially when I discovered he was coming from Detroit - I would have thought there would be more around than that. He is making a video about architecture for Yale, and had been specifically asked to include some footage of the Divisumma. He'd evidently had trouble finding one in the US but - thanks to this blog - he found mine.

He came by today and spent half an hour or so in the sunshine, shooting very close-up footage of the keyboard and the case. And, when he's finished putting it together, Divisumma will be a movie star!

Halfway to Upside Down

2010-09-06T21:57:00.000-07:00

When I started this blog it was with the intention of writing regularly about aerobatics - hence the title. But you know how things are... I got involved with learning to fly the heli, which dramatically reduced the amount of acro I fly.

But today I flew the Pitts, and I took my camera with me. It's hard to take pictures of aerobatics while flying solo, I can tell you - trying to fly a tricky manouver left-handed while holding the camera with the right and trying to get it to focus on the right thing.

I did however manage to get a nice picture of a knife-edge. That's when you hold the airplane in a 90 degree bank, which means that the wings aren't providing any lift. So the only way to maintain altitude is to hold a very steep fuselage angle and let the fuselage provide the lift, which means LOTS of top rudder - in the Pitts S2C, full top rudder. Of course the drag is huge, so you slowly lose airspeed and eventually you will fall out of the sky. But you have long enough to take a nice pic, like this one.

Full size photo, and others from the same flight, here.

BBC News RIP

2010-09-06T13:04:00.000-07:00

For years I've got my daily news fix from the BBC news site. It was well arranged and would give instant access to the top world stories and, when I was feeling nostalgic for good ol' blighty, the UK. There were always a few interesting tidbits. And it was free, too.

Then, a few months back, some genius in marketing (I presume) decided that the page needed a New Look. Ever since, I just can't find any news on it. It's not just me, I have a colleague (Iranian) who has exactly the same feeling. Somehow, they just messed it up. I guess all the same news is still there, but it has lost its interest and salience.

Today I signed up for a subscription to The Times (the London one). Goodbye BBC news. I'm sorry to have lost a friend.

Flying under the Golden Gate Bridge

2010-08-30T20:31:00.000-07:00

I did it! On Sunday we flew the helicopter under the Golden Gate Bridge. It was pretty exciting. The picture shows us just approaching the bridge from the east. We're at 100 feet, about half the height of the deck.

The sight-seeing operators do this all the time. I was astounded when I first saw the red Jetranger that does these flights around the city zoom under the bridge. It's quite legal. There's a special exemption to the regulations that allows helicopters to fly pretty much anywhere without the distance requirements that apply to airplanes. (Take a look at FAR 91.119 if you're interested).

I'd discussed it with my instructor a few months ago. The plan was to fly through the SFO Class B airspace up to the city, under the bridge, then back again by the same route. Unfortunately though, since the incident earlier this year, SFO Tower is being much more cautious about Class B transitions. During the busy part of the day, they are generally not available. And indeed that's what San Carlos tower told us . Our second plan was to cross the hills at route 92 over to the coast, and go round the Class B. Although it was cloudy, there was plenty of room under the clouds to do this.

But then San Carlos handed us off to Norcal Approach, who cleared us through the Class B at 3500 feet. I've been given this transition before and it works well. Sometimes you have to ask for it, a bit like Special VFR. So we climbed before we got to the clouds, flying over the undercast and also the SFO departure corridor. It was very pretty, the airport and the city were both visible through gaps in the clouds, although I didn't manage to get a decent picture.

Then a very rapid decent once we were past the Class B surface zone, down to 700 feet and ready for the big adventure. We went over the south end of the bridge, slowed right down in a left descending turn, and at a stately 40 knots or so passed under the bridge at 100 feet. It was over all too quickly.

Although it's perfectly legal, we were both a bit nervous that people might react badly, so we climbed well away from the bridge, back over the south end, and then across the Bay to Oakland for the transition back south.

More pictures on Flickr.

Audrey Hepburn and the Eiffel Tower - twice!

2010-08-30T12:02:00.000-07:00

There are two movies that feature both Audrey Hepburn and the Eiffel Tower (well, I suppose there could be more). One is well known and you will find it by googling those two subjects - Funny Face, one of her hallmark movies. It's a great movie, which we saw at the Stanford Theater last night. (One of the really nice things about living near Palo Alto is this movie theater, lovingly and expensively restored and specializing in classic movies, mostly pre-1960).

By coincidence, we had watched the other one the previous night at home. Any guesses? It's the Lavender Hill Mob, one of the greatest of the British Ealing comedies, from 1951. The main characters are Alec Guiness and Stanley Holloway, with other classic stars from the time like Sid James and Sidney Tafler. The Eiffel Tower has a central role in the plot, including a fantastic scene where the two heroes run down the emergency staircase. And Audrey Hepburn? In 1951 she was still an unknown. She has a tiny walk-on part at the beginning. You wouldn't recognize her unless you were looking. It was only when the closing credits rolled by that I saw her name.

Going back to Funny Face, there are some wonderful period-piece scenes. For avaiation fans, the scene of the TWA Constellation taking off is enough to make the whole film worthwhile. Then they land at Orly, where on short final they fly over Notre Dame and the Champs Elysees. Funny, this never happens when I go to Orly.

Another great detail is the Isetta, a 1950s "bubble car". It's a weird little thing, with a single door which is also the front of the car. For the movie they gave it a funny pweep-pweep horn, which the real thing didn't have.

o hisashiburi desu nee!

2010-07-30T21:59:00.000-07:00

Just looked at my blog and realised it's months since I wrote anything. (The title is Japanese for "it's been a long time, hasn't it!"). A lot has happened in that time. I got a wonderful hexapod (six-legged) robot from Micromagic Systems, with lots of intentions to write software to do all sorts of clever things. I figured out the math for how to make it go where I want to, which involved re-learning a lot of things that I never really learned in the first place, like matrix math. I got a lot of code written, too.

But then real life intervened - I gave myself a new work programming assignment, and I really can't do two big, new pieces of software at the same time. So I haven't touched that in several months.

The maze solver got put to one side when the hexapod showed up. I'd realised that you just can't track the rotation angle of the 3pi by monitoring the motors. So I had to make the compromise of requiring mazes to always have 90 degree angles in them. I started to make the necessary software changes when, again, real life got in the way.

Then we went travelling, first to Japan where we had the immense good fortune to attend a friend's traditional Japanese wedding, then to France on holiday.

I'm still flying - I've got (at least somewhat) back in the groove with the Pitts, and I fly the heli once a month or so. My precision autos are getting better, and there are times when I think that autos are the most fun you can have with or even without your clothes on. (Maybe that's just a sign of advancing age though!). I've also started work on my CPL-ASEL (commercial license, that lets me get paid for flying, should I find anyone willing to do so), though that hasn't gone very far for all the above reasons.

The Divisumma still sits on the corner of my desk, with its very attentive operator. It still seems to be working.

And that's about it, four months of my life summarised in a couple of paragraphs. Hope to get time to keep this blog a bit more up to date in the next few months.

Robots: Pololu 3pi

2010-03-21T13:52:00.000-07:00

In the last couple of months I began to get a bit frustrated, technically speaking. The big programming project that I've been working on for the last year has finally come to an end - now we need to find some customers for it before we decide what to do next. So I scratched my head and decided that maybe it would be interesting to find out more about robotics.

A week later I'd read a lot and decided what I wanted to do to get started. I sent off an order to Trossen Robotics (there are of course quite a few suppliers, but these people seem to have a very good range) for a few bits and pieces from the Phidgets range, which are a great way to do robotics direct from a regular PC. I also ordered a Pololu 3pi, to get some quick experience with embedded programming. The 3pi is a small, round (10 cm across) robot made for following lines, with two independently-powered driving wheels, some infra-red sensors to see the lines, a microcontroller, some buttons and LEDs, and a tiny display.

I was amazed by how easy it is to work with the 3pi. I've never done anything with tiny microcontrollers before - my day job involves embedded programming too, but with a 400 MHz PPC with 1GB of memory, comparable to a PC. The 3pi has an Atmel 328p, which has 2 KBytes of DRAM and 32 KB of flash to hold the program. Compared to the PDP-8 I worked on in 1973, this is luxury. But when you're used to writing complex frameworks in C++ with Boost, it takes a bit of a mental gear-change.

Atmel provides a really excellent C compiler, actually GCC, and an IDE, loosely based on Visual Studio. It takes less than half an hour to download and install them, download the examples from the Pololu site, and be up and running. Then it was off to Office Depot (which I hate - never shop anywhere with "Depot" in the name - but they are just around the corner) for some paste board and black tape. Within a couple of hours I had my 3pi running round mazes, using the sample code.

"Every schoolboy knows" that if a maze has no loops in it, you can solve it by just turning left (or right) at every opportunity, until you reach the goal. So the interesting challenge is to solve mazes that do contain loops. The wall-hugging strategy will never actually get stuck in a loop, but it won't find the goal either, except by chance. (Actually it does kind of loop, because eventually it gets back to the start of the maze, and since it can't recognize that, it just starts over).

Solving looped mazes requires knowledge of absolute position. (If you have a useful upper bound on the size of the maze, I believe you can do it without such knowledge, but it's a lot harder to figure out how, and will take a lot longer to find a solution). If you know where you are, then you know when you reach an intersection you have already been to, assuming you have enough memory to keep track. (Remember, only 2 Kbytes of DRAM). The 3pi has no explicit position measurement, not even encoders on the motor shafts. The only way to track position is to integrate the motor speeds. It took me about a week to come up with working code to do this. It would be much easier to do in floating point - but the cost in both code space and computation time is unacceptable. I absolutely hate doing scaled integer math, it's a constant trap of overflows and underflows and just plain getting it wrong. But it's the only way.

Once I eventually got the code to work, I was surprised by how little of the available compute power it took. According to the simulator that comes with the Atmel IDE, keeping track of position every 10 mS takes about 0.5mS, i.e. 5% of the CPU. That's good, because it leaves plenty for doing other things, such as following a line, and keeping track of the maze structure.

Although the IDE includes a simulator, in the end I had to get the code working with Visual Studio as well, so I could get a "brain dump" of the computations at every 10 mS interval without having to step through them with the debugger. It's very easy to do this - just create Visual Studio and Atmel IDE projects in the same directory, and they work off the same source files. Of course a few #ifdefs are required to get things to compile in both environments.

One complication I ran into is that the motor speed response is not linear with the applied voltage - which is not a surprise. At lower voltages, a greater proportion of the power is used to overcome friction. Also, the two motors are not absolutely identical, so if you apply the same voltage to both of them, the 3pi gradually curves around. I wrote some code to calibrate the motors, which for the moment is a manual process although it would be a lot better to automate it. With the motors calibrated, it looks as though it should be possible to get a position estimate that is reliable to within about 1%. That should be good enough for mazes up to a couple of meters across.

That's about as far as I've got, at the moment. I'm now in the middle of writing the code to actually solve the maze. This is logically tricky but at least it doesn't involve any fixed-point math. More on that later, if and when I get the whole system to work.

Divisumma Diary - It Works!

2010-03-19T11:17:00.000-07:00

The Divisumma has been sitting on a corner of my desk in full working order for a couple of weeks now. There's a picture of it at the bottom, complete with case and very attentive full-time Divisumma Operator.

Progress since the last posting involved several more variations on the Battle of the Springs, and a lot of lubrication. The problem with springs is that they age, losing some of their strength. Replacing them with new springs is a risky business, because in many cases the exact strength of the spring is important.

I mentioned before the problem that multiplication sometimes gave the wrong answer. At the same time, I noticed that the key that allows you to transfer the result of a multiplication to the multiplier register - for example when computing powers or strings of products like 123 x 456 x 789 - no longer worked.

The Divisumma uses a clever trick to speed up multiplication. If it needs to multiply by 4 or less, it uses repeated addition. For 5 or more, it instead adds an extra one to the next digit, then subtracts. So to multiply by 9, it subtracts once, then adds an extra one to the next digit. If you multiply by 99999, it only performs two actual arithmetic operations, effectively multiplying by 100000 and subtracting 1. The effect is to halve the time to multiply, on average. As you can imagine, the mechanism to do all this, purely mechanically, is frighteningly complex. As ever, I'm lost in admiration for the designer of the machine, Natale Capellaro.

The error in the multiplication could be explained if in certain cases, a value was added instead of being subtracted. To cut a long story short, the problem turned out to be where I had replaced a spring, the long spring 277 from the Battle of the Springs, by a stronger one. It completely solved the problem there, but unfortunately the static end of the spring is not really static, it's a little hook on the end of something that moves. The stronger spring was holding it a fraction of a millimetre away from its correct resting position, and that was enough to block two pieces of the machine: the lever that decides whether to add or subtract, and the lever that causes the multiplication result to be transferred.

So I had to revert to the original spring, and solve my Battle of the Springs problem another way. The service manual says that in certain cases, to get things to work, you should "act upon" various cam followers and the like. So indeed I "acted upon" the little lug 150a, until it stayed out of the way of lug 286n when it was supposed to. It's now twisted quite a lot, I'm sure more than the spirit of "acting upon" would demand, but the machine now works.

Another spring problem caused an equally odd problem. Sometimes, an add or subtract would trigger a total operation, wiping out the value just entered. Curiously, this was related to the length of the number just added. If it had just one digit, the problem didn't happen. If it had 12 digits, it always happened.

The immediate cause of the problem is the mechanism for printing the result of a multiplication or division. This uses the total machinery, and it does it by pulling down the total key at the end of the cycle. It's almost as if a big arm reached out of the machine and pressed the button down. The arrangement for triggering this is amazingly complex, with flimsy levers that run the whole length of the machine. It turned out that the mechanical shock to the machine when the digit carriage was restored, was just enough to trigger one of the pieces of this.

The mechanical stresses in the machine are pretty high. There is very little time to restore the digit carriage within the overall quarter second or so main cycle - maybe 50 milliseconds, to move something quite heavy through 5 centimetres or so. So when it hits the stop, it is moving fast and there is a lot to absorb. Designing these machines to work reliably, considering everything that was going on, was a lot harder than just getting one to work on a bench in the lab.

Anyway, the fix in this case was just to use a new, stronger spring. Problem solved.

Then it was time to put the machine back in its case. The first step was to replace it on the steel base. It hadn't been too hard to get it off - there are just five bolts that hold them together. But getting the holes lined up again was nearly impossible - they have to be correct to within less than a tenth of a millimetre or the bolts just don't engage with the nuts. And two of these bolts are completely inaccessible. Luckily I had just bought these bolt-handling tweezers from Micromark, which were exactly what it took to weave the bolt through the levers and get it to the right place. Even so, for one of them I had to hand-turn the machine to a certain point in a division cycle, to move a lever out of the way and give me access. I can't imagine how they did this assembly in the factory, there must be a trick that I missed.

After that it was just a matter of putting the case back on, cleaning things up, replacing the ribbon, and now I have a perfect piece of retro-computing sitting on my desk. I even use it occasionally, instead of firing up the Windows calculator.

Flying the Helicopter

2010-03-18T21:35:00.000-07:00

It's been a long time since I wrote anything about the heli. I've been flying it generally about once a month since my checkride. That means training and currency flying - apart from the one flight with my wife, everything has been with my instructor. And that means lots of autorotations. They're one of the most fun aspects of flying helis anyway, plus you need to be good at them in case the engine ever stops.

Most practice autorotations are done down to the flare, about 10 feet above the ground, and then you re-apply power and fly away, or hover. That's because the greatest danger - mainly to the aircraft - is in the very final phase. But if it ever happens for real, you need to know how to get the last bit right too. That's called a "full down auto", when you actually land. I've done quite a few of those, my instructors hands poised ready to take over in a heartbeat if necessary. But actually the last couple I've done have been fairly good, resulting in a gentle touchdown and a short roll-out, or I suppose it would be more accurate to call it a scrape-out, considering the absence of wheels.

Another variation on autorotations is the 360 degree auto. This is what you would have to do if you had just flown over the only viable landing spot in sight. Chop the power, then execute a steep (30 degree or more) banked autorotating turn, keeping a sharp eye on rotor speed, airspeed, altitude, and the outside world, all at the same time. The trick is to touch down exactly under the spot where you chopped the power. Let's say I'm getting there.

And then last weekend, something completely different. We've been talking about doing a mountain checkout for a while - we had it arranged a couple of months ago but on the day it was snowing hard in the mountains. Last Sunday, the weather was perfect, so two of us together our instructor flew up to Tahoe, then on to Reno and Truckee before returning home. I flew the outward leg, about 90 minutes from Palo Alto, first in a straight line to Jackson and then following Route 88 up to Kirkwood before cutting across to the pass and then a steep descent into Tahoe. Much easier in the heli than in the plane, where you just about invariably end up flying out over the lake to lose altitude. It was a beautiful flight, with the unparallelled visibility that you can only get from a heli.

We had the heli deliberately loaded up to gross with some ballast, so we got to feel what a high-altitude take-off and landing is like. Also, at altitude you need to watch power carefully, since the limit is lower there. But overall it was fairly straightforward. We went on to Reno, and then I flew the leg from there to Truckee. And learned something important.

In an airplane, your take-off options are limited. You use a runway, and there aren't many of them, and the tower always tells you which one to use. It's something they take seriously! But a heli is capable of taking off from anywhere you like - the ramp, a runway, a taxiway, the grass - and most towers are happy to let you do so, as long as they approve it. But at Reno we ended up with a misunderstanding - the tower expected me to do something different from what I actually did. Nothing bad happened, and they were nice enough about it, but it was an excellent lesson. In the heli at a towered airport, always make certain that what you are about to do is what the tower is expecting you to do.

Flying around Tahoe was a great piece of heli-tourism too. The views were wonderful, especially on the legs where I was sitting in the back, and could take some pictures. Here are a couple to be going on with while I get them all organised and onto Flickr.

Divisumma Diary: A Little Lubrication

2010-02-10T22:27:00.000-08:00

My Divisumma worked, somewhat at least, straight out of the carton. With some very minor attention, it briefly performed all four functions, despite sitting unused in a cupboard somewhere in Italy for probably three decades or more. Not everybody is as lucky. There's an excellent article here (in French) about the restoration of a Tetractys, the Divisumma's big brother (with practically identical internals). The author describes all the lubricants as having gummed up to the point where he had to clean and relubricate it extensively before it would even do basic addition.

I asked a couple of people how best to lubricate the machine, and they both recommended the same thing: AeroKroil. So I sent off for some, and a few other products from the same source while I was at it. Meanwhile, my Divisumma was not working very well at all, culminating of course in the catastrophic jam. There seemed to be no lubricant present, except some dark, dried-out grease in a few places.

When the AeroKroil showed up, I squirted tiny amounts of it into every bearing I could find. What a difference! It was now much easier to turn over by hand, and seemed to run smoothly under power as well, although after The Jam I had become very reluctant to use the motor anyway. I also came across an article in Italian which, after Google translation, said "Lubrication life is entrusted to 4 types of lubricants, each optimized for the kind of friction generated in the appropriate place: dense oil, fat, low viscosity, fat infusions and grease molybdenum disulfide Molykote.". This seemed like a mistranslation - it's hard to imagine them using lard - and indeed it turns out that "grasso" is Italian for both grease and fat. However there's still no indication what the "low viscosity fat" and "fat infusions" actually are. Molykote seemed easier to track down, but there's a bewilderingly huge variety of greases under this name. Also, since it's an industrial product, most suppliers want to sell it to you in 20kg drums or packs of 10 tubes. Luckily I found this supplier. Perusing the list, the BG-20 grease seemed about the best fit, so I sent off for some of that too.

At the same time I was dealing with the machine's various other caprices and dysfunctions, such as The Battle of the Springs. Finally, the Molykote showed up. I spent an hour or two applying tiny beads of it on the end of a nickel spatula (the extremely useful result of the two weeks I spent studying Chemistry at university) to everything that slides - all the cams, the various sliding bearings, and of course the motor and its gearing. That, and plenty of AeroKroil, have made a huge difference to the smoothness of things. One little indication: if you've ever used a Divisumma, you may have noticed that at the end of every operation it makes a gentle "clunk, clunk" noise. That's the one cam that turns directly off the motor output, not through the clutch, bouncing as the motor runs on. When I first got the machine, I barely got three clunks. Now I get eight or nine - an indication of how freely it is now running.

There have been plenty of other little adventures too. For example, for a while when it multiplied, it would sometimes add two to the value of the multiplier - so 9x1 was 11, and 7x1 was 9. I figured it must have been a feature for working with sales forecasts. It's fixed now - it turns out to have been an improbable side effect of The Battle of the Springs. I'm sure you can't wait to read about it.

The Upgrade: Help! I can't get help!

2010-02-10T22:01:00.000-08:00

Now that my VPN problem is solved, my Windows 7 system seems to be working reasonably well. I still can't find the IE cache, or rather I can find it and some things are there but a lot of other things aren't. And videos always start jerkily in the first few seconds, but then they get smoother. All of this is survivable.

But I did come across yet another "what were they thinking" a few days ago. I have quite a lot of old software on the machine - my very own Winlife32, dating back in its origins to 1992, for one. I have a version of "Corel Essentials" that I bought nearly ten years ago. I don't use it enough to justify going and spending $100 on a newer version, but it's handy when I do need it. PhotoPaint is an OK-ish inferior version of PhotoShop, with the great advantage compared to PhotoShop that I know how to use it. And CorelDraw is absolutely indispensible for making jam labels. So the other day I needed to annotate the pictures for my Divisumma Diary. I wanted to draw some arrows on them, which ought not to be difficult to do, but I couldn't find the trick. For once Google was no help, so I was driven to the ultimate recourse, the documentation. Or rather, since there isn't any, the online help. So I clicked the help button. And my brand new Windows 7 system said "I don't know how to open this file", or words to that effect. It turns out that brand spanking new Windows 7 can't open those crufty old fashioned .HLP files that came out with Windows 3.1. Which is a major pain if you have applications that still work that way, like my Corel stuff or indeed Winlife32.

Ah! But then there's an option that says "would you like to download WINHLP.EXE for Windows 7?" So I said yes, and was led through a maze of twisty little web pages, until I got asked which of several files I wanted to download. There was no basis for choosing, and the one that looked most likely (it had 32 in the name where the others had 64) downloaded then said, "this program will not run on your system". Eventually random selection found the right one. And then Corel help worked fine. So does WInlife32 help, although I have now discovered that the API it uses for drawing doesn't work on Windows 7 or Vista, so I guess I'll have to figure out what gratuitous change Microsoft has made there and figure out the fix for that.

So, some genius in Redmond decided that although there's a perfectly fine version of Winhelp that runs on Windows 7, and there are undoubtedly thousands of old apps out there that want to use it, they wouldn't put it on the (huge and bloated) distribution. Instead you have to go through an arcane process to download it, whereupon it works fine. What were they thinking???

Oh, and it turns out that there is no way to draw arrows with PhotoPaint, at least not the version I have, short of drawing the heads by hand, line by line. By the way there's also no way to draw a simple oval line with transparent fill, such as you might put around something to draw attention to it. I guess maybe I really should get one of the cheaper flavors of PhotoShop.

The Divisumma Diary: The Battle of the Springs

2010-02-05T16:32:00.000-08:00

Last time, I had just managed to repair the disastrous jam which tore out a tiny rod deep in the heart of the machine. With much patience and ingenuity, I'd finally got it back in place, and the machine briefly seemed to work. Since then, it has spent a lot more time not working than it has working. Currently, everything is fine except that, like a math-phobic small child, it refuses to do divisions.

I spent a long time dealing with a problem which is really a great illustration of why these machines were so fiendish to service. I'll call it "The Battle of the Springs". The picture shows the two protagonists, and the second picture is a closeup of the actual battlefield, which as so often is far from the comfortable homes of the instigators.

First, a little Divisumma anatomy. Everything that happens in the machine is driven by the main camshaft. In one machine cycle (about a quarter of a second) it turns round once, and drives a single basic operation such as an addition or a subtraction. Exactly what happens depends on which key was pressed to start the cycle. Well, that's almost true. For the total and subtotal functions, there's something called the "extra cycle" which happens prior to a normal machine cycle, to preset various things. It arranges for the printer to leave an extra blank line and print in red, it disconnects the keyboard from the internal operation of the machine, and a few other things. Incidentally, it's the machinery for the extra cycle that causes the "clonk, clonk, clonk" that you hear at the end of every operation.

Another piece of the Divisumma's anatomy is the "memory", the place where quotients are built up during division, where the multiplier is held for multiplication, and which can also be used just to store an intermediate result. This register has a very different physical structure from the normal arithmetic register. It lives at the back of the machine, behind the "executors" which move values between the keyboard, the arithmetic register, and the printer. Some operations need to place a value into the memory, while others need to retrieve a value already there.

And now finally to the battlefield. Operations which need to read the memory value initiate this by releasing the long lever labelled 286 in the picture, so that it can be pulled towards the back of the machine by the spring 277. (The numbers come from the Olivetti service manual). That movement ultimately results in the vertical racks of the memory register being moved into contact with the executors at the right time. Because this is a bit like a total, in this case an extra cycle is performed, and the movement of lever 286 is permitted by the extra cycle machinery, which among many things moves forward the little tab 276m.

With me so far? But there's one problem with all this. This is just a normal operation, so although the result is to be printed, it should not be in red or have double spacing, which the extra cycle would normally do. It's the little tab 150a which, when it drops down, causes these things to happen. 276m moves forward in the extra cycle and 150a drops down. By happy chance, all these things are in the same place. So all that's needed is that when lever 286 moves backwards, the little lug 286n slides under lug 150a, which can no longer drop down, and the value is printed normally. Perfect.

Except that this is actually a race between two springs. The long spring 601 pulls 150 down (gravity could do the trick, but nothing in the Divisumma depends on gravity. It will certainly work on its back, and quite probably upside down too, although I haven't tried it). Meanwhile the short spring 277 pulls 286 backwards.

It's probably about time I told you what problem I was actually having. Sometimes, when I pressed the total key, I got a value of 999999999999, and the current value wasn't cleared. As you can imagine, it took quite a lot of tracking down to figure out what was happening. And as you can guess, I ended up in the Battlefield of the Springs, which is a long way from anything else to do with the total operation. And here's why. When 276m moves backwards, lever 286 has to follow it, and stop 150a dropping down, before 150a makes any movement. If it doesn't, 150a effectively wedges 286n and stops it moving. The result is an impasse - 150a doesn't drop, but also 286n doesn't move backwards. That means that the memory operation doesn't read the memory, but it also stops lever 286 from being fully restored to its normal position. And, via some other complicated linkages, this also means that the total function remains disconnected from the arithmetic register. So the result of this contest of strength is that the machine simply stops working. It just takes one tiny movement of lever 286 to put everything back where it should be, and the machine will work normally again. But for the office worker in 1960, her precious Divisumma was broken. Time to send it to the men in brown coats. You can see why there were so many of them in those days.

Those readers who are following this with the service manual in hand (admittedly improbable) will have noticed that I've diverged slightly from the straight and narrow here. I've commented before on how excellent these manuals are, but in this case Homer nodded. There is no spring 601 in the manual! And lever 150 is really lever 150', and it doesn't have a lug 150a! How can this be? What I think happened is that Olivetti enhanced the original double-spacing mechanism, which was much simpler, but my manual reflects the older design. The web has a surprising number of pictures of the insides of the Divisumma 24, and they all the seem to be the same as mine. This is a problem though, because nothing describes exactly how all these extra pieces are meant to work together, so it's largely guesswork.

Now that I'd finally diagnosed the problem, what could I do about it? The apparent reason was that spring 601 was too strong compared to spring 277, so the obvious thing to do was to put a stronger spring at 277. However I didn't have any suitable springs. I sent off for a pack of 200 small springs (lots of people sell this kit but I'm pretty sure they're all the same one), but while I was waiting I thought, it doesn't actually matter whether the total is printed in red, so why not just stop lever 150 from dropping at all, ever. So with a piece of wire I fixed it to a convenient place at just the right height. That was a real mess! It turned out that the place I fixed it to also moves, and that was enough to mess things up completely. I'd broken one of the Divisumma cardinal rules, which was to forget just how incredibly inter-related everything is in this machine. Lifting lever 150 a bit too far put it in the way of something else, and almost nothing worked correctly.

When my box of springs showed up, I eagerly replaced spring 277 with something stronger. I carefully turned the machine over by hand, and as the extra cycle did its thing and lug 276m slid forward, lever 286 gracefully followed it, preventing the evil tab 150a from dropping down into its path. YES!!! Problem solved, through ingenuity and creativity. How happy I was!

Briefly. Because after that, the total stopped working again, intermittently, but differently - instead of showing all 9s, it would show some apparently random value. That was when I discovered inertia. What happens is this. When the memory read cycle is over, the extra cycle machinery gets put back into its normal position, which means among other things that lug 276m gets pushed forwards and hence lever 286 also. This should restore it to its normal position. Visually, it does. But it just doesn't go quite far enough to latch in place. This leaves it free to move forwards during a total cycle, ultimately resulting in the memory register being engaged. The actual effect of this is that each digit, as printed, is the lower of the value in the memory and the value in the register. It isn't obvious to figure this out!

This time, the problem was that spring 277 was too strong. It turns out there is a tiny gap between the two lugs 276m and 286n. With the original spring, there's enough inertia to move lever A the extra fraction of a millimeter so it engages. But the stronger spring stops this, so the machine doesn't work again, but differently.

To adjust the machine, sometimes the service manual tells you to bend the lugs. So, I thought, the next step would be to tweak the exact position of lug 286n to get things to work properly. I did, and that, I rather think, is why my Divisumma will no longer divide. But that's a story for later.

The Upgrade - Part 2, Mainly getting the VPN to work

2010-02-02T16:11:00.000-08:00

Well, once I got Windows 7 running, everything seemed fine. For a while. For a stand-alone machine, Windows 7 is OK. Personally I don't find it any better or worse than XP - different sometimes, but not noticeably better. The graphics are all different but honestly they don't make any real difference. It's kind of cute the way taskbar items give you a preview before you expand them, but again it's not exactly life changing.

I can't comment on whether the performance is better than XP, as some people claim, because the machine was so badly broken before that it isn't a fair comparison. Some things are however much worse. For some reason re-sorting a large directory can take ages - tens of seconds - if the content consists mainly of images. I suspect it is re-extracting the thumbnails from the files. It's a typical example of trying to be gee-whizz helpful and just making the user experience painful, which unfortunately is fairly characteristic of Windows.

It was when I tried to install the VPN so I could use the machine to work from home that things started to go downhill rapidly. Like many companies, we have a Cisco PIX (well, it's called something else now) which provides VPN services among other things, and the Cisco VPN client. Getting hold of the Windows 7 client and getting the VPN to come up was amazingly easy. So far things were looking good. I could SSH to my Linux system, and I found a so-so free X server and that worked too. For some reason I stopped there, and it was a few days before I tried the next step.

Which was... to open a Windows Explorer window on the Linux filesystem. We have (just like lots of people) a server that runs Samba. When I tried to connect, it just hung for a long time and eventually said it couldn't reach the server. What's more, this time, I couldn't get SSH to work either. Using Wireshark, I could see that packets were being sent into the VPN client, but they were never coming out on the wire side. And in fact, the client's statistics window showed an increasing "discarded packet" count. It's impossible to find out why the client discards packets. The log shows nothing, even when you turn the logging level up to maximum for everything.

At this point I gave up for a while on the regular VPN client, and tried using the SSH client. This was amazingly easy to install. But it didn't work. I eventually discovered that even though I had my work network set as "trusted" in the ZoneAlarm firewall, I still had to turn the global internet security level all the way down, effectively disabling the firewall altogether. The SSH VPN came up that way, but of course this is way too dangerous as a normal setting. So much for that solution.

Luckily, I was able to ask for help from a security expert at Cisco, who parenthetically remarked that it's just about impossible to get the VPN to work with the ZoneAlarm firewall running. Aha! I turned it off, turned on the Windows firewall, and now everything except remote mounting my Linux server worked fine. So, lesson #1: don't use the ZoneAlarm firewall if you want VPN to work.

More Wiresharking showed that this time, packets were getting through to the wire, and were getting replies. Yippee! Strangely, the Windows file client was trying to connect to port 80 (i.e. HTTP) and getting an immediate reject from the server. I have no idea why it tries to connect to port 80. Another interesting thing I saw was that after a couple of failed attempts at connecting, Windows just sulks. When you click on "connect" it immediately comes back and says "can't" without even sending any messages. I suppose it has cached the failed result, but it's a real problem when you are making configuration changes to try and get it to work.

Then I suddenly remembered something. Windows remote file access uses a protocol called SMB. SMBv1, used up to XP, is a truly awful protocol. It sends literally hundreds of messages, and waits for the round trips, for even the simplest operation. That's why accessing a remote file system over a wide-area connection is so slow and painful. Finally, after a decade or so, this problem was addressed in Vista, with SMBv2, which supposedly makes things better. However Samba still only speaks SMBv1. Now Vista, and Windows 7, are supposed to auto-negotiate, i.e. if the server can't do SMBv2, they fall back to SMBv1. Supposed to, but evidently they don't. If the client doesn't get the answer it's expecting when it tries SMBv2, it just sits and waits an irritatingly long time, then tells you it can't do it. So, the obvious thing to do is look for a way to tell the client always to use SMBv1. I looked everywhere in the "advanced" configuration, but nowhere is there a checkbox that says "don't even think about using SMBv2".

So I Googled, and I found an excellent page on an Israeli IT-help site that says exactly what to do. It just involves typing two unbelievably complex commands at the Windows shell. However this isn't the whole story, because Windows 7 has this great idea that even if you have Administrator privilege, you don't actually get that privilege unless you ask nicely. So after you have typed this humungous command, the response you get is "access denied". Luckily, Google had the answer for that, too.

So the whole procedure for disabling SMBv2, assuming that you have Administrator privilege, is:

Right click on the "Command Window" icon, then select "Run as Administrator".
Type:
sc config lanmanworkstation depend= bowser/mrxsmb10/nsi
sc config mrxsmb20 start= auto

Don't miss out the spaces after the equal signs - they're important. Someone at MS evidently skipped parser school.

So, after a very frustrating couple of sessions, I can now fully access my work system from home. But (a common refrain I'm afraid), why is it so hard?

That wasn't all though. When I was physically rebuilding my system, I kept noticing that the CD/DVD drives stopped working. Each time, I'd remove all their cables, plug them back in again, and when I rebooted, they worked fine. The cables didn't seem loose or damaged, so it was a bit surprising, but I figured I must be disturbing things somehow. Later, I finally figured out why my DVD drive refused to write DVDs... it isn't a DVD writer, just a reader and a CD writer. Well, that was easy to fix, so it was off to Fry's for a fancy new SATA DVD drive. That was actually an amazingly simple and smooth installation. Absolutely nothing went wrong at all.

Then, while I was trying to solve my VPN problems... the CD drives wouldn't work. When I inserted disks, the lights would flash, and sometimes the drawers would pop open, but nothing showed up for the disks themselves. Now though, they use completely separate cables and adapters, right to the motherboard. So it can't be a cabling problem. Finally, I rebooted the machine... and they worked just fine. Turns out (thanks again to Google) that Windows 7 has a well-known bug that sometimes, CD drives don't work. How on earth did that get through the QA process?

On Upgrading - or "what were they thinking?"

2010-01-18T11:25:00.000-08:00

My desktop machine at home has had creeping software sickness for a long time. It was new about 4 years ago and has gradually got less and less likely to succeed at any given task. Visual Studio crashes quite often, which I've never seen anywhere else. Google Earth kind-of runs but is pretty much unusable. And so on. I've no idea why these things happen, there's no logical reason why software should wear out, but it seems that it does.

I resisted Vista for all of its sorry lifetime, wisely as it turned out. But everyone seems to be saying nice things about Windows 7, so I decided it was time to upgrade. At the same time I decided to swap the 250 Gbyte hard drive for a brand new 2 terabyte one. Not that I have anything that will use up that much disk space, but 250 GB was getting full.

The machine is something called an HP Media Center. It was a distress purchase, my previous machine had scrambled its hard drive and I had exactly 10 hours to get something important done, so I just rushed out to Fry's and bought what they recommended. Until now, I'd never had any reason to take the covers off. When I did, I found that the hard drive was totally, utterly inaccessible, hidden inside the machine, and mounted vertically (rather than horizontally as usual). No normal screwdriver could access any of the mounting screws. I tried undoing various other things, but nothing gave access to the screws. Eventually I figured I could just put the new drive loose on the floor of the box. But, since I don't give up easily, I wondered how the factory had ever managed to assemble the machine. More on that later.

So, having swapped the drive cables over to my brand-new, unformatted 2 TB disk, it was time to install Windows 7, which I had (legitimately) on a USB memory stick. (As an aside, in about 1994 I worked on one of the very first video-on-demand projects, at DEC. Part of the system was a 1 TB drive farm, which occupied four full-size racks. Of course the whole project was more than a decade ahead of its time, and flopped completely).

Since I had a new disk, I actually wanted to do an install from scratch. But still there is a great mystery here. Microsoft built Windows 7 because the adoption rate of Vista was so low - I've seen estimates of 20%. Yet you can only do an in situ upgrade from Vista. So if you're one of the 80% who never moved from XP, you're out of luck. Or rather, Microsoft is, because that's who loses money if people don't upgrade. XP is a perfectly fine system, I would not have upgraded but for my "software wear-out" problem. Most not-very-technical users will be completely put off by the complication of a from-scratch installation, followed by reinstalling all of their apps, and in all probability just won't bother. So what was Microsoft thinking when they made this decision?

So, I booted from the memory stick, Windows 7 started and everything seemed fine. Then it put up a screen asking me which disk I wanted to use. When I selected my new drive, I got an error, "cannot create system partition". That's dumb, I thought, surely a from-scratch installer should be able to partition and format a disk? Still, I was able to rig things up so that I booted from the old system, with the new disk connected. That way I could format the new disk from XP. Then I restarted the Windows 7 system again. The same thing happened.

It took a lot of frustration, much bad language, and a lot of googling before I found the solution to this. It turns out that since the early days of Vista, the installer has not been able to cope with having more than one drive or partition visible to it. If it does, it gives this misleading error message which gives absolutely no clue as to the problem, and which will certainly stop most people dead in their tracks. Yet MS had at least a year to fix this, preferably so installer could handle multiple connected drives, but failing that at least to give a better error message. Once again, what were they thinking?

Once I had this figured out and unplugged everything except the target drive, everything went fine. The system installed remarkably quickly, and subsequent reinstallation of all my apps went smoothly too. It took quite a while for further problems to emerge.

Meanwhile, I hadn't given up on the disk mounting problem. I picked up various tools - stubby screwdriver, offset screwdriver - from my local TrueValue, but none of them could get all of the hidden screws. Finally a trip to Fry's got me a tiny ratchet offset screwdriver with interchangeable bits. Using that, and a lot of missing flesh from my hands, I was able to remove the drive and install the new one. Still I wondered how on earth they did this at the factory? Then by chance I tidied my office and found the original hardware manual for the machine. This revealed the secret latch which allows the whole drive bay to be dismantled and removed. Of course it is much easier that way, but there would be no way to identify the secret latch without the manual.

Windows 7 ran smoothly for a while, but then I ran into a very odd problem. Double clicking on certain system-ish folders gives the message "Access Denied". Now, I am the only user of the machine, and of course have Administrator privilege, so what is going on? I tried changing access rights, but of course since I didn't have access in the first place, that didn't work. Finally, some extensive googling found a way round this, which involves going into the deepest depths of "advanced options" in the security screen, and changing ownership of the parent directory. It's at times like this that Linux is so attractive - although the commands are arcane, it is always reasonably easy to figure out what is going on. With Windows, it's either easy or next-to-impossible. The dumbed-down user interfaces give you few alternatives when things are partly broken.

So now, my new system seems to be working fine, and does very nearly everything my old one did. (There are still a few things to reinstall). One mystery is a system-created "shortcut" (i.e. softlink) which points to itself. This confuses Explorer no end, but since I have no idea what the folder is for, I daren't delete it. I also still haven't managed to find the cache directory for IE8 - I've found one of them, but only some files show up there. That's another mystery.

Divisumma 24

2010-01-09T13:40:00.001-08:00

It's arrived! My circa 1960 Olivetti Divisumma 24 electromechanical calculator has arrived from Italy. It's in good condition although so far I haven't got it fully working.
First a little background. When I was about 17 (a long time ago!) I worked one summer at an advertising agency doing analysis of some surveys. For this I was equipped with a Divisumma 24. This was, for its time, an amazing machine. It could not only add and subtract, but also multiply and divide, for numbers up to 12 digits. It was very compact, unlike most such machines, and also a very nice piece of industrial design. It fascinated me, and I was especially intrigued as to how it worked. I wrote to Olivetti, and they kindly sent me a green booklet describing in complete detail how one of the simpler machines - just an adding/subtracting machine - worked. A while later I wrote to them again, thanking them profusely and practically begging them for information about the Divisumma. Obviously my letter went to someone else, because this time I got a reply more-or-less accusing me of stealing the book and asserting that Olivetti would never, ever, under any circumstances, give such information. So that was that.
I still have that little green book, and a few weeks ago I took it out of the bookcase and re-read it. I was as intrigued as ever about how the Divisumma worked. A lot has changed in 40 or so years - Olivetti no longer really exists, for one thing, and nobody has used mechanical calculators for three decades. So with Google's help I discovered this source for Olivetti manuals, and a great many others too. A few days later a parcel showed up with seven volumes of green-covered manuals, and I started reading. It's not that hard to understand how it works, but what is truly mind-boggling is that someone was able to design it. Even allowing that it was a process of evolution from simper machines, it is still extremely impressive. Unlike software, which has some structure to it, these are three-dimensional puzzles, where two completely unrelated functions that just happen to need to do something at the same time, can share the machinery. (For example, the mechanism that arranges double line-spacing for totals just happens to be in the right place to reset a lever that controls the way the registers engage for memory operations).
Of course reading these manuals just whetted my appetite again. A bit of searching showed someone in Italy who had one to sell. I paid about $150 for it, but of course the postage - even surface mail - nearly doubled that. And this week it arrived. The seller had done a super job of packaging, including a wooden framework to protect the machine inside the box. It was complete, apart from one minor knob on the keyboard, and the power cord. It even had a roll of yellowing paper.
I rigged up a power cord for it (don't tell the safety nannies about this) and plugged it in. Unfortunately, it didn't work - one of the keys was jammed, and the motor just ran without anything happening. Not really a surprise, but a bit of a disappointment. So then it was off with cover. That comes off surprisingly easy, no screws, just a couple of toggles.
The insides of the machine are in remarkably good condition. There was some fluff around the keyboard, but no rust or damage. All the metal parts seem to be cadmium plated, which means they look the same as the day they were assembled.

I need to say something about the manual, too. The Olivetti technical manuals are also works of art. Every tiny aspect of the machine's operation is described in great detail, with beautiful engineering drawings for every page showing the various levers, bridges, pins, racks, springs and so on - just those relevant to the current description. Once you've digested that, they also include complete instructions for assembling a machine from scratch. The numbering of the parts is consistent through the seven volumes, so once you read in chapter VIII (as befits an Italian product, much use is made or Roman numerals) that lever 73 is key to some particular function, whenever you see lever 73 later on, you know exactly what it does.

So with the help of the manual, I was able to figure out what was jammed, and unjam it. And the machine worked! Well, briefly. For about fifteen glorious minutes it would add, subtract, multiply, divide, and move numbers to and from the "memory" . Even the 30-year old ribbon worked, well enough to read anyway.
But then disaster struck. I started a divide, but instead of the usual kerchunk-kerchunk of the running machine there was just a faint hum, the horrifying noise of the stalled motor. I unplugged it and tried a few things, and eventually it unjammed. Phew! But then I realised things weren't working right, and when I looked, I realised that disaster had struck. Right in the very heart of the machine, a frame had got twisted so badly that it had let a metal rod fall out. This was a totally inaccessible place, without completely dismantling the machine. I was mortified. Just when things had been going so well (I had never really expected the machine to be working), and now, just so much scrap metal. I could almost have wept.
I studied the manuals again, and very carefully, removed the paper roller and associated bits and pieces, saving all the parts in little plastic bags. There was only one problem - a circlip which in the time-honored tradition flew across the room. Luckily I found it easily. That gave me access to the mechanism, from above. Then the real fun started. I found that I could just get a pair of tweezers between the individual elements of the register, and that way I could hold the rod in more or less the right place to insert it. But actually getting it into the holes at either side of the frame was another story. Let's just say it took a long time. Finally the trick was to hold everything in place with wire (actually green garden wire, which gave a certain rusticity to the whole operation). After several hours I had it back in place.
Very carefully I turned the machine over by hand. It seemed to work correctly. But then I discovered how delicate these machines really are. Just like an exceptionally old wine, which is wonderful when opened but undrinkable within ten minutes, so it seems that my early success with the Divisumma was not to last. Every operation revealed some new misalignment or jam, and when that was fixed, another one showed up.
There is really a lost trade and craft here. Back in the day when these machines were commonplace, every town had numerous little back-street workshops, where bent-backed mechanics in brown coats would pore over them when they jammed or refused to work. I'm sure it was a common occurrence. In the few weeks I was using the Divisumma all those years back, it jammed once and had to go for repairs. Probably every machine was in the shop every few weeks. "Oh yes," the brown-coated old guy would say, "looks like the returbitating cam has shifted again, just move it a bit and tighten up the unflurging spring while I'm at it..." and the machine would be as good as new... for a few more weeks.
Anyway, my Divisumma definitely needs some more tender loving care before it will be dividing again. It's a wonderful piece of engineering, and I'm very lucky to have it. It's certainly something to fill those long winter evenings!

Ergonomics, Cars and Turn Signals

2009-11-25T00:14:00.000-08:00

There's a book called "Turn Signals are the Facial Expressions of Automobiles", which deals at length with the kind of dysfunctional man-machine interfaces that annoy all of us from time to time. If it's true, then the automobiles made by Opel (the European arm of GM, at least for now) must have eaten something very nasty.

Turn signals (direction indicators for my UK readers, if any) have been operated in the same way ever since flashing lights replaced pop-up arms that stuck out of the side of the car, which is to say the early 1950s. You push the little arm up to turn left, down to turn right, and it stays there, making the corresponding lights flash, until either you straighten the wheel up after the turn, or you turn them off manually. It's simple, it's intuitive, and it's easy to build.

It's a shame that, along the way, manufacturers have never been able to agree about which side this otherwise universal control should be. So every now and then, when you rent a car, you find yourself turning on the wipers instead of signalling. Even so, it's immediately obvious what you've done, and what to do about it. Since turning the signals on and off is never an emergency, it's no more than a minor nuisance.

Sometimes it takes real genius to look beyond the mould of convention and make the leap to a new and far better way of doing things. Unfortunately genii are rare, and when the non-genius tries the result is rarely a success.

Such is the case at Opel (and their UK badge-engineered variant, Vauxhall). If something has been in use for over half a century, works well, and is familiar to every driver on the planet, then surely it must be time to come up with something gratuitously different. And so they have.

My guess is that someone on the production side worked out that the mechanism that holds the arm in place, and returns it to the neutral position as the wheel turns, represents a few dimes that could be saved in the production of every car thanks to the miracle of electronics. Who needs springs and cams when you can do the same thing with some silicon?

And so if your rental car turns out to be an Opel you will find that the indicator stalk is sprung so it always returns to the neutral position. To turn left, you push it up, the silicon equivalent of a cam and a spring remembers it is there, the signals blink, but the memory is strictly non-mechanical. This is surprising at first, but you barely notice. You make your turn, and when you straighten the wheel out afterwards another miracle of the electronic era turns the signal off. So far, so good.

The problem comes when you make a lane change. In the US of course this problem doesn't arise. People rarely signal at all, and never (in my experience) for a lane change. But in Europe people still do, and it seems polite to follow suit. Since there are no large steering wheel movements, the signal does not automatically cancel. You have to do it yourself. You can't move the stalk back to the neutral position, because it's already there. So what do you do? Well, the only option is to push it in the opposite direction, i.e. downwards if you were signalling left. This works, in that the left signal stops signalling. Unfortunately, what happens now is that you are signalling right instead. Given that you probably just pulled into the left lane, this will certainly confuse following traffic (if they pay any attention to these things). So now, how do you stop the right signal? Only one choice... push the stalk upwards. You can guess what that does. At this point, you just have to hope that the police are not close behind you, because if they are they will certainly pull you over and give you a breath test. (Well, maybe not if they are driving an Opel too, because then they will understand the problem).

I can say from experience that there exists some combination of circumstance and action that does completely turn off the indicators. You don't have to make a 360 degree turn on the highway, although it may start to feel like the only solution. However it's not something you can ever figure out or remember. Sometimes, some amount of waggling the stupid arm stops the signals, sometimes it doesn't. I've even heard of a case - certainly not from personal experience, as I'm sure you will realise - where a driver has become so frustrated at the stupid @#$^^%$@# thing that they have ended up yanking the stalk right off. (It's made of some feeble soapy plastic, that breaks very easily. No doubt this saves another cent or two on the production. I've also heard - not that I would know personally - that superglue will hold it in place well enough to last until the next rental).

My first experience of this technological leap was several years ago, in a rented Opel Zafira. It was a terrible car; if blogs had existed at the time, I would certainly have written more about it. But this the worst of its many sorry misfeatures. However I'm pleased to say the engineers at Opel are not deterred by any kind of temporary setback. My son is now learning to drive, in a Vauxhall. He swears that he will never, ever buy one - because of their bizarre and dysfunctional indicators. (And no, I hadn't told him what I thought of them).

It's a reassuring constant that when people, and companies, make mistakes, the last thing they will do is correct them, since that would be showing weakness. Instead they persevere, in full denial that they could ever have done anything wrong. Occasional lapses from such behaviour (such as the Austin Allegro's square steering wheel - and no, I'm not making it up - which indeed was corrected very quickly) are satisfyingly rare.

Your Neighbors will be Astounded by...!

2009-11-24T04:12:00.000-08:00

Like everyone in the US, we are bombarded by glossy catalogs for things we don't want. Every day brings two or three of them: clothes, household nick-nacks, guy stuff, garden stuff, just about any stuff you can think of.

These catalogs are in fact very sophisticated examples of applied psychology. For 99% of the things in them, you have lived your entire life without realising you need them, and you're unlikely to suddenly decide you need them all of a sudden now. So the motivation to purchase has to be something else. The most frequent appeal is to the common human desire to appear better - more sophisticated, stronger, more intelligent - than the next guy or gal. It's amazing how often the headline "Your Friends/Neighbors/Workmates will be Astounded by..." appears. "Your Neighbors will be astounded to see this giant, fully illuminated, solar powered inflatable Polar Bear". "Your Friends will be astounded by this 15-place table serving in genuine Orinocan Sustainable Mahogany". And so on.

Another frequent appeal is an application of "X does Y, so if you do Y too, you'll be X too". The crudest example of this is the scantily clad young ladies who adorn car adverts, and real cars at shows. The implication is of course that if you buy this car, suddenly your life will be filled with desirable nymphets. It's obvious to even the densest person that this can't possibly be so, that having a Geo Metro Sports Edition on your driveway is not really going to expand your life like this. Yet it works, or I suppose it must or people wouldn't still be doing it.

Last week's crop of catalogs produced an extension of this that I hadn't seen before, though, pushing the notion to new extremes. It includes a pen - a pretty expensive pen, at $70. And the headline says: "Straight from Italy, the Pen I Drove Around Ferrari's Fiorano Test Track at 185 mph!". (We can leave the question of why every word Begins With Capital Letters for Another Time).

So, let me get this straight. I should want this pen because someone had one just like it in their pocket while they were driving at 185 mph. The implication must be that if I have a pen like this, then I'll have as much fun as if I was driving at 185 mph. Can this appeal possibly really work? Are there people out there who say, gosh, I must have one of these because someone drove at 185 mph with one in their pocket? Or who buy them as gifts, telling the bemused recipient, "You'll love this pen, someone you've never heard of was driving at 185 mph with one in their pocket?"

Why stop at pens? Why not socks, or underwear, or denture retainer? "Straight from Malaysia, the Pack of Tissues I Flew in a Search-and-Rescue Helicopter!" (I think that's generally considered a fairly manly occupation).

Maybe I should just get used to human nature, and the clever ways of exploiting it for profit. But I just can't.

PPL-H!

2009-11-07T16:52:00.000-08:00

Well, almost exactly one year after my "discovery flight", I passed my PPL-H checkride and became a qualified helicopter pilot. To my pleasure and surprise, my wife - who has never been at all keen on helis and until now has never flown in one - agreed to be my first passenger. So last weekend I did my first helicopter "Bay Tour", departing from Palo Alto towards the coast, with a slight detour to fly over the Montebello Ridge vineyards so she could see them from the air. Then out to San Gregorio, up the coast over Half Moon Bay, over the Golden Gate and back over the city. I've done this numerous times in airplanes, but the view from the heli is much better and of course you're lower too. The flight was a success from all points of view, with a very happy passenger who agreed - despite her attachment to our plane - that this was even better.

One of the hardest parts of flying helis, for me anyway, was to get a really smooth pickup. Hovering seems impossible at first, but after about five hours it's under control and getting better. Autorotations are fun, and if you're used to landing the Pitts then they don't seem too dramatic. Getting them right is tricky and takes some fairly aggressive manouvering, and (in my opinion) you can never practice them enough. But in the end, pickups were the hardest.

Here's the problem. While you're sitting firmly on the ground, the position of the controls isn't affecting anything. Hence, you have no feedback about whether they're in the right position or not. As soon as the skids completely leave the ground, you're flying, even though you're only an inch off the ground, and everything depends on the position of the cyclic and the pedals. A touch too much left cyclic and right pedal, and you're whizzing off to the left while spinning round to the right - not good, especially if you're in a confined area or on soft ground.

Instructors and textbooks say the same thing - lift the collective slowly until you're "light on the skids", correct the control position, then lift further. Easy to say, harder to get a feel for. It's just like learning coordinated turns in a plane - the instructor says, "but surely you can feel it, without even looking at the ball?" And you say (or at least I did), "errr, no, I can't". Now I can feel it, after over 1000 hours and 100+ hours of aerobatics, but it took a while.

So my experience was, you so-so-gently pull up on the collective, waiting to feel the heli come alive as the weight comes off the skids. And then it starts to turn or to move, and in an instant of panic you yank up the collective and you're three feet off the ground and you've zoomed off several feet from your starting point, before you brought things back under control. This is not the way to pass a checkride, something my instructor made very clear to me.

The fix, for me, was an exercise where I deliberately set the cyclic and the pedals way off where they should be, then, as the collective came up, gradually corrected them as I felt things starting to move. After a few times doing that, I'd overcome the panic instinct to yank on the collective, and after that it was just practice. It's very satisfying - now that I can feel what "light on the skids" means - to make small corrections just before the thing leaves the ground, and have it move smoothly and vertically through those first few inches.

The other hard thing, getting close to the checkride, was slope work, where you delberately have the weight partly supported by one skid and partly by the rotor. The amazing videos you see of helicopters touching a steep hillside, yet still flying, are all about this. Again it's all about feel for the controls during the critical moments, and tiny movements of the collective. It's a real feeling of victory when you can hold the heli, half on the ground and half off, then sooo gently let it down onto the other skid. And even better when you can smoothly lift it off again. It takes a lot of practice.

Finally, a couple more pictures from our Bay Tour. There are more on Flickr.

The Blue Cogwheel of Death

2009-10-25T20:18:00.000-07:00

Last week I had the great good fortune to be invited to the Computer History Museum's Annual Fellow Awards event. I first visited the Museum when it was in Boston, shortly after it opened there, and I again find myself living close by. The collection is truly amazing - if you haven't been and you find yourself nearby, you should definitely go.

And of all the amazing items, the most outstanding is the Babbage Difference Engine, courtesy of Nathan Myhrvold, who paid the London Science Museum to build him one and then lent it to the Museum. It weighs four tons, and is eight feet high by about twelve feet long (I didn't have a tape measure with me so this may not be accurate). In principle it's extremely simple - repeated addition can be used to calculate any polynomial whose degree does not exceed the number of stages in the machine, and hence to approximate a great many interesting functions. However given that it is built with 18th century technology, the reality is anything but simple. To anyone who admires mechanics and machinery, it is truly a thing of beauty - four tons of gleaming brass and steel and cast iron, thousands of cams, levers, wires and pulleys. To sit down and design it from scratch now would be an achievement; that Babbage did so over two centuries ago is almost beyond belief.

The machine is operated by a large crank handle, which is turned slowly and evenly by a trained operator. Sadly though, when I was there to see it, a short way round the first turn the handle jammed solid. No amount of gentle persuasion would shift it. The docents (guides, for any British readers I may have) told us that this happens occasionally, and when it does, the machine needs to be reset. As you may imagine, this involves a lot more than pressing a button and waiting a couple of minutes. Screws have to be loosened, cams carefully reset to their starting position, and goodness only knows what. In any case it takes way too long to do in front of an audience.

I feel quite honored to have seen the first computer ever built, crash.

Millennium

2009-09-21T18:31:00.000-07:00

Wow! I just finished the third volume of Stieg Larsson's Millennium trilogy. What an amazing piece of writing! I really think it's the most absorbing, entrancing fiction I've ever read (and I've read an awful lot of it). Unusually for me, I read it very slowly, savoring every sentence - normally I race through fiction, but with Millennium I wanted to make the most of it, especially since I know there will never, ever be another one.

It was hard to find all the volumes. The first one I found quite by chance, through an Amazon recommendation. Then when I wanted to read the second volume, I found that it isn't available in paperback in the US. I managed to get a "grey import" of the UK edition (which apparently is a different translation). But the third volume isn't available in the UK or the US, and won't be for some time. Luckily, though, it has been available in French (and Spanish) for quite a while, so I bought the French imprint from amazon.fr. I was concerned that it would "feel" different in French, but I needn't have worried - the author's style comes through in exactly the same way in either language. The French translation has a few oddities in it, but it's nicely done and enjoyable to read from a literary point of view. (Although there's a huge blog full of nit-picking criticisms by people with no life, who say they couldn't possibly read a translation where the subjunctive future pluperfect is used incorrectly, and so on - which of course is entirely their loss).

The pivotal character is of course the extraordinary Lisbeth Salander. Obsessive, brilliant, almost pathologically introverted, hacker extraordinaire, gifted with a photographic memory, and completely fascinating. She is pivotal - but not always central. In the first book she is an important, but not central character - everything revolves around Mikael Blomqvist. The second book is really hers, with Blomqvist often absent for a long time. Then in the third book, her circumstances (I won't say more, read it for yourself) keep her out of the main action for nearly the whole book, and Blomqvist is back on center stage. Until the climax of the whole series, the courtroom scene at the end, where she destroys one of her principal enemies (she has plenty) in a single blow.

Sadly, Larsson died shortly after finishing the third volume. It is rumored that the fourth volume was well advanced, but his heirs have said it will never be published. There are quite a few places where you can see loose ends left to be picked up later, and we'll never know, now, what Larsson had in mind - did the awful you-know-who really put the pictures on his computer, or were they planted by one of Salander's hacker friends? I'm sure he was going to tell us. Maybe it's better to have three extraordinary books, and be left panting for more, than to have ten (his original goal) that gradually deteriorate into repitition and mediocrity, which so often happens with long series. (Think how keen Conan Doyle was to be rid of Holmes).

In any case, a front-runner for "best read of a lifetime". Now I just have to wait for the English-subtitled movie versions to appear, and maybe re-read the third volume when it finally appears in English.

Ergonomics and Supermarkets

2009-09-14T10:33:00.000-07:00

This morning I stopped in the Safeway next to the hotel I'm staying at, to get a couple of things. The staffed checkouts were quite busy, so against my better judgment I decided to use the self-service checkout for my three items.

I should really know better. With the honorable exception of ATMs - which somehow they got right early on and and which have stayed right ever since - just about ALL of these self-service booths seem to have been expressly designed to be impossible to use successfully without help. Which is why, for example, you will always find airline staff hanging round the self-service machines at airports. (Although they do mostly seem OK now, but for a long time they would always have some trick to catch you out just when you thought you were getting there. For example, the British Airways machines wanted to check the card used to buy the ticket - impossible if you had a company ticket issued through a travel agent. And I NEVER got a Lufthansa machine to work in the days when I flew with them just about every week).

Anyway back to Safeway. To my amazement, it let me scan all three items. As long as I ignored the irritating voiceover telling me about all the other things I could do, it seemed to be going well. Then came time to pay. I slid my card through the debit card reader, keyed my PIN, tapped "confirm", and all seemed well.

Except that the actual checkout screen said "amount paid $0.00". I pressed the help button and the store manager came along. Turns out you FIRST have to tell one screen how you want to pay, and THEN use the separate payment machine. It took several minutes to straighten it all out, made more complicated by the fact that I refused to scan my card again since I'm sure that I would then have ended up paying twice.

Now, how hard would it have been either:

a) to figure out that when you slide your card, you probably want to pay by card, so accept the payment

b) or if that is really too hard, then disable the card reader until you have made a payment choice?

In the end it all ended OK, in that I left the store with the items I wanted and didn't get arrested for shoplifting. But why does it have to be so painful?

Mount Diablo and the Morgan Territory

2009-06-01T20:39:00.000-07:00

After nearly ten years in the Bay Area (is it really that long?), there are still spectacular things that we have still to do. This has been a frustrating weekend for flying - my plane is in the shop having a new gadget installed, and my favorite Pitts wasn't flying either for its own reasons. I've flown over Mount Diablo numerous times, so it seemed like a good idea to go and see it close up.

A look at the map showed an interesting road passing east of the mountain, under the flight path that I've taken so many times between Livermore and the practice area. This is very rough terrain, and most of us have decided that in the Pitts, with its brick-like gliding ability (glide ratio about 3.5:1), it's best avoided since there are really no good options if the engine stops. The road is called Morgan Territory Road, and it is truly spectacular.

Livermore stops abruptly crossing I-580, and from there northwards is country as empty and seemingly isolated as the back country east of Paso Robles - which really is isolated! This is under my usual return path into Livermore, and as it often happens, it looks very different when seen from the ground. Soon an easily-missed right turn takes you into Morgan Terrtory Road itself, and shortly afterwards it becomes a single-track road, rare indeed around here. It twists and turns under a canopy of trees, giving occasional glimpses of the main Diablo peak, soon reaching a parking lot and trailhead. We didn't have time for a long walk, but it was a beautiful place.
After a picnic, we continued round the mountain. Eventually we came back into some dull suburbs, though constantly dominated my Mt Diablo. It's a long, slow, twisty road up, climbing 3500feet, with magnificient views south to Livermore and west to the bay. Then, suddenly, you're at the top, and the views are in all directions. Of course I've had the same view many times, but it's different when you have your feet on the ground.

This is definitely one to add to the "favorite roads" list, along with the Nacimiento-Fergusson Road, and the long drive down the San Andreas fault from Hollister to Ojai.

chirari-to (ちらりと)

2009-05-24T13:03:00.000-07:00

One of the many things that makes learning languages fascinating is the completely different way that other languages view and express the same thing. Even between similar languages like English and French, this can sometimes result in words that are absolutely impossible to translate without a lot of context, moche for example (probably most simply glossed as "not very nice", but it can be "ugly" and a whole range of "not nice"-like meanings).

With completely unrelated languages like English and Japanese, this happens all the time. Thanks to my teacher, I just came across an especially nice one, chirari-to, written only in hiragana (ちらりと). It's so hard to translate that most Japanese-English dictionaries don't even have it as an entry. If they do, they just give it in one or two fixed phrases like chirarito miru, to glance at.

But in fact, it can be used in made-up phrases and has a stand alone meaning all of its own, which is something like "glancingly". Combined with "to hear", chirarito mimi ni iru, it means "to overhear by chance". My teacher agreed that chirarito nioi o kagu (i.e. with "to smell") could mean "to catch a whiff". She didn't think that chirarito yomu (with "to read") is something people would say, but then I found it on someone's blog (chirari to yonda manga - a manga that I happened to read).

Which makes me wonder how far you can take this. I wonder whether in the right context you could use chirarito korosu, "to kill by chance and without it being really important", as some samurai might say about a peasant he'd beheaded for being in the way on his journey home? Best yet would be chirarito haru, since haru (broadly, "to stretch") is another of those impossible-to-translate words with dozens of meanings depending on the exact context. It's enough to give a translator nightmares.

(Apologies for the romaji, when I tried to type hiragana I got very unexpected results and it was just too hard to do).

Off Airport

2009-05-18T20:40:00.000-07:00

The two main reasons for using a helicopter, rather than an airplane, are (a) to land where there is no runway and (b) to hover in place. So, naturally enough, these form part of helicopter training. Tonight we did a bit of both. Both are harder than they look. We went out to a remote area and landed on hilltops and on gravel tracks. After circling round to look for animals, power lines, fences, people, houses and so on, you try to land on a hilltop. Compared to an airport (a) you don't know the altitude, so you have to judge everything by eye (b) it's surrounded by hills and other inconvenient obstacles (c) typically there are trees whose distance from the target you have to judge from afar (d) surfaces look deceptively smooth until you get up close. So when we finally got to a landing hover above our first chosen location, we realised that the grass was eighteen inches high. Probably you COULD land there if someone's life was at stake, but it's too risky for a training exercise - you have no idea what is under the grass until you get the skids on the ground, and there's always the risk of tail-rotor damage.

Off to the next place. We find a gravel track winding around the contours of some hills, and make a couple of approaches to it. But the chosen landing spot looks awful small until you get very close in. It's hard to believe you can get this big helicopter (relatively speaking) into that tiny postage stamp of gravel, surrounded by trees. I confess that as my instructor flew the short final I had visions of our rotor blades thwacking into the adjacent bank (forgot to mention that part). In fact there was plenty of room, or adequate anyway. But then you have to get off again, with said bank directly in front and trees on either side. No problem (if, like my instructor, you have 10,000 hours of heli time) - lift off into a 5 foot hover, 180 degree pedal turn, and swoop back off the way you come. But it takes nerve and experience - which of course is what the training is all about.

We survived all that, and went off to practice high-altitude hovering - OGE is the technical term, short for "out of ground effect", i.e. far enough away from the ground that it doesn't interfere with the down-draft from the rotor, which provides extra lift when close to the ground. It's surprisingly hard to enter a hover smoothly. For a fixed-wing pilot the idea of stopping just doesn't come naturally, so as the airspeed gets down to 40 knots or so it already feels like you're standing still. It takes very careful control of the cyclic and collective to get to zero airspeed without losing control of altitude - either climbing or descending. We were doing this just under a Class B shelf, which made it more important not to gain altitude - as my instructor pointed out rather forcefully when I got to within 100 feet of it.

Then you have to stand still. The ground is a long way away when it comes to judging position to within a few yards, and a 25 knot headwind didn't make it easier. The GPS helps a lot, although you have to remember that a ground speed of 10 knots can be in any direction - the GPS doesn't know which way the heli is pointing.

So after a bit of practice at that, we took advantage of our hover to practice "settling with power" also called "vortex ring state". This is one of the peculiarities of helicopters. If you descend too fast with very low airspeed, the rotor falls into the downward-rushing column of air that it is creating itself - and stops providing any lift. Recovery, as long as you have enough altitude, is similar to stall recovery in an airplane - push the nose down and accelerate out of it. If you don't have enough altitude, you will hit the ground very hard - which is why it's best to practice at altitude (also like stalls).

Then back home. If all goes well, my next flight will be another solo.

Helicopter Solo!

2009-05-16T15:42:00.000-07:00

Well, I did it! Today, after exactly 24 hours of helicopter dual, I soloed. You might reasonably wonder what consititutes a solo in a heli... three two foot hovers? But it's just like fixed-wing, three times round the traffic pattern. Except that each take-off involves a clearing turn, so actually it counts as six landings.

As with my other solos (fixed wing, and the Pitts), I was so concentrated that I didn't remember to be nervous. That's probably a good thing. Disappointingly, I didn't get to see the Collings B-17 taking off from adjacent Moffett Field during my solo - things seem to be very quiet there this year.

The first five hours or so of heli training are about the most humbling experience you can have, in flying anyway. The aircraft seems completely uncontrollable as you zoom backwards, forward and sideways, desperately trying to stand still and failing utterly. Then, when you've oscillated too far from your spot, a 10,000 hour instructor takes the controls and with perfect precision plops you back in exactly the right place, before saying "your controls". And it starts all over again.

Actually flying the heli isn't too bad, at least if you have airplane experience. In flight it behaves pretty much the same, except that it is a lot more sensitive to the slightest movement of the controls.

Most of my 24 hours dual time, once I could take off, hover and land with reasonable accuracy, was spent on emergency procedures. Mainly, this means autorotations. A heli will fly just fine without an engine, though not for long, and unlike an airplane, it doesn't need a runway. I think autos are probably terrifying at first - unless you are used to landing the Pitts, in which case they seem very gentle.

Now on to 10 hours of solo, 3 hours of night flight, and more emergency procedures...