failure is the chance to do better next time

My fortune cookie this weekend said failure is the chance to do better next time.

As with life, so with sports. To reach out beyond what we are accustomed to, we have to (at least, gingerly) explore the discomfort zone.

Recently, I've been having problems getting my body to accept something beyond that 10K run at 7.0 mph, which I know I can repeat fairly comfortably (see Natural Training).

So I tried pushing things a bit last week.

However, running at 8.0 mph only got me 6 minutes before entering the red (anaerobic) zone (On the edge).

So I tried again on Saturday, this time at 7.5 mph.

As you can see, I basically tried to run in blocks of 10 minutes at 7.5 mph (1.33 miles) to keep the heart rate (HR) from entering the red zone. Then walk for 5 minutes inbetween to allow the HR to drop.

How do we evaluate this? Well, a measure of fitness that's commonly employed is to see how fast the HR drops during recovery (the faster it drops the fitter you are). What happened was this:

1. At the end of the first block, my HR was 164 bpm. It dropped nearly 60 bpm to 106 bpm during the recovery. An encouraging sign.
2. 2nd block, I started to run into trouble. My HR hit 170 bpm by the end of the 10 minutes. This is in the anaerobic zone for me. The HR dropped 55 bpm to 115 bpm during the 2nd recovery zone.
3. 3rd block, I knew I was done. After 8 mins @ 7.5 mph, I was already back up at 170 bpm. So I shut it down.

My body's wimpy response to the 0.5 mph (1 km/hr) increase was kinda disappointing. Total fail.

I tried to do these 10 minute blocks as a natural interval workout, hoping to get my 10K in. Five 10 minute blocks would have gotten me nearly 11 km. Unfortunately, I stopped just shy of 3 blocks.

As any cyclist who has trained using a powermeter knows, time spent in the red zone is always costly. You have a fixed and limited number of minutes in that zone for your body. Every minute spent there will cost you aerobically in the end. And distance running is all about aerobic capacity.

So I knew every minute I spent at 170 bpm (rather than at 160 bpm) would hit me hard on the next block to come. And as I hit 170 bpm just 7-8 minutes into the 3rd 10 minute block, to paraphrase my friend Barry Dattel, there's no way, no how I could have completed the 5 blocks I had planned.

I have my excuses ready of course: it was a busy first week. I taught 4 times. Many meetings. And prepared and gave a colloquium talk as well. So by Friday I was all in the red, so to speak. Plus my body is not yet in marathon shape. Blah, blah, blah... but ultimately unconvincing.

After the frustration with the treadmill, I completely over-compensated and took it out on the spinning bike. This was workout #8 below:

I spun 615 kcal in 43 minutes (rate of about 860 kcal/hr). Average HR on the bike was 142 bpm, significantly higher than my normal relaxed time on the bike. In addition to the 467 kcal from the treadmill, the total kcal was nearly 1100 kcal. Yes, the workload was a bit high for a short workout. (1500 kcal would be a medium workout in my book.) But hey, it was a Saturday.

Taking a step back, the fortune cookie's optimistic interpretation of events, namely failure is the chance to do better next time provides perspective. We should see opportunity in every failure. There's a silver lining in every cloud and all that.

Case in point, this week an aftermarket part was not working properly anymore in my car.

Monday morning, I dropped my car off at the place that originally installed the offending part, but I had nobody to give me a ride to work.

I brought out my race bike, which has scarcely turned a wheel in anger since PacTour Elite Southern Transcontinental - and that was back in 2007, nearly 4 years ago (see Day 17).

And I rode the five miles or so to work in 100F heat. To my surprise, I felt fine. Arriving at the office, I smiled and thought: hmm, maybe I am not doing so bad after all. 106F for the afternoon trip back. But hey, round trip, it'll be ten miles. Good preparation for tonight's tilt at the treadmill.

Epilogue

As I promised, I tried again today after work.

Monday's results are obviously significantly better than Saturday's. Instead of three 10 minute blocks at 7.5 mph, I got nearly four 10 minute blocks in at 7.5 mph. (And getting in the target 5th can't be far away.) As a result I burned 615 kcal on the treadmill instead of 467 kcal on Saturday. Plus another 321 kcal on the spin bike afterwards gave me a satisfactory Monday total of 936 kcal. I call that a good start to the week.

Obviously, there are no scientifically valid improvements in aerobic capacity that are realizable in only 3 days.

The red line is the heart rate.

1. As the graph shows, I reached 165 bpm at the end of the first block. It dropped to 110 bpm (55 bpm delta) on recovery.
2. 2nd block, I reached 170 bpm, dropping to 119 bpm (51 bpm recovery delta).
3. 3rd block, I reached 172 bpm, dropping to 125 bpm (47 bpm recovery delta).
4. 4th block, I hit 172 bpm by minute 7, so I shut it down at minute 8. If I had persevered, the recovery would have been worse again.

These HR numbers are much as the same as on Saturday. There is no training effect possible so quickly.

So why was I able to last longer 2nd time around? Well, immediate gains are the result of neuromuscular adaption to running at 7.5 vs 7.0 mph. Basically, the muscles in the legs first get used to firing at an increased rate.
(Incidentally, same thing is true for weight room workouts. Initial gains cannot be from muscle size adaptation.)

The other effect is learning to survive at the higher heart rate (HR). Back when I used to train on the bicycle, I called it "learning to live in the 170s", i.e. getting used to the HR staying in the 170s bpm range for an extended amount of time. Of course, as aerobic improvement accrues, the sustained HR will drop for any fixed speed.

on the edge

On the edge of beginning a new semester (tomorrow I begin teaching), today is a last chance to get a triple workout in.

I ran on the treadmill, hopped onto the Spin bike, and finished with a 2 hour table tennis training session all in one evening after work. I'd probably be lucky to get a single short workout in every other day once things are in full swing around here.

On my previous blog entry (natural training), I mentioned that now that I can knock out a 10K run comfortably, it was time to bump up the speed and see how far that can take me.

It was time to put my money where my mouth is.

I began with a 10 minute warm-up, basically walking at 3.0 mph to get the circulation going but including a 2 min 7.0 mph rev up of my cardio engine.

Then I bumped the speed up to 8.0 mph on the treadmill (1.0 mph more than I've been running at recently). The Nike+ download reported an average pace of 7'13" min/mile but as you can see I didn't quite manage to complete mile one.

Adding in a 5 minute cooldown, with the warm-up that's a 20 minute workout. Very short, but since I went anaerobic, it's not wise to push it.

The HR graph tells the real story.

The average HR was 154 bpm (a fairly low number, I sustained an average of 161 bpm on the comfortable 10K workout) but this is totally misleading.

By minute 6, my HR had busted through my comfort zone. As you can see, I hit 169 bpm and the pace felt unsustainable. In the discomfort zone, I probably could have pushed it for a bit longer, but since I'm following the natural training method, I shut it down.
(The idea is that without forcing myself to follow a formula, I should be able to naturally adapt and delay the onset of unsustainability significantly by simply repeating this 8.0 mph workout once a week.)

However, since I only burned 99 kcal in 6 minutes, it was onto the spin bike. So I tapped out an extra 458 kcal in just over half an hour under the video guidance of the ultra-cool video dude with hair gel who never seems to sweat during the ride:

After that, I managed a good 2 hour table tennis training session. And the luxury of this third workout of the evening was only made possible because I shut down the 8.0 mph treadmill workout before I managed to tire out and trash my legs. As Dave Edmunds sings (see YouTube):

Midnight, I'm a-waiting on the 1205
Hoping it'll take me just a little farther down the line
Moonlight, you're just a heartache in disguise
Won't you keep my heart from breaking
If it's only for a very short time
...

Like that orange Nittaku training ball in the first picture (which incidentally came to a rest of its own accord in that position), I'm living on the edge hoping it'll take me just a little farther down the line...

natural training

Usually, I train with some goal and set intermediate goals along the way. For example, in running, one simply increases mileage a little each week. That way, the body is given time to adapt and injuries are less likely. Then at the end of a few months, run that goal event: usually a marathon. Magazine training plans are usually like that.

However, one doesn't necessarily need to be a slave to a pre-planned training regime. While it is true one must continually push a bit in order not to stagnate, one could also let it happen naturally, i.e let your body set the ramp factor.

For example, consider the above graph. I do a short workout twice a week at the gym (Mondays and Thursdays) immediately after work. I try to keep the total workout to a modest 1000-1200 kcal. Any more than that, it's not a short workout because I may not be fresh the next day.

Monday	Tuesday	Wednesday	Thursday	Friday	Weekend
short run #1	ping pong	ping pong	short run #2	rest	long

(Tuesdays and Wednesdays, I need to hop around the table for ping pong, so my legs can't be heavy or trashed. Actually, I could do a heavy workout on Thursday since Friday is my rest day.)

Anyway, here's the theory: you run as much as you feel you can comfortably do twice a week. The difference is this: under my old training regime, I'd finish the planned distance no matter what: I'd use a bit of mental willpower and gut it out if necessary. Sounds a bit old-fashioned. With this new scheme, I do the same total workout (in terms of kcal) so I don't need to feel guilty I'm slacking off.

I simply just move the workout to the spinning bike (to burn off the remaining required kcal) as soon as I feel I'm having to utilize willpower to stay on the treadmill. (And since I used to be a serious ultra-distance cyclist, the spin bike portion takes zero willpower and is always easy to complete - after all, I'm mostly sitting down.)

"The treadmills: the blue zone"

"The spinning bikes: the red zone"

So the blue bits on the graph are from my treadmill, and the red zones are from the spin bike. Together, the kcal stack up as shown on the y-axis. For example, you can see I only ran for 310 kcal (for 20 minutes) for the first workout on the graph, but then topped it up with 420 kcal on the spin bike. By session #7 (last Thursday) I burnt 740 kcal on the treadmill (I ran a bit over 10 km), and I only did 310 kcal on the spin bike: total just over 1000 kcal. Since I am getting progressively fitter on the run, so it takes up a progressively larger slice of the workout. The key is that the ramp up is dictated by how I feel rather than sticking to some formula.

So in 7 sessions, I'm up from lasting 20 minutes comfortably on the treadmill to over 50 minutes (at the same speed). All well and good, but how do you know when to get off that treadmill? Obviously, it's easy to decide if you are running flat out. You get off because you can't hold the pace anymore. But what if you are running at an easier pace? Well, I use my heart rate (HR) and perceived exertion (RPE) to decide.

As documented in a previous entry (see iPod Nano 6G), I use an iPod Nano with the Nike+ kit. Not only do I get to listen to my specially-prepared run music playlist, it also records the pace and my heart rate. For example, here's session #7 (from last Thursday).

As you can see, it reports an (effective) flat line for the pace as expected since I set the treadmill to an easy 7.0 mph and never touch the controls during the workout. (Small variations are probably due to momentary changes in stride as I grab some water or due to limitations of the Nike+ transmitter.) The HR graph is more interesting though:

As you can see, my HR ramps up fast and then settles down, averaging 161 beats per minute (bpm). This is an aerobically sustainable level for me. There is some tilt upwards to the graph (as I slowly dehydrate and retain body heat). I think I end up around the 164 bpm level.

I could have kept it going but I had completed the target 10K (6.2 miles).

In addition, my left ankle was dripping with blood (see left) from chafing. My running was so relaxed muscle-wise, occasionally the right heel would scrape the left ankle. I have to control that right heel movement a bit more in future. But it's good my center of gravity was tightly localized.

If my HR keeps rising, it will eventually move into the anaerobic zone (unsustainable). So we can use a HR limit (e.g. 165 bpm) plus RPE (e.g. "damn, I feel terrible!") to decide when to stop. Otherwise, I'd push myself deep into the red zone - which would be a pyrrhic victory of sorts. In other words, unworthwhile and actually counterproductive since I won't be able to recover fresh for the next day.

A quick word on the spinning bikes. I really like the latest generation.

The spinning bikes are a pleasure to use. These newest ones have a large touch screen. I can follow the very cool (recorded) video of the guy. If he gets out of the saddle, I follow his lead. If he jumps, I jump. I subconsciously follow his pedaling cadence. And at the end, when he stretches, I just follow him. The electronics provide a suggested HR zone. However, since it's a spinning bike, I get to set the resistance knob.

Here is an example of a workout. I rode for 42 minutes and burnt off 530 kcal. I spun at an average cadence of 91 (no mashing). And my average HR is 130 bpm - much lower than on the run, an indicator that it was an easy 530 kcal for me.

Now that I've gotten up to my 10K the comfortable way, what's next? After all, as I mentioned at the beginning, one must continually push a bit in order not to stagnate.

Since I do this workout twice a week, the key then is to bifurcate my efforts.

• For workout one, I can bump the speed up to 7.5 mph or 8.0 mph and start from 20 minutes again (or until my HR goes in the red zone) to build speed.
• And I can simply run for longer for workout two. For example, back in May before I lost my fitness, I tapped out a 1 hour 45 minute run at 7.0 mph (see A half marathon). Alternatively, it's better to do the longer run outside (as soon as it dips below 95-100F in the afternoons in Tucson).

Curiously enough, I was not particularly happy with that half marathon run back then. At this point, I'd be ecstatic to be able to nail that down. It's all relative... :)

bad sneakers, do it again

Imagine, if you will, my car smoothly eating up the miles in the desert on I-10. I'm on my way home from Los Angeles.

A mere press of the window lift away, it's over 110F (43C) but inside, it's deliciously cool and quiet. As the sun reddens behind me, I settle into the cooled seat which will be my home for the next six and a half hours. Relax, connect up the iPhone, in seconds I'm down in the world of Steely Dan's sophisticated mix of jazz and melancholy.

As the doubtlessly well-informed reader that you are, you already know I'm listening to them from the title of this blog entry. I reflect on what has been a good visit.

Five names that I can hardly stand to hear
Including yours and mine, and one more chimp who isn't here
...

That's how the song Bad Sneakers begins. For a moment, I think back to New Jersey days. Those Butterfly 808 table tennis shoes pictured above look brand new but are over 12 years old. Having languished in a dark box for all that time, they are essentially new, unused after I ordered them back in the late 1990s.

I figured it's about time these "sneakers" emerged into the light again. Since I've restarted table tennis, I've stuck to my marathon running shoes for cushioning and comfort. I've been re-purposing my Newton Sir Isaac neutral guidance trainer (see picture at the bottom of my prior entry A Half Marathon). You may be surprised to know these Butterflys (538g) are heavier (and offer less protection) than my Newton neutral racers (500g).

(Out of all the sports I dabble in, my bicycle racing shoes are by far the lightest, at around 320g. Yup, one Butterfly shoe is nearly two bicycle shoes.)

But, as the saying goes, it's very much horses for courses. Those Butterfly shoes have a very low heel (to protect the ankle) and (by design) a natural rubber sole that balances the ability to slide and make small adjustments with grip. With respect to cushioning (to protect against fatigue), let's just say they work best with a professional (cushioned) floor that the best clubs have.

As long as we're talking numbers here, have you ever wondered why cars report the outside temperature but never the inside?

As you can see, the automatic climate control is set for 74F (23C) but the actual temperature at head level is 32.3C (90F) with 12% humidity (it's the desert).
(Notice the fan is barely running. In other words, the car thinks it has reached the correct temperature.)

Back to the coolness of Steely Dan (and Do it again):

You go back, Jack
Do it again
Wheels turning round and round
You go back, Jack
Do it again
...

There are no quick fixes in table tennis (or life for that matter). Old habits are ingrained into muscle memory and die hard. Once we recognize our technical problems, it takes considerable work (i.e. time), dedication (i.e. make time) and a certain kind of optimistic stubbornness (it will come, it will work out, give it time). Those who promise quick and easy fixes are mere charlatans.

When you know she's no high climber
Then you find your only friend
...

I'm not completely sure what some of the songs mean, but people have said this breakout tune of theirs is about addiction, second chances and the inevitability of fate. I muse, hmm, this is my 2nd time around at table tennis. You could say it's an addiction. But I'm not sure what fate has to do with it...

Perhaps then my occasional weekend trips to Los Angeles are about a leap of faith.

1. Dedication: I have to drive 13 hours round trip.
   I prefer it to flying because of convenience. Flying basically involves nearly 5 hours each way counting time spent getting to the airport, parking, going through security, waiting around, renting a car etc., only actually saving 3 or 4 hours round trip. So far, unless I have been driving all night, I have been able to arrive alert and relaxed enough to take a lesson after a short break.

2. Work: to maintain quality, a lesson lasts only 90 minutes.
   Why? Well, multiball is physically demanding. I go through three shirts (sweat saturated), two bottles of water, a Gatorade and a towel in just 90 minutes. Willing the body to continuously move, move, move to get to a position and hold balance, stop momentarily and get the shot off. It's not about mindless or brute force strength. The coaching is intense: every improperly played ball (never mind the missed ones) gets reviewed and highlighted. Nothing is missed. Correct form is everything. Struggling with yourself on a conscious level is also mentally taxing (not to mention, frustrating, at times).

3. Optimistic stubbornness: that's a few sessions per weekend maybe once every other month or three. (That's pretty far from the ideal of training 3 or 4 mornings a week.) It's a legitimate question to ask whether anything can be learnt and retained in such short bursts. Without constant feedback and application, one tends to revert back to old habits.

At this point, I can report that I am finally starting to see glimpses of the promised land. As I mentioned earlier, give it time, it will come. My body is beginning to react and move differently. Because how you move and maintain balance is everything, it affects all aspects of the game. Tension has been transferred from the shoulders into the forearm. Suddenly I go from looping two or three balls to twenty or thirty in a row. They're not all perfectly minted, mind you. But now and then I feel my center of gravity doing the right thing. And it's definitely not the shoes :)

Fundamental change is possible, and on my long drive through the desert, that thought makes me happy.

For match practice, I also play in LATTA's Saturday afternoon's advanced league. Typically, I take a lesson in the morning at 10:30am. Then break for lunch at noon.

(There is a cheap chinese eatery nearly directly across the road. They also serve protein shakes not on the menu.)

Back to the club at 2pm, it's a round robin group format. You play 5 or 6 matches. Because it's such a large club with several leagues, the level on Saturday afternoon is good, all are hardened and experienced players. Competition is ferocious. No match is easy. As I can easily see myself losing every match, I'm amazed sometimes I even manage to win any. After the round robin phase comes the single elimination phase. It all finishes around 7pm.

For example, last Saturday I played a total of 32 games (including 10 straight with a guy who wanted to learn my style and me, his). By this point, I've put in close to 7 hours. I'm tired and ready to book myself a long massage. Despite some lingering fatigue, the underlying fitness is there. I know I can recover in time for tomorrow morning's 90 minute three shirter.

On Sunday morning, I'm a bit stiff but after warming up, it's time to see if recalcitrant muscle memory can be bent a bit more until it yields. After the lesson, it's noon time and I've made special arrangements to play my 1600 nemesis, Zhu Min Ming (see earlier LATTA entry). I've never even managed to take a single game off her but she is kind enough to offer her number and say practice with me. Her consistency and control game is way ahead of mine and her long pips backhand isn't easy to handle. However, this time there is a titanic struggle with long rallies. No matter how many loops I bang in, she always has an answer. But somehow this time is different. I finally manage to prevail and come out 3-1 ahead.

I'm carrying my fortune cookie from last Sunday.

It's an epochal moment. I know I have passed some test.

(I have a sneaking suspicion she makes sure she is perennially underrated. She is almost at 1900 now and needs to start discounting again.)

At lunch, she calls me and says someone is waiting to play you.

It's like something out of an Indiana Jones movie or kungfu novel. It seems Zhu Min Ming has passed me onto challenge number 2. His name is Duc Loi, and he is over 60. Plays every day. Can chop away from the table. Can stay at the table and work you over with placement. His speciality is attacking with the long pips. He is rated over 2200. I am warned even 2400 players lose to him first time around. Of course, I lose 0-3. Like Zhu Min Ming before, he offers me his card and says call me to play when you are in town. I know he is a benchmark. 過三關 and all that. It's definitely like something out of a movie.

Appendix

One of the advantages of Los Angeles is that it is a big city with large ethnic populations. Population density makes for convenience. You can find everything within a 3 mile radius. The Monterey Park/San Gabriel "ghetto" is one such enclave. And local competition ensure low prices and high quality. It almost fully mitigates against the traffic, lack of green, open space and poor air quality that characterizes the area.

There are 3 table tennis clubs run by high level professionals within a mile or so of each other. I've never been to the others. The LATTA is the largest.

I have also established a routine here. I know where to go for what. I typically book the same hotel or another one. I trade off convenience against an oasis of green and quiet perched up on a hill. Either way, it's just five or ten minutes more.

Saturday morning, I go for dim sum before my 10:30am lesson.

Always at the same place (海寶潮粵海鮮酒家), they're open early. No queuing up. And more importantly, they always have my lotus seed paste buns (蓮蓉包) correctly configured with egg yolk. (I'm very particular about that.) And it's pretty good but not expensive. (Okay, whaddya expect, it's no match for 稻香.) Still, no reason to go anywhere else.

I always go to the same massage place (快樂按摩) straight from the club. I choose them because they have a shower so I can clean up first. $25/hr plus tip is the going rate around here.

I know where to go for cheap food. I have a favorite noodle place, a decent thai place. Even the $12.95 sushi place is unpretentious and the taste is fine. Back in town, they'd make a big deal about it. Here, it's just another place. (Okay, it's not Sushi Say (寿司清) Tsukiji main branch (築地本店) in Tokyo, but this is frickin' Southern California, whaddya expect?)

And for the observant, yes, that sign shows $9.99 for one hour foot massage. Amazing isn't it?

A tip of the iceberg

This blog has always been about the non-work component of my life: e.g. marathon running, cycling or working out in general. Some topics are obviously terribly geeky and of limited general interest, e.g. table tennis or spherical panorama photography.

Recently, I've been mainly working and working out. On the work side, one of the things I sometimes do is write computer programs that implement (mainly scientific) theories that I'm interested in. In particular, I'm interested in theories of grammar. Note that I keep mentioning this word "interesting" again. I suppose I'm kinda lucky in that in some sense there is a component of my work that would also be a hobby of mine. The programming side does not necessarily stem from practicality: I simply enjoy making these theories become tangible. I am a professional computer programmer. But I like to think it's my hobby as well.

As with hobbies, let me quote Alan Parsons Project 's Limelight and say "maybe the prize is small," but the time and effort that goes into parts of a programming project is sometimes ridiculously disproportionate.

To illustrate this point, let's take a simple example. I'm currently rewriting a program maybe for the 4th or 5th time in its lifetime. I've gone from hosting the program initially on a Symbolics 3650 Lisp machine (with microcode assist for Prolog), then a Sun Sparc workstation, then Apple's Powerbook (PPC), and finally to a Macbook (Intel). (I apologize in advance for the super-geekiness in this post. In an attempt to not lose the plot, I've put the most dire and geekiest parts in separate paragraphs with small fonts.)

As an aside, the Symbolics machine (see here) provided a programming environment that is truly unequaled. How can you match a system that provided seamless full source clickability from your application, through the windowing and underlying operating system, all written in Lisp? Or a keyboard that provided Hyper, Super and Meta as well as a Control key?

The core of my program implementing the theory I was interested in hasn't changed very much apart from porting considerations. It's mostly written in Prolog, a logic programming language that is way past its heyday, plus bits in raw C for efficiency. (Remember, computers used to be so much slower.)

Over the years, I've had to port over various flavors and wrestle with differing foreign language interfaces, from Quintus (commercial), to SICS's Sicstus (semi-commercial, restrictions on runtime environments) finally to University of Amsterdam's SWI (freeware). Freeware at last! These days, I've also taken out the C bits for portability. As anyone can tell you, dynamic library linking is simply no fun when OS libraries are patched and released all the time.

The part of the program that has changed greatly has been the user interface component. Unfortunately, it has been completely rewritten from scratch a number of times. That came with the lack of standardization. The latest rewrite, the subject of this blog post, uses the Tcl/Tk toolkit with native Aqua Mac OS X support. Here is a sample of what it looks like now:

I'm a design minimalist at heart when it comes to user interfaces (as well as other things). I believe user interface complexity can be overwhelming; it's best hidden from view and enabled only when necessary to the task. Note that there is only one button visible ("clear trees"). In the relatively austere-looking window above, there are additional menus and detailed information hidden from view that can be displayed by clicking on or passing the mouse through relevant display objects. In this blog post, I'll concentrate on programming the behavior of a single detail, the blue sideways tab near the top-right of the window.

Tcl stands for Tool Control Language, a simple interpreted programming language. Tk is a graphical user interface toolkit that can be used with Tcl. In contrast to native toolkits from Microsoft or Apple, Tcl/Tk is multi-platform and well supported natively. Moreover, due to its interpreted nature, you don't need to recompile or relink your application each time you make changes. There are other viable choices, e.g. Java and Swing from Sun Microsystems (before it was bought out by Oracle), but Tcl/Tk has that spare minimalist look to its programs that appeals to me.

Initially, the user interface was written in Lisp, effortlessly borrowing the tree layout and mouse clickability facilities that came with the Symbolics programming environment to render syntax trees. Then it was onto the X Window System on Unix. Remember X11R3 and X11R4? The C-based XView libraries plus Slingshot extensions was supported by Sun Microsystems and implemented the OpenLook interface guidelines. X11 for Mac OS X allowed recompiled executables to run on the Mac. However, generational changes in the X11 libraries as Mac OS X morphed over the years resulted in niggling lost functionality in the best case and random core dumps in the worst. Moreover, it didn't mesh well with the arguably slicker native Mac OS X graphical user interface.

A menu of commands (useful mostly when starting up or quitting) is hidden under that blue tab.

As a design minimalist, I don't want those commands unnecessarily cluttering up the user interface or inconsistently taking up valuable screen real estate.

But if you mouse over that tab, the tab extends out a little to the left, and the command menu pops up as shown here.

However, when the program is first launched, this design minimalism results in a seriously over-sparse window. See below.

It's not terribly clear how to proceed or what to press. So, as a helpful hint, I thought it'd useful to automatically pop that menu up at launch, as if the tab had been manually "moused over", and the user can see he/she should select something intrinsically useful from the menu like "load defaults", which in turn will lead to more relevant interface components materializing.

It's not really necessary to explain in detail how this programming is accomplished. (See below.) It's the idea behind it that matters; in other words, that hinting at startup might be helpful and appreciated by the user.

The way to elegantly pop up the menu without writing much additional code is simply to simulate moving the pointer over the blue tab. User interface code is event-driven. Therefore, we write code that "binds" or associate actions with specified user interface events.

For example, if the pointer moves over the blue tab, a logical <Enter> event is generated. If we have previously associated the menu with <Enter> for that tab, the menu will pop up reactively to moving the mouse (or finger on the trackpad) to the right location. So at startup, we pretend that the user has moused over that tab simply by posting <Enter> blue tab to the event queue and letting the user interface code do its work.

For those curious, here's what it looks like in Tcl/Tk:

1. label .f.msg.tab -image tab_blue -width 23 -height 32
2. bind .f.msg.tab {popup_tab %X %Y}

Line 1 says .f.msg.tab is a label displaying the blue tab, and line 2 says we've bound popup_tab to an <Enter> blue tab event. In our startup code, we post the event virtually by simply saying:

event generate .f.msg.tab -x 10 -y 10

(x and y are the screen coordinates.)

We would have nothing further to say except that, unexpectedly, the obvious code doesn't work. In particular, not only the menu doesn't automagically pop up, worse still, it generates an incomprehensible and untraceable error as shown below:

$ wish interface.tcl
popup_tab 622 87
bgerror failed to handle background error.
    Original error:
    Error in bgerror: wrong # args: should be "text id"
^C

(Here, wish is the name of the program that interprets Tcl programs. And interface.tcl is the file that contains the source code that implements the graphical user interface shown earlier.)

Never mind about the aesthetics, programs that bomb out aren't acceptably user-friendly. Like the tipping point of an avalanche, this error generates a cascading fury of activity by the programmer that only ends when the dust has settled and a new equilibrium has been reached. Well, for the programmer the choice is simply to give up now or concentrate furiously - lest you lose that train of thought - and doggedly follow the chain of inquiry to its logical conclusion and eventual solution, at which point 10pm has somehow morphed into 4am or 5am accompanied by blurry eyes and stiff, aching shoulders.

(And that's of course if you're fortunate. If you're not, you're out one night's sleep and the problem still stands.)

I'd like to think it's not necessarily obsessive compulsive behavior, rather it's the nature of the beast that forces these debilitating all-night debugging sessions. Having been completely immersed in the details of chasing the bug down for hours, a lot of short-term memory has been committed to the task at hand. You can't just simply pull the plug or switch off in the middle of it all; it's remarkable how many of those details committed to short-term memory will vanish in the dark of the night. Next morning, you could be faced with the overhead of considerable detective work to retrace and piece together again the current half-analyzed state of the puzzle. You could try to write down everything you've done so far before going to bed (to explain things to tomorrow's you), but it's far easier just to get on with it and burn that midnight oil.

The hunt for an explanation and a solution

So how do we track this one down? Well, for a start we have to be reasonably sure we haven't made a mistake in the code. We first confirm, that without the virtual event, the menu does pop up reliably. From the Tk toolkit's perspective, it's desirable that there should be no semantic distinction between a simulated and an actual event.

This example I've chosen is not as trivial as one might expect. The ability to post virtual events as if they were truly real is an important and valuable feature for user interface toolkits. For example, this would allow demos to be easily built. And seeing is understanding sometimes. The program could automatically show the user how to perform certain actions.

Understanding the behavior

The first clue to the puzzling behavior comes when I instruct the program to delay posting the virtual event by a few seconds, and I click to bring the program's window to the foreground before the event is posted. Ee bah gum, the menu pops up without the error message! Unfortunately, I can't really always ask the user to foreground the window quickly at startup before the virtual event posts; if he/she could grok that, they probably wouldn't need or necessarily appreciate the menu hint in the first place. Saying this is a feature not a bug would also be a serious cop-out. We are better than that (I hope)!

Going to the documentation

The next step is to look at the Tk documentation for generating virtual events:

The last line provides the clue we need to gain traction in our inquiry. It states "Certain events, such as key events, require that the window has focus to receive the event properly." Of course, this doesn't match the error message we received: Error in bgerror: wrong # args: should be "text id". And "receiving the event properly" is kinda wishy-washy and not properly defined. Plus a mouse or pointer event isn't really the same thing as pressing a key on the keyboard. But an experienced programmer will smell blood immediately.

The reason why this is a good clue because the program works when we delay the virtual event and click first to foreground the window, thereby giving it focus.

Going to the comp.lang.tcl.mac mailing list

Okay, we need to be able to write code that forces focus to be assigned to our program window as soon as the process is started. If we can do this before the virtual event is posted, we'll have no error message. Unfortunately, despite perusing the Tk documentation frantically, there appears to be no way to force Mac OS X to do this. Do we give up? No, of course not. A good programmer instinctively knows someone out there must have encountered and bitched about this problem. It'll just require some efficient and judicious application of the right search terms to ferret out the discussion. Let's go to the web then.

In the dark days before graphical user interfaces, Safari browsers and the World Web Web, there was Usenet. People could post questions and issues to a hierarchy of newgroups, the contents of which would be asynchronously transmitted on the Arpanet from host to host. Given the limits of storage back then, a typical hosts would retain a few weeks or months of these sometimes valuable discussions. Nowadays of course, Google is your friend, and all these discussions have achieved immortality, being permanently stored and publically available to all and sundry.

comp.lang (computer languages) is a sub-node in this hierarchy, and comp.lang.tcl is a subgroup that deals specifically with the Tcl language. And comp.lang.tcl.mac is a subgroup that deals with Tcl on the Mac platform.

Our instincts are correct. Someone named Steven back in 2009 asked the following relevant question on comp.lang.tcl.mac:

And someone gave an authoritative and informative reply:

The informative reply requires a bit of decoding though. tclCarbonProcesses extensions and a teapot?

The first thing a good programmer would do would be to check if these extensions might already be available on his computer. After all, Mac OS X already occupies several gigabytes on his hard drive. Unfortunately, as the following dialog shows, it isn't.

$ wish
% package require tclCarbonProcesses
can't find package tclCarbonProcesses
% package names
ttk::theme::classic http tcl::tommath tcltest ttk::theme::default Ttk ttk::theme::aqua msgcat Tcl ttk::theme::clam platform tile Tk ttk::theme::alt

Not to worry, we'll just have to download and install this tclCarbonProcesses extension package.

tclCarbonProcesses extensions and a teapot

This is where things start to get a bit hairy. I didn't have luck with the repository teapot.activestate.com. However, there is a wikipage for tclCarbonProcesses.

But the web link provided is broken. But the file tclCarbonProcesses.tcl seems to be all there. So we have the source. And it's written in Tcl.

But wait a moment, this is no ordinary Tcl program. tclCarbonProcesses appears to be a Critcl wrapper for Mac OS X Process Manager services. So we need critcl first. And of course, it's not installed on our computer. And what in heaven's name is critcl anyway?

A critcl wrapper

More googling:

Critcl, which tclCarbonProcesses.tcl requires, is an acronym for Compiled Runtime in Tcl. Very cute. It allows C code to be inline embedded in Tcl. I have a vague uneasiness on many fronts about this, but having come this far, let's press on.

Let me briefly explain the source of my unease. The reason why I'm using the Tcl/Tk toolkit is to achieve platform independence. And I've also tried to eliminate linking in specific C code for portability reasons as well. tclCarbonProcesses.tcl is both platform-specific (Mac OS X) and apparently embeds C code. Moreover it appears a bit kludgy and might not be supported or work in future editions of Mac OS X.

Furthermore, it appears from the critcl manpage that critcl is supplied as a Starkit. And we need something called Tclkit in turn to install and run it.

Tclkit

Are you still with me here gentle reader? To recap, I need to download and install Tclkit because critcl needs it. And I need to install critcl because tclCarbonProcesses.tcl needs it. And I need the tclCarbonProcesses package because it allows me to set focus for my program window. And I need to set focus because Tcl needs it to support virtual events without raising a bgerror.

At this point, even a brave and intrepid programmer might think it's prudent to reassess the situation. After all, experience tells him there could easily be trouble getting any of Tclkit, critcl and tclCarbonProcesses to install and work properly. And none of them can be skipped or omitted: all of them are required in order to get the programmatic focus he so desires.

(The programmer has also started referring to himself in the third person, rather than the first. It's okay. He is feeling somewhat detached from the real world by this time.)

Moreover, the programmer ruefully rubs his chin and notes that even if all of the packages are installed and working properly, and his program manages to grab focus via a command, the error might still be there. After all, who is to say there is no difference between focus achieved manually (via a mouse click) or programmatically? This is not some utopian environment in which events, real or imaginary, get to live harmoniously together. It's the real world of imperfect computer programming languages and flakey windowing systems. In other words, there is no guarantee the fix will work anyway.

But it's getting late in this almost poker-like game where the stakes keep getting raised. He realizes there are far too many single points of failure in this scenario. But he is in too deep to back out. He has wasted hours already. Fully committed, it's all or nothing.

He finds there are two version of the binary executable tclkit for darwin (Mac OS X) floating around on the internet. He optimistically installs version 2.

$ ls tcl*
tclkit-darwin-univ-aqua 2 tclkit-darwin-univ-aqua.gz

$ mv tclkit-darwin-univ-aqua\ 2 tclkit
$ chmod +x tclkit
$ mv tclkit ~/bin

$ which tclkit
/Users/sandiway/bin/tclkit

Next, he find there are two versions of critcl, critcl.kit.sh and critcl2.kit.sh. He downloads both and tests critcl2.

$ sh critcl2.kit.sh -test
exec = /Users/sandiway/bin/tclkit
prog = /Users/sandiway/Downloads/critcl2.kit.sh

As the above test indicates, critcl2 seems to pass a simple self-test. So he tries to process tclCarbonProcesses.tcl using critcl2:

$ sh critcl2.kit.sh tclCarbonProcesses.tcl
critcl2.kit error: No compiler found

It throws up an error message saying "No compiler found". This is strange. Since critcl allows C code to be inline embedded in Tcl, it must be the C compiler that it can't find. And the C compiler of choice is gcc.

C Compiler gcc

The programmer is starting to question his own sanity at this point. Shurely shome mishtake? He knows he has used gcc on this machine before. How come it's no longer there?

He investigates this rather dubious situation using those two powerful Unix commands which and locate.

$ which gcc
$ locate gcc

WARNING: The locate database (/var/db/locate.database) does not exist.
To create the database, run the following command:

sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.locate.plist

Please be aware that the database can take some time to generate; once
the database has been created, this message will no longer appear.

The command which turns up nothing so it's not in his executable path. locate is particularly unhelpful, claiming the hard drive needs to be indexed for it first to find things. The programmer demurs since he has a half-full 640GB hard drive in his laptop, and that would take a very long time indeed to index.

He finally sorta locates it in /Developer/usr/bin. And runs a simple check using the "-v" flag. Despite the verbiage reported below, he is not fooled by the self-test. In fact, he is genuinely and deeply troubled. He knows for a fact that he has installed Apple's Xcode and used gcc before. Why then has it stopped working?

$ /Developer/usr/bin/gcc -v
Using built-in specs.
Target: i686-apple-darwin10
Configured with: /var/tmp/gcc/gcc-5666.3~6/src/configure --disable-checking --enable-werror --prefix=/usr --mandir=/share/man --enable-languages=c,objc,c++,obj-c++ --program-transform-name=/^[cg][^.-]*$/s/$/-4.2/ --with-slibdir=/usr/lib --build=i686-apple-darwin10 --program-prefix=i686-apple-darwin10- --host=x86_64-apple-darwin10 --target=i686-apple-darwin10 --with-gxx-include-dir=/include/c++/4.2.1
Thread model: posix
gcc version 4.2.1 (Apple Inc. build 5666) (dot 3)

Again, he goes to the web. Yes! Someone has bitched about this exact problem.

Yes, just like the poor sod with the handle Lolmaniac, this programmer used to run Mac OS X 10.5 - codenamed Leopard - on this laptop. And like Lolmaniac, he had upgraded to 10.6 - aka Snow Leopard. So he needs now, in the middle of the night, to dig out his Snow Leopard installation dvd.

Snow Leopard

By the grace of some tremendous and most merciful deity, although he now longer can remember whether or what he had for dinner tonight, he remembers where he put his original Apple Snow Leopard disk.

Triumphantly, he inserts the dvd in his laptop drive, and half an hour and a few hundred megabytes later, Xcode for Snow Leopard, and therefore, gcc now live again on his computer. He notes with a disproportionate amount of satisfaction and glee that the which command now finds gcc on the most righteous path, i.e. in its anointed and proper location, the hallowed /usr/bin.

$ which gcc
/usr/bin/gcc

Update: well, the hit wasn't really just several hundred megabytes. Fact-checking the next day using du, I see that /Developer is 2.0 gigabytes large, though I confess to being unclear whether earlier versions remain or are included in that total...

$ du -s -h /Developer
2.0G /Developer

Plus of course, next day, Xcode wanted to download a 650MB update to all that..

Back to critcl

After that several hundred megabyte gigabyte sacrifice, the programmer returns to trying to run critcl on the tclCarbonProcesses package. The build fails and it crashes with an almightly thump of error messages.

$ sh critcl2.kit.sh -pkg tclCarbonProcesses.tcl
Target: universal-macosx
Source: tclCarbonProcesses.tcl
Tue Aug 09 02:59:51 MST 2011 - /Users/sandiway/Downloads/tclCarbonProcesses.tcl
gcc -c -arch i386 -arch ppc -isysroot $SDKROOT -mmacosx-version-min=$osxmin -DUSE_THREAD_ALLOC=1
-D_REENTRANT=1 -D_THREAD_SAFE=1 -DHAVE_PTHREAD_ATTR_SETSTACKSIZE=1 -DHAVE_READDIR_R=1
-DTCL_THREADS=1 -DUSE_TCL_STUBS -I/Users/sandiway/.critcl/universal-macosx -o /Users/sandiway/.critcl/universal-
macosx/v20_f082abebe39115d7e5eb1db8a2e0a5c9_pic.o /Users/sandiway/.critcl/universal-
macosx/v20_f082abebe39115d7e5eb1db8a2e0a5c9.c -O2 -DNDEBUG
cc1: error: missing argument to "-mmacosx-version-min="
cc1: error: missing argument to "-mmacosx-version-min="
lipo: can't figure out the architecture type of: /var/folders/G+/G+85fomYEnS8Eu68w9O4JU+++TI/-Tmp-//ccUVY2kt.out
ERROR while compiling code in /Users/sandiway/Downloads/tclCarbonProcesses.tcl:
child process exited abnormally

critcl build failed (/Users/sandiway/Downloads/tclCarbonProcesses.tcl)
Files left in /Users/sandiway/.critcl/universal-macosx

Normally, this single point of failure would end his efforts here tonight, but remember he downloaded two versions of critcl. He abandons critcl2, and fires up critcl version 1. Mercifully, it completes without emitting a single error message.

sh critcl.kit.sh -pkg tclCarbonProcesses.tcl
Source: tclCarbonProcesses.tcl
Library: tclCarbonProcesses.dylib
Package: /Users/sandiway/Downloads/lib/tclCarbonProcesses

As the above dialog indicates, he finally now has a dynamic link library called tclCarbonProcesses.dylib. This is pretty good news. He needs to test it though. But first, where should he put this library so the Tcl interpreter (wish) can automatically find and load it?

Diving through unfamiliar parts of the Tcl documentation, he finally locates the piece of information he is looking for in pkg_mkIndex, a Tcl command which he has never needed to invoke, and hopefully, never will.

The pkg_mkIndex documentation tells him there are four steps he must follow in order to make the Tcl command package require (that loads in optional packages) operate smoothly. But he is vastly experienced and not so easily sidetracked by wanton instructions (or so he believes). He finds the critical clue in the documentation of step 3:

It teases him like something terrible out of that bad Dan Brown novel "The DaVinci Code". In other words, it doesn't exactly tell him what he is looking for, but it tells him he can find out by inspecting the value of the Tcl variable $tcl_pkgPath:

$ wish
% puts $tcl_pkgPath
/System/Library/Frameworks/Tcl.framework/Versions/8.5/Resources/Scripts ~/Library/Tcl /Library/Tcl /System/Library/Tcl /System/Library/Tcl/8.5 ~/Library/Frameworks /Library/Frameworks /System/Library/Frameworks

Looking through the list of directories, and not wishing to customize any more than absolutely necessary, he settles for putting tclCarbonProcesses under ~/Library/Tcl:

$ pwd
/Users/sandiway/Downloads
$ cd lib/
$ ls
tclCarbonProcesses
$ mv tclCarbonProcesses/ ~/Library/Tcl/

And so it came to pass, the Tcl interpreter (wish) loaded tclCarbonProcesses when requested to do so, and it also managed to front the interpreter and assign it focus.

(You see, in the picture above the commands were entered using the Terminal window. Normally, the window you are interacting with, e.g. typing, will have focus. But notice that only the (blank) Wish window has colored decorations in the header. Those non-grayed decorations signals that it has focus.)

Hinting

The programmer decides to insert the setFrontProcess code into his graphical user interface program. The moment he has striven for through the long night has finally arrived. He fires up the program and it returns the same original stupid error message.

Undeterred, much like repeatedly wacking a misbehaving tv set that is on the fritz, he fires up the application again. And it works. And has continued to work ever since.

Now, gentle reader if you have made it so far, you must be wondering if it was all worth the trouble. Technically speaking, depending on how the application might be deployed - maybe not, but it affirmed this programmer that he still got what it takes. You see, during regular semester time, he plays professor and would never have a block of undistracted time large enough to tackle such a task.