Amusing story from the back office...

I haven't had a chance to describe what it is I do for Amazon, and this story needs some background, so here goes.


I'm a software development engineer (read: "Code Monkey") in a group which variously goes by the names Customer Master Systems/Service and Identity (you might say we have a bit of an Identity crisis, but you wouldn't because we'd have to hurt you for such a pun). We're basically responsible for all of the core customer information (names, addresses, etc., though not credit cards or bank accounts) -- storing it reliably, making it available to the various applications inside the company that need it, and enforcing things like privacy policies.

The last bit is surprisingly non-trivial -- keep in mind that Amazon runs a number of third-party sites like Target, The Bombay Company, and the NBA store, whose privacy policies may state that they won't share personally identifiable information with third-parties. It's up to us to make sure the bits associated with, say, your Target account information never mix with the programs running for Bombay or even Amazon itself (!). At the same time, we're constantly monitoring for suspicious activity -- we're often able to shut down hacked accounts before the real owner even knows they were hacked. Multiply this by a few million users on the site at any given moment and you have a lot of bits that need to go from point A to point B without crossing points C, Q, or Z.

So many bits, in fact, that, even though we have some of the fastest network equipment in the world, we're always looking for ways to reduce the load. One of our other engineers implemented a way to offload a significant chunk of this traffic while maintaining all the other features and constraints. We tested it, beat on it, stomped all the bugs we could find out, and declared, "This is good. Let's deploy it."

So we pushed the new software live. We watched the traffic on it grow, and it seemed stable. Things went this way for 12 hours or so, and nobody noticed a thing.

Then a bug was filed. It said, "Hey, 1-click isn't working!" Worse, the bug was from Jeff Bezos.

For those who don't know, Jeff is the founder and CEO of Amazon. He's one of the inventors of 1-click (where you just click on a single button next to each item you want and they magically appear on your doorstep -- you don't need to explicitly check out, fill out payment info, etc., if you already have an account). Whether 1-click is a significant contributor to our bottom line is irrelevant. The rule is: Thou shalt not break 1-click. The corollary is: If you do break 1-click, you scramble to fix it.

You definitely do not leave 1-click broken for 12+ hours and let Jeff (no, he doesn't need a last name inside the company) find the bug for you.

(We did actually test this, incidentally; it's one of those cases where it was only intermittently broken and happened to work in the test case. These are usually the worst kinds of bugs.)

Anyway, I was one of the three engineers on the call fixing it. I was finding reproducible cases where it was broken and transcribing the phone call. Another engineer was actually doing the fix. The third was verifying the fix -- that is, he was going around to all of the broken cases I found and making sure 1-click did work.

Today, he came into the office and said: "Hey, Dave, remember the 1-click bug from last week?"
Me: "Yes..."
Him: "You know how I was testing and making sure it worked?"
Me: "Yes..."
Him: "Well, last night I had about 40 items show up on my doorstep."

The office pauses, then bursts out laughing.

For those who are fans of MythBusters, I have located the video of the water bottle jet pack from the Japanese game show which inspired last week's episode. (2 MB Windows Media Player file)

Today marks a monumental point in my employment at Amazon:
On Tuesday, November 8, 2005, I got my first "They're all top priority" message.Thank you, thank you. The Dilbert calendars will be appearing momentarily. Don't forget to tip your server.

Ok... so it wasn't quite that blatant. The actual phrasing from my director was, "I'm not sure that 'best effort' support is suitable here." But it qualifies.

I'm getting wiser (and mellower) in my old age, though. My reply was to outline the consequences for the other project I'm working on and to confirm that he understands and accepts this.

Cat lost... cat FOUND!

Robin has been found!

He somehow made his way down the hill to some folks who live in the woods across the way -- just beyond our search range, and a fair bit farther than I thought he would have gone. Now, what the hell prompted him to travel so far is a bit of a mystery (and will remain so until he learns to talk)... but we're just glad to have him back.

Nice family he stayed with, though somehow I doubt that he and their Yorkshire terrier became good friends.

Ah, relief. I can sleep easy tonight. (I haven't the last two nights.)

One of my kitty cats is lost. I haven't seen him in 18 hours. :-(

This worries me because my cats don't spend much time outdoors, so if he's just wandering out there...

Outlook enhancements

When responding to a meeting invite, Outlook gives you three options:

• Accept
• Tentative
• Decline

I need more than that:

• Accept enthusiastically because I'm not missing free food.
• Accept because I have nothing better to do.
• Accept because I'll end up in a lot of trouble if I don't.
• Accept because I want to look like I'm taking you seriously but I'm planning on skipping the meeting at the last second.
• Tentatively accept because I'm too chicken to decline and I'm hoping something else comes up.
• Decline because it's more comfortable to sleep in my cube than in your meeting.
• Decline because the last thing I need is more meetings and I don't fear any retribution from you or your team.
• Decline because what you're proposing is a damn waste of time.

In addition, I want a checkbox to appear if any of the "Decline" options are selected: [ ] Decline with prejudice.

I think we've learned an important lesson here.

Do not have advisers whose names are in the form "Z. Name Lixxy", where Z is an arbitrary initial and x is an arbitrary consonant.

Another interesting article -- this one real, though.

Remote Control Device 'Controls' Humans

We wield remote controls to turn things on and off, make them advance, make them halt. Ground-bound pilots use remotes to fly drone airplanes, soldiers to maneuver battlefield robots.

But manipulating humans?

Prepare to be remotely controlled. I was.

Just imagine being rendered the rough equivalent of a radio-controlled toy car.

I missed this headline last week, but it's important to bring up -- , let us know how the situation is going there.

Study Reveals Pittsburgh Unprepared For Full-Scale Zombie Attack

There are days when Amazon operates like a well-oiled machine. The gears are meshing and spinning, orders are clicking away, shipments are shipping, users are happy, and the engineers are destressed.

Alas, today does not appear to be one of those days. We appear to have a Category-4 Monday here.

Ah, those wacky Brits. They're toying with embedding MP3 players into breast implants. I guess the rectal fax machine is next...

Today was my third day of commuting to work via ferry. Still enjoying it. I've been using those ~30 minutes eastward to get a head start on work e-mail so far. The westward trip is spent admiring the nighttime view of Seattle.

Forget the odd political battles, the fact that it's overrun by old folks, and the humid weather. The fact that I'm lower on the food chain than a gator is reason enough to keep me from Florida, and yet they are now being challenged for dominance by Burmese pythons.

My ThinkPad just gave up the ghost. Looks like some kind of severe power issue -- it was making arcing noises with the battery. Without the battery and plugged into the wall, it runs for a few minutes and then just freezes.

Alas, I discovered that it's out of warranty by a few days. :-(

Fortunately, Amazon provides us with laptops, so I don't think I'll be replacing it anytime soon.

For novelty's sake, I'm posting this from the middle of Puget Sound, aboard the M.V. Wenatchee (ferry). Talk about wireless! (High packet loss, though...)

I finally won one of those free Diet Cokes today. I'm actually a bit bummed -- I had a terrific conspiracy theory going about the 8th floor US2 Coke machine, but now it's ruined. It's a conspiracy to ruin my conspiracy theories!

:-)

We're a bit annoyed because we drove+ferried over to Bainbridge (that's $13.30 each way) to sign papers to get a storage place (after Tamara called and confirmed that they'd be waiting for us), only to find them all locked up and closed. Grr. (Normally, I'll be biking+ferrying over and using a monthly pass, which is $70/month rather than $26/day.) I guess we're going to have to go back tomorrow -- but to take a lease with a different place.

Heh... Just found this old entry (thanks to a meme posted by :

Must... get... out... of... Pennsylvania... before... it... sucks... my... life... force... out...

Lucky or unlucky?

The Diet Coke bottles in the vending machine here advertise that one out of every four 1L bottles will win a free 1L bottle. At this point, I have purchased 10 bottles of Diet Coke here and none of them are winners.

Assuming that winning bottles are randomly distributed, the chances of this happening (i.e., none of them being winners) is just under 6% (0.75¹⁰ == 5.63%).

I am starting to get a bit suspicious about this contest.

Maybe such contests are void here and they've "pre-voided" the bottles for us? That would be even sneakier...

Very windy in downtown Seattle today. This isn't normally something normally remarkable until I realized: I don't think I've felt the wind once in Seattle until now. (Ferry trips, of course, excepted.)

I'm now doing some web UI coding (dynamic forms and all that jazz) for the first time (beyond toy projects). A lot of folks hype this web stuff (XML, HTML, Javascript, etc.) as being the next step in technology. What they fail to mention is that it's a step backwards. If I misspell something in C or Python, the compiler or bytecode interpreter will complain and point out my mistake. Javascript and HTML? Gah, who knows who long it will go before something is wrong, happily passing null or undef values around.

And don't get me started on HTML, XML, SGML, and their ilk. Especially XML. It combines the power of Peewee Herman and the simplicity of a nuclear reactor with the readability of a Victor Hugo novel.

Back in Seattle. The worst week of my life is now behind me. I hope I never again have to console a mother crying, "They've taken my baby!"

As awful as funerals are, they can be very beneficial. It brought us to accept that Jane is gone. For many of us (myself included), the night after the funeral was the first night of decent sleep in a week. Certainly, there's a lot of healing left, but most of our ghosts have been dispelled.

Jane, you will be forever missed.


Jane Eileen Cuthbert
1982-2005

Thanks for all the messages of support. It means a lot to me. I'm still doing a bit of Dr. Jekyll and Mr. Hyde, one moment calm and placid, the next a bit of a basket case. I'll be better after the funeral (Friday) and the weekend are over, and there's a bit less media attention on my family.

Tam and I had an excellent time on Sunday hiking with , , and the Inimitable India, as well as their trustworthy canine companions Clyde and Shorty. It was nice to get outside on such a gorgeous day, and it was fun to get to know such like-minded (and very likable!) people. Thanks, guys!

Chicago Sun Times is full of assholes

"Maureen thinks she has an injured knee and injured rib," Martin McNicholas said. "She started to cry because people died on the train with her. She said she saw fingers, brains, and it looked like someone had exploded."

That "someone who exploded" was Jane, you jerks. The same Jane whom I was inner tubing with on a lake a summer ago. Whom I took to dinner on a trip out to Monterey a few months ago. Who got amusingly drunk with her mother at my wedding.