Monthly Archive for November, 2003

Mammoth, day 3

Last night both Buster and Coco slept on my bed. They were both on one side, leaving me about 1/3rd of the width of the bed to myself. I’m actually pretty used to that from whenever a girl sleeps in my bed, that’s usually how it ends up anyway.

I set my iPAQ to wake me up at 5:30am as we wanted to get an early start today. I checked the ftp and all of the photos were successfully uploaded.

I took a nice long hot shower and got dressed. While waiting for Larry I packed my big duffle bag and got my backpack ready for the hike today. It looked as if it snowed most of the night and was still snowing right now. I also had a Myoplex smoothie since we’re not having breakfast today.

Because it is most likely going to be very cold, I decided to dress warm for the trail today. I wore blue-jeans, the tight synthetic stretchy material Nike white shirt, an Abercrombie t-shirt, and my Abercrombie long-sleeved wool sweater.

We left the house around 7am and got to the trailhead around 7:45am. While driving down the little road ascending to the trailhead, Larry mused that the road may be closed due to the snow. Fortunately it wasn’t closed and we arrived at the snow-covered parking lot at the trailhead. We were the only ones there.

The Mosquito Flats (Larry calls it ‘Rock Creek’) trailhead is around 10,000 feet and the car temperature gauge said it was 8 degrees outside. That’s damn cold. The trail started out in the shade due to the position of the rising sun and the mountains.

Despite the cold temperature, I was pretty comfortable except for my fingers (which were sheaved in gloves). The whole area was caked in snow which was quite a treat. The last time we were here, it was summer and there really wasn’t any snow. Fortunately the snow wasn’t deep enough to be a problem so our Lowa hiking boots repelled the snow quite nicely.

Rock Creek trail follows rock creek (as the name implies). The creek goes through several mountain lakes which is really nice. The first lake we came to was mostly frozen. We each picked up a hefty rock and threw them onto the frozen lake – the rocks didn’t fall through the ice.

Eventually the sun rose high enough in the sky to hit the trail. Once we were in the sun, the hike became much more enjoyable. I regained feeling in my fingers and the sun shining against the ice-kissed foliage was a sight to behold. Larry brought along the tripod again today and took full advantage of it. We stopped for quite a while at several different spots.

One place we stopped was a large clump of shrubs with ice & frost on the leaves. The way the sun was hitting the leaves gave a sparkling menagerie of lights which was begging to be macro photographed. Most of the creek was frozen with a few patches of flowing water. There were also interesting icicles and frozen spots that proved to be great photographic subjects.

For most of the trip I had my Sigma 15mm fisheye lens on my camera and took many wide-angle photos of the snowy meadows and snow-covered mountains in the background. I had a great time being a little creative with the fisheye lens, photographing the landscape with a large object in the foreground (like a rock or stump or something). Because we got an early start, the sky was clear.

We eventually turned around when we saw ominous-looking clouds building behind us in the valley below. On our way back we passed two other groups of people.

It was really cool to go on this trail early in the morning and the snow was an added bonus. I think we both got a nice bounty of some good photos, between my wide-angle landscapes and Larry’s tripod-assisted macros.

When we got back to town, we agreed to go to Roberto’s again for the third day in a row. It’s just so good that it’s not bad to have it so often. Like before, we both had the carne asada burrito.

Once back at the house, we set a 2pm departure deadline and packed everything up. We ended up leaving at 2:10pm. On our way out of town we stopped at the video store to drop off the movie.

From Mammoth Lakes, we headed south on highway 395. We descended and went through Bishop, then Big Pine, then Lone Pine. Lone Pine is the staging area for Mt. Whitney. The whole time we followed along the eastern sierra mountain range. Eventually we came out of that and into a desert area. The sunset was really nice against the flat desert area.

For most of the long car ride home I was thinking about blood clots. I’ve been reading a lot about people who sit on airplanes a long time who develop blood clots in their legs and die. I hope that doesn’t happen to me!

When we got to San Diego, Amy had the pumpkins lit on the front porch. Her and Rachel did an excellent job carving the pumpkins with designs from ‘winnie the pooh‘ characters.

I fixed myself some leftover spaghetti and worked on Larry’s computer. I downloaded and installed Ad-Aware. I’ve never really used it before, but it was simple. I simply updated the ‘definitions’ and ran a full system scan. It found over 160 ‘objects’ with infections. I had it clean everything up and rebooted.

I also set up an FTP-link from the WinXP box to Larry’s iBook and moved all of the photos from the laptop to the PC.

Balboa Park

I slept in the very comfortable guest room that Larry and Amy have set up with the fluffy full-sized bed and Egyptian cotton sheets. I ended up oversleeping until around 7am PST.

I woke up and fixed myself a smoothie and puttered around for a while. Tucker and Lily were watching a video on ‘big trucks’ featuring monster trucks, garbage trucks, fire engines, etc.

Around 9am Larry and Amy decided that we should head down to the Balboa Park area to visit the museums and have lunch at Prado. I packed up my bag and changed into my ‘airplane clothes’. We loaded the suburban with the stuff and the three kids.

I’ve never been to Balboa park so I didn’t know what to expect. Of course anything we do in San Diego turns out to be fun so I wasn’t worried. While we were driving into town, we passed a nice stretch of highway with a grassy trail along the side. I’ve always noticed this before but asked about the area. It turns out that this area is the San Diego Zoo and Balboa Park.

It was still early (around 9:30am) so not much was open. We parked and walked around among the buildings. Tucker was obsessed with two little seeds he found on the ground. Larry explained that Tucker loves to grab onto tiny objects and then it’s a big tragedy whenever he loses them.

We swung by a nice botanical garden area with a pond outside. Inside a large building was the fauna area.

After that we went to the science museum (around $7 per adult) and spent about an hour in there. I think Tucker and Lily had a nice time with all of the scientific stuff inside.

At 11am we walked over to Prado for lunch. It was a nice restaurant with attractive young blonde girls greeting and seating people. I had the special which was the apple-wood smoked bacon risotto with bay scallops.

After lunch, they dropped me off at the airport. It wasn’t that crowded and I had no problems checking my bag and getting listed on the flight. I got to sit up front in business class (seat 5G).

The flight was very enjoyable. I flipped between ‘The League of Extraordinary Gentlemen‘ and ‘American Wedding‘. Even though I’ve seen American Wedding already, there are some great scenes to see again (notably, the bachelor party scene).

I think I overindulged in alcohol on the flight. I can’t remember how many crown & cokes I had but it was around four or five. In the future, I don’t think I’ll have that much.

When I got home, I uploaded the rest of my photos and spent way too much time editing them in Photoshop. I didn’t get to bed until 4am. I set my alarm for 8am so I could at least get four hours of sleep. I know that I’m going to pay for it tomorrow when I’m feeling like crap.

I created a photo album for the Mammoth trip and broke it down into three sub-albums – one for every trail we went on.

All work and no play makes Jeff a dull boy.

I’ve been so busy that I’m writing this journal entry five days late from memory.

We were told that all code changes for production are to be in by tomorrow.

This afternoon Jegan came by and told me that he finished writing a helper class, ‘RPWriter’ which will assist in the creation of special log files. A series of environment variables control weather or not the logging is enabled, how often files are created, and other things relating to the filename.

I worked on incorporating this functionality into the four translators. I worked on one first and went through a few coding/testing iterations. When I was pretty happy with the results I shared it with Casey. When we were both looking at it, we discovered a problem with the timestamp on the filenames. We made some tweaks to include a date stamp in the filenames as well.

It took longer than expected to finish the work. Casey was also working on a last-minute change to one of the point-of-sale services too. Apparently there was a requirements gap and he had to make some very last-minute changes. We both had our code changes wrapped up around 10pm.

Because we had a change to the ‘common library, I had to do a ‘top-down’ build of everything. It was definitely going to be a long night. Before I left the office, I kicked off the first part of the build and drove home.


(I snapped a photo with my camera as I rode down the elevator at 10pm)

As soon as I got home, I continued working. I ran into a snag. The build server process wasn’t distributing the built-binaries correctly. There was nothing I could do until someone from release engineering could take a look at it. I finally collapsed in bed around 3:30am. I didn’t get a chance to have dinner.

the day from hell

I woke up at 6:30 and immediately resumed work. Bob O’Brien from release engineering fixed the distribution problem. The build server couldn’t find the Tivoli endpoint on the network due to some DNS changes yesterday.

I worked straight until about 8:30 at which time I took a shower and drove into work.

It was pretty anti-climatic to finally get all of the code into system test as early as possible only to find out that they couldn’t begin testing yet. There were yet more DB2 errors.

The issues didn’t get corrected until the afternoon. Since I only had a few hours of sleep, I was pretty out of it today. I did sit on the integration-test bridge calls and assisted where I could.

Jegan offered the activate last night’s changes in the integration environment, so I finally left around 4:30pm.

It was a dreary rainy day. As soon as I got on I-285, I hit gridlocked traffic. What is normally a 30-minute commute turned into a 2.5-hour commute. At one point, it took me thirty-minutes to go 0.5 miles – on the interstate highway.

This was definitely an exercise in patience and self-discipline. I don’t think I did too well.

I went to bed again today without having dinner.

all work and no play makes jeff a dull boy

While yesterday was supposed to be the absolute last day for code changes, the ‘MTR’ group in my team had some critical bugs to fix.

I had a decent amount of sleep last night so I was a little sharper today.

Because of the late code-fixes, I was at work until about 10pm again doing all of the builds and getting the code into system test. We decided to wait until tomorrow morning to activate the code in integration.

I had some snacks for dinner tonight before going to bed.

photo friday: ‘Downtown’

Today’s theme is ‘Downtown


(click on the photo for a larger version)

aLL WoRK aND No PLaY MaKeS JeFF a DuLL BoY

I’ve been so busy lately that I’ve put off doing my 2004 benefits stuff. It’s due in by the 12th. Last year our medical and dental benefits used to be free. This year they started charging us, due to the rising health-care costs. Now for next year, my medical premiums are going to be going up 276%. I’m not too happy about that.

Around 10am I started work on activating the code in integration while the MTR team was working on yet another critical code fix.

Today is Scott’s 30th birthday. A large group of people were taking him out to lunch. I was invited but couldn’t go because I was in the middle of the activation. Around 1pm Alex (who was busy working on the MTR changes) and I ran to Wendy’s to pick up some lunch. We brought it back so we could continue working while we ate.

This afternoon we had a large ‘PRB’ meeting with all of the managers from the various groups with a stake in the project. Alonzo (my director) went over all of the outstanding issues and the current status.


(Everyone intent on meeting our delivery date)

At one point the meeting got pretty intense when there was discussion around the weekend schedule. The System Test team wanted to wait until 6pm Saturday to test today’s fixes and tomorrow’s fixes. Carole & others wanted to run a batch of tests tonight and then another batch tomorrow. The reasoning is that if we discover a problem with the fixes tonight we’ll have more time to fix them than if we wait to discover them tomorrow night. Finally we decided to test twice.

Again I worked late. The MTR team was busy making some last-minute code fixes. Some of us joke that they do ‘extreme programming’ by cramming five people into a cube until the bug if fixed.


(extreme programming?)

Once they completed their changes, I began building all of the code. When I was about half-way through it, Sameer came by and said that they discovered another problem which promoted me to begin the build over again. This cost me about an hour or so.

During one of builds, I watched Sameer and Rodney play a game of chess while we waited. It was a pretty good game. They were about evenly matched and it developed nicely. Unfortunately I left around 9pm so I didn’t see the conclusion of the game.

When I got home, I continued working. I finished the build and got it into system test. Once the system test team validated the changes, I activated them in integration. I finished up around midnight.

aLL wOrk anD no PLAY maKEs JEff a dULl Boy

As soon as I woke up at 9:30am I dialed into work. I checked the health of integration test and to my horror I saw that we had tons of core files. The 5GB log file system on each of the six nodes was at 100% due to all of the core files being created.

It was readily apparent that we had a major problem. I paged Carole and John with an update. I also paged Rodney. Rodney and Sameer were in the office this morning. I explained the situation to them and started sending over core files for them to look at. I also sent the stack trace which wasn’t very helpful.

During this time I was in constant contact with Rodney and Sameer. Eventually they thought they found the ticket number of one of the tickets causing the core dumps. They requested that ODE send them the transaction for that ticket number.

Unfortunately ODE could not find that ticket in their error queue. I think Rodney took this to mean that there were _NO_ tickets at all in their error queue. He then came to the conclusion that the tickets were eventually making it through the system when retired enough times. This would mean that the problem wasn’t always reproducible – meaning this could be an intransient memory issue.

Even though they didn’t know what was causing the problem, Rodney and Sameer requested that I rebuild one of the components to see if re-linking would fix the problem. I agreed but also suggested that I temporarily enable logging so we can capture the message causing one of the next core dumps. They agreed to this and I set out to do both tasks.

Around this time Mehdi from ODE sent us an email with the 976k logfile from the error queue. It looks like Sameer and Rodney misidentified the ticket number causing the problem. I suggested that they take a look at one of the tickets in the file Mehdi sent.

About an hour later I got a page from Rodney. They finally found the root cause of the problem causing the core dumps. The code fixes were checked in and they requested that I build the two components as soon as possible.

I labeled, merged, and kicked off the build of the code. It didn’t look like I was going to have it ready by 3pm.

Once 3pm rolled around I called into the bridge line for our scheduled 3pm status conference call. I gave everyone an update and then listened to the managers go over the schedule for the rest of the day. The game plan is to wait for me to activate the current build in system test. They are going to do a ‘comprehensive’ test in system test so we don’t have the snafu like we had last night. We agreed to have another bridge call at 7pm tonight.

While I was working today I watched ‘The Shining‘ and then after that I watched ‘Rich Girls‘ on MTV.

Around 4pm I had the code deployed and activated in the system test environment. While I waited, I paged Chris A. and Tim B. from mid-tier engineering to have the binaries deployed to the six integration servers. It wasn’t until around 6:30pm that they validated the results in system test. I paged John to see about activating in integration. Have gave me the go-ahead.

It’s a tedious, time-consuming process to activate code in integration. I have to do it one node at a time. I start with the last node and work my way up. By the time I joined the bridge call at 7pm I had only completed the first node. I continued to work while on the conference call.

Finally around 8pm I finished activating and they turned the ticket flow back on. I monitored the system for the next hour and didn’t see any more problems. There were no core dumps and everything seemed smooth.

I paged Carole and John with an update. Before logging off for the night, I entered my time for this week into PlanView. This was by far my longest week of my career. I logged 73.5 hours.

2003 Developers Conference

Instead of going in to work this morning, I headed over to the Georgia International Convention Center near the airport off of Camp Creek parkway. For a few weeks now there have been advertisements in the lobby and around the building about a ‘Developers Conference’ for all ‘developer’ family of employees within the organization. They have been making a pretty big deal about this and have been presenting the banners and advertisements with a comic-book style of artwork.

When I arrived around 7:45am, I got in line for registration. Because all of the people there were developers and managers, I recognized many people. In fact, Casey was next to me in the registration line.

While registering, they gave me a bag with paperwork including a brochure booklet that was presented much like a comic book. I also got a 32-meg flash memory keychain device for filling out a survey.

As soon as I got through registration I went into the main conference room for the 10am keynote speech. Casey, Brian, and Rodney saved a seat for me. The keynote speech was interesting.

First to speak was Mike Childress, a senior vice-president. He talked about the IT industry in general as well as shifting our focus more to ‘engineering’ style development instead of ‘hacking’ style development. In order to achieve this, we use a standard set of tools and methodologies.

After Mike spoke, the second speaker was Carey Stewart (director of Architecture). Carey did a role-playing thing describing an evil secret force that has been infiltrating our ranks. He unveiled this threat by pulling a giant sheet off of a large stage prop which was a gigantic robotic spider. The demonstration continued with a developer ‘firefighter’ to try to eliminate the bug with a fire extinguisher. Unfortunately it didn’t phase the spider. Next, a team of new firefighter developers armed with the new Windows XP suite of products used their high-tech tools to kill the spider. Instead of using a fire extinguisher, they used some sort of flash-bang device. There was a loud explosion and lots of light and confetti from the ceiling. I wonder who is footing the bill for all of this? Carey also talked about something very interesting. He said that for windows development we are going to be moving to .NET and C#. This was very interesting to hear because I sent him an email about 1.5 years ago asking if there was any plan to move to .NET and C#. At the time, his response was that we had no intentions of moving in that direction. I guess things change.

After the speeches, I headed over to the ‘book room’ where Borders set up some booths. For attending the conference, Developers get a free copy of ‘XML Programming Bible‘ as well a free book of our choice. I decided to get ‘.NET Patterns‘.

I walked around to some of the other rooms where different vendors had booths set up demonstrating their products. There wasn’t a whole lot new for me to learn at the conference.

Around 11am I went back to the office and worked for a couple of hours before going home. Craig planned on arriving today around 2pm so I wanted to be home before he got there.

When Craig finally showed up, we went to Fuddruckers and had a late lunch. We caught up over lunch and then went back to my place. I helped Craig unpack his car and get settled in. He brought his old computer that he’s going to be giving to his cousin Jeff. I helped him rebuild the OS and get it cleaned up. He also brought all 25 Harry Potter book 5 audio CD’s as well as the Lord of the Rings trilogy.

Around 7pm we left my apartment and headed out to the movie theater. We saw ‘Matrix: Reloaded‘. Unlike some of the reviews, I thought it was a good movie. I thought there was indeed some closure at the end. Some of the parts were pretty slow but I still liked it.

After the movie we stopped at the package store and bought some liquor. Craig got some of those Bacardi single-drink things and I got some vodka and Red Bull.

Back at my place we played Warlords 4 all night until around 4am before going to bed.

wasting away being a warlord

The original plan was to wake up early and go hiking around Kennesaw Mountain this morning. Reality was that we ended up sleeping in until around 11am.

When I woke up, I found Craig playing Warlords in the living room. I joined him (since it’s a hotseat game) and before we knew it, it was late. We planned on going to Athens today to visit Jenn but because of the time, we didn’t want to get caught in the afternoon traffic. We decided to forego Athens.

Around 6pm we headed out for dinner. I’ve been itching to go to Houston’s for a long time now because I haven’t been able to – and I really like Houston’s. We had a leisurely and enjoyable dinner.

After dinner Craig really wanted to go to a gentleman’s club but I wasn’t up for that.

mardi gras

I woke up around 6am today and went into work, leaving Craig asleep. I worked for half a day. Originally I was going to come into work this afternoon because the original plan was to take revenue pipeline to production this evening. There was a problem yesterday which caused us to push our “go-live” date to tomorrow.

When I got back home around noon, Craig was awake playing on the computer. We left around 12:30 for lunch. I took Craig to Smokey Bones off Ashford-Dunwoody road. I had a full rack of baby-back ribs. It was truly delicious! During lunch we watched the Kasparov vs. X3D Fritz chess match on ESPN2. I was surprised to see ESPN covering a chess match.

Back home I watched a little more of the chess tournament while Craig played on the computer.

Around 3:30pm we left and drove to Phipps Plaza mall to see the special director’s cut of Alien. It wasn’t too exciting. I’ve seen it before but it was nice to see it on the big screen. I like Alien’s better.

After the movie we walked around the mall for a while before going back home.

Back home we played Warlords a little (I think I’m seeing a pattern here). We used to play an older incarnation of this game as well as ‘Heroes of Might and Magic‘ back when we were in high school. It was a lot of fun to play in large groups because of the ‘hotseat’ feature in which players trade-off taking turns.

We were going to head out for dinner around 8pm but I convinced Craig that we should delay because Survivor comes on at 8pm and my VCR broke so I can’t tape it. We watched Survivor and then left for dinner.

For dinner we went to Copeland’s around 9pm. I had a large salad with honey bacon mustard dressing. It was pretty good.

Back home Craig packed up his bags and I took a power nap.

Around midnight Craig and I left for a gentleman’s club called Mardi Gras that’s actually pretty close to where I live. When we got there, we had to valet park. There was a $10 cover per person. We found a table and ordered some drinks. Craig got a beer and I got a Red Bull & Vodka, the cost of both together was $19. Mardi Gras is a full-nudity club with a few stages. Craig went up to the main stage to tip a well-endowed dancer. When he got back he complained that unlike San Antonio, the dancers here don’t do a special private dance for you when tipping them. After observing another table get a lap dance, Craig commented that there doesn’t seem to be much contact – possibly due to them being totally nude. Craig theorized that if they were wearing something the lap dances may contain a little more contact.

The dancer that Craig tipped from earlier came by and Craig ordered a lap dance. Her routine was pretty much like all of the others – very scripted and predictable. I think she had implants.

We didn’t stick around for much longer after that. Craig was pretty disappointed in the ‘quality’ of the place but I can’t really judge since I don’t have enough experience with that sort of thing. It may just be a difference between Georgia and Texas.

photo friday: ‘Work’

Today’s theme is ”Work


(click on the photo for a larger version)

baby shower

I planned on going into work late today because the scheduled activation of the Revenue Pipeline application isn’t scheduled to begin until 7pm.

Craig needed to leave early because he’s driving 11-hour to Chicago today. I helped him get his stuff together and he left around 6:30am.

After Craig left, I set my alarm for 10am and went back to sleep. I woke up at 11 and got ready for work. While I was getting things together I got a page from Karen. Karen is my vice-president’s administrative assistant. She paged me asking me to go to Brent’s office. This presented a challenge because I wasn’t even at work yet. I paged her back explaining that I’m coming in late today due to the 7pm load.

I checked my email and noticed a nice email from ‘Linda’ regarding my website:

“All I can say is, You are very ,very gifted.I really enjoyed looking at all your photos.Thank you for sharing them with all of us”

It’s nice to get a compliment like that!

I arrived at work around 11am and settled in. Today at 11:30am in the 10-East conference room we are having a baby shower for Scott and his wife, Krissy. Everyone brought in food and we all pitched in to buy a gift.

I had my backpack with me, and so I had my camera. I slapped on the 15mm fisheye and took it along with me. During the shower I took some photos. It was tricky lightning conditions because pretty much everyone was back-lit since the bright sun was shining through the windows along the wall. This made many photos of people with their backs against the window very tricky. When it came time for some of the photos of Scott and Krissy opening stuff, I really could have used something a little tighter than the 15mm. I tried to make the best with what I had though.

I stopped by Brent’s office after the party and he gave me instructions to carry out some covert activities.

As soon as the baby shower was over, I began working tweaking the script I wrote to archive log files. I decided to change the way in which files are identified to move to the archive server. Previously I was only comparing the current and previous hour against the hour timestamp in the filename. Now I’m making an embedded perl call to compute a custom datestamp n-number of hours back. There is a snag however. Our ‘standard’ HPUX 11.00 installation comes only with an ancient version of perl (4.0). This prevented me from using any of the packaged solutions found online. Fortunately Jegan helped me come up with a command that worked within the confined of perl 4.0. Once I achieved this functionality it was trivial to expand it to the ‘rolloff’ capabilities of the script to automatically delete archived logfiles over x-number of days old.

The production load of Revenue Pipeline 1.2 was going ahead of schedule today and it looked as if the piece Jegan and I need to work on would happen earlier than the scheduled time of 7pm. Indeed, we managed to begin at 4pm. Jegan had everything pretty much under control because he and I have been doing activations in integration so much we can pretty much do it blind now.

While Jegan worked on the activation, I continued to work on and polish up the cron script. I did some extensive testing and then loaded the new version into integration.

The activation in production went smoothly and there were no problems. The ticket flow won’t be turned on until tomorrow morning at 7am so most people have cleared out of the office by now. I was still working and noticed a nice sunset which I snapped a photo of:

I worked until around 7pm. When I got home I basically collapsed and went to bed.

go live

I woke up around 5:30am and got ready for work. It was unusual for me to drive into work while it’s dark outside. When I arrived, Carole was already in her office. I let her know that I had arrived and settled into my desk.

I saw that the sun was beginning to rise, and the cloud cover was just right to result in a spectacular sunrise. I grabbed my camera and headed over to the 11-East conference room and closed to door (to block out and glare of lights against the windows). I snapped a few photos of the sunrise with both my 15mm fisheye and 28-135mm rectangular lens.

Back at my desk, I hopped on the 6:30am bridge call. Jeff Chambers was running the call and quite a few other people were online too. Originally we were supposed to turn on the ticket flow at 7am, but there was some sort of hawk/rendezvous problem last night that they needed to resolve.

Jegan came in while we were waiting. I checked the health of my hourly logging script in integration and everything seemed fine, so I installed it on all six servers in production.

We didn’t actually get the ticket flow turned on until about 8am. As soon as the tickets started to flow through the system, ODE reposted some errors from DB2. At the same time, we dropped a core file on one of our tuxedo nodes. The output translator in the north was the culprit. I ftped the core file to the development server and loaded it up in the debugger.

Analysis of the stack trace didn’t reveal anything spectacular but I was able to identify the ticket number of the ticket that caused the error. I emailed the stack trace to Rodney who was at home but on the conference call.

While we were working on this, Vann from middleware came up to the 11th floor and wanted to know if any of us wanted breakfast. He went to McDonalds and got me a sausage & egg biscuit. It was pretty good.

Back on the bridge call, Rodney said that the problem we are seeing is exactly the same problem he saw and fixed before. He suggested that the configuration was wrong. I verified the configuration across all six nodes as well as comparing them with integration and saw no differences. I emailed the results to Rodney and over the bridge call we discussed this and theories about what the problem may be.

Carole came by a short while later and suggested to me that we keep the detailed discussion off the call because there is a large audience listening – including our vice president. She told me that she also paged Rodney with the same thing. Her words made sense and I see the wisdom in what she said. She wasn’t telling us to cover up the fact that we were looking at a problem, but rather to solve it in private – that’s what all of the other groups are doing.

I extracted the ticket from one of the log files but I was unable to reproduce the error in the development environment. Around this time Rodney declared that he found the source of the problem. The Electronic Ticketing group was sending duplicate messages with duplicate keys which were causing our process to die. This type of condition isn’t ‘supposed’ to happen, but it did. Later we found out that it was an unintended side-effect of the temporary DB2 problem we had. Sameer called me later and we chatted about the problem. Both he and I agree that our app should never drop a core in a situation like that. At the very least we should throw an exception. Sameer said that he is going to code that fix on Monday.

That was the only problem we had today with the tuxedo applications. About thirty minutes after we worked through the DB2 and core dump problems, we saw severe performance degradation. Research revealed that one of the oracle servers were under massive disk IO. We tracked this down to a query which was doing a full table scan every transaction. In integration we had the same type of problem but corrected it by creating an index.

It took some time (dealing with the oracle DBA) but we finally decided that the index was missing from production. We were told that it would take some time to correct so we turned off the ticket flow into the application.

Casey had also come in the office this morning. While we waited, Carole came by and took lunch orders. There were about ten of us in the office at the time and we decided to order pizzas from Johnny’s. I went with Casey to pick up the five pizzas we ordered. We ate lunch around 1pm in 11East.

The DBA got the index created and we turned the ticket flow back on. The performance problem we observed earlier went away.

The rest of the day was pretty smooth with only a few more bumps. As the afternoon wore on, I saw a sunset out of the west windows and captured it with my camera. It’s strange that I saw both the sunrise and sunset today from the 11th floor.

I finally left work around 6pm. This was much later than I originally planned. I made plans to meet Jenn after she gets off work today (7pm) for dinner. I raced home so I could change and the immediately headed back out to Lenox mall.

I met Jenn in the Christmas-decorated store where she works. She looked great! She was wearing a short black skirt with thigh-high black boots and a powder-blue turtleneck sweater. To top it off, she was wearing cool pointy-frame black glasses which looked really good on her.

We decided to walk to a restaurant nearby called Milan. We sat outside and had tappas. One of the things we had which was quite memorable was bacon-wrapped dates.

1&1

Yesterday I discovered an offer for three years of free web hosting by a company called 1&1. The deal is 500 megs of storage and 5GB of bandwidth/month free for three years. It looks too good to be true but everything I read about it (from skeptical people on the net) indicated that it’s the real deal. I, of course, signed up and registered a new domain: leased-territories.com.

The 1and1.com ‘control panel’ interface isn’t that easy to use and they don’t give many DNS options when dealing with your domain, but it’s free. The space should be sufficient enough for me to mirror my entire site over there (once I clean up my photo gallery).

I brainstormed today about how I could achieve mirroring billimek.com on leased-territories.com and came up with the following ideas:

1) The phpBB and movabletype MySQL databases will need to be dumped, copied, and loaded into the mysql server at 1&1.
2) The entire html directory tree on billimek.com will need to be copied over to 1&1. A great way to achieve this is to use the ‘rsync‘ utility. It does an intelligent synchronization, utilizing compression during transport and only updating the parts of files that have actually changed.
3) Because the two sites will have different domain-names, there will be considerable challenges in configuration files, database rows, etc containing a hard-coded billimek.com domain name for the site which will break things when being run under a different domain name.
4) 1&1 only runs a POP-based mail server which is unacceptable to me. IMAP is far superior so currently the mail server functionality must remain on my linux box at home.

This is an exciting little project for me because I’m going to be learning a lot of new things as well as getting an opportunity to practice migrating my site to a hosting provider – something I’m probably going to be forced to do when I move because I’ll probably not be able to retain a static IP address.

It was a pretty busy day at work day. As soon as I got in the office, I jumped on the bridge call and we reviewed the status of revenue pipeline 1.2 production. Things are going pretty good, at least from the tuxedo server application standpoint.

Immediately following this was an hour-long meeting with all of the managers of the project. It was a short meeting today – we just went over the status of the system and the major issues.

Instead of going to lunch today I had a lunch-meeting in 12-East. We discussed the reservations system sync processing. It looks like we have around 13 million transactions to run through the system so that we get synchronized with the res ‘cache’.

I spoke with Larry who invited me out to San Diego over thanksgiving. I checked the flights and it actually looks doable to go out there early Thursday morning and return sometime on Saturday. I also checked flights to/from Austin around the same time and they didn’t look nearly as good. I may spend thanksgiving in San Diego.

My lease is up at the end of this month. Six months ago I renewed my lease with the intention of buying a house between then and now. I still haven’t gotten around to doing that. I now have a choice. I go month-to-month and pay a whole lot more per month, or I renew for six months and pay only a little more per month. I’ll probably renew for another six months and start getting serious bout home buying.

This evening I upgraded my photo gallery software to the latest version (1.4.1-RC3). The new version is pretty cool because they have many different ‘skins’ one can choose from.

One of my personal tasks has been to prune my photo gallery. Until a few months ago, I posted monster-sized photos to the albums which eats up a lot of space and bandwidth. I thought the built-in resize option in software would allow me to resize all of my original-sized photos, but that didn’t work. I then looked into using some command-line utilities to resize but that wasn’t working either (it was too tedious and didn’t work correctly). I finally found a great tool called ‘myresize‘ which is a plugin for gallery which will automatically resize all of your photos to a specified size. I ran this and it worked like a charm. My disk-space usage went from 650MB to 280MB after I resized everything to 800×600.

After resizing, I went through each album and pruned out photos that I didn’t think were necessary. I’ve used a lot of discipline with the later albums so there wasn’t much work to do with them, but some of the older ones had way-too-many photos.

As I was going through my albums, starting with the oldest ones first, I noticed an interesting trend. The further back I went, the worse the quality of the photos. Of course, most of the stuff before 2002 was film (scanned). However I think the quality of the camera (as I’ve upgraded my digital camera a few times since 2002) could have something to do with it too.

phpBB and movabletype

I’ve been listening to the Harry Potter and the Order of the Phoenix audio CD on my drive to and from work. It’s such a treat because Jim Dale does a great job narrating the different voices. My commute to and from work are quite enjoyable now.

Today was a fairly slow day at work. Things are moving along in production. We had to turn off the ticket flow for several hours so the DB2 DBAs could do reorgs on some of the tables to speed things up.

I upgraded phpBB to the latest version (2.0.6). After upgrading it, I did some more work on the synchronize script to mirror everything from billimek.com to leased-territories.com (hosted on 1&1).

With a little effort, I was able to mirror the ‘netheril‘ BBS on the new host. I even have the script set up to extract the database from my local mysql databse, upload it to leased-territories.com, massage the mysql dumped file to replace the domain name, and re-insert it into the remote database.

Also this evening I upgraded movabletype to the latest version (2.64).

I went to bed early because I was up so late the night before.

find /html -type file -name ‘*.php’ | xargs sed ….

In the morning revenue pipeline core team meeting, we discovered that there was an issue with a lot of missing coupons in the RPAD database. It turned out that the extract wasn’t working correctly and many coupons were being skipped.

I made a lot more progress with my domain mirroring project. I managed to mirror everything from billimek.com to leased-territories.com. I spent most of my time polishing up the sync scripts to get everything working correctly.

There are still a few snags I need to work out though. While I can ‘sed-out’ the hard-coded names in the database, I’m still at a loss at how to handle the domain being hard-coded in individual files. It turns out that there are many such occurrences in various files. While doing a ‘rebuild’ from movable type, most of the hard-coded domain names get fixed based on the configuration value in the database. Unfortunately all of the ‘image’ pages remained the same as they are not tied to the database and are not ‘rebuilt’.

This poses an interesting challenge. There are over 370 such files in jeff/images containing a hard-coded domain name. If I want to change domains, I’m going to have to sed those files as well. Maybe a recursive sed using ‘find /html -type file -name ‘*.php’ | xargs sed ….’ would do the trick? I’ll have to investigate this further.

photo friday: ‘Gatherings’

Today’s theme is ‘Gatherings


(click on the photo for a larger version)

mysqldump

I came to the revelation that I have a problem if I want to move form the 1&1 mysql database to my local server. Locally phpbb is in its own database and movable type is another database. On 1&1 I have only one mysql database to play with. So when I do a mysqldump from that database on 1&1, I have a file with both sets of tables. I can’t cleanly split this to two separate files to use on the two separate databases on turing.

I finally came up with an answer, in involved a weird hack using the following commands:

mysqlshow -h${DB_HOST} -u${DB_USER} -p${DB_PASS} ${DB_NAME} phpbb_\* | awk ‘{print $2}’ | grep phpbb

and

mysqlshow -h${DB_HOST} -u${DB_USER} -p${DB_PASS} ${DB_NAME} mt_\* | awk ‘{print $2}’ | grep mt

These commands will return a list of only the tables I want. I can then use them embedded in the mysqldump commands in my ‘backup_database.sh’ script on the 1&1 host:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
#!/bin/bash
 
# DB host
DB_HOST="xxxxx.perfora.net"
 
# DB name
DB_NAME="dbxxxxxxxx"
 
# DB user
DB_USER="dboxxxxxxxx"
 
# DP password
DB_PASS="xxxxxxxxxx"
 
# phpbb
mysqldump --opt -h${DB_HOST} -u${DB_USER} -p${DB_PASS} ${DB_NAME} `mysqlshow -h${DB_HOST} -u${DB_USER} -p${DB_PASS} ${DB_NAME} phpbb_\* | awk '{print $2}' | grep phpbb` > ${HOME}/dbxxxxxxxx_phpbb.sql
 
# mt
mysqldump --opt -h${DB_HOST} -u${DB_USER} -p${DB_PASS} ${DB_NAME} `mysqlshow -h${DB_HOST} -u${DB_USER} -p${DB_PASS} ${DB_NAME} mt_\* | awk '{print $2}' | grep mt` > ${HOME}/dbxxxxxxxx_mt.sql
 
# fix permissions
chmod go-rwx ${HOME}/dbxxxxxxxx_*.sql

My ‘restore_database.sh’ script on 1&1 looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
#!/bin/bash
 
# database server
DB_SERVER="xxxx.perfora.net"
 
# database name
DB_NAME="dbxxxxxxxx"
 
# database username
DB_USER="dboxxxxxxxx"
 
# database password
DB_PASSWORD="xxxxxxxx"
 
# from
FROM="http:\/\/billimek\.com"
 
# to
TO="http:\/\/leased-territories\.com"
 
# phpbb_db
#gunzip -f phpbb_db.txt.gz
#sed -e s/'cookie_domain','billimek\.com'/'cookie_domain','leased-territories\.com'/g -e s/'server_name','billimek\.com'/'server_name','leased-territories\.com'/g phpbb_db.txt > phpbb_db.txt.new
#mv phpbb_db.txt.new phpbb_db.txt
#mysql -h${DB_SERVER} -u${DB_USER} -p${DB_PASSWORD} ${DB_NAME} < phpbb_db.txt.new
mysql -h${DB_SERVER} -u${DB_USER} -p${DB_PASSWORD} ${DB_NAME} < phpbb_db.txt
#gzip -f phpbb_db.txt
#rm -f phpbb_db.txt.new
 
# mt_db
#gunzip -f mt_db.txt.gz
#sed -e s/${FROM}/"${TO}"/g -e s/"\/var\/www"/"\/kunden\/homepages\/14\/dxxxxxxxx\/htdocs"/g mt_db.txt > mt_db.txt.new
sed -e s/"\/var\/www"/"\/kunden\/homepages\/14\/dxxxxxxxx\/htdocs"/g mt_db.txt > mt_db.txt.new
#mv mt_db.txt.new mt_db.txt
mysql -h${DB_SERVER} -u${DB_USER} -p${DB_PASSWORD} ${DB_NAME} < mt_db.txt.new
#gzip -f mt_db.txt
rm -f mt_db.txt.new
 
# fix permissions
chmod go-rwx ${HOME}/*_db.txt*

I’m pretty happy with the results. The sync scripts are almost done.

I did more pruning of my photo gallery today. I removed all intermediate images (640×480) so that I only have the thumbnails and the ‘original’ 800×600 images. This reduced the disk-footprint of my entire album tree to 165MB. Not bad considering how large it was before.

I ended up staying at work late today. Carole came by and told me that they were assigning my pager as the primary ‘on-call’ duty page for revenue pipeline until further notice. This means that I should be the first to get paged for problems. Lovely.

Throughout the day Casey was looking at another potential problem with HRRC. I helped him out by extracting tickets from our DTMO logging system. It’s worked out really well that we implemented that.

At first it looked as if we were overstating revenue by a great deal due to this bug. We were prepared to make an emergency code deployment to production to fix the problem, but as they looked closer at it, they determined that it only affected 2% of 1% of all tickets. Based on this, management decided to hold off on implementing the change until after the Thanksgiving code freeze.

After work I drove up to Athens to visit Jenn. We watched ‘The Bourne Identity‘.

DNS pointing

It’s been an unproductive weekend and I lament that I didn’t spend much time outside since both Saturday and Sunday were beautiful. I’m sure I’ll make up for it though.

I managed to wrap up my synchronize scripts so I’m pretty happy with that. Actually they were pretty much done, but I didn’t finish up the last parts because I knew I already ‘solved’ them.

Instead, I worried about transferring the DNS registration of billimek.com.

First, ‘sync-to.sh’ is used to synchronize all of my local host’s content to the stuff on 1and1.com:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
#!/bin/bash
 
# local directory 
LDIR="/var/www/html"
 
# remote directory
RSYNC_DIR="/kunden/homepages/14/dxxxxxxxx/htdocs"
 
# excludes file - this contains a wildcard pattern per line of files to exclude
EXCLUDES="/root/rsync.excludes"
 
# the name of the backup machine
RSYNC_SERVER="leased-territories.com"
 
# username on the remote server
RSYNC_USER="uxxxxxxxx"
 
# command-line options
OPTS="-rlptDvz --delete -e ssh --exclude-from=${EXCLUDES}"
 
#####################################
 
# sync cgi tree
rsync $OPTS /var/www/cgi-bin ${RSYNC_USER}@${RSYNC_SERVER}:${RSYNC_DIR}/html/
 
# sync html tree
rsync $OPTS --exclude=**/cgi-bin/** /var/www/html ${RSYNC_USER}@${RSYNC_SERVER}:${RSYNC_DIR}/
 
# backup local databases
/root/backup_database.sh
 
# transfer databases
rsync $OPTS /root/phpbb_db.txt ${RSYNC_USER}@${RSYNC_SERVER}:${RSYNC_DIR}
rsync $OPTS /root/mt_db.txt ${RSYNC_USER}@${RSYNC_SERVER}:${RSYNC_DIR}
 
# remotely restore databases
ssh -l ${RSYNC_USER} ${RSYNC_SERVER} ${RSYNC_DIR}/restore_database.sh

rsync.excludes contains:

1
2
3
4
5
6
**/config.php
**/mt.cfg
**/mt-db-pass.cgi
html/jeff/gallery/.htaccess
**/usage
**/php.ini

Now, to move things in the opposite direction, I crafted a similar script, sync-from.sh:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
#!/bin/bash
 
# local directory 
LDIR="/var/www/html"
 
# remote directory
RSYNC_DIR="/kunden/homepages/14/dxxxxxxxx/htdocs"
 
# excludes file - this contains a wildcard pattern per line of files to exclude
EXCLUDES="/root/rsync.excludes"
 
# the name of the backup machine
RSYNC_SERVER="leased-territories.com"
 
# username on the remote server
RSYNC_USER="uxxxxxxxx"
 
# command-line options
OPTS="-rlptDvz --delete -e ssh --exclude-from=${EXCLUDES}"
 
#####################################
 
# sync cgi tree
rsync $OPTS ${RSYNC_USER}@${RSYNC_SERVER}:${RSYNC_DIR}/html/cgi-bin /var/www/
 
# sync html tree
rsync $OPTS --exclude=**/cgi-bin/** ${RSYNC_USER}@${RSYNC_SERVER}:${RSYNC_DIR}/html /var/www/
 
# backup local databases
/root/backup_database.sh
 
# backup remote database
ssh -l ${RSYNC_USER} ${RSYNC_SERVER} ${RSYNC_DIR}/backup_database.sh
 
# transfer remote databases
rsync $OPTS ${RSYNC_USER}@${RSYNC_SERVER}:${RSYNC_DIR}/dbxxxxxxxx_phpbb.sql
 /root/dbxxxxxxxx_phpbb.sql
rsync $OPTS ${RSYNC_USER}@${RSYNC_SERVER}:${RSYNC_DIR}/dbxxxxxxxx_mt.sql
 /root/dbxxxxxxxx_mt.sql
 
# restore databases
/root/restore_database.sh

Originally I was planning on just changing (via mydomain.com) the DNS ‘A’ record for billimek.com to point to the IP address for my host on 1&1. Unfortunately it wasn’t that simple. 1&1 has some lame check to give an error if you try to hit your webspace from domain not in their system.

The only way to remedy this is to transfer the DNS registration with my registrar to 1and1.com. I did this Sunday afternoon.

By the time Sunday night rolled around, the change started to propagate through the root nameservers. I logged into my old Kennesaw shell and did a nslookup to find billimek.com pointing to 1&1′s nameservers. Lynx took me to the correct location so I sent myself a test email and got a bounceback with some strange error:

1
2
3
4
5
6
----- Transcript of session follows -----
... while talking to mx00.1and1.com.:
>>> DATA
< << 550 relaying to <jeff@billimek.com> prohibited by administrator
550 5.1.1 <jeff @billimek.com>... User unknown
< << 503 Valid RCPT TO <recipient> must precede DATA</jeff>

I figured that I needed to wait longer so I went to bed not too worried about this.

new tower

When I came into work today I did my daily check on the health of the six production tuxedo servers. At 10:11am the exchange valuation engine dropped a core on dpd22y, just as I was checking the ULOGs for errors.

Jegan did a quick dump of the stack trace and I started investigating. I loaded the core up in wdb and quickly located the ticket number. Back in production I ran the ticket-search script to look through all of the DTMO logs and found a candidate file. I ftped it over to devel and extracted the DTMO for the particular ticket. I ran the DTMO in sapitest and was able to successfully reproduce the problem.

I printed out a dump of the objects and determined the structure of the ticket:

NEW ticket containing two coupons (both sales).
OLD, exchanged ticket containing four coupons. All have voided states. The first two are sales and the last two are exchanges. The problem is with the first two being voided. We’ve never seen anything like this before.

Jay stopped by to ask about James and before he left, I showed him the ticket. He agreed that it looked weird that voided coupons were exchanged because that’s normally not supposed to happen.

I looked at the code and immediately saw the problem. We don’t populate a special coupon container if non-exchanged coupons are voids. Unfortunately this resulted in an empty container. In another method, we loop through the container to look at each coupon’s state. No check is done before that for loop to ensure that the container contains something. This, in turn references a null pointer and we get a signal 9 and core dump.

The fix is simple: Do an upfront check and then return false. As I was about to code the fix, it was 11:30 (time for lunch), so I decided to do it when I get back (since the problem & fix were already identified).

Brian, Casey, and I went to Chick-Fil-A for lunch today.

This morning email to jeff@billimek.com was still getting bounced back with the same error as yesterday. The 1&1 ‘control panel’ reported that billimek.com was completely switched over to their DNS and that everything should be working fine. I sent an email to the 1&1 support team. The last time I sent them an email it took five days to get a response. I’m not too happy about the prospect of having a broken email address for that long.

Fortunately around 1pm the email problems seemed to fix themselves.

After lunch, a few guys from field services came around to replace our desktops with towers. This is a good thing. They screwed up our order the first time and gave us XP desktops with only 256MB of RAM but with a developer’s suite. This meant that we had no free physical memory once all of the background crap loaded up. The new towers are a bit faster, have a secondary 120-MB HD and 1 gig of RAM. Much nicer.

The bad part about this is that it left most of us without a workstation for several hours while they migrated our data. While we waited, I gave an inquisitive Narender a demo of my phone. Today is the first day the new FCC rule regarding cell-phone number portability takes effect and a few people are looking at changing providers and getting new phones. Apparently the Nokia 3650 is still free after rebate on Amazon. While demoing the phone for Narender I used the camera and snapped his photo.

I ended up staying at work until 8pm. I was busy getting my settings restored and everything configured the way I like it.

renewing lease

This morning we had a meeting with ODE to talk about ‘parsing errors’. Carole called the meeting, but she was out of the office today. We all sat around the table, not sure exactly what to talk about since Carole wasn’t here. We talked about errors in general and eventually someone from ODE mentioned that they are dropping 700-800 tickets a day due to parsing errors. I couldn’t believe my ears. Earlier this year we (the apps team) were told that we cannot lose any tickets whatsoever. This came up earlier because we wanted to have a special ‘error queue’ where bad tickets (maybe one a week) go in order to be looked at later. We were getting beat up over this because supposedly no tickets can be dropped and that the entire system needs to stop before we continue on. Now to hear that ODE is dropping 700-800 messages a day completely baffles me. I don’t know how they can get away with this type of behavior. It was a very frustrating meeting.

James, Brian, Casey, Alex, Anil, Bob, and I went to the Greek place for lunch today. I really enjoy the gyro wraps there.

After lunch Jegan and I had a long meeting with Dave M. and Nadeem from middleware. The middleware group wants to get out of the business of tuxedo administration and is trying to transition all of the responsibilities over to us. I don’t know how I feel about this. I don’t want to get typecast into a rut-of-a-job doing Tux support for a long time.

I left a little early today so I could stop by the leasing office and renew my lease. It is with great sorrow that I have to renew for six more months. I should have been in a house long before now.

When I was renewing, the girl asked if there was any way I could do a 7-month renewal instead of 6-motnh because a lot of people are moving out in six months. I explained to her that I might be willing to do that if we could revisit the monthly payment. She agreed to come down $16. Not much, but not bad.

My $150 Amazon.com rebate for my Nokia 3650 phone finally came in the mail today.

I did laundry and cleaned my apartment.

pre-thanksgiving dinner

Last night I saw a post on slashdot talking about spam filters being reviewed. Many of the posts were complaining about the fact that an old version of spamassassin (2.40) was being used in the comparison. They all said that the latest version (2.60) is much better. I checked my system and noted that I’m running 2.55. I decided to upgrade.

One of the things linux advocates bleat about is how great the ‘RPM’ package system is. My experience hasn’t been so rosy. I’m running a pretty unmodified recent release of Mandrake. I downloaded the three RPM’s required for Spamassassin 2.60 and attempted to upgrade. I got a ton of cryptic errors complaining about missing dependencies. The errors didn’t tell me what packages I was missing – it just gave me a list of some obscure perl function names.

Is this supposed to make things easy? Granted, I’m not looking for something as simple as a Mac but this is ridiculous. I often have these types of dependency problems when dealing with packages in linux. It baffles me that people actually think that linux is ready for mainstream desktop use. In fact, since 1999, advocates have been saying that linux will ‘take over the desktop in six months’. It’s still not there.

I wasted an hour or two doing more research only to find out that that I need to force the upgrade because the perl modules are already on the system (you can verify this by doing a simple ‘perl -e ‘require packagename;”). Maybe one of these days a simple task like upgrading a version of spamassassin won’t require someone to waste so much time.

As soon as I came into work today, I coded the exchange core dump fix. I worked out the solution on paper yesterday and implemented it this morning. I tested it, and then created a ClearQuest problem and defect and checked the code in.

Casey asked me to build the ‘edd2′ package because he made a fix to HRRC. After I built that, I built my exchange change and deployed the new versions to system test.

I skipped lunch today because I was planning on leaving early. Indeed, a lot of people were clearing our after lunch. I left at 2pm.

When I got home, I took a nap because I was tired from being up all night trying to accomplish a simple task of upgrading my spam filter. It was one of those nice kinds of naps where you don’t really fall asleep but you’re teetering on the edge and drooling. I like that feeling almost as much as sleep because it’s so relaxing.

A couple of weeks ago Mom invited me over to her house tonight for Thanksgiving dinner. Jenn called me last night and said that she’s going to be in town today so I invited her along. Jenn showed up around 3pm and we left for Mom’s house.

Dinner was really nice. The house was clean and tidy and like always Mom did a wonderful job with everything. We had a turkey breast, corn-bread stuffing with bacon, yams with pecans on top, green beans with bacon and cheese, cranberries, buttery rolls, cherry cheesecake, and chocolate pecan pie.

After dinner we all watched Tomb Raider 2.

thanksgiving

I woke up at 6am this morning. I took a shower, dressed, packed, and left for the airport. It was still dark outside and the roads were mostly deserted.

I parked at the airport park & ride. I was a bit surprised to see so many other people parking too. When I got to the terminal and checked in, I noticed that it was very busy. I was expecting today to be a very quiet day since all of the major travel was supposed to occur yesterday.

The security line was very long so I went to the employee entrance and flashed my ID to get in the special shorter security line.

I ended up with first class (seat 6A). I was pretty tied so I took a short nap before they served breakfast which consisted of an omelet, sausage, fruit, croissant, and yogurt.

Today’s movie was ‘Pirates of the Caribbean‘. I wasn’t too impressed and took a nap instead of watching all of it. The last hour of the flight I read my book. I can’t believe how gripping it is. I’ve really enjoyed the entire Wheel of Time series so far.

As we were landing in San Diego, I noticed a thick fog-like haze all over the place. It turns out that it’s ash from all of the fires a month ago. The Santa Anna winds are blowing westward, so ash from all of the fires inland is now blowing back west.

Larry, Tucker, and Lily picked me up. On the way back to the house we stopped at a liquor store so Larry could pick up some Vodka and Peach Schnapps.

When we got to the house, Amy was cooking dinner. She was basting a gigantic turkey which looked yummy. She was using some kind of buttery herbal paste to spread all over the turkey.

Amy’s parents were also there.

My pager started going off about some database problems in production. These were automated messages. I logged into my computer at home and then from there I logged into work and looked around the prod servers. Everything appeared to be fine, so I guess that it was a temporary problem.

We had dinner at 2:30pm local time. It was a huge spread. In addition to turkey we had yams, stuffing, potatoes au gratin, green beans, rolls, and cranberries. I really liked the potatoes and stuffing.

I ended up eating a lot to the point where I was feeling bloated. If I could only often eat that much, I think I could expand my stomach and gain some weight!

As to be expected, I was very tired after dinner. I took a nap.

Later this afternoon I helped Larry with his computer. One of his hard drives was giving him trouble and he wanted to try to recover some files from it. While I was using the computer Larry wanted to plug the power cord into the HD. I suggested that we turn off the computer first, but he didn’t seem concerned.

As he plugged in the HD, sparks flew and the computer sort of shut down. The lights were still on, but it was turned off otherwise. He hit the power switch and nothing happened. This didn’t look so good. He unplugged the computer and then turned it back on. Fortunately it booted up ok and there were no obvious problems. Lesson learned: don’t attach internal drives while the computer is on – they are not ‘hot-swappable’.

We ate apple pie for desert and Tucker, Larry, and I went out to the video store to rent a movie. We ended up getting ‘The Italian Job‘. I didn’t watch it and went to bed early because I was still very tired.

torrey pines

My pager woke me up at 5am because of the 8am EST automated status page. I went back to sleep for about 45 minutes.

When I woke up again, I took a shower and dressed. For the next hour or so, I wrote in my journal. After that I had a smoothie for breakfast.

I was a bit cold so I climbed back into bed (Buster climbed up on the bed too to keep me warm) and read my book.

Around 9am Larry, Tucker, Lily, and I left for the Torrey Pines state reserve. When we got there, we parked near a trail and started out.

Torrey Pines is a pretty cool area. I’ve been here once before with Larry a few years ago. It’s an area full of pine trees sitting on cliffs above the Pacific Ocean. There are actually about seven or eight different trails in the park. We only went on one which was 0.6 miles long. Tucker and Lily are just under three years old so they can’t walk great distances with ease.

When we left the trail area, we drove down the hill to the beach just underneath the cliffs. Off to one side were a lot of surfers. The waves were in good form today. Larry suggested the waves were so good because of the westward winds.

We walked around the beach for a while and then headed back to the car. From there we drove into Del Mar and went to a place called ‘Dogs Beach’. Larry pointed out a large compound sitting on the beachfront. This is house owned by one of the founders at his company. The guy had to buy six beachfront houses in order to build his large house.

Dogs beach was pretty cool because it’s an area where people can let their dogs roam free without a leash.

From there we drove to a restaurant in Del Mar for lunch. I had a Caesar salad. Del Mar is a nice area, much like La Jolla.

On our way home, Tucker and Lily were pretty tired. In fact, Tucker fell asleep on the drive home. When we got home, Larry put Tucker and Lily to sleep in their beds.

A short while later my cousin John and his girlfriend, Michelle, came over. They drove down from Irvine, CA. Larry invited them to watch the TexasA&M football game. The game was pretty fun, but UT ended up stomping A&M.

At one point we all had some leftovers from Thanksgiving dinner. I really enjoy those potatoes.

During the game, a girl named Jenny came over. She’s a babysitter they hired to help with the kids. She took Tucker and Lily to the park.

Around 5pm Larry and Amy left for a dinner they had planned with some friends.

A short while after that, John and Michelle left too.

I spent an hour or two going through the photos I took today. I took them all in RAW mode so I had a good opportunity to see how I like the new Photoshop CS RAW conversion utility.

When I selected some photos to keep, I uploaded the edited versions to my computer at home. I also read some more of a book lying around called ‘The Worse-Case Scenario Survival Handbook: Travel’. It was interesting because in the glossary in the back of the book are useful phrases in different languages. Some of the phrases are:

May I use your belt as a tourniquet?
Do you know a place where I can hide?
How fast can this car go?
You will never make me talk.

sun & sand

My pager woke me up at 5am again this morning. I went back to sleep after reading the pages. If you don’t acknowledge the page, it will keep beeping until you do.

I had a Myoplex smoothie for breakfast and worked on my journal in the study while everyone else woke up.

Around 9am Jenny, the babysitter, came over to take care of Cal. The rest of us packed into the suburban and headed to the beach.

We went to the Torrey Pines State Beach. I was here with Larry yesterday but it was more crowded today. It was pretty nice outside. The sun was shining and it was comfortable wearing shorts and a t-shirt. I had my camera with me the whole time (28-135mm lens). We stayed at the beach for a couple of hours before leaving for lunch.

For lunch we went to a place called ‘Robertos’ near the beach. Despite being the same name, it isn’t the same as the wonderful ‘Robertos’ in Mammoth Lakes, CA. I had a breakfast burrito with eggs, cheese, and ham. It was served on a hot corn tortilla.

After lunch, they took me to the airport. Unlike Thanksgiving Day it wasn’t too crowded. I headed to the restroom and changed out of my shorts/t-shirt/sandals and into my airplane clothes.

Once I was checked in, I headed to the gate and approached the gate agent to request a window seat. I recognized the guy working the gate. He was the same guy I asked about a window seat the last time I was out in San Diego a few weeks ago.

While I was waiting for the plane to board, I realized that I left my cell phone charger back at Larry’s house. I called the house and told Amy who said that Larry would be happy to fedex it to me on Monday. It’s going to suck not having a charger for a long time. I’m going to have to keep my phone off and use it very sparingly.


(looking out the window at gate 40 in SAN – my plane (DL 156) is on the left)

I ended up with a business-class seat (3G). Only about half of the business class was occupied (the seat next to me was empty). Unfortunately the movies were still on the November cycle so it was all the same stuff as before.


(shortly after takeoff)

I finished my book and got some sleep.




Powered By Site5.com