More Life Lessons from VMS

 

One of the first tasks as the new computer operator was to help write queries for Oracle RDB that ran on the VAX.  This was my first exposure to SQL, database management, etc.  The interesting thing is that OpenVMS has a database manager directly built into the OS called RMS.  You could "query" your OS just like your data.  Almost like DMVs in SQL Server today...but we had this in the early 1980's!  

 

I learned "SQL" but didn't even know it was SQL.  I was the "go-to" guy whenever someone needed an ad-hoc query.  "I need a list of adjunct professors who did not return their building keys after the semester."  No problem, I can do that.  "I need to know which alumni have not given to the pledge drive this year."  No problem, give me a day.  I was writing so many ad-hoc queries that I wrote a little macro in WordPerfect for VMS (yes, there was such a thing) that would help a user schedule queries for off-hours.  

 

I really did not understand how valuable I had become.  I was NOT a CompSci major.  For some reason I had this notion that I would make a good high school history teacher.  

 

During my junior year I applied for a job working for Electronic Data Systems.  At the time the only "toll-free" numbers started with 1-800.  Suddenly the RespOrg system, which is responsible for toll-free numbers, was running out of available numbers.  I was recommended for the job based on my experience with VMS and Oracle RDB.  The job was to extend the system to support 1-888 numbers.  Two of the first 888 numbers in existence were registered in the RespOrg system to me.  1-888-WENTZEL (1-888-936-8935) and 1-888-ASSHOLE.  I essentially "squatted" those two numbers.  The former I held on to until 2001 when I was tired of paying the yearly registration fee.  The latter I lost because I forgot to pay the yearly fee.  The RespOrg system, at the time, ran much like the domain name registration system does today.  I later found out Howard Stern had purchased the rights to 877-ASSHOLE, reportedly for many hundreds of thousands of dollars.  

 

Oh well.  

 

The entire RespOrg project was solely an issue of data management.  How do we take a hierarchical database and extend it for flexibility?  But I *still* didn't realize I was doing data management and "SQL".  

 

My first real job out of college was as a help desk tech.  My cube-mate was using TeraData and was having problems with her query.  I leaned over and helped her with her filtering and she asked if I knew "SQL".  I wasn't really sure, "but I know how to do THAT."  It turned out that THAT was SQL.  

 

My value was soon recognized and I was promoted into a better job, writing queries.   

 

SQL and data management isn't going anywhere, regardless of the NoSQL movement (if it is a "movement", is it "religious" or "bowel"?).  

 

 

Somehow I keep digressing from VAX/VMS.  

 

The VAX 11/780 that we had in college never required rebooting.  I was "in charge" of this beast and I religiously patched the software whenever we got a "kit" in the mail from DEC.  But it never needed to be rebooted.  

 

One time my junior year we were hit with an internal email-bomb.  We allowed people to write basic email macros that let them to do things like auto-reply to emails and set up "distribution lists".  We didn't put a lot of security around this so eventually a CompSci major (really, a "script kiddie") got his kicks by writing an email-bomb that put a SEND command in an infinite loop using auto-reply.  I was contacted when people started complaining that their emails were being delayed-on-send by a couple of HOURS.  

 

It probably took me 20 minutes to realize the problem was the macro and then disable it.  I expected response time to improve, but it didn't.  The VAX was crawling along trying to empty its mail queue.  I didn't think it was a big deal, I assumed it would eventually clear its messages...which were all failing delivery anyway.  It wasn't long before the VP came along and asked me why I didn't reboot it yet.  

 

"Um, you can reboot a VAX?"  I had never done that.  I rebooted my IBM PS/1 daily.  But I didn't know how to reboot something the size of a washer/dryer.  I wasn't really a "system operator" and had not prepared for that.  Remember, I was not a CS major and my formal VAX training involved RTFM.  I was just a quick-study liberal arts guy.  I knew how to restore VAX files from tape, but that's it.  I told the VP that I would reboot it and he left.  I never did reboot it.  I let the queues clear naturally.  VMS was such a stable OS that there are stories of VAX systems being up for decades between reboots.  Assuming you don't do something totally stupid, and you keep your system properly patched, follow good security principles, and do not over-engineer your software, rebooting should be a rarity.  

 

I still have co-workers today that believe it is necessary to reboot a SQL Server every 30 days.  Why?  "The error log gets too big to open up so it's just easier to reboot which gives me a new error log."  ...Or..."It's just good to clear out the old memory once in awhile."  Yeah, a cold cache is better than a warm cache.  BTW, these are enterprise-class servers.  

 

 

Eventually we migrated the 780 to a VAXstation later my junior year.  The "microVAX" was a "minicomputer" with the form factor of a desktop PC.  The 780 was the size of a refrigerator/washer/dryer combo.  

 

The process of migrating the data required one reboot of the 780.  Although I was highly trusted considering I wasn't a CS major, the VP wanted to shadow me.  When it came time to issue the reboot command I still didn't know how to do it.  Why didn't I research it last time?

 

VP:  "I thought you rebooted the VAX after the mail-bomb."

Me:  "Oh I did, I just can't remember what command I used."

VP:  "Do me a favor and type 'show sys' at the prompt."  

 

 

(example show sys output)

 

Damn.  I was caught.  You see, "show sys" will tell you the last time a VMS system was rebooted.  I slowly typed the command.  The response indicated the last reboot was 7 years ago.  I'm not kidding on that either.  7 years.  

 

The VP scowled.  

 

Me:  "I guess the reboot didn't 'take'."

VP:  "Don't lie to me, you didn't reboot, did you?"

 

It was at this moment when I learned that "lying" and "professionalism" are mutually exclusive.  Yes, I 'fessed up.  But did I really "learn" my lesson?  

 

More on that in my next post.  

 

 

 

 

One of the first tasks as the new computer operator was to help write queries for Oracle RDB that ran on the VAX.  This was my first exposure to SQL, database management, etc.  The interesting thing is that OpenVMS has a database manager directly built into the OS called RMS.  You could "query" your OS just like your data.  Almost like DMVs in SQL Server today...but we had this in the early 1980's!  

 

I learned "SQL" but didn't even know it was SQL.  I was the "go-to" guy whenever someone needed an ad-hoc query.  "I need a list of adjunct professors who did not return their building keys after the semester."  No problem, I can do that.  "I need to know which alumni have not given to the pledge drive this year."  No problem, give me a day.  I was writing so many ad-hoc queries that I wrote a little macro in WordPerfect for VMS (yes, there was such a thing) that would help a user schedule queries for off-hours.  

 

I really did not understand how valuable I had become.  I was NOT a CompSci major.  For some reason I had this notion that I would make a good high school history teacher.  

 

During my junior year I applied for a job working for Electronic Data Systems.  At the time the only "toll-free" numbers started with 1-800.  Suddenly the RespOrg system, which is responsible for toll-free numbers, was running out of available numbers.  I was recommended for the job based on my experience with VMS and Oracle RDB.  The job was to extend the system to support 1-888 numbers.  Two of the first 888 numbers in existence were registered in the RespOrg system to me.  1-888-WENTZEL (1-888-936-8935) and 1-888-ASSHOLE.  I essentially "squatted" those two numbers.  The former I held on to until 2001 when I was tired of paying the yearly registration fee.  The latter I lost because I forgot to pay the yearly fee.  The RespOrg system, at the time, ran much like the domain name registration system does today.  I later found out Howard Stern had purchased the rights to 877-ASSHOLE, reportedly for many hundreds of thousands of dollars.  

 

Oh well.  

 

The entire RespOrg project was solely an issue of data management.  How do we take a hierarchical database and extend it for flexibility?  But I *still* didn't realize I was doing data management and "SQL".  

 

My first real job out of college was as a help desk tech.  My cube-mate was using TeraData and was having problems with her query.  I leaned over and helped her with her filtering and she asked if I knew "SQL".  I wasn't really sure, "but I know how to do THAT."  It turned out that THAT was SQL.  

 

My value was soon recognized and I was promoted into a better job, writing queries.   

 

SQL and data management isn't going anywhere, regardless of the NoSQL movement (if it is a "movement", is it "religious" or "bowel"?).  

 

 

Somehow I keep digressing from VAX/VMS.  

 

The VAX 11/780 that we had in college never required rebooting.  I was "in charge" of this beast and I religiously patched the software whenever we got a "kit" in the mail from DEC.  But it never needed to be rebooted.  

 

One time my junior year we were hit with an internal email-bomb.  We allowed people to write basic email macros that let them to do things like auto-reply to emails and set up "distribution lists".  We didn't put a lot of security around this so eventually a CompSci major (really, a "script kiddie") got his kicks by writing an email-bomb that put a SEND command in an infinite loop using auto-reply.  I was contacted when people started complaining that their emails were being delayed-on-send by a couple of HOURS.  

 

It probably took me 20 minutes to realize the problem was the macro and then disable it.  I expected response time to improve, but it didn't.  The VAX was crawling along trying to empty its mail queue.  I didn't think it was a big deal, I assumed it would eventually clear its messages...which were all failing delivery anyway.  It wasn't long before the VP came along and asked me why I didn't reboot it yet.  

 

"Um, you can reboot a VAX?"  I had never done that.  I rebooted my IBM PS/1 daily.  But I didn't know how to reboot something the size of a washer/dryer.  I wasn't really a "system operator" and had not prepared for that.  Remember, I was not a CS major and my formal VAX training involved RTFM.  I was just a quick-study liberal arts guy.  I knew how to restore VAX files from tape, but that's it.  I told the VP that I would reboot it and he left.  I never did reboot it.  I let the queues clear naturally.  VMS was such a stable OS that there are stories of VAX systems being up for decades between reboots.  Assuming you don't do something totally stupid, and you keep your system properly patched, follow good security principles, and do not over-engineer your software, rebooting should be a rarity.  

 

I still have co-workers today that believe it is necessary to reboot a SQL Server every 30 days.  Why?  "The error log gets too big to open up so it's just easier to reboot which gives me a new error log."  BTW, these are enterprise-class servers.  

 

 

Eventually we migrated the 780 to a VAXstation later my junior year.  The "microVAX" was a "minicomputer" with the form factor of a desktop PC.  The 780 was the size of a refrigerator/washer/dryer combo.  

 

The process of migrating the data required one reboot of the 780.  Although I was highly trusted considering I wasn't a CS major, the VP wanted to shadow me.  When it came time to issue the reboot command I still didn't know how to do it.  Why didn't I research it last time?

 

VP:  "I thought you rebooted the VAX after the mail-bomb."

Me:  "Oh I did, I just can't remember what command I used."

VP:  "Do me a favor and type 'show sys' at the prompt."  

 

 

(example show sys output)

 

Damn.  I was caught.  You see, "show sys" will tell you the last time a VMS system was rebooted.  I slowly typed the command.  The response indicated the last reboot was 7 years ago.  I'm not kidding on that either.  7 years.  

 

The VP scowled.  

 

Me:  "I guess the reboot didn't 'take'."

VP:  "Don't lie to me, you didn't reboot, did you?"

 

It was at this moment when I learned that "lying" and "professionalism" are mutually exclusive.  Yes, I 'fessed up.  But did I really "learn" my lesson?  Read on. 

 

 

The most important responsibility I had was backups. The 780 had a 9-track reel-to-reel backup system.  It would take about 12 tapes and 3 hours to take a full backup of the system, which I did religiously every Saturday morning. There wasn't much to do after mounting a new tape except wait to mount the next one. I would use the time to get in a little "paid studying."  After all the backups were done I would put the tapes into two cases, each weighing about 20 pounds, and would walk them about a half mile to the offsite storage facility, which was really an old dorm room converted to store lots of tapes.  I then determined which 12 tapes needed to be walked back to the computer room to reenter the backup rotation.  That's quite a bit of exercise. Thankfully for a lazy guy like me we only have to do that once a week. We did daily incremental backups, which still needed to be walked to the offsite storage facility, but rarely did those backups span more than a few tapes.

 

I called this my "vaxercise."  

 

When we migrated to the microVAX we used nice little DAT tapes for backups.  

 

Imagine carrying 12 heavy reel-to-reel tapes vs one littel DAT cartridge.  I can correlate my weight gain to the VAX migration.  

 

 

The big 'ol VAX 11/780 needed two giant air handlers.  These were monstrously huge systems.  One of my jobs as a computer operator was to ensure the VAX was backed up.  This required an hour of my time nightly and about 3 hours every Saturday.  My work study program included summer employment so I got to live on campus for free.  I loved living on campus, free to do whatever I wanted and I didn't have to go home to my parents like all of the other kids.  The big problem was that my dormitory had no air conditioning.  I MUST HAVE A/C.  After sweating for a few nights during a record heat wave I realized I had a possible solution, I could simply take a pillow over to the data center and sleep amongst all the big mainframes.  I was going to have to be there for backups anyway.  

 

If you've been in a large data center you know that the noise can be deafening.  But I thought unbearable noise pollution was better than unbearable heat.  

 

The first night I got no sleep.  I didn't realize JUST HOW LOUD the air handlers could be.  So the next night I tried earplugs.  The noise situation was tolerable, but now the floor was too hard.  Within a week I had my bedding just about perfect and was able to get a good night's sleep.  I could curl up between the wall and the VAX and prop my pillow on the side of the VAX.  My alarm clock, surprisingly, was still loud enough to wake me and I would pack up my bedding and take it with me.  

 

This arrangement worked well for two summers.  But almost immediately I started to get sloppy.  Instead of taking my pillow and blanket back to my dorm everyday I started hiding them in closets or in corners.  I was confident no one would find me.  

 

As I said earlier, we eventually migrated the 780 to a VAXstation.  This needed far less A/C so we were able to shut down one of the handlers soon after we powered down the 780.  I could now sleep in the computer room without earplugs!  

 

So, we had this giant 780 sitting around that was no longer needed.  It was so old the college couldn't sell it either.  What a perfect place to store my bedding! Then one day I got a call in my dorm to come over to the computer room immediately.  I did and there was the VP with a few strangers looking at the 780, staring at the bedding stuffed inside.  It turns out, no kidding, that a museum wanted the 780 and the college was donating it.  When the museum reps looked inside they saw my bedding.  The VP immediately realized this was my doing, and called me over.