Friday March 21, 2008

Upgrading to Mac OS X Leopard

Disclaimer: This article documents the problems and solutions we encountered during our recent home server migration from Mac OS X 10.4 (Tiger) to 10.5 (Leopard). Unless you want to use Mac OS X for a DNS/Mail/Web/... server, this will probably not be a particularly relevant article. And, unless you are technically oriented (and preferably, Unix-literate), you may find some parts of it confusing.

Since December 2006, we have been running our home server system on a Mac mini, running MacOS X (Tiger). Specifically, this is the "consumer client" version of Mac OS X, not the specially labeled (and vastly different) "Mac OS X Server" product.

Recently, said system become unacceptably slow and Rich's investigations with top(1) revealed Very Large Numbers of pageouts. This told us that we needed more RAM and (perhaps) a faster hard disk. Because the mini is not designed for this sort of expansion, we purchased a gently-used Power Mac G5 (2.0 GHz, dual processor, 4 GB RAM).

Then, because it seemed like "The Right Thing To Do At The Time", we installed Mac OS X 10.5 (Leopard) and prepared to migrate our back-end tools and web applications. After all, we've each been running Leopard from a "user perspective" for a few months. It's stable. How bad could the upgrade be?

Whenever you find yourself asking that question, slow down and reconsider. The upgrade worked, eventually, and it was the right thing to do. However, it was not accomplished without considerable frustration.

Web and email services were offline for approximately 4 and 12 hours, respectively. MySQL access was offline for over two days. (I admit, I had to go to work during that time, so it could have been back sooner. :) My weblog finally came back online on the third day.

We're up and running, with many fewer pageouts and a more spritely system. We learned a lot. One of the things we learned was that upgrading a server to Leopard is nontrivial and it's not even, for the most part, Leopard's fault.

POP3

We use POP for email. A POP daemon isn't included in Mac OS X, so we needed to install one. Again, how hard could it be? I've successfully installed a POP daemon before.

This time, however, I couldn't get the daemon to launch on system startup. I finally ended up using MailServe, a GUI mail setup application from Cutedge Systems. Solving two problems with one application, we could also use MailServe to ensure that the Postfix mail delivery system was launched at startup.

I have to say, I'm very impressed with the work Cutedge has done. MailServe is extremely easy to use. The configuration files are all installed in /usr/local/cutedge. In retrospect, getting POP (and Postfix) going was the easiest part of the upgrade.

Apache 2

Mac OS X Leopard ships with Apache 2. I've been avoiding upgrading to Apache 2, but now it would be more work to move back to version 1.3 So, Apache 2. I'd heard rumors...

In actuality, everything seems to be working better than I anticipated. The only real problem we ran into was that Apache 2 starts with a more restrictive configuration than was found in Apache 1.3. What this meant for me was that our TWiki didn't run using the configuration and directory structure we had before. The Apache error log was full of errors:

   client denied by server configuration:...

Eventually, I solved the problem by moving one directory, twiki/pub out of cgi-bin and into the DocumentRoot, renaming it to twikipub. Then I modified the TWiki configuration file to match and TWiki ran as expected.

Moving on, I started finding similar (but different) errors for other pages within our web. After a while, the easiest thing to do was to "turn off" the more restrictive Apache configuration settings.

I'm sure that someone had an excellent reason for disallowing images or symbolic links under cgi-bin, but ultimately, I chose the path of least frustration. It was simpler to disable two lines in httpd.conf than to rearrange all of our files and directives to suit the more restrictive set of features.

   <Directory />
      Options Indexes Includes FollowSymLinks MultiViews
      AllowOverride None
      #  Order deny,allow
      #  Deny from all
   </Directory>

PHP 5

Leopard ships with PHP 5. Tiger used PHP 4. We expected some changes to be required.

So far, however, the only thing we've had to do was to set

   allow_url_include = On
in /etc/php.ini to allow "<? include..." statements to work. That was relatively painless.

MySQL 5

We run several applications that use MySQL databases. Unfortunately, the binary distributions did not (as of this writing) include a package for Mac OS X Leopard on PowerPC. (Lame!!)

We had been using MySQL 4.1 and this seemed as good a time as any to upgrade to MySQL 5. I tried the Tiger distribution, using the Leopard setup hints posted by Sam Bauers. These were extremely helpful and allowed me to get MySQL running.

Unfortunately, Movable Type couldn't talk to MySQL yet; it needed the Perl DBD::MySQL module.

DBD::mysql

I downloaded and built DBD::mysql from cpan. Then I ran make test.

32/34 tests failed. :-(

Much searching later, I had tried (and discarded) many recommended solutions. The one remaining possibility wasn't even proposed as a solution - only a very strong recommendation:

It is a good idea to rebuild and reinstall the Perl DBD::mysql module whenever you install a new release of MySQL. The same applies to other MySQL interfaces as well, such as the PHP mysql extension and the Python MySQLdb module.
This recommendation can be found at line 5921 of the INSTALL-SOURCE "readme" file in the MySQL source distribution. A similar recommendation is buried in the DBD-mysql documentation as well.
However, if you need a C compiler, make sure that it is the same C compiler that was used for compiling Perl and MySQL! Otherwise, you will almost definitely encounter problems because of differences in the underlying C runtime libraries.

MySQL (Take 2)

So, I downloaded and built MySql 5.0.51a from source. It built, tests passed, I installed it.

Still, all was not smooth sailing from this point. Some problems:

  • The binary distributions install all files under /usr/local/mysql. The source distribution scatters the installed files across the filesystem. Where did they go?

  • There's a make_binary_distribution script included in the source distribution. This creates a tar archive which unpacks into /usr/local/mysql. Better! However...

  • Many of the programs (and the tests :-( ) still expect the "other", source-installed, paths. Some links helped:
       sudo ln -s /usr/local/mysql/share/mysql /usr/local/share
       sudo ln -s /usr/local/mysql/lib /usr/local/lib/mysql
       sudo ln -s /usr/local/mysql/include /usr/local/include/mysql
    

    Another option, it turns out, is to explicitly configure make_binary_distribution. See Mysql bug 35424 for details.

  • Once I had MySQL running, the next "strong recommendation" is to run mysql_upgrade
    mysql_upgrade should be executed each time you upgrade MySQL. It checks all tables in all databases for incompatibilities with the current version of MySQL Server. If a table is found to have a possible incompatibility, it is checked. If any problems are found, the table is repaired. mysql_upgrade also upgrades the system tables so that you can take advantage of new privileges or capabilities that might have been added.

    Unfortunately, the bin/mysql_upgrade script expects a ./.libs/mysql_upgrade binary that didn't get packed up for installation by make_binary_distribution. Lucky for me, I still have the source tree available.

       sudo cp -R .../src/client/.libs /usr/local/mysql/bin
    
    After that, the upgrade ran smoothly.

Movable Type

Once MySQL was running, with the databases upgraded and DBD::mysql installed, Movable Type was able to run its database upgrade routines and... finally!... my weblogs were accessible again. Rah!

I had to reinstall a few more Perl modules but, in the grander scheme of things, this was a minor concern.

crontab

For reasons known only to Apple, the location of the cron/tabs directory has changed in Leopard. So, if you upgrade from Tiger, crontab data is suddenly "not there".

I did a web search and learned that Apple has moved cron/tabs from /var/cron/ to /usr/lib/cron/.

    sudo mv /var/cron/tabs/* /usr/lib/cron/tabs

Lessons Learned

We're up, we're online, and our software is closer to "State of the Art". The upgrade wasn't easy, but it wasn't impossible and it was The Right Thing To Do.

Google is decidedly your friend and the Web is the best advancement in communication since the written word. We would never have made the progress we made, in only a few days, without the ability to search for — and find — the answers to our questions!

I can't thank those intrepid explorers enough for paving the way, ferreting out solutions, and posting them to the WWW. This article is therefore my contribution (paying forward) to the corpus of knowledge.

Upgrading to Mac OS X Leopard ( in category Computerware , SciTech ) - posted at Fri, 21 Mar, 23:34 Pacific | «e»


Post a comment

Any posted comments will be viewable by all visitors. Please try to stay relevant ;-) If you simply want to say something to me, please send me email.

All comments will be moderated. Thank you for your consideration.