DrunkenBlog

In Fear of Security Update 2005-007

A few hours ago, AppleInsider posted that Apple's newly released Security Update 2005-007 had major problems when it came to 64-bit apps, namely that it completely broke them:

Wolfram Research, makers of the popular Mathematica software, began informing its customers of the issue in an email on Tuesday. The company said the security update disables its flagship Mathematica software: "Due to an error on the part of Apple, this update prevents any 64-bit-native application from running. In particular, this means that Mathematica 5.2 will not run on any G5 system if it has installed this Security Update

MacInTouch apparently contacted Apple and confirmed the issue. The same email AppleInsider was working from ended up on the Mac Enterprise list in its entirety yesterday evening...

** Important notification for users of G5 Macintosh systems running OS X 10.4 **

At approximately 8 p.m. PDT on Monday, August 15, Apple began automatic distribution of Apple Security Update 2005-007 for Mac OS X 10.4.2 (Tiger).

Due to an error on the part of Apple, this update prevents any 64-bit-native application from running. In particular, this means that Mathematica 5.2 will not run on any G5 system if it has installed this Security Update.

This problem was discovered by our testing procedures a few hours ago, and Apple has now assured us that they have stopped automatic distribution of Security Update 2005-007 at this time.

If you did not install Security Update 2005-007, then you will not be affected. If your Mathematica 5.2 successfully launches and performs any computation (such as 2+2), then this also means that you have not been affected.

If you have been affected, then Mathematica 5.2 will generate a MathLink error when you try to do any computation with it. (If you run MathKernel directly from the command line, it will crash at startup.)

Apple has informed us that there is no workaround for this problem.

Apple is investigating the problem at high priority, and intends to distribute a new Security Update in the very near future. This update will correct the problem and allow Mathematica to run successfully.

To run Mathematica 5.2 today, you must temporarily disable its 64-bit capabilities. You can do this by running the following commands in the Terminal:

cd /Applications/Mathematica\ 5.2.app/Contents/MacOS cp MathKernel MathKernel.bak lipo MathKernel.bak -remove ppc64 -output MathKernel

If you are unable to run the script above, an alternative is to use an earlier version of Mathematica. The problem with Apple Security Update 2005-007 affects only 64-bit applications; Mathematica 5.2 is the first 64-bit-native version of Mathematica.

Note that when Apple has made the corrected Security Update available, and you have installed it, you must reverse the procedure above by running the following commands:

cd /Applications/Mathematica\ 5.2.app/Contents/MacOS mv MathKernel.bak MathKernel

If you do not do run these commands, Mathematica will not operate in optimized native 64-bit mode.

We regret the inconvenience caused by this problem, and hope that as soon as Apple has corrected the problem you will continue to enjoy the outstanding performance of Mathematica on 64-bit Macintosh systems.

Sincerely,

The Technical Support Team Wolfram Research, Inc.

P.S. Should you require further technical support for this problem,
Apple has informed us that you should contact them

As you can see the screenshot, it's still going out. And yes, I know I am behind, I reinstalled from scratch again yesterday to try to document things in a more scientific way. For the record, science is saying Tiger can bite me. I tried to check 40 minutes ago to see if the update was still there, but Software Update stalled and had to be force quit, which completely locked up the GUI layer, which forced a hard reboot and a round of DiskWarrior "just in case" while I hung around.

I'll admit I was pretty damn pissed, as this was the second time Software Update has done this to me in 10.4, but MarsEdit tempered my anger by saving the last big chunk of the post...

The first time it was trying to check in the background, and popped up the little thing in the Dock where it was going to show me there were updates, but then wouldn't let me switch to it... and trying to force quit it locked up the GUI too.

This situation brings a few things to mind...

• Luckily, there aren't that many native 64-bit apps on the platform so the pool of people who will be affected will be segmented and easily identified. Wolfram mentions native, and the frameworks are touched by this update, but Mathematica is still dead via the CLI. It's unknown if this will affect someone running a 64-bit version of say, MySQL will be unaffected, and unfortunately I know only one person running a 64-bit version on OS X and they aren't going to install it to see.

• Of those who are affected, they'll be severely affected. It's nasty. I.E., while Wolfram has a workaround for running the program slower on your hardware via the GUI, you're out of luck when it comes to using it via the CLI.

• With adequate testing procedures -- including an adequate testing pool -- a bug of this magnitude should never get out of Cupertino, let alone down the chain and into our systems. Since adequate testing procedures would have caught it, you couldn't blame someone for assuming Apple's testing procedures are inadequate for the type of software they're shipping.

• Memory is a funny thing, and we have a tendency to gloss over the bad and remember the good. Unfortunately, my memory is telling me that updates going out and yanked the same day or next -- with little explanation of the problem or what to do if you already installed it -- is entirely too common with OS X.

• Reversion is way past due in Mac OS X, and an update like this only shows how important it is. Reversion is what would allow you to tell the system to uninstall this security update, or any update if you are having problems.

  I.E., if you install something like this and have major problems -- or it automatically installs in the background -- your only real way to get close to a reversion process is to boot from CD, do an "Archive and Install", and then install updates to get to where you were before. It's good to do nightly backups.

• It's nice to see that they're supporting 10.3.9 in this update for the client and server, but it's still decidedly not nice that we have no idea how much longer they'll continue to do so. Repeat after me: Apple will never make be taken seriously in the enterprise market until they tell people how long they will support what they're buying.

• If you look through the tech note for S.U. 2005-007, Apple's documentation for updates isn't yet perfect, but I feel as though I should note again how much it's improved. If you go back through the archives, or have been around for awhile, this is something I harped on for a long, long time.

  I almost hate to give them props for improvement here, because I think it -- and more specifically their entire process for dealing with security issues and bugs -- could still use major work and I'd hate for people to think the issue is taken care of. It's not, but they're miles from where they were and its worth rubbing them a bit for it.

The other thing that's coming to mind is the idea of too much, too sexy versus taking your medicine all at once:

• Too much, too sexy

  You're probably not aware of it, but Macs get hacked all the time now, although ~95% of the hacks occur Macs acting as exposed servers somewhere. Platform differences can make them immune from some attacks, but not all, and I'm often forwarded sites served from a Mac that are defaced because a script kiddy got in.

  Often times, this has as much to do with a flaw in the software they're using -- say a bulletin board software -- but often its due do flaws in what Apple is rolling into their OS. I.E., in this update there updates to SquirrelMail and ping and traceroute as well as major issues for those using LDAP and fun things where a flaw in OpenSSL can allow someone to grind your server off the grid.

  Now, this security update is huge. It not only touches a lot of apps, it makes sizable changes to CoreFoundation and Appkit which can affect any app running on top of them.

  The problems with it being teh huge is twofold:

  1. If you need any of the above updates, but installing this will kill apps on your G5-based systems you also need in order to get work done, your only choice is to run an insecure system until Apple fixes the issue.

  2. Flaws don't get fixed at the same time, and if you are waiting for a security update with 30+ fixes, that means you're waiting on some of the updates in general. This is becoming increasingly unacceptable as the time between a problem being found and exploits in the wild keeps going down. If you need that Kerberos update but don't care about Bluetooth, or vice-versa, you're out of luck.

  If you are lucky, you might be able to find a workaround, or do away with what Apple ships and install your own, but that can create its own headaches and isn't an option for a whole lot of what Apple is shipping.

  The obvious solution to this is to break the updates out into individual updates that target the service they're affecting. I.E., roll the Apache updates into one, and the Bluetooth updates into another, and the CoreFoundation updates into another, and let people pick and choose what they want to install and when. The time lag for the flaw-to-fix goes down, and administrators and users are able to avoid what will screw them up without giving up the rest, so people have more secure systems across the board.

• Taking your medicine all at once

  Of course, those being affected by Too Much, Too Sexy aren't the only school of thought, or rather the only aspects that have to be kept in mind, and there are some problems with breaking out all of the updates. For starters, Mac users like to laugh about launching Windows and seeing that it needs to install 120 different updates to their 20, but there are others...

  1. What Apple is doing now -- lumping all these updates into one -- is easier for them, which means its less expensive. I.E., Apple does do some internal testing, and while the things I'm digging up are making me wonder at its thoroughness, there are internal processes patches have to go through before they go out. For the way they're set up now, running one patch through the QA process is much easier than running 30 through it.

  2. A lot of users -- especially the ones Apple says they're trying to target -- need updates broken out for the reasons I gave above, but I'm sure Apple if Apple did it they'd get just as many requests for them not to do it.

     Both groups of users are using Macs to try to get things done, instead of personal use, but it's how they go about dealing with updates that causes the problem. Generally, in an organization of any size where computers are being used to get stuff done you don't just install an update. It first has to be qualified.

     I.E., if you are responsible for supporting 50 Macs running x software, and a patch for Mac OS X comes out, you first test it under the apps your business or school needs to run to make sure MatheMatica, Quark, Office, networking functionality or even some learning game doesn't stop working. You have to do this, because an aberrant patch could work fine for most users, but a change in Safari could mean none of your employees could log into the intranet.

     If your company depends on say, Microsoft Office, and they release 40 little patches instead of 2 larger patches, each one has to go through the company's testing process before it can be rolled out, which means its more expensive and many of them would much prefer to deal with bigger rolled-up updates. The lack of being able to roll back updates, and the fact that software for administering Macs is somewhat anemic compared to other platforms only exacerbates the problem.

Of course the flip side of this is that my font bug for Office 2004 may well be fixed, but I'm waiting on it because they have 10 other things that they want to include in one larger update. This stuff is never simple.

If forced to choose between the two interests I'd go with the Too Much Too Sexy side, because of the two they are the ones that are actually screwed with little options or recourse. The other side has options, they're just annoying and require more effort and expense, but honestly I'd rather not have to choose sides.

When forced to deal with two competing yet valid interests with little overlap, if you try to take a hammer to it a lot of stuff is going to squish out at the sides, which means an elegant solution is needed.

 Posted by drunkenbatman
    August 17, 2005, at 03:27 PM