DrunkenBlog

Behind the Red Shed, with Jonathan 'The Wolf' Rentzsch

Every developer community has its "Doctor Who's". Faces who show up at all the conferences, always seem to have their hands in the cool stuff, and everyone in the know knows their name but not a whole lot of others do.

Jonathan Rentzsch is one of those people, and is a guy who wrote his own preemptive multitasking engine for the Classic Mac OS that benched in 400% faster than Apple's built-in Thread Manager for copying files. For Mac OS X, he created a solution to allow developers to do things they couldn't otherwise do called mach_inject and mach_override -- similar to Unsanity's APE -- and released it under the BSD license to be incorporated by anyone.

As a special easter treat, Rentz agreed to do the blog and let me pick his brain about a whole range of subjects including, but not limited to: mach_inject/mach_override software, WebObjects, Apple and enterprise, code optimization, programming languages, Core Data and even the rarely-discussed Mac software casting couch...

What's your coding background, and what led you to focusing on the Macintosh?

I started coding on a TRS-80 (16 kilobytes baybee), with a brief stint on a TI-99/4A until it got hit by lighting (not while I was on it). It would still boot, but it would only print the semicolon character ad-infinitum. While sad, its definitely inside the top five list of interesting ways of how my machines have died.

I was a total DOS guy. My Tandy 1000 wasn't going to be able to run the upcoming Windows 3.0, so I was figuring out how to score my next box. Then my brother-in-law gave me his old Mac 128K that he didn't need (he upgraded to a Mac SE). I thought Macs were toys, so I just set it beside my "real" PC. At the end of the week, I realized I hadn't powered up the PC in a few days. This old Mac was so vastly better, there was no comparison.

So while I didn't purchase the Mac when it first came out, I've been using them since System 0.9 on the original hardware. I focus on the Macintosh simply because it has the best software of any platform I know, and I want to be a part of it.

Interesting... A lot of people who liked to hack at software moved away from Apple kit when the Mac came out, just because things were so locked down, yet you seemed to gravitate towards it early. You weren't bothered by the closed-box aspect of the Mac?

I didn't mind the closed aspect because the little 128K was so vastly more capable than my "open architecture" PC. A good example is the mouse -- I had to purchase a serial card and serial mouse for my PC and get it working, versus the 128K's built-in mouse. By the way, that original 128K still resides in a pseudo-shrine in the Red Shed.

I also realize I can no longer spell "pseudo". My finger have now hard-wired "sudo". Thanks, Unix! I also cannot uppercase "cat" for the life of me.

What hardware do you use now, and do you have any non-Macs around?

I live off a PowerBook. I totally live the PowerBook lifestyle. Between a condo, office and the farm, ongoing presentations at PSIG and CAWUG, train rides, plane rides and on-sites, it's just easier to keep everything inside one machine that goes with me and has anything.

My current hardware is Titanium PowerBook G4 800 MHz 1GB RAM/80GB HD. It's long in tooth, but I haven't been impressed enough with the current PowerBooks to give this one up.

When/if the PowerBooks go dual core, get a G5, or go with a higher-resolution display, I'll buy a new model. That, and each major OS release makes my old hardware go faster -- not exactly the way to push newer hardware, Apple (grin).

Because we have to get this out of the way, what's the story behind the 'wolf' part of your name?

My full name is Jonathan Wolfgang Von Rentzsch. My last name is pronounced "wrench". I really like my name. It scales from a casual (perhaps even grease-monkey) "Jon Wrench", to a friendly "Wolf" (which most of my programmer friends use), all the way up to Country Club territory with the "Von".

Sattelite photos of your business address seem to show no shed, let alone a red shed, but rather you're directly across from a parking lot. So where does the name came from?

Those photos are old. We bulldozed the professional building and plopped the 100-year old shed there. Update your files. Oh, and don't tell the EPA.

There is indeed a real red shed in Wisconsin, on the small farm I grew up on (no farming, but we had goats, chickens, horses and cats). The photo masthead on rentzsch.com is a fairly recent photo, after I revved the interior and put up the siding. It finally got electricity just a few months ago. Woohoo 20th century technology!

You're from Wisconsin? 'Wolfgang Von Rentzsch' is a cool as hell name now, but one has to wonder if it wasn't a recipe for disaster on a Wisconsin playground... at what age did you stop cursing your parents?

Just because I have that name, didn't mean I had to advertise it. My school mates knew me phonetically as "John Wrench" -- no muss, no fuss. It was in my college days that I started using the Wolfgang part.

The 'Von' stays in the back pocket for now, maybe in my 30s or 40s. Of course, this line of reasoning opens the possibility that perhaps I have even more components of my name that I'm not currently advertising. Let the conspiracy theories commence!

Anyone who has seen your work knows you've obviously got the chops for software -- so why is there so little end-user software of yours I can point to?

The problem is two-fold. First, consulting is very good to me. For a while now I've been wanting to write more end-user software, but folks just keep tossing me interesting projects. I'm a total sucker when it comes to getting paid to pick up a new technology and then do something cool with it.

The second problem is separate, but related to the first. In general my BigCo clients write into the contract that I can't mention them or their projects. That's kind of lame on my end, but I understand why they do it -- they don't want me using their name to promote my image (or, as I like to imagine, they want to keep me their secret weapon).

So while there's little software I can explicitly point you towards, I'm pretty sure you're using software today that I had a hand in creating or updating.

You must have some sense of just how hard it is for someone to get work programming for the Mac, let alone staying a successful and independent consultant. Half of the Mac developer community is wondering just who you had to sleep with to go from hobbyist hacker to where you are today?

It wasn't so much the sleeping around as it was the blackmail that got me to where I am today. But I'll gloss over it all with a slick Hollywood biography to gloss over my evil rise to power, like "Bugsy" or "Working Girl".

I got started programming the Mac with HyperCard. HyperCard was programming crack, and the first hit was free. I see a lot of parallels with Cocoa today, but it's not nearly as easy to get started with Cocoa+Xcode+Interface Builder as it was with HyperCard.

I read Tog on Interface and Scott Knaster's books, and that's what sealed the deal. It became clear programming Macs was to be the best in the business -- writing insanely difficult software all in the name of making it easier for the user.

There's a market for Mac software talent, but it's small and closely knit. The best way to break in is to publish cool code and articles/papers/blog entries and attend the conferences.

What inspired you do create mach_inject and mach_override, and how would you explain what it is?

At MacHack 2003, a small group of leading-edge developers got together to discuss how to retake control over this new operating system we know as Mac OS X. The idea was to share our individual pieces of the puzzle, and form a coherent model and learn what was possible and worthwhile and what wasn't.

I realized at that meeting that we, as a community, needed a common open-source implementation of an extension technology. At that meeting I promised to deliver one. It's not as dramatic as it sounds, since I'm pretty sure no one at the table took my promise seriously. But I took it very seriously.

We needed an open source implementation because there's a lot of ways to do it wrong, and I wanted to make sure anyone who saw anything wrong could at least tell us about it, if not offer a fix himself. I also knew the license would need to be free and nonviral. I'm not one to go ask someone else to work hard and then give it away, so I decided to hunker down and do it myself.

I didn't really know what I was getting into, so I just kept on digging away. Reading code, writing code, trying designs. Finally it settled down into two separate projects that, like chocolate and peanut butter, are great by themselves but are also great together.

mach_override (and yeah, the underscore in the name bugs me too) is a source code library that allows replacing ("overriding") one function with another. The old function is still around, so you can call through to it if you'd like.

It's like the old days of trap patching, except mach_override works with both system-supplied functions and your own application-defined functions. If mach_override is the chocolate, then mach_inject is the peanut butter.

mach_inject allows one process to load and execute code in another process, at runtime. This can sound really scary to people, but it really isn't.

For example, the Unix permissions model doesn't allow code injection across user boundaries. So, a normal user on the system can't inject into a root process and 0wn the entire machine.

So, together, mach_inject allows you to write code and slip it into the applications you've paid for, while mach_override allows you to then take control of your software at a low-level, and extend it to do things the original authors may have never even thought of.

Both are hosted the the SourceForge Extendamac project (Extendamac was formed before I wrote mach_*). Right now the project is mostly focused on mach_* technologies, but I always make it very clear that we're open to other submissions of cool extension technology. I certainly don't see mach_* as the last word on the topic.

Can you give us an idea of who uses it in their software, and how?

mach_* seems to be most popular in workspace managers. At one time, there were four or five of them all using mach_*. I think we're down to a smaller number today, though -- I know at least one has left the market. Two version-control-system clients that integrate with the Finder use mach_override.

Then you have the smattering of other users: Stuffit Deluxe (they were able to get rid of their kernel extension), Default Folder (Jon Gotow's ultra-handy Open/Save panel enhancer), QuicKeys X2 and X3 (though X3 is less reliant on it now that they're using Apple's Accessibility APIs).

I also hear through the grapevine of other users who never told me directly they're using the packages. While I prefer to know who's using the packages and for what, the license does allow anyone to use it and never tell me.

Can you give an example of why it would be necessary for a developer to go outside the bounds Apple has set for them and go overriding another developer's functions to get what they're after?

All software has limitations. Sometimes those limitations are technical ("we don't know how to do that"), sometimes they're creative ("we never thought you'd want to do that"), sometimes they're political ("we won't allow you to do that").

Overriding allows outside developers to work around limitations. Where you see it the most is extending existing software's functionality or fixing behavior. Software is "soft", right? Overriding keeps it malleable.

One concrete example of overriding to extend existing software are the version control Finder extension projects (CVSFinder and SCPlugin). Flat out, the Finder doesn't know version control. These plugins want to do things like dynamically badge file icons based on their status.

The Finder doesn't know how to do that -- it only knows how to ask Icon Services for the badge information. So the developers discovered the right bottleneck, and have it return the information they look up themselves. Ta-da, you have a version-control-aware Finder.

Other times, overriding is handy for testing software. For example, you may want to test your code path when a system function returns an obscure error, like an I/O error.

Now, you could mount your disk over the wireless network and turn on three leaky microwave ovens and your portable phone to induce those errors, but it's better for your health and sanity when you can just override the system function, having it return the error on demand.

You mentioned MacHack -- now called ADHOC -- which is sort of a geek-mecca-cum-crucible. For those who have never been to Dearborn Michigan, can you give us an idea of what it's about and what it's like to participate?

Adhoc is about one thing: coolness. It has four facets:

1. Cool people
2. Talking about cool stuff (papers/sessions)
3. Showing off cool ideas (hacks)
4. Coming up with your own cool ideas and trying to get them working (your hack(s))

That's it. It's very informal, most of it made up on the fly, by really smart people whom you can communicate with at very high bandwidth. Unlike WWDC, which is Apple's firehose, Adhoc is a distributed firehose about what your peers think is cool, not just what Apple thinks is cool.

It's also known for its marathon hack contests where people setup their machines, throw down a gauntlet, and see just what coolness they can hack together within two days to show off at the end. Out of all the hacks you saw, what's really stuck in your head?

I just adored Andy Bachorski's and Nat McCully's "BrickPoint" from 1998. It put a game of breakout inside MacsBug. Any hack involving MacsBug automatically gets cool points in my book, but this one was inspired.

Also inspired was "Appearance Mangler" by Jon Gotow and Greg Landwebber. It overrode Kaleidoscope (a Mac OS 8-era Themeing System) to randomly pick from all your loaded themes for each UI element drawn. So your scroll bar would be Gizmo, your title bar platinum, etc. Until the window redrew, then it would change again. Funtastic.

More recently, Gorman Christian's "Interface UnBuilder" from 2003 was under-appreciated. It allowed you do drag and drop user interface elements from one running Cocoa app to another and it would still work! Whoa.

It looks like hacks using the new PowerBooks' Apple Motion Sensor will be hot this year.

So on the one hand, code-injection allows for some really impressive customizations and empowers a lot of developers, but are you also opening a pandora's box for other developers who are trying to debug their apps? I.E., there are rumors that if a bug report comes into Apple with Unsanity's APE or your code-injection showing up in the log, it's automatically discarded...

I would hope it's only a rumor. If Apple ships a software update that leads to crashes for everyone who has APE installed, that's really something Apple needs to know about.

I'm pretty sure it's just a rumor for mach_*, since the current version doesn't leave any explicit indicator of its existence in the backtraces. It has to do with the low-level nature of how mach_inject works -- it just copies over a block of code from one address space into another, which tends to strip the symbols you'd see in a backtrace.

The downside of mach_* is when innocent developers get bug reports and spend cycles tracking down a problem that isn't their fault. However, that's always been the case, long before APE or mach_* was written. mach_* has the potential to make it worse, but mach_* has also been proven to enable a class of software which may not have been written at all.

Obviously, I come down on the side of having more control over the software I own, otherwise I wouldn't have written mach_* in the first place.

So it's basically a tradeoff -- potential weirdness versus not having that functionality at all -- but let's jump back to mach_* and APE stripping the symbols one would normally see in a crash log. Several developers I pinged found it maddening; is the fact that the symbols have been stripped an indirect sign that your software is there, and is there no way to have it not happen?

There's two stages to injection: the "booster rocket" phase and the "get real work done" phase.

When code is squirted across address spaces, it lands in a rather primitive environment (specifically, a mach thread, which is the OS primitive used to make up pthreads, NSThreads, Thread Manager threads, etc). A lot of the system and Cocoa APIs don't work since they assume they're running on at least a pthread, and try to do wonky things like read+write thread-local variables.

The general technique is for this code to be small and do very little. All this "booster rocket" code usually ends up doing is to load another bundle from disk, in a normal fashion.

So, in practice, while the injected code may be stripped of symbols, the real meat comes from a normal bundle, and that should have its symbols intact because by then you're walking in through the front door.

Considering the functionality has been created independently twice, should Apple be providing their own hooks for this type of functionality for developers?

Technically, yes. Politically, definitely no.

mach_* is all about providing a level of control to developers that Apple does not/will not provide. It's not that Apple is being "mean" here -- they have legitimate concerns about stability and developer support load. It's a hard sell to management to allocate resources to support something that would make their jobs harder, requiring yet more resources in the future.

So, having mach_* outside of Apple means no one at Apple has to sign off on it. Being outside of Apple, that means Apple would need to explicitly allocate resources to kill it. And if I do my job right, that return on investment will never make business sense.

There was a time when WebObjects was really the only thing of its kind, but time has passed and the space it served isn't a vacuum anymore. To put it bluntly, is WebObjects still relevant?

When I picked up WO in 2000, I told pretty much anyone who listened that while WebObjects is the most advanced application server out there, that open source would catch up with it inside five years.

Yet here we are in 2005, and there's still nothing close. Believe me, I've been looking. Read the WebObjects developer mailing list for a recap of the treatment WebObjects developers got at WWDC 2004.

I would love to get off WebObjects and replace it with something open source. It would make web application development pitches easier if I never have to mention the dreaded "A" word. I have clients who will simply shut the door if I mention Apple's name, even today.

So I keep an eye on projects like Hibernate, Cayenne, Tapestry and Ruby on Rails. And yet, each time I start a new project, I do the math and rediscover WebObjects will deliver better software in less time. Lord, I wish it weren't true, but it is.

But that wasn't your question. Your question was "is WebObjects relevant"? As a commercial application server: no. It hasn't been for a long time.

No, WebObjects is only relevant if you're on the hook for writing lots of web applications fairly quickly. There's an definite escape velocity however -- the learning curve is steep, so it really only makes sense if you are currently or planning on becoming a professional developer.

Those WO archives are pretty brutal -- Mac community circa Amelio years brutal -- you get a real sense the canaries are dying in the mines. From someone who has years invested, what really sets it apart from what is available, and why is it dying off?

After all these years, WebObjects is still ahead in two critical areas:

• EOF
  (Enterprise Objects Framework)
• DirectTo*
  (DirectToWeb, DirectToJavaClient, etc.)

EOF allows you to define classes in a way that's language agnostic (remember WO supported both Objective-C and Java for a while), datasource agnostic, and application target agnostic. This sounds like needless abstractions, but it's not.

Objective-C is just a cruddy language to do data modeling, so it's great not to do it there (Java is just slightly better, and it still a really poor choice -- this is what Hibernate gets wrong). Data modeling just doesn't belong in general purpose imperative languages.

Funny as it sounds, data modeling doesn't belong in your database either. Relational Database Management Systems are too low-level -- they're the nuts-and-bolts of how things are stored. When designing and thinking about the app, you want to be able to think "these two entities are related in a bidirectional to-many", not "I have three tables, one of which is really just glue to get these other two tables related to each other".

Datasource agnostic means you're not locked into a database vendor. Case in point: I originally developed my WebObjects-based weblog on FrontBase, originally deployed on OpenBase and later switched deployment to MySQL+InnoDB. I didn't change one line of code.

Application target agnostic means the classes don't have to know how they're going to end up being used. In a desktop application? Fine. Web app? That works too. Web Services app? Sure. All at the same time? Rock on.

EOF is currently unmatched, but Cayenne is a very promising open source project that's helping to close the gap.

DirectTo* widens the gap. You start with your EOF data model, and give it to the frameworks. Out pops a functioning app. The app can create, edit, search and delete all rows of all your tables. This application generation happens at runtime, by the frameworks introspecting your data model. Gives me goosebumps.

By necessity, the application will be generic. You'll want to add buttons, remove fields, change spellings, etc. That's where the DirectTo* rule engine comes in.

When generating your app, the frameworks don't follow a hard-coded path. Instead, they ask the rule engine questions. What entities can I see? What attributes do they have? What component is responsible for editing this instance's "birthdate" attribute?

DirectTo* comes with sensible default rules, which are stored separate from your code and data model. You supply your own overriding rules that customize generation of the application. It's all quite beautiful, and I can't wait until Cocoa gets something like it.

I wouldn't say WO is dying off -- it's more in stasis, waiting its turn for attention from Apple. Now that Core Data is done, hopefully that attention isn't too far off.

Alright then -- not dead -- just resting. The obvious question that comes to mind is whether they'll have a developer base when they do decide to pull it from cryo?

They've already lost lots of developers. With the price drop from $50K, they lost many of the the high-priced consultants. With the move from Objective-C, they lost many of the old-tymers. With the current stagnation and catastrophic WWDC 2004, they've lost still more.

The last time fresh blood was infused into the community was with the price drop. Apple needs to do something radical to spark interest again. I know lots of folks would love it if Apple open-sourced WebObjects.

Something will have to change soon, because Apple needs bodies for their own WO apps, and having WO on the market is where they get their talent.

Why do you think you meet so much resistance -- you almost imply prejudice -- when you try to suggest a solution from Apple to some of these clients? And, are these small business, corporate, or enterprise clients?

I see resistance at all levels, for different reasons.

Small businesses tend to have the least resistance -- they're the most open to "I don't care how you do this, just get it done fast, cheap and good." The small business resistance is mostly "uhm, this isn't Microsoft?" and they barely know what Java is.

I win these guys over with price: I buy them a copy of WO myself, and show them open source web server software and open source database software. Then I show them what Microsoft is charging for Software Assurance nowadays. I show them I'm cheaper today, and they pay nothing forever more until the day they want more features. That's versus the ongoing Microsoft tax, lock-in and poor security track record.

With large corps, I can just tell them I'm pitching a J2EE solution. Their buzzword detector is satisfied, and I move on. But it's also large corps that don't care if a solution is better. Instead, it's more about minimizing risk. And a small pool of talented highly paid contractors is more risky than the legions of body shops of "standard" Java programmers.

Once the software is successfully developed and deployed, if the situation fits, I'll mention that the software is deployable on Mac OS X. It helps if the CEO has an iPod (grin). In my experience, in organizations with zero Apple installed base, purchasing of Apple hardware always follows deployment of Apple-compatible software.

That is, I've never seen someone hot to trot on Xserves and picking up WO as they go along. It's always been: deliver a successful software story, and then maybe Apple hardware will be considered in the future.

WebObjects has been called 'Cocoa for the Web'... A set of rich frameworks which were accessed via Objective-C. What did you think of Apple's move away from Objective-C to Java for WebObjects?

Even with 20/20 hindsight (well, maybe 40/40 right now), it's still unclear if WebObjects/Java was a mistake or not.

It's quite possible WebObjects would have simply completely died if it remained Objective-C. But WebObjects is hardly burning up the world as pure Java nowadays, is it?

Ignoring the business decision... Did coding for WebObjects in Java instead of Objective-C cause developers to lose anything in terms of features or functionality? Was anything gained?

Yes, we lost categories, which some WO folks used heavily to add business logic to their classes.

But we gained automatic memory management, which is really nice. We also gained access to the large Java library universe and a much better deployment story. One binary, every platform.

If we let the logic bottom out, it leads us back to Cocoa. As a developer, do you believe you'd lose anything if you Objective-C were dropped in favor of C#, Java or another object-oriented language? Would anything be gained?

It was clear Apple was backing away from Objective-C for a while due to developer backlash. It's also clear that period is over, and Apple is now keen on exploiting Objective-C's unique abilities to their advantage (Cocoa Bindings being a good example -- to pull it off in Java or C# would be... interesting).

Objective-C's big weaknesses today is its lack of a Windows story, its lack of automatic memory management and its method call syntax, in that order.

That said, my eye isn't on Java or C# to put the knife in Objective-C's back -- it's Python. Python addresses all the issues and PyObjC makes it easy to use Python with Cocoa. Indeed, PyObjC can do things Objective-C can't do by itself.

The geeks are lauding Python for its technical merits, but C# and Java programmers are being grown on trees... To go back to your large-corp example on why Apple-kit can be an impossible sell, what about talent pool? And, um, isn't Python damn slow? :)

We can't win the body count issue. We shouldn't try. It was like when Apple was trying to compete with Wintel with Mac cloning. As Chris Espinosa put it, that was a noble way to go out of business. Apple is doing well again today because it switched gears and is now leading the charge, not playing the industry's game. I just wish some of that thinking would rub off on WebObjects...

So, as non-Wintel programmers, we'll never win by being like them. Our best tactic is to be better. Better necessarily means different. We need to continually look out, find the low-hanging fruit on the bleeding edge, and carry it to the cutting edge. That's what we've been doing for the past few years, and that's why Windows is in our taillights again.

Windows is so backwards that the ancient Objective-C/AppKit was better. C#, while not aimed at Objective-C per se, gets them closer. So let's widen the gap again. It's highly arguable if Python is "better" than C#, but from a control-your-own-destiny angle, Python is a complete slam dunk. Python works well on *nix, Java, .NET and Mac OS X. It's open source. It's sane.

But I won't argue it's fast. It's usually just not so slow you care. (grin)

When you go through your body of work, you can spot some trends... like what seems to be your obsession with threading and how processes interact with data. You don't write software to allow for preemptive multitasking in OS 9, nor papers on atomicity for IBM on a lark. What about these areas has sucked you in?

Red Shed Threads -- the package I wrote that enabled preemptive multithreading on the Classic Mac OS -- was born when I was on a project to implement a telecom server on sets of Mac IIci's.

A machine had its two built-in serial ports and a four-port NuBus serial card. Keeping five modems full, running a complicated protocol like ZModem, all off a 25MHz 68030. That pretty much requires threading.

The alternative was using asynchronous callbacks. It's definitely possible, but hard to do with a stateful protocol like ZModem. It would be like reciting Homer's The Oddessy from memory based on instructions you'd get in the mail.

You'd get a note referring to a specific line/verse and a duration -- it's your job to reconstruct the poem, figure out where you left off, and only answer what the note explicitly asks for. As an added bonus, some days you'd get 20 such notes, while other times you'd wait a couple of weeks in-between.

So, I decided it would be less pain to just write my own high performance threading engine than to deal with that maze. I first wrote it for the 68K, and rewrote the kernel for the PowerPC (that's when I picked up PowerPC assembly).

Strange as it sounds, Red Shed Threads may even have a place on modern Mac OS X and Cocoa.

Implementing sheets is kind of a pain since you have to work with callbacks. Eliminating callbacks is pretty much why I wrote Red Shed Threads in the first place. But to date it hasn't been enough pain for me to consider it seriously.

I picked up my atomic knowledge by trying to make preemptive threads play nicely with each other. It's one of those things where other folks were way ahead of me, but I didn't have access to their literature. So I pretty much reinvented the wheel. Which I don't mind too much -- it's a great way to make sure you understand something.

Help me out here... aren't 'callbacks' when you take a function you've written and give it to another function as an argument, which does its thing and then 'gives it back' when it's done? How do you eliminate the need for that?

Yes. You don't eliminate the need for callbacks, you just eliminate the inconvenience. Basically, your callback becomes nothing more than a reusable "wake up this thread" routine.

So your thread calls the system, goes to sleep and unwinds the stack. Later (or immediately -- Red Shed Threads handles that case too), the system calls the callback, which resumes the thread from within the callback. Bango, you're back where you started, but without having to pass around explicit state blocks (refcons, userInfos, etc.).

Mac OS 10.4 is bringing in this new-fangled technology called Core Data. What is Core Data, and can you give a real-world example of why it is cool?

Brent gave a nice quickie intro to Core Data in his interview. In my eyes, Core Data is the single most important feature of Mac OS X 10.4. Which is telling, since it certainly doesn't get top billing (although I will confess Core Image is as sexy as hell and a genuine breakthrough). Core Data's purpose is to handle your application's data model, and the effect is profound.

Right now Cocoa doesn't have a data model, and boy does it ever show. You write tons of boilerplate imperative code to try to fill in all the details your computer would be very happy to deduce for you given just a handful of declarations. Because you have to write reams of code, your software develops premature rheumatism -- it becomes unnecessarily hard to make changes. Every line of code is a liability.

Remember I came to Cocoa after I had WebObjects under my belt. WebObjects comes with something called "Enterprise Objects Framework" (EOF -- and no, not End Of File). It's the precursor to Core Data. So the pain of such explicit coding is very obvious to me. I can't wait to write dozens of apps against Core Data, because it's so brain-dead easy yet powerful.

I think perhaps the best real-world example of why Core Data is cool is that if you use it right, you get undo for free. Now, Cocoa made implementing undo much easier than it used to be. Core Data makes it free. You do nothing, and you get a very respectable undo/redo implementation. That so rocks.

And it's the tip of the iceberg. For example, I don't see a reason why you couldn't get basic user interfaces for free (WebObjects has such a technology known as DirectToWeb and DirectToJavaClient). I wouldn't be surprised if Core Data apps don't get AppleScriptablity for free-to-cheap circa 10.5.

Right, so a developer is able to abstract himself from all that glue code, which will make developers smile. So what is not cool about Core Data?

That it isn't Enterprise Object Framework [from WebObjects]. OK, that sounds harsh. Let me explain.

EOF used to be written in Objective-C. Then, as part of the WebObjects/Java push, it got ported to pure Java. At first I thought Core Data was re-badged EOF/Objective-C from a few years back. I was rapidly disabused of that notion by the team (grin).

No, Core Data is from-the-ground-up new, fully taking advantage of modern Mac OS X. And that's the crux of my concern. EOF was proven and mature. Granted, it was also crufty, big and hard to get your head around.

I'm concerned that the result of Core Data is to better implement what we already had, versus pushing forward with something altogether new/better. Now maybe rewriting the foundation from scratch will enable a faster velocity in the future, so that in three year's time you break even and the rest is gravy. I know developers feel better working on a modern code base, and you can't discount job satisfaction.

It boils down to what's inside the black box.

If Apple is truly thinking+executing long-term (four+ years), Core Data was absolutely the right thing to do. But if that discipline isn't really there, then they would have done better to cut the risk and ship what they had, which was already ahead of everyone else.

Another thing that seems to stir your brain is code optimization. When you go through the various aspects of the system we're using, from the kernel to the frameworks to third party code, how much room for improvement is there on the platform?

Lots. Mac OS X is a huge system. There's an old programmer motto: make it work, make it right, make it fast.

The early systems struggled just to make things work, and try to catch up with where Mac OS 9 was (of course, Finder X will seemingly forever be lame by comparison). 10.1 and 10.2 were evidence of Apple moving from the "make it work" to the "make it right" phase. 10.3's speedups came because Apple was ready to move to the "make it fast" phase.

10.4 is faster still, although we may see some regression to "make it right" since they're hoisting fundamental aspects of the OS to add finer-grained locking to the kernel. And that's just with the code Apple ships.

It's funny, here you have an article ['Align your data for speed and correctness'] where I'm talking about how to save microseconds with data alignment, but even I totally abuse the system.

I have a nasty habit of doing things like using Cocoa's Key Value Coding support to cut a loop into a statement.

Now, the loop isn't really saved -- the framework has to do it for me. Worse, the framework has to parse a string to figure out what to do, and then turn those strings into message-sends.

It's just grossly inefficient. But it's actually the right thing to do.

Remember, each line of code is a liability. And unless Shark.app pinpoints it or it's just obviously slow, that means I'm delivering more software at less cost.

How important is the compiler in the process, and from a developer point of view, what is the state of the compilers on PowerPC?

Compilers are hugely important. Every time GCC gets a little better, the entire OS gets faster, the entire machine gets faster. Even if you just code at the scripting level, you still want a great compiler so your interpreter runs your scripts as quickly as possible.

Concerning compilers, programmers care about two issues, and they're almost mutually exclusive:

• Compilation speed
• Generated code quality

The faster your software is compiled, the shorter your turn-around cycles, the faster you can complete a feature or kill a bug. Here Metrowerks still just absolutely trumps GCC [GNU Compiler Collection].

While Apple is still working on making GCC faster, it's clear it's not going to approach Metrowerks' speed anytime soon. So, to Apple's credit, they changed the rules of the game by leveraging distcc to distribute compiles across multiple machines and putting predictive compilation in Xcode.

While predictive compilation is definite win, in my experience the distcc advantage has been almost academic. In environments you'd think they'd be using it, they aren't, for a number of reasons.

It's off by default (but I'm on Apple's side here due to the security implications), you really need to be on a wired 100Mb/1Gb network (802.11G isn't really good enough), folks are selfish and don't want to slow down their machine to share the group's compilation load and finally you need to match the exact version of Mac OS X across the machines.

It will be interesting to see if GCC 4's support for auto-vectorization bears any real-world performance benefits. On paper it's interesting and yields a real boost for its target scenario. What remains to be seen is if there's enough of that code lying around and if it's executed often enough to show a noticeable kick.

If nothing else, auto-vectorization could be used as Altivec training wheels. Write your straight C code, compile it, disassemble it, read the entrails and use it as a basis for your own hand-rolled Altivec code (bonus: Shark loves to provide you with advice to make it faster).

I know GCC's code generation used to be nowhere nearly as good as Metrowerks', but it's been steadily improving over the last four years. We'll see how GCC 4 stacks up, but I'd say even today GCC is on par with Metrowerks in the common case.

What are your thoughts on the state of the G4 and G5 architectures? Some developers seem to be disappointed in the fact that it uses the older Altivec engine, rather than the one found in newer G4 chips...

I'm impressed the PowerPC is still alive and kicking. During the gigahertz run-up wars, I pegged PPC for dead. I think it's mostly an artifact of Intel botching Itanium and IBM dropping the fab-for-rent idea and focusing on the PPC.

I'm not sure if the fact it uses an older engine has to do more with the 970's lead time or that IBM+Apple thinks people really just don't care. Maybe it's a mix of the two. I mean, I've never heard anyone outside of vector programmers and processor geeks even mention it.

So let's look at another aspect -- Hyper-Threading and Multi-Core CPUs. Is threading, and the issues it presents for developers, starting to come into its own on the Desktop?

Oh yeah. It's getting harder to speed up execution of single instruction streams, so you're seeing more processors allocate their transistor budget towards branching out (heh, a bit of a pun) by handling more than one instruction stream at a time.

I don't have inside knowledge, but I think a dual core G5 is coming sooner rather than later. And I don't expect Apple to suddenly drop MP in favor of dual core, which means we'll have a dual core/double proc mainstream PowerMac. Four-way boxen for the masses. Well, the Photoshop/Final Cut Pro masses, initially.

10.4's finer-grained locks will help a lot to wring performance from such a box (running 10.3 on such a box, even if the kernel were to be tweaked to acknowledge the additional processors, would likely leave the 3rd and 4th processors mostly idle).

Programming four-way boxen at the application level really isn't harder than programming for a two-way box, unless you really care about performance (then you start working with the OS to make sure your threads stay out of contention and stuff like that). There's about four levels regarding writing concurrent software:

1. Single threaded
   No clue about multiple threads. Probably most software today, which is actually OK, since we have multiple processes, and each process can go onto a different processor.
2. Cooperatively threaded
   Like the old System 7-era Thread Manager. You have threads, but your program is in control of when they're run, and no more than one is ever running at one time.
3. Preemptively threaded
   This covers both a preemptive time slicer on a uni box, and a MP box (I'll waive my hands a little here since you do have to worry more about memory coherency on a MP box than a uni box).
4. Declarative and message-passing concurrency
   This one jumps out of the box, and it's non-mainstream. I don't know if or when this will go mainstream, but I do know writing high-performance preemptively threaded software is difficult, easy to screw up, and hard to get right. Systems like Erlang and Mozart make it much easier, but they require rethinking your traditional straight-line imperative code.

Cocoa is kind of weird concerning threading. It supports multiple threads, but the first thread you spawn, Cocoa takes notice and sends out an application-wide notification that the app has gone multithreaded. If your app was a Star Trek episode, it would be the Klingon ship decloaking right off starboard bow with a system-wide RED ALERT and raising of shields.

You also can't use most of Cocoa from nonmain threads (the "main thread" is the thread that handles events like mouse clicks and key presses). It's really not bad, as it's typically not the UI you want to thread, it's calculations and I/O.

Speaking of weirdness with Cocoa and threading; are all of the Cocoa API's even thread safe yet? Or is that what you're hinting at with the secondary threads business?

I treat Cocoa threading the same way I treat C/C++ operator precedence: I don't even try to get it in my head, I just always add the explicit parentheses.

Chunks of Cocoa are thread safe and chunks are not. The chucks vary between OS releases (albeit with a trend of making more and more thread-safe). I don't even track it, I just do everything GUI-ish on the main thread. It's easier and always right. NSObject's '-performSelectorOnMainThread:withObject:waitUntilDone:' is your friend here.

What is this memory-coherency business you speak of, since you have to worry about it on MP boxes and in the (hopefully) near future those will become a given?

Processors have caches (smaller areas of faster memory) to help performance, so they don't have to reach out to RAM each time they want to read or write one little small nugget. The problem arises when two or more processors have their own copy of the same data, and one processor changes it. Now the caches on the other processors are "stale".

If you did nothing, you fall into a scenario called last-write-wins. Whichever processor writes to memory last, "wins" (by overwriting the other processor's changes).

When you use these special atomic instructions to write to memory, your processor calls a huddle with the other processors and they work out whose caches are to be marked invalid. This keeps everything coherent.

Unfortunately these huddles aren't free -- it takes time for the processors to communicate, update the caching machinery and finally memory (atomic writes have to go out to memory almost immediately since everyone has to start from scratch and re-read the data again to get on the same page of the playbook).

So there's a tension between performance and correctness here, but that's nothing new. The key thing for MP programmers to keep in mind is to know when to use C's 'volatile' keyword. Also check out gcc's aliasing flags, which gcc 4 enhances.

(You also have to worry about memory coherency on non MP machines, because those also have ICs which read and write to main memory, but this tends to be handled in the OS -- it's not something application programmers typically see.)

You've been around the scene for a long, long time. How is being a programmer in 2005 different from 1995? How do you think it'll be different in 2015?

Hey, I'm only 28 -- you make it sound like I invented the vacuum tube. My friends pulled off things like System 7 and the original Macintosh -- they're the real senior engineers here.

Being a programmer in 2005 is not much different than 1995. The IDEs are better, the languages are mostly garbage collected, but that's it. We're still doing the same stuff. Cocoa development is notably easier than Carbon development, but Cocoa isn't notably better than where NeXT was in 1995, so I won't consider that progress.

I think the most important thing is that Java, C# and Objective-C are pushing dynamic programming across the board. This is huge, and it's mostly under the radar.

There's been a bias against rich runtime information in desktop apps for a long time, mostly owing to older machine's crippling resource constraints and widespread of use of C/C++. Today's Java, C# and Objective-C apps have real runtime objects under the hood.

A lot of us already know what that means, and the type of software that it enables, but you're going to see that realization become widespread over the next ten years. I can't wait.

You mentioned garbage collecting, and I can't help but notice that Cocoa's own Objective-C is one of the OO holdouts that doesn't have it. Depending on who you ask this is completely unnecessary because it has 'reference counting' and 'autorelease pools', or it's hurting developer adoption and causing software pushed onto the world to be buggier than it needs to be...

I'm a strong advocate of making it easier to write correct software. As programmers, we spend way too much time beating on the same wheel when we could be working on the next thing and actually push technology ahead.

Garbage collection mostly solves the problem of invalid references and leaking of memory resources, so I'm in the GC camp.

That said, GC is not a complete correctness slam-dunk. That's because traditional GC only manages memory resources.

When you have an object that represents a nonmemory resource (perhaps a remote TCP endpoint), you've ceded control of that objects' lifetime to the nondeterministic GC. When it comes down to it, I feel we're overusing nondeterministic GC when we can fully work out what the lifetime should be, and flagging anything else as an error.

I've really have come to appreciate C++'s RAII (Resource Acquisition Is Initialization) idiom. It starts to feel transactional, with a guaranteed ordering of operations even in exceptional cases. C#'s "using" clause is a promising mix of the two systems, but I haven't used it enough to give it a definitive thumbs-up.

So, reference counting is deterministic, which can come in handy. That said, it isn't used much -- autorelease pools rule the day. It's just easier to not have to specify lifetimes, which is a failing of Objective-C.

Wow, that answer was wandering, so I'll nail it down: Objective-C would be far better off with GC.

Do you think there are any aspects of developing on OS X that are underrated? Are any overrated?

Overrated is the chick-appeal. "Hey baby, I write Mac OS X software" isn't the slam-dunk one would expect. Doesn't stop me from trying, though. I hear the iPod engineers have better luck.

I would say Cocoa is simultaneously underrated and overrated. Underrated in that it really is the best application framework out there, single-platform nature, warts and all.

Overrated in that Cocoa is just a framework. It's a great one, sure. But your nontrivial app is guaranteed to hit something that Cocoa doesn't handle, and that means you're going to write code. And you're going to have to know your problem domain backwards and forwards, and figure out how to extend Cocoa cleanly, which often involves exploratory coding.

Cocoa won't do all your work for you -- you're still going to have to be a great programmer to write great software.

If you could push a feature request, or perhaps a bug on your wish-list to the top of Apples' radar for OSX, what would it be?

Kill Finder X. It's really that bad. But don't just resuscitate Finder 9 -- do better. Finder 9 always needed a toolbar, for example. Finder 9 needed a plugin architecture. Finder X does trounce Finder 9 in terms of built-in search, though.

There are people out there who know what the next Finder looks like. Apple needs to get out of their way and let them deliver it.

You seem like a guy who prefers to refactor rather than rewrite, so I'm going to assume you wouldn't say that lightly, and we'll just proceed on the assumption that Finder X is a complete clusterfuck. How does a company known for its innovation, craftsmanship and software skills let something like Finder X out the door?

Finder X is the compromise between the Mac OS folks and the NeXT folks. Neither won, everybody lost.

Oh my god, the entire bastardized notion of switching from metal to aqua and hiding the sidebar when clicking on the toolbar chiclet in the upper right-hand corner.

Bonus: notice how if you click on the extreme right of the chiclet and try to switch back, you fail -- the window theme switch moved the chiclet slightly to the left and now you've got to follow it. Gag. Folks, this type of stuff makes Gnome look good.

I don't know how Finder X shipped. Someone high enough must be in love with it that the normal human interface vetting+feedback process didn't/couldn't take place.

If you had to give one last talk, or write one last paper, what would you choose to talk about?

Well, it turns out "Dynamically Overriding Mac OS X" was my last paper for MacHack/Adhoc. I initially wrote papers for MacHack because I had more time than money.

That inverted a while back, but I kept on writing papers because I enjoyed it and wanted to contribute back. I really thought my "Intro to WebObjects 5" paper was going to be my last paper, but my desire to fulfill my promise for Extendamac brought me back.

I prefer writing for my own website since it reaches more people than conference proceeding tend to. That and the final product is exactly what I created, for better or worse (having editors is a double-edged sword).

My hypothetical last paper or talk would probably be nontechnical. There's always lots of literature on the technical topic de jour, but there's much less on the personal strategies behind the world's greatest programmers. That's a shame, because those strategies tend to be timeless.

I know there's obvious similarities among the top 100 software engineers alive today, it's more of an articulating them to bring the topic to the conscious level.

By personal strategies, do you mean how they approach life, or how they approach problems? :)

How they approach problems, but isn't life our one biggest problem? (grin) In my superficial experience, some patterns I've noticed is that great programmers tend to be humble and don't have their ego in their work.

I get the sense that in order to get to great software, you have to iterate through bad software designs. If your ego is tied up on an early bad design, you never become great. So you learn to throw away your best paragraph and move on.

In the spirit of the site, what's your alcoholic beverage of choice?

I don't drink much, but I will admit to a Kahlua Colada streak last time I was in Maui.

Since I'm interviewing freakin' mormons lately, we'll try another personality traceback. What are the top tracks currently on your playlist?

I keep a short (100-song) "Infatuation" playlist, where newly purchased music goes into as well as my old favs, so here's the top ten:

1. Paint it black, The Rolling Stones
2. The Want, Android Lust
3. KOMPRESSOR DOES NOT DANCE, KOMPRESSOR
4. What We Need More Of Is Science, MC Hawking
5. Life in Mono, Mono
6. Sharp dressed party, ZZ Top vs Pink
7. Loneliness (Klub Mix), Tomcraft
8. Something Nation Army Going On, The White Stripes vs Frida
9. 1985, Bowling for Soup
10. Silver Screen Shower Scene, Felix da Housecat

(grin) I'll let you add the iTMS affiliate links to pay for your site's bandwidth...


drunkenbatman Addendum: I really owe Rentz a big round of thanks for allowing me to abuse his non-billable hours to the extreme that I did. We tried to do something a little different with this one, and -- even if everything didn't stick -- I'm hoping if you got this far you came away richer for it.

There's a lot of meat above, but we really only just skimmed just about every talking point, which is why I give you linkage:

• Tales from the Red Shed is Rentz's personal site, and ground zero for whatever he's working on.
• The web version of the original Dynamically Overriding Mac OS X, which goes into detail on how the mach_* works and how to incorporate it into your app. ADHOC has the original paper in .pdf form, although the name has been changed.
• An archive of slides to accompany the Dynamically Overriding Mac OS X paper which were used at the original MacHack presentation.
• The Extendamac project, which hosts the source code to mach_inject and mach_override.
• A list of known software projects using mach_*.
• If you're curious on learning more about Atomicity, Rentz's original 1999 paper Concurrent Data Access Without Blowing Up has much on 'The Window of Death'. There's an updated version, Save your code from meldown using PowerPC atomic instructions up at IBM's developerWorks site. The IBM is briefer, has many less tidbits, and has all the 80s music references removed (Aha is in there, I swear). Even if you're not a hardcore developer, chances are you can glean much from the older.
• If you're curious about learning more on data alignment, IBM developerWorks has Rentz's updated (Feb 8th) Data Alignment: Straighten up and fly right, which as my cliffnotes gave above is about structuring your data for improved speed and correctness. Again, even if you don't grok it all, you're bound to pick something up. I also found D.J. Bernstein's notes on this helpful.