Under The Microscope

Well, Shoot

Update (August 30th, 2011): Airfoil 4.5.7, featuring Instant On 4.1, is now shipping. You can download it from the Airfoil page. With this update, Lion users will be able to grab audio from already-running applications, as well as from the System Audio device. Get it now!


Approximately 48 hours ago, we shipped version 4.5.6 of Airfoil. Unfortunately, we shouldn’t have. Since shipping, we’ve learned that the Instant On component of Airfoil has a critical incompatibility with several third-party applications on Mac OS X 10.7 (Lion). To be clear, Airfoil itself causes no problems – this issue only occurs with version 4.0.0 of the optional Instant On component. This issue may cause crashes with applications including DragThing, SpamSieve, and others1.

Because of this issue, we’ve rolled the Airfoil update back. For the time being, Airfoil 4.5.5 is once again the latest version of the app. Please continue reading for more information.

What To Do If You’ve Installed Airfoil 4.5.6

Most importantly, let’s resolve any issues you might be seeing. If you’ve installed Airfoil 4.5.6, you can and should immediately roll back to version 4.5.5, by following these steps:

Step 1: Open Airfoil and choose “Install Extras” from the Airfoil menu.

Step 2: If the Instant On component shows version 4.0.0 is installed, click the Gear menu, and choose Uninstall. Once complete, your computer will need to restart

Showing Instant On is Installed

This is what you’ll see if Instant On is currently installed.

If Instant On is not installed, just move on to step 3.

Showing Instant On is NOT Installed

This is what you’ll see if Instant On is not currently installed.

Step 3: It’s critical that you complete the above two steps. Once you have, delete your existing copy of Airfoil.

Step 4: Download Airfoil 4.5.5 from our site, install it, and use it.

Step 5: Watch for future Airfoil updates to restore Instant On’s functionality to Lion2. These updates will be even more heavily tested prior to shipping.

We hope you’ll accept our sincere apologies for both the hassle of rolling back, and any crashes you ran into. We’re working around the clock to get Instant On functioning on Lion. Unfortunately, despite our testing, it was not yet ready when it was released.

More Details

While we can’t undo the problems caused by this update, we can provide full transparency as to what went wrong.

Airfoil 4.5.6 was to be the first version to provide full Instant On support for Mac OS X 10.7 (Lion). As you may know, the Instant On component provides two powerful features: the ability to capture audio from already-running applications, and the ability to capture all System Audio at once. The release of Lion broke all previous versions of Instant On, and a tremendous amount of time and effort was required to restore this functionality. Our users were eager to have this functionality back, and we were eager to provide it.

We got this working several weeks ago, and tested it throughly. In addition to our own testing, we provided developmental versions to dozens of users who wrote in looking for Instant On’s functionality on Lion. After discovering no issues in our testing, nor hearing of any problems from our users, we released Airfoil 4.5.6 on August 18th.

Starting yesterday afternoon, the 19th, we began to receive reports of problems from users of several different applications. We also spoke directly with James Thomson, developer of DragThing, about the issue. We determined there was indeed a problem caused by Instant On running on Lion, and set about gathering more data to determine its exact nature.

Now, one day later, the nature of the problem is clear. The root cause of the issue is a change to Mac OS X, which alters the order in which critical library functions load. This can place an application in a state incompatible with Instant On, which then leads to the application crashing. While this issue only crops up with a handful of applications (older software that uses non-standard entry points), the bug is still caused by us. It’s affecting users, so we felt compelled to pull the update. Shipping any kind of incompatibility with other third-party applications simply isn’t acceptable.

What’s Next?

Unfortunately, while the nature of the problem itself is known, the solution will require some time to implement. In addition to increased testing, the update will require a redesign of the way Instant On works to fully resolve the problems we’re seeing.

For now, while the basic functionality of Airfoil (as well as Audio Hijack Pro and Nicecast) works, you will still need to restart applications to grab their audio, and System Audio capture will not available. We’re continuing to work tirelessly to get Instant On working with Lion, and hope to have more news soon.


Footnotes:

1. A full list of affected applications follows:

• AirRadar

• Amnesia

• Apimac Timer

• Carbon Copy Cloner

• CastAway

• CDFinder

• CleanMyMac

• ColorNavigator

• Comcast Bandwidth Meter

• ControllerMate

• DiskCatalogMaker

• Doxie

• DragThing

• eMail Bounce Handler

• Find Any File

• GoFlex

• Google Earth

• HP10BII Business Calculator

• HTML-Optimizer, HTML-Optimizer Plus, HTML-Optimize Pro

• IntelliJ IDEA

• iCash

• iDupe

• iFFmpeg

• iPulse

• iVolume

• JollysFastVNC Pro

• KeypassX

• Lion Cache Cleaner

• LogMeIn

• Mac Bluray Player

• MacPilot

• MacTracker

• MaxBulk Mailer

• OfficeTime

• OmniGrowl

• Playback

• PowerTunes

• Processing

• Quay

• Radiologik DJ

• Radiologik Scheduler

• Scrivener

• SecretSync

• ShareTool

• Sitemap Automator

• Skype 2.8

• SmartSleep

• Snapz Pro X

• SpamSieve

• TeamViewer

• Tweet Adder

• Ultralingua

• Vuze

• Weatherman

• Web Site Maestro

• WhatSize

• Xscope

• Yummy FTP

If we hear reports of issues with other applications, we’ll add them here.  

2. If you’re still running Mac OS X 10.6 (Snow Leopard), Instant On 3.1 works just fine with Airfoil 4.5.5. Install it from the Install Extras window, under the Airfoil menu. 

37 Responses to “Well, Shoot”

  1. Peter Witham says:

    Thanks, I thought I was going crazy. I appreciate the honesty and solution to get us back to ‘normal’, take your time Airfoil is still one of my favorite all time Mac iOS tools.


  2. Chris says:

    Thanks for the heads up. I’d opened a support ticket with LogMeIn about their app constantly crashing, good to be able to finally find out what the problem was. I’ve suggested they let other users know as well


  3. Joshua says:

    Thanks for the info!

    That explains why Find Any File started crashing… D’oh! Check to see if the Skype build before v5.x is also affected. I tried launching it tonight and it was crashing just like FAF. Ended up installing the latest (ugly) version of Skype. :(


  4. Alex says:

    Thanks, I thought I was going crazy too, trying to find out what was causing this. Incidentally, I also found Google Earth 6.0.3.2197 to have the problem.


  5. tlakarpi says:

    Caused also Lion Cache Cleaner 6.0.2 crash.


  6. Martijn says:

    Incompatible with Skype 2.8 too.


  7. Rosyna says:

    Yeah, I think it’s called something like, “32-bit ASLR”.


  8. RemyLeroy says:

    What if I have installed Airfoil 4.5.6 and INstant ON 4.0 on MacOS X 10.6.8?

    Should I revert to 4.5.5 too? or the crashes only occur on Lion?


  9. Niklas says:

    I´m running 4.5.6 om Lion with no problems and I do like to keep the instant on functionality. I assume that the new version coming will have version number 4.5.7 or higher allowing me to stay on 4.5.6 for the time being waiting for a update to arrive?

    If I would run into problems at least now I know what to do =)


  10. PJL500 says:

    Airfoil is a 5-star app. Thanks for the fix. My machine is out from under the 4.5.6 update turmoil now. Looking forward to Lion Instant On and more good stuff.


  11. Thomas Tempelmann says:

    Doh! I just spent a few hours trying to track this down with two of my Find Any File users, even writing a general guide on tracking such conflicts (see http://www.tempel.org/FindingCrashCausesOnOSX ).

    And when we finally found the culprit, I am coming here to report this, only to find you already know. Well, good :)

    I just wished I had been informed right away.

    Anyway, thanks for posting about this so openly, and good luck getting this fixed properly soon and without too much more hassle. I know how it is when half the world seem to be waiting at your doorstep and you’re freaking out inside from a mix of shame, frenzy and sleep deprivation.


  12. Thomas Tempelmann says:

    On a side note, I also once had to resolve a conflict between DragThing and Find Any File (actually with any app made with Real Studio), and he (James Thomson) was very helpful and responsive then, too. Cheers to that!


  13. glgray@me.come says:

    I believe that another incompatible app is Comcast Bandwidth Meter from Tisza LLC.


  14. deo says:

    macpilot 4.3, sharetool 2, mac blu-ray player are incompatible !


  15. Paul Kafasis says:

    RemyLeroy: We recommend downgrading on both OSes. As noted, on 10.6, you can install Instant On 3.1.

    Niklas: Yes, the next version will be 4.5.7 or higher. It’s up to you if you wish to keep running Airfoil 4.5.6, but we certainly do NOT recommend it.

    Thomas Tempelmann: I’ve often thought the app should have a message window, so we could send urgent messages like this to all users. However, in this case, even that might have missed, if you didn’t open Airfoil. We definitely spread it as loud and far as currently possible – hopefully it gets to everyone who needs it.


  16. Rick Moen says:

    As stated earlier, your honesty, quick response, and solution is what makes yours a good company. I will continue to recommend your products to my friends.


  17. Jose Nieves says:

    Add ControllerMate to the list of affected apps, crashed on launch – too bad I didn’t see this before I wiped my HD, reinstalled and restored from backup.


  18. Bob Martin says:

    Add OmniGrowl and the Logitech Webcam software.

    Thanks for fessing up — I’ve supported your stuff for years, and I’ll continue to do so.

    –bob


  19. Stan Busk says:

    The problem also affects iCash and MaxBulk Mailer.


  20. WebWerner says:

    Tweetadder 3 , TeamViewer, Max Bulk Mailer , Max Email Bouncer crashes also with this bug!


  21. Leo Newball says:

    So, is AirFoil the main problem, or is the Instant On capability a part of the issue as well? For instance, if I’m running Instant On 3.1, would that also cause conflicts with some of the listed applications?


  22. RAGE Software says:

    This bug also affects Sitemap Automator.


  23. loudestnoise says:

    I can also report I was unable to launch Doxie, the document scanner app after installing Instatn On in Airfoil 4.5.6


  24. Mike Bombich says:

    I’m getting reports from Carbon Copy Cloner users as well. FWIW, I think this implication is inaccurate:

    > While this issue only crops up with a handful of applications (older software that uses non-standard entry points)…

    CCC, for example, isn’t “older software” nor does it do anything non-standard as far as memory allocation is concerned. Perhaps it would be more accurate to say that this affects applications that continue to support Tiger? Not knowing the underlying cause of the problem, that’s only a guess…


  25. Jeff Johnson says:

    Mike: To be clear, this is our fault. The affected apps are not doing anything wrong. The terms “old binaries” and “non-standard entry point” come from comments in Apple’s open source code. They are in no way intended to denigrate the affected apps. We were just trying to distinguish which apps are affected. You can email us for more information.

    Rosyna: That’s not actually the problem. It turns out to be very subtle.


  26. Norbert M. Doerner says:

    I just heard from a customer that CDFinder 5.7.3 is also affected:

    http://www.cdfinder.de

    CDFinder is also still Carbon based, so maybe there could be a common point there?

    Anyway, thanks a lot for making this blog entry and describing the issue!


  27. Pierre says:

    Please add Carbon Copy Cloner to the list of affected apps.

    PB


  28. Patrick Stein aka Jolly says:

    I sure understand that those bugs happen. I don’t have a problem with that – bugs do happen.

    But NOT writing the developers of the named and therefore known applications did cost us ( I’m one of them ) a lot of support time. I’m quite p***sed that you did not inform me, even though you did know that my software was affected. I do have an email on my website unlike you.

    After receiving a couple of support mails I tried to figure out what the bug was, but before the program was even loaded into memory the dyld crashed. So it was some other software linking into every program on the Mac. I got here cause one customer did a reinstall of his system and installed dozens and dozens of applications one at a time.

    Would be nice to fully disclose what the problem is. I have no knowledge of

    Patrick Stein aka Jolly

    Addition to the list:

    JollysFastVNC – all versions (appstore and website versions) crash.
    SmartSleep – all versions
    ScreenRecycler – probably
    WhiteList – probably


  29. Christian Schildhorn says:

    Really would appreciate more direct communication to other developers. I’m THE ONE customer!!


  30. Joe Ranieri says:

    Jeff: I’m curious, would you consider posting technical details? If I had to hazard a guess, you were relying on all of the image’s initializers before you added your dyld image load hook? or perhaps just the important bits that crt.o normally does?


  31. Paul Kafasis says:

    Leo Newball: Instant On 4.0.0 itself, on Lion, is the issue. I’ve edited the post to reflect that. Instant On 3.1 will only run on Snow Leopard, which will be fine.

    Patrick Stein: We’re certainly sorry you ran into the issue – we’ve been speaking with devs as we’ve confirmed the issues with their software. Your app was only recently added to the list, and we haven’t had a chance to contact everyone yet. We do have email right on our site though – our contact form goes directly to us. Please get in touch that way if you’d like more details – we’re happy to work with you.

    Any app devs: We’ll likely post more technical information on this issue here on the blog when we post our fix (hopefully later this week), but get in touch if you’d like more details.

    Joe Ranieri: We’ll likely have more information posted in the future, along with our fix.


  32. Daniel P. says:

    Thanks for the heads up – I can report I was unable to launch iFFmpeg after the update. http://www.iffmpeg.com/


  33. Ken B. says:

    Processing SDK environment is also incompatible.


  34. Michael Ventura says:

    hm, i don’t run any of the affected applications on the machine where i need instant on and i’d really like to try this, but it seems like it is impossible to get it now. i don’t suppose we could get a direct link in the comments here so that it doesn’t hit general release but can still be used in the meantime by people that know the risks?


  35. CM Harrington says:

    So how are you getting around the sandbox/Powerbox situation in Lion?


  36. pl_svn says:

    please add to the list of incompatible apps:

    – PowerTunes
    – WhatSize

    and, please, as you have your customer’s email addresses, should this happen again *send all your customers a note*!
    got mad for days and involved WhatSize and other developers too in troubleshooting this nasty issue!

    thanks


  37. Paul Kafasis says:

    Michael Ventura: You can try out the beta, linked in the update at the top of this post.

    CM Harrington: Thus far, we’ve run into no issues with sandboxed apps.

    pl_svn: Unfortunately, emailing our thousands of users really wasn’t a viable solution. This problem has affected far too many people, but because the software was available for just 48 hours before being pulled, it was a very tiny fraction of our users. Sending an email to all of our users would have been incredibly counter-productive. Instead, we posted this everywhere where the update had been posted, and more.


Comments for this post have been closed. Thanks for reading!


Our Software