Basic Proof of Concept

<< Prev: Introduction       Next: Loading Firmware >>

I figured a good first step would be to create a kext that recognizes the hardware it's supposed to apply to and loads and unloads successfully. That would be the most basic proof of concept. Also a good sanity check — if I can't get that far, I should quit now.

It turns out this part can all be done in OS X-land.

Project Setup

Not wanting to make this any more difficult than I had to, I started with Xcode.

I created a new project, with type OS X / System Plugin / IOKit Driver. That gave me some pretty empty boilerplate code. About all I did to customize it initially was change the header file from .hpp to .h because I haven't really seen .hpp files elsewhere and .h seems to work fine.

Matching Hardware

These cards are all PCI-based (PCIe, m.2, etc.). The PCI device infrastructure lets you specify a list of compatible PCI vendor and device IDs in the Info.plist for the kext, and it will start the driver when matching hardware is present. All I needed for that part was the list of device IDs. I found it here (just scroll past the license). That code actually shows the device ID and sub-device ID, since the vendor ID in both cases is Intel (shockingly, 0x8086).

I used that big list later for matching all the specific cards the code was supposed to recognize, but just to match the driver with the correct product families, I started with the IDs in the Info.plist like this (notice they all end with the vendor ID of 8086):

<key>IOKitPersonalities</key>
    <dict>
        <key>AppleIntelWiFiMVM</key>
        <dict>
            <key>CFBundleIdentifier</key>
            <string>org.opentools.${PRODUCT_NAME:rfc1034identifier}</string>
            <key>IOClass</key>
            <string>AppleIntelWiFiMVM</string>
            <key>IOProviderClass</key>
            <string>IOPCIDevice</string>
            <key>IOPCIPrimaryMatch</key>
            <string>0x08b18086 0x08b28086 0x08b38086
                    0x08b48086 0x31658086 0x31668086 0x095a8086
                    0x095b8086 0x24f38086 0x24f48086</string>
            <key>IOProbeScore</key>
            <integer>1000</integer>
        </dict>
    </dict>

The only challenge was testing it on my MacBook Pro, which doesn’t have a suitable card.

Fortunately, I found a workaround for this stage. Normally, OS X only loads one driver for each device. However, by specifying an IOMatchCategory in the Info.plist, you can get a separate driver to load for each "category". So I could add the device ID of some hardware I actually had in IOJones, with a bogus IOMatchCategory, and then load the driver for my non-Intel WiFi hardware. It would never work for real, but it was enough to let me prove that the driver would load and start, and therefore emit messages to the system log.

The changed plist entries look like this (notice the non-Intel ID at the start of the list):

<key>IOPCIPrimaryMatch</key>
    <string>0x12421b21 0x08b18086 0x08b28086 0x08b38086
            0x08b48086 0x31658086 0x31668086 0x095a8086
            0x095b8086 0x24f38086 0x24f48086</string>
    <key>IOMatchCategory</key>
    <string>Foo</string>

Finding the Kext

Well, I'm getting a little ahead of myself. First, I tried to load the kext with "sudo kextload [kext file]". Rather, I tried to try that. But I couldn't find the kext it built.

I guess I'm used to every other build environment on Earth, which puts the thing you just built right there in your project where you can then go on to try it out or work with it. Xcode, in contrast, stuffs it in a directory you'd never find. It puts a bunch of random characters in the path just to make sure it's unpredictable location. Because, you know, it would be so dangerous if you could actually find the thing you just built.

OK, I guess it just moved my cheese. Anyway, right-clicking the kext name under "Products" in Xcode lets you "reveal" it in Finder. Then I dragged it to someplace more useful.

SIP and Unsigned Kexts

Of course, then when I tried to load it, it wouldn't load or start. I hadn't set up my code signing infrastructure in Xcode. So for now, I disabled System Integrity Protection on my development machine. There's an easy command for that — "csrutil disable". Unfortunately, it has to be run by booting to Recovery with Command-R on startup, an then opening Terminal to run the command. I was annoyed by this reboot, but at least I knew it was coming (more on that part in a later post).

OSBundleLibraries

That got past the signature problems, but the kext still wouldn't load:

$ sudo kextload AppleIntelWiFiMVM.kext
Password:
/Users/ammulder/temp/AppleIntelWiFiMVM.kext failed to load
- (libkern/kext) validation failure (plist/executable);
check the system/kernel logs for errors or try kextutil(8).

The log then reported:

2/23/16 3:49:37.347 PM com.apple.kextd[45]:
/Users/ammulder/temp/AppleIntelWiFiMVM.kext
is invalid; can't resolve dependencies.

That reminded me of the Info.plist section that was supposed to declare dependencies. So I ran the tool to generate that block of code. (If there's a tool for that, why do you have to write it instead of the compiler or kext loading system just using the tool automatically? Good question.)

It gave me this:

$ kextlibs -xml AppleIntelWiFiMVM.kext
 <key>OSBundleLibraries</key>
 <dict>
  <key>com.apple.iokit.IOPCIFamily</key>
  <string>2.9</string>
  <key>com.apple.kpi.iokit</key>
  <string>15.3</string>
  <key>com.apple.kpi.libkern</key>
  <string>15.3</string>
 </dict>

I put that into my Info.plist instead of the empty OSBundleLibraries it defaulted to.

File Permissions

Then I tried again:

$ sudo kextload AppleIntelWiFiMVM.kext
/Users/ammulder/temp/AppleIntelWiFiMVM.kext
failed to load - (libkern/kext) authentication failure
(file ownership/permissions); check the system/kernel
logs for errors or try kextutil(8).

The log wasn't much more helpful:

2/23/16 5:28:51.886 PM com.apple.kextd[45]:
Can't load /Users/ammulder/temp/AppleIntelWiFiMVM.kext
- authentication problems.

But it turns out this means the kext needs different file permissions:

$ sudo chown -R root:wheel AppleIntelWiFiMVM.kext
$ sudo kextload AppleIntelWiFiMVM.kext
$ sudo kextunload AppleIntelWiFiMVM.kext

That did it! Now the kext loads and unloads successfully. Checking Console.app, I see:

Feb 19 14:08:57 sulaco kernel[0]: AppleIntelWiFiMVM::init
Feb 19 14:08:57 sulaco kernel[0]: AppleIntelWiFiMVM::start
Feb 19 14:08:57 sulaco kernel[0]: AppleIntelWiFiMVM Found card
definition for Vendor 0x8086 Device 0x095a
SubVendor 0x8086 SubDevice 0x9010 Revision 0x59
Feb 19 14:09:07 sulaco kernel[0]: AppleIntelWiFiMVM::stop
Feb 19 14:09:07 sulaco kernel[0]: AppleIntelWiFiMVM::free

That 0x095a device is the card in my test machine! It loaded and recognized the hardware and then shut down on command; so far so good.

Code & Configuration

The code and XCode project for the progress described here is in GitHub:
https://github.com/ammulder/KextTest

<< Prev: Introduction       Next: Loading Firmware >>

Introduction

Next: Basic Proof of Concept >>

I recently started a project to port drivers for the current Intel WiFi chips from Linux to Mac OS X. I can’t claim this was especially necessary: most Macs come with functional WiFi, and there are already third-party cards that are either compatible out of the box or work given appropriate drivers. Still, it’s time to learn something new, and this hits a couple of bucket list items for me — writing something in a different language, learning Linux kernel development, and doing something meaningful in the OS X/iOS programming environment. 

A bit of research turned up the following:

The Good

• The "iwlwifi" Linux driver is just plain part of the Linux kernel sources, so it's publicly available and actively maintained
• In order to operate, you need both hardware and firmware. During startup, you load the firmware into the hardware, and then it works. While this process arguably doesn't belong in the "good" list, the firmware is available for download. The license appears to allow you to load it on OS X, though potentially without patent coverage. That's OK as far as I'm concerned, because it's really just a hobby project.
• Others have ported Ethernet (though not WiFi) drivers from Linux to OS X, with source code available. This means I have code for some working OS X Ethernet drivers to refer to.
• Apple has some starter documentation on Creating a Device Driver
• Apple's documentation refers to an IORegistryExplorer tool to browse your current hardware configuration, but I didn’t seem to receive it with Xcode. There seem to be copies available from various sources around the net. But perhaps better, there’s an open-source equivalent called IOJones.

The Bad

• The Linux code is in C, while an OS X driver will be an IOKit kext (kernel extension), in C++. While that’s not explicitly bad, and in fact is much more compatible than if one was in Java or something, it does make for a bit of an impedance mismatch. I guess the real problem is that now I have to learn two new styles of programming at once. I've theoretically used C before, but that was maybe 20 years ago and I never did anything "real" with it.
• The Linux and OS X APIs are different in a number of nuts and bolts type areas — macros and functions used for logging, memory allocation, synchronization, and etc. as well as the basic data structures provided. It means there are a lot of differences scattered throughout the code.
• The iwlwifi supports devices with both the older dvm and newer mvm firmwares. I am happy to simplify by only supporting mvm for now, but in many cases the dvm code is mixed in (just disabled by #ifdef blocks if I choose not to support it).
• Conveniently, there is a free "book" covering the Linux wireless architecture. Sadly, most text other than descriptions of specific structs was never written, and it looks to have been abandoned more than five years ago. The wireless wiki looks more promising, although it's not completely up to date either. Comments in the source code it is, then.
• Published books are available covering both OS X and Linux kernel and driver development (e.g. 1 2 3). However, they all seem to be several releases out of date.

The Ugly

• While all the Linux APIs are public, Apple has not released a wireless API. There are APIs available for PCI devices and Ethernet devices, but not wireless. My guess is that they figure the demand for third-party wireless devices is low, and their code is probably a bit messy; not worth cleaning up and documenting for external release. This has a couple of implications:
  • Just plain more work because all the generic wireless plumbing has to be implemented as well as the driver itself
  • Besides the driver itself, the project will ultimately need to include a user-space API and tools to display and prioritize available wireless networks, choose one, specify and store the security key, and etc.
  • Once finished, the device will appear as a generic Ethernet device instead of as a native wireless device in System Preferences / Network, and will not use the wireless menu bar control to change WiFi networks.
  • As an example, Realtek has created their own app for devices using their USB wireless chips. It’s bad — overcomplicated and annoying to use. It comes with a dysfunctional menu bar icon. It wants the main app to be on all the time. Yuck!

Bottom line, this is going to be a challenging project. But it should be good for learning.

Next: Basic Proof of Concept >>