How-To – Use a 3Dconnexion device inside an OS X application written in Swift

Bitte teilen! Please share! Share on FacebookShare on Google+Tweet about this on TwitterShare on LinkedInPin on PinterestEmail this to someone

Welcome to my how-to guide on how to implement a 3Dconnexion device in Swift inside your own OS X application. In this extended article I want to guide you through the process of how to setup a Xcode project for 3Dconnexion, import their framework written in C, and initialize the driver. You will be able to register callback methods the „swift way“, to receive data from the device. I will also discuss hiccups that you will face, while trying to forcefully join C and Swift. This article is accompanied by the source code that you can download for free from GitHub and my first AppStore App to demonstrate the implementation.

Because you have clicked on this article or searched for it with a web search-engine of your choice, I think you know what a 3Dconnexion device, Swift, or OS X is. For all that stumbled on this article by accident, but are still reading this lines, let me give you a very brief explanation what I’m talking about in the spoiler-box below – everybody else can use the following table of contents (click to expand) to jump directly to the more involved sections. Have fun reading, good luck coding, and please be so kind to leave a constructive comment regarding either the content, or my bad English – so that I can improve this series and my skills.



Prerequisites and project setup

Everything I will tell you from this point on is based on my own experience, while implementing a little application to monitor one of my 3Dconnexion devices. You can get the source code for this application on GitHub, or directly using this url

https://github.com/MartinMajewski/ToolShelf-4-3Dconnexion.git

I will not go into every detail of my implementation, but highlight some of the parts I find helpful for dealing with 3Dconnexion’s framework.

You can also download the accompanying app on the App Store. Read more about it at the bottom of this article.

AppleAppStoreBadge

 


Download the SDK – or not

Although 3Dconnexion provides what they call „SDK“, it is not necessary to have it present during this guide. The SDK does not contain anything that has to be installed or available during implementation or runtime. It just provides you some examples, but very little documentation. If you read this series you will probably have a much more information, than I had reading the docs – especially in terms of using Swift.
All you really need is the latest 3Dconnexion driver installed on your machine, actually. This driver provides the framework file that you need to be referenced inside your Xcode project. To get the latest driver please visit the driver download site at 3Dconnexion.com. By the time I am writing this guide the latest driver is 10.2.4, dated to 10/09/2015.

However, if you want to investigate how the driver works, especially how you have to structure certain control calls, the SDK is a good resource – effectively the only one you will get, except for this tutorial, as it seams.

To get the SDK, head over to 3Dconnexion, register a developer account, wait for acceptance and download your copy for OS X. My current SDK is at version 1.0.3 (May 2015 release).


Start a new Xcode Cocoa Swift project

I am using Xcode 7.2, thus every advice relates to this version. As we are using Swift 2.1, you have to have ElCapitan (OS X 10.11.*) installed.

To start from scratch open Xcode and start a new project. You can start with a command line application, if you’re not planning to go beyond a pure proof-of-concept implementation. I prefer to start a storyboard driven Cocoa GUI application. Here I can also use CLI printouts with the print function for debugging and testing. I am also all set to develop a more advanced GUI bases application, without too much extra effort. The storyboard configuration makes designing GUIs much easier.

Either way, use Swift as programming language and save your project to a desired location.Bildschirmfoto 2015-11-20 um 11.44.30


Import the Framework

Open the project preferences by clicking on the topmost folder inside the Project Navigator. Inside the General tab find the Linked Frameworks and Libraries category and add the framework which is normally located at

Status can be set either to Required or Optional – I haven’t noticed any difference.

Bildschirmfoto 2015-11-28 um 23.11.43

Note that it is mandatory to have the driver installed, to run the final application. Weak linking the framework is not possible directly from Swift (haven’t figured out a native way – for my workaround see the section Weak-linking the framework (C workaround)), so your application will crash if the framework is not present.


Create a bridging header

Being a C library, Swift is not able to access the framework automatically. You have to create a bridging header that imports the framework headers into your workspace. The integrated Swift converter in Xcode will then make all C symbols available to Swift. Sadly it is not able to perform the conversion on all symbols that are commonly used in C. I will explain later what to do with values, such as „four-byte-character-integers“.

Inside your project create a new header file via

⌘N → OS X Pane → Header File

name it 3Dconnexion-Bridging-Header.h and import both framework header files.

Next, you have to tell Xcode to use the bridging header.

Head over to the project preferences once again, choose the Build Settings tab and start typing „bridg“ into the search field. You“ll get the settings pane called Swift Compiler – Code Generation and have to insert the path to the the previously created header file into the Objective-C Bridging Header field, like this:

Replace every part of the above string corresponding to your actual path and project. Xcode will resolve the placeholder variable as soon as you hit enter.

Bildschirmfoto 2015-11-20 um 13.29.35


Weak-linking the framework (C workaround)

When you get a look at the SDK documentation PDF file, you will see at page 12 the advice to weak link your framework reference. This way you can check whether the framework is present (driver is installed), or not, and react accordingly at runtime. Basically, you simply check if the pointer of an API function directs to an actual function in memory at runtime, or not. If not (pointer is null) the required driver is not loaded and you have to handle this situation.

In Swift however there is no such a thing as a null pointer, except you have declared your object as Optional (?-operator). In case of the 3Dconnexion driver, none of the imported and converted symbols are declared as Optionals, simply because this mechanism is not part of C and therefore not delegated to the Swift converter.

So as long as there is no native way to check the existence of a framework at runtime in Swift (if you know it better, please inform me and I will update this guide and the source code), you have to provide some C code to do the job. This C code makes no use of exotic syntax or pattern, thus it gets adopted by Swift nicely and the embedded functions can be used by Swift with no problems.

So here is how the resulting C file should look like. It can be placed wherever it makes sense in terms of file structure inside your project. I will describe the components in a second.

Inside the C file you can see the extern expression, which is the (Objective-) C way of telling Xcode / OS X that there is no guarantee that the reference to the framework will have a valid value during runtime. Basically it states that the public API’s function symbol SetConnexionHandlers could lead to nowhere. We will use SetConnexionHandlers in a minute to register our App to the driver.

The last step is to implement the simple function isConnexionDriverAvailable, that checks if the mentioned symbol points to an existing function in memory, or to NULL. The result of this check is returned to the caller as boolean. We have to perform this check inside C, to be able to use pointer arithmetics of this kind.

Now we have to make the C function available inside Swift. This is done exactly the same way as we did it with the 3Dconnexion API: with a bridging header file. To avoid extra work, use the already existing bridging header from the Create a bridging header section. Remember to include the bridging header on top of the new C file, as shown above. Add the following line beneath the two import statements inside the bridging header.

The resulting bridging header should look like this

 

Since isConnexionDriverAvailable is declared inside the bridging header, we can now access this little helper from Swift code. It will return true if the driver is present and loaded and false otherwise.


Activate Sandboxing

The last step before we can actually start using the device within our app, is to enable Sandboxing and USB support. To do so, head over to the Capabilities tab inside your project preferences and switch on App Sandbox, as well as check the USB checkbox on the Hardware category.

Bildschirmfoto 2015-11-28 um 23.30.30


Fiddling with C and Swift

If you followed the steps until here, you are good to go, implementing your 3D Mouse into your application and use its data. This is the focus of this section. But before I show you which lever to switch and what knob to turn, you should be aware of some pitfalls using the public API.

The most obvious one that you will encounter frequently, is the usage of pointers and standard C-types. C is not as strict as Swift, when it comes to value assignments or the absence of valid values. Basically every value can and will be (mis-) interpreted somehow in C and this behavior is frequently observed in 3Dconnexion’s API. For example you find several integer constant definitions that use four one-byte characters as values, such as this switch:

Swift is not able to convert this kind of values into usable Int32 pendants and sadly Xcode does also not provide any warnings or errors. So you will probably not notice anything until you need this particular switch constant. Xcode’s auto completion simply doesn’t show „kConnexionCtlSetSwitch“ and it is also not available as known symbol inside Swift.

In the example above ‚3dss‘ is actually the for 8-bit representation of 862221171. The ConnexionClientControl function, where this value is used, expects an uint32_t type (or UInt32 in Swift). So to use the switch you could memorize 862221171 and write it manually… Or you can use two things I created to simplify life:

  1. I created a function that converts four-char values into UInt32 raw values.
  2. I migrated ConnexionClient.h into a Swift conform representation and did all the conversion for you, which results in ConnexionClient.swift that you can load from GitHub. As soon as you provide this file to your project you can access all constants without hassle. Please be so kind to leave my signature block intact.

But!

When you take a closer look at the ConnexionClient.swift file and compare it to the original ConnexionClient.h derivate you will possibly notice that there are some structs missing. These are the ConnexionDeviceState and the ConnexionDevicePrefs structs. It not like I haven’t tried to convert them. In fact I have and I tried long and hard to make it work, but once again it is the type-system as well as how Arrays are implemented in Swift that makes it impossible (for me, for now) to achieve a satisfying result. So I made the decision to discard both structs from the Swift-version and use the automatically converted structs. Since both these structs are working properly, it is no big deal. Sadly you have to provide the original ConnexionClient.h file still.

What happened exactly?

  1. Swift arranges Integer values in a way that leaves a gap of 16bit „here and there“. I tried to get an answer to this behavior at StackOverflow. You can read more about it over there – http://stackoverflow.com/questions/33424316/swift-converts-cs-uint64-t-different-than-it-uses-its-own-uint64-type/33428437#33428437 – if you like.
    Long story short: ConnexionDeviceState uses an uint64_t variable time to provide a timestamp, but the Swift pendant that is UInt64 was filled with nonsense. This was because between the preceding variable value and time is a mysterious 16 bit long gap. The timestamp value starts inside this gap but is not entirely accessible via the time variable. In fact „time“ only has the remaining 48 bits of the timestamp value set and its remaining 16 bit are not used for the timestamp, but they contain the values of the first two array filed of report, which consists of 8 uint8_t types. So these two fields are blended into the time variable. Therefore, every other of the remaining variables is shifted 16 bits further from the expected position. If you print every value you can clearly see that.
    One workaround is to sacrifice the time variable and introducing e.g. a tuple of two 32 bit fields. The gap vanishes and everything is aligned properly.
  2. Swift’s array type is a library class and it’s handled „by value“, therefor not accessible via a C pointer. This was just a small issue, because C’s arrays can be represented as tuples in Swift.

So having that said and introduced the ConnexionClient.swift we are good to go.

The next steps will show you how to start and stop the connection to your 3Dconnexion device and how to handle received data..


Connect your client to the driver

To receive any data from your device, you have to first register your application aka client to the data queue of the driver. You get an Id that you have to keep during the lifetime of your application, or at least until you close and clear the connection to the driver. Inside an OS X Cocoa application I found the best place to perform the following steps automatically is inside the viewDidAppear and viewDidDisappear methods of your main ViewController, that you have to override. But you are free to perform the following steps wherever you like – e.g. on button click. The needed C function definitions can be found inside ConnexionClientAPI.h header file.


Set ConnexionHandlers aka callback function pointers

First you have to tell the driver which callback functions it has to use, when broadcasting device-events like axis value changes, pressed button Ids, and configuration changes. The responsible function for this is:

As you can see it uses custom function types, also found in the ConnexionClientAPI.h file. If you look closer, you’ll notice that it calls for function pointers. As with Swift 2.1 there is an concept for this to bridge the gap between C and Swift, but sadly it does not work here. Swift uses the „convention(c)“ syntax to define a pointer to a function of a specific signature. ConnexionMessageHandlerProc becomes

which seems to be a legit type alias to use. However, you cannot use it as type for a raw Swift function, simply because Swift functions / methods are solely identifiable (aka typed) by their signature. But a function of an equivalent signature as ConnexionMessageHandlerProc is not identical to this „type“. Xcode tells you that your function with the above stated signature cannot be passed to the SetConnexionHandlers function, because you need a C function pointer and you can only form it by a reference from a ‚func‘. So following this hint you have to do something like

But this will not work, because your message handler function is not of the same type. Interestingly Xcode will, if you try this approach, move the same error message to the SetConnexionHandlers call.

The next idea would be to cast the function to the ConnexionMessageHandlerProc type. Because functions are also first-class citizens you would expect that it could be somehow possible. But its not, simply because Swift does not recognize the Signatures as equivalent. So you cannot cast your function’s reference to the needed type.

This is not the whole truth. There is a way to cast a function to a specific type in Swift: Closures.
Closures are simply anonymous functions, i.e. functions without an identifier name, but with an implementation body. Not having a name doesn’t mean that it does not exist in memory, so you have the function somewhere located at your heap. To access it you need what? Exactly, you need a pointer. And here is Swift’s inconsistency regarding pointers, because you can assign a Closure-definition to an identifier aka variable that can be casted, because it is a type unsafe pointer, fully compatible with C. That took me long to figure out.

Here is what you have to use, fill with life and toss to SetConnexionHandlers

As you can see, I used deviceId instead of productId, because I found it more intuitive. Actually, productId seems to be technically more appropriate, because this variable identifies a product type (SpaceMouse Pro, SpaceNavigator, etc.) and not your specific device. A particular device can be distinguish through its device Id and port address
So if you have e.g. two SpaceNavigators connected simultaneously, you will get the same Id regardless of which particular device is active (to distinguish them, you have to use the USB address).

Sadly 3Dconnexion is also not consequent at this point, because inside the ConnexionDevicePrefs structure they also use deviceID (with a capital D) and describe its meaning analogous inside the comments:

I think this was even the reason why I begun to use deviceId in the first place. Whatever…

… the main point is, that it causes no problems in regards of casting the closure.

The fourth parameter defines if you want to handle the incoming data inside a thread different from your main UI thread. This should be considered in a more rendering intensive application, but set this parameter to false for now. I haven’t worked with multithreading inside Swift, yet.


Register your Application aka Client

Next you have to register your application to the driver, so whenever it is in focus it will receive data from the previously provided callback function. Here is the C function responsible for this

RegisterConnexionClient needs four parameters. The first and second are redundant, so it is up to you which one you want to use. I recommend to use the first one called signature, which is a UInt32 value generated from a four character string that you have to provide at your project setting.

This string is located in the Custom OS X Application TargetProperties pane at the Bundle creator OS Type code field. If you are an OS X developer you will also know this field as the CFBundleSignature value (you can display it as such by right-clicking into the pane and selecting Show Raw Keys/Values). Type in any four character string that serves best as abbreviation for your application name.


Bildschirmfoto 2015-12-02 um 18.00.05

 

CFBundleSignature

To retrieve the value of this field you have to call

You can embed this into an if statement to ensure that appSignature is not nil when you pass it to RegisterConnexionClient, like this

You can see two things here:

  1. appSignature is a String but an UInt32 is expected
  2. I use a static helper function I wrote to convert the four character string into an UInt32.

As mentioned in Fiddling With C And Swift, a four character string is common in C, but not directly useable in Swift. So use this function to save yourself a lot of time. What this function does is basically to encode every Unicode character (Swift uses UTF8 encoding by default) to ASCII, using its representing UInt8 value, adding it to a UInt32 variable, and pushing it 8 bits up to make space for the next UInt8 bit-sequence.

The other, redundant way to identify your application would be by providing the application’s full name. Inside C this would be very easy, except figuring out what the „full name“ actually is, in order to pass a String to the function. A C-String is basically just a pointer to the first char inside a sequence in memory, and every following char is just an eight bit offset addition to the predecessor character. In Swift this is not the case and leads to many new questions that I don’t want to discuss here. Just use the previous approach and you’re good to go.

The third parameter is the method of how your application will interact with the driver and the device. You can chose from the two options (note that I am using my swifty ConnexionClient.swift file)

and

TakeOver will let you decide how to configure and use the device. Everything you want from your device, such as calibration and precision you have to set inside your application. Plugin on the other hand uses the driver’s preference that you can set inside the global OS X configuration panel.

Sadly both options currently cause some problems.

  • TakeOver sends data to the application, but seams to ignore most of the configuration commands. That’s why my example application is so limited at the moment.
  • Plugin does not send axis or button data at all – or at least I haven’t figured out how and where to grab the data (thanks to the very limited documentation – it’s more likely to find a needle inside a haystack).

So I stick with the TakeOver variant for now, because this way usable data will be sent from the device.

The last parameter requires a bit mask that you can find under ConnexionClient.Mask. You can concat different masks by using bitwise OR ( pipe character | ). For now, you can stick with ConnexionClient.Mask.All, which activates broadcasting of every event.

As return value of the RegisterConnexionClient function, you get a unique client ID if everything went fine. Save this Id, because you need it during the whole lifespan of your device interaction.

Here is the resulting code snipped in Swift. Please note that I encapsulated the whole process inside a function called start that is located inside a helper class. This class uses a singleton pattern that I often use while developing in Java for Android and it seams to work very good in Swift, too. Have a look at my entire code on GitHub if you want to see how it’s done.
I also use the weak linking of the framework, as discussed in Weak-linking the framework (C workaround), at this point right before I call the first framework’s function, in order to to avoid a runtime-crash.

The actual start function uses exception handling via throw. I left this out of this example – for simplicity.

There are two more lines of interest

This is the first time you configure your device. In this particular case we say that we want to receive events from all buttons pressed and from all axes. The clientId is necessary, to tell the driver who is requesting this behavior. If you have several applications running simultaneously, using this framework, every application can have a different setting configured.

Maybe you ask yourself why I am casting ConnexionClient.Mask.AllButtons to UInt32. This is because 3Dconnexion has implemented this field a an uint64_t type, but their function asks for a 32bit unsigned integer. Basically the value of ConnexionClient.Mask.AllButtons  is zero, but its representation is 0xFFFFFFFF. I have no clue what they where thinking here. Every other button mask is also 64bit. Weird!


Stop driver and perform cleanup

Before digging into data-usage, I show you how to properly shutdown the driver, in order to avoid any memory leakage or unforeseen consequences.

It is a simple process. Just call the following functions in the given order:


Data usage

During the client registration process, three callback functions were defined in the form of named closures and passed to SetConnexionHandlers. Whenever your device has something to tell to your application, it will be done by one of those callbacks. We already have everything set to use the data passed to the callbacks and only need to implement the data handling.


Message Handler Callback

Values send through the Message Handler callbackThis closure implements the ConnexionMessageHandlerProc type and handles every event produced by the device’s buttons and axes. So this is where your application gets its interaction data.

Here you see a code skeleton that classifies every event. Making sure that it is a DeviceState message, the msgArgPtr pointer can be safely casted to the ConnexionDeviceState struct and its data can be made available via the memory property of the UnsafeMutablePointer container class.

Furthermore, PrefsChanged and CalibrationDevice messages are intercepted. I haven’t done much with this message types, yet. Perhaps I will write another short tutorial on how to use them, after testing them productively for some use-cases.

 


Added Handler Callback

The second callback closure is used to identify if a device was connected to your machine. If so, this callback is called and the deviceId (or productId if you prefer 3Dconnexion’s taxonomy) is passed to you. As mentioned before, the deviceId specifies a device type, not an individual device. How to identify a individual device, I will cover in a later short tutorial.


Removed Handler Callback

The third callback is the counterpart of the added handler callback. It informs you whenever a device was unplugged from the machine.


ConnexionDeviceState: Get axis and button values

As mentioned in the Fiddling with C and Swift section, it is currently not possible to provide you a clean and native Swift conversion of the C struct. But Swift’s internal mechanism is smart enough to give you a Swift way of accessing the properties. So if you want to access, for example, the device‘ address and you use state, as shown in the Message Handler Callback section, you do it with the dot-operator, like

that will give you the USB address at which your device is registered. You saw another example of this in the code skeleton for the Message Handler Callback, where I accessed the command property to distinguish between axis and button events.

Most interesting at this point is how to handle the axis array (I will leave out the report array here, because it provides very unintuitive data, that I don’t understand by that time).

The C array is automatically converted to a Swift tuple. Therefore, no advance operations of the Swift array implementation can be used. Nevertheless, tuples are great performance-wise and you can access every element in Swift with the dot-operator and the element’s index. However, you can put it without the index directly into a print command to have it pretty printed to the terminal.

Axis tuple

Here is the meaning of the axis array elements.

Button bit-mask

Every button on your device is associated with an Id. However, an Id doesn’t come in its own variable or inside an array element.

The documentation on this made me wonder for a moment

Reports which buttons, if any, are pressed. Buttons are assigned 2n values, i.e. 1, 2, 4, 8, 16, and so on. The buttons field is a sum of all the values being pressed. For example, if button 1 and button 2 are both pressed, the buttons field will be 3. When all buttons are released, the buttons field will be 0.

It really couldn’t be described more complicated!

So lets be more specific: Every active Id is encoded as an one into the bit-mask buttons.

So to access the particular button Ids of an active button you have to cancel out all other one bits and read the integer value of the remaining one bit on its position inside buttons.

For my sample application I wrote these little helper functions to do the dirty work for you:

The IsButtonActive function checks if a given button Id exists inside the provided button bitmask, and returns true if so, and false otherwise. BitMaskOne is just a constant with value 0x0001.
Because every Id is a value in the power of two, the ordinal representation of the Id is its position in the bit-mask. So you can bitwise-shift every bit by the ordinal value of the Id of interest until its bit is at the least significant bit (LSB) position. Now you can bitwise-and the bit-mask with the BitMaskOne value canceling out every other bit and only leaving the LSB in its original state: One if it was already one, and zero if it was already zero. Check this value again against BitMaskOne (i.e. value 1) and you get your boolean result.

You can embed this function into a small for-each loop and get truth-values for all possible 32 buttons

The second function converts the bit-mask into a Swift array, so that you can access the buttons by their ordinal Id and lookup if the activity status is true or false

It performs the same comparison against BitMaskOne, in order to check if the LSB, which corresponds to the current button Id of interest, is active (one after bitwise and) and set the corresponding position at the resulting array to true or leave it at false (default value of the array elements ofter initialization, set with repeatedValue: false). Then the hole bit-mask is bitwise shifted towards the LSB, putting the next button Id position at the LSB.

If you know even more efficient and elegant ways to handle a bit-mask like this, please let me know in the comments below.

 


ToolShelf-4-3Dconnexion App

3DconnexionLogoIn addition to the GitHub repository and the free source code you can also download my application on the App Store.

I am proud to introduce my first AppStore App called ToolShelf-4-3Dconnexion that implements all the topics discussed in this guide. I will extend the application and will write more articles about the features, as well as update the source code frequently. If you have any suggestions on how to improve the application, or want something to be implemented differently please let me know. Please also rate and leave a comment on the AppStore, but please don’t be too mean! 🙂

AppleAppStoreBadge


Conclusion and Outlook

This was a long piece of text, wasn’t it? It took me some while to figure a few things out regarding Swift and C and it took me even longer to write it down. But it was a fun and educational ride that made me really like Swift. Hopefully this guide can help you get your implementation working much faster than it took me having only sparse information and a not very satisfying official documentation.

Swift was made open source just two days before the release of this guide, so I expect that some of the information here will be obsolete in a few weeks or months. But that’s a good thing and will hopefully bring this very elegant and powerful language further.

Please leave comments on this guide, report mistakes and correct me in terms of my bloody bad English. 🙂

See you!

PS: special thanks go to Andreas Braun for reviewing this article – excluding this line here! 😛

Ein Gedanke zu “How-To – Use a 3Dconnexion device inside an OS X application written in Swift

  1. Pingback: Short Note: Embedding a private framework within your OS X app bundle – MartinMajewski.net

Schreibe einen Kommentar

Deine E-Mail-Adresse wird nicht veröffentlicht. Erforderliche Felder sind mit * markiert.