Apple announced a new machine learning framework called CoreML during WWDC 2017. It allows developers to import custom machine learning models to their application, and run them locally and safely on the device through an easy-to-use API.

With CoreML, you can now very easily provide features powered by machine learning algorithms with minimal effort. In fact, no specific machine learning knowledge is necessary if you are able to get access to a model. Once you import the model into your application, CoreML will abstract all the algorithms and logic behind the model, and provide you with a public API that you can consume from your application.

Apple’s goal is to make machine learning integration and utilization trivial, and let developers focus on user experience.

To facilitate even more the integration of models, Apple created a Python tool — Core ML Tools — that can convert models from the most popular machine learning platforms, and open CoreML to developers who want to use custom models. Since the converter tool is open source, we can expect more platforms to be supported in the future. More details on the Apple website here.

Prototype

This prototype requires Xcode 9 and iOS 11 if you want to run the app on your device.

I put together a simple application using CoreML and a ready-to-use model provided by Apple on their website for image recognition. It was really easy to import the model; and with a few lines of code I was able to display the image’s dominant object category name.

Import The Model

After you created your project, the first step for using CoreML in your application is to import the model. You can find ready-to-go models on the Apple machine learning page here. After you download it — I used ResNet50 for this example — drag and drop the file inside your Xcode project.

When you click on the model file, you’ll see details about the model, like the type, size, and more importantly the inputs and outputs. ResNet50 input requires you to provide an image file with a size of 224x224px. The output will return the most likely image category (dominant object type), as well as a dictionary of probability for each category.

Xcode generates automatically the Swift code that you will be able to use from your application in the background, and let you know when it’s available.

Use The Model

Once the model file is embedded in the project, open your view controller file, and create your model:

let model = Resnet50()

Write the function that will take an image (ResNet50 requires a CVPixelBuffer type for the image), and call the prediction function on the model and pass the image:

func objectName(forImage image: CVPixelBuffer) throws -> String {
    let output = try model.prediction(image: image)
    return output.classLabel
}

Now you can call this function and start using the model output:

do {
    let name = try objectName(forImage: imagePixelBuffer)
    print(name)
} catch {
    print("Could not recognize this image... 😔")
}

The full prototype is available here.

Conclusion

You saw how easy it was to import a machine learning model and start using it right away. Apple did a great job abstracting all the complex logic and providing a comprehensive API to the developers.

It is too soon yet to know if this approach will allow developers to implement great features powered by machine learning, or if this simplicity will be limiting.

I am personally very excited about CoreML and all the possibilities. With already a great amount of open source models available to everybody, I am confident that developers will pick up on machine learning on Apple platforms to enrich their user experience.