Awesome Machine Learning DEMOs with iOS
We tackle the challenge of using machine learning models on iOS via Core ML and ML Kit (TensorFlow Lite).
Contents
- Machine Learning Framework for iOS
- Baseline Projects
- Application Projects
- Create ML Projects
- Performance
- See also
Machine Learning Framework for iOS
- Core ML
- ML Kit
- fritz
- etc. (Tensorflow Lite,
Tensorflow MobileDEPRECATED
)
Flow of Model When Using Core ML
The overall flow is very similar for most ML frameworks. Each framework has its own compatible model format. We need to take the model created in TensorFlow and convert it into the appropriate format, for each mobile ML framework.
Once the compatible model is prepared, you can run the inference using the ML framework. Note that you must perform pre/postprocessing manually.
If you want more explanation, check this slide(Korean).
Flow of Model When Using Create ML
Baseline Projects
DONE
-
Using built-in model with Core ML
-
Using built-in on-device model with ML Kit
-
Using custom model for Vision with Core ML and ML Kit
-
Object Detection with Core ML
TODO
- Object Detection with ML Kit
- Using built-in cloud model on ML Kit
- Landmark recognition
- Using custom model for NLP with Core ML and ML Kit
- Using custom model for Audio with Core ML and ML Kit
- Audio recognition
- Speech recognition
- TTS
Image Classification
Name | DEMO | Note |
---|---|---|
ImageClassification-CoreML | - | |
MobileNet-MLKit | - |
Object Detection & Recognition
Name | DEMO | Note |
---|---|---|
ObjectDetection-CoreML | - | |
TextDetection-CoreML | - | |
TextRecognition-MLKit | - | |
FaceDetection-MLKit | - |
Image Estimation
Name | DEMO | Note |
---|---|---|
PoseEstimation-CoreML | - | |
PoseEstimation-MLKit | - | |
FingertipEstimation-CoreML | - | |
DepthPrediction-CoreML | - |
Image Segmentation
Name | DEMO | Note |
---|---|---|
ImageSegmentation-CoreML | - |
Application Projects
Name | DEMO | Note |
---|---|---|
dont-be-turtle-ios | - | |
WordRecognition-CoreML-MLKit(preparing...) | Detect character, find a word what I point and then recognize the word using Core ML and ML Kit. |
Annotation Tool
Name | DEMO | Note |
---|---|---|
KeypointAnnotation | Annotation tool for own custom estimation dataset |
Create ML Projects
Name | Create ML DEMO | Core ML DEMO | Note |
---|---|---|---|
SimpleClassification-CreateML-CoreML | ![]() |
A Simple Classification Using Create ML and Core ML |
Performance
Execution Time: Inference Time + Postprocessing Time
(with iPhone X) | Inference Time(ms) | Execution Time(ms) | FPS |
---|---|---|---|
ImageClassification-CoreML | 40 | 40 | 23 |
MobileNet-MLKit | 120 | 130 | 6 |
ObjectDetection-CoreML | 100 ~ 120 | 110 ~ 130 | 5 |
TextDetection-CoreML | 12 | 13 | 30(max) |
TextRecognition-MLKit | 35~200 | 40~200 | 5~20 |
PoseEstimation-CoreML | 51 | 65 | 14 |
PoseEstimation-MLKit | 200 | 217 | 3 |
DepthPrediction-CoreML | 624 | 640 | 1 |
ImageSegmentation-CoreML | 178 | 509 | 1 |
WordRecognition-CoreML-MLKit | 23 | 30 | 14 |
FaceDetection-MLKit | - | - | - |
📏Measure module
You can see the measured latency time for inference or execution and FPS on the top of the screen.
If you have more elegant method for measuring the performance, suggest on issue!
Implements
Measure📏 | Unit Test | Bunch Test | |
---|---|---|---|
ImageClassification-CoreML | O | X | X |
MobileNet-MLKit | O | X | X |
ObjectDetection-CoreML | O | O | X |
TextDetection-CoreML | O | X | X |
TextRecognition-MLKit | O | X | X |
PoseEstimation-CoreML | O | O | X |
PoseEstimation-MLKit | O | X | X |
DepthPrediction-CoreML | O | X | X |
ImageSegmentation-CoreML | O | X | X |
See also
- Core ML | Apple Developer Documentation
- Machine Learning - Apple Developer
- ML Kit - Firebase
- Apple's Core ML 2 vs. Google's ML Kit: What's the difference?
- iOS에서 머신러닝 슬라이드 자료
- MoT Labs Blog
WWDC
Core ML
- WWDC2019
- WWDC2019 256 Session - Advances in Speech Recognition
- WWDC2019 704 Session - Core ML 3 Framework
- WWDC2019 228 Session - Creating Great Apps Using Core ML and ARKit
- WWDC2019 232 Session - Advances in Natural Language Framework
- WWDC2019 222 Session - Understanding Images in Vision Framework
- WWDC2019 234 Session - Text Recognition in Vision Framework
- WWDC2018
- WWDC2017
Create ML and Turi Create
- WWDC2019
- WWDC2019 424 Session - Training Object Detection Models in Create ML
- WWDC2019 426 Session - Building Activity Classification Models in Create ML
- WWDC2019 420 Session - Drawing Classification and One-Shot Object Detection in Turi Create
- WWDC2019 425 Session - Training Sound Classification Models in Create ML
- WWDC2019 428 Session - Training Text Classifiers in Create ML
- WWDC2019 427 Session - Training Recommendation Models in Create ML
- WWDC2019 430 Session - Introducing the Create ML App
- WWDC2018
Common ML
- WWDC2019
- WWDC2018
- WWDC2016