ProgressiveNet

Official implementation of the paper "Progressive Transmission and Inference of Deep Learning Models", to appear at 20th IEEE International Conference on Machine Learning and Applications (ICMLA 2021).

[arXiv] [demo]

Youngsoo Lee¹, Sangdoo Yun², Yeonghun Kim¹ Sunghee Choi¹

¹ School of Computing, Korea Advanced Institute of Science and Technology(KAIST)
² AI LAB, NAVER Corp.

Abstract

Modern image files are usually progressively transmitted and provide a preview before downloading the entire image for improved user experience to cope with a slow network connection. In this paper, with a similar goal, we propose a progressive transmission framework for deep learning models, especially to deal with the scenario where pre-trained deep learning models are transmitted from servers and executed at user devices (e.g., web browser or mobile). Our progressive transmission allows inferring approximate models in the middle of file delivery, and quickly provide an acceptable intermediate outputs. On the server-side, a deep learning model is divided and progressively transmitted to the user devices. Then, the divided pieces are progressively concatenated to construct approximate models on user devices. Experiments show that our method is computationally efficient without increasing the model size and total transmission time while preserving the model accuracy. We further demonstrate that our method can improve the user experience by providing the approximate models especially in a slow connection.

Examples and Demo

Image classification Model (MobileNetV2):

Object Detection Model (SSD-MobileNetV2):

Demo: You can try using our pre-built models in https://prev.github.io/progressivenet

Getting Started

ProgressiveNet is implemented based on TensorFlowJS, which is an open-source hardware-accelerated JavaScript library for training and deploying machine learning models. Our implementation interacts with the TFJS core engine, by passing the intermediate weight matrices to the TFJS and receives the approximiate results in the middle of the transmission.

How to install

$ npm install progressivenet

How to use (Classification Model)

import { ProgressiveNet, Classifier } from 'progressivenet';

const pgNet = new ProgressiveNet({
    modelUrl: '<model_url>',
    numProgressSteps: 8,
});
const img = document.getElementById('my-img');

pgNet.loadSequentially((model, isLast, progressStep) => {
    const classifier = new Classifier(model);
    const predictions = await classifier.classify(img);
    console.log(`Result from step #${progressStep}: ${predictions}`);

}).then() => {
    console.log('All models are transmitted and inferenced');
}

How to use (Detection Model)

import { ProgressiveNet, ObjectDetector } from 'progressivenet';

const pgNet = new ProgressiveNet({ modelUrl: '<model_url>' });
const img = document.getElementById('my-img');

pgNet.loadSequentially((model, isLast, progressStep) => {
    const detector = new ObjectDetector(model);
    const predictions = await detector.detect(img);
    console.log(`Result from step #${progressStep}: ${predictions}`);

}).then() => {
    console.log('All models are transmitted and inferenced');
}

You can see the detailed code in demo. It is written in TypeScript and implemented with jQuery.

Models

ProgressiveNet requires a dedicated model file for the progressive execution (inference).
We provide MobileNetV2 and SSD-MobileNetV2 models in the Release.

Generating custom model

Rather than implementing a training code for ProgressiveNet, we provide a converter for generating progressive model from static TensorFlowJS model.

Towards making your own model beyond our pre-converted models, try visiting converter directory for advanced usage. Converter is implemented with TypeScript and and we provide command line interface(CLI) to convert TFJS models into progressive models.

If you want to apply ProgressiveNet into your own service with your own models, you can follow the instructions below:

Writing the training code with TensorFlow.
Train the model and save it in a TF format (either SavedModel or Keras).
Convert your model into a TFJS model by following the guide.
Convert the TFJS model into Progressive model by following the instruction on our repo.

License

CC 4.0 Attribution-NonCommercial International. The software is for educaitonal and academic research purpose only.

Prev / progressivenet