Increased App Size

Question

Increased App Size

lks-nbg opened this issue 3 years ago · comments

After implementing google_ml_kit, the app size is increased by ~85 MB in debug mode and ~44 MB in release mode. I investigated this with the app size tool and found that all models/resources are always included, even if they are not used. So it would be great to be able to configure which resources are used and and which can be excluded.

The firebase_ml_vision plugin had the same issue (firebase/flutterfire#4767) and I was able to adapt the workaround of excluding unused resources to google_ml_kit and wanted to share it. In my case I am currently only using the language detection feature therefore I don't need pose detection, etc.

The workaround for Android looks like that and increases the app size by less than 3 MB:

android {

    // some other code 

    buildTypes {
        release {
            // some other code 

            aaptOptions {
                ignoreAssetsPattern '!mlkit_pose:!mlkit_label_default_model:'
            }
        }
        debug {
            // some other code 

            aaptOptions {
                ignoreAssetsPattern '!mlkit_pose:!mlkit_label_default_model:'
            }
        }
    }

    packagingOptions {
        exclude 'lib/**/libtranslate_jni.so'
        exclude 'lib/**/libdigitalink.so'
        exclude 'lib/**/libxeno_native.so'
        exclude 'lib/**/libmlkitcommonpipeline.so'
        exclude 'lib/**/libbarhopper_v2.so'
        exclude 'lib/**/libclassifier_jni.so'
        exclude 'lib/**/libface_detector_v2_jni.so'
        exclude 'lib/**/libtensorflowlite_jni.so'
        // exclude 'lib/**/liblanguage_id_jni.so' → required for language detection
    }

Since a workaround does exist, this is not a pressing issue, but I think many developers would appreciate some sort of configuration option :)

Bharat Biradar · Answer 1 · Mon May 24 2021 01:36:51 GMT+0800 (China Standard Time)

@lks-nbg Thanks for pointing it out. I'll add this in the docs.

Adrian Casabuena Sáez · Answer 2 · Tue May 25 2021 07:12:07 GMT+0800 (China Standard Time)

Hi @lks-nbg
Which one should I comment to use image label?

Bharat Biradar · Answer 3 · Tue May 25 2021 11:38:12 GMT+0800 (China Standard Time)

@adriancsbna libclassifier_jni

Adrian Casabuena Sáez · Answer 4 · Wed May 26 2021 00:50:31 GMT+0800 (China Standard Time)

Hello @bharat-biradar.

I have applied it but the size of the App has not decreased.
What should I add in ignoreAssetsPattern?

Malay Agrawal · Answer 5 · Wed May 26 2021 04:39:09 GMT+0800 (China Standard Time)

Yep, this did work it reduced my app-release.apk size from 323MB to 86MB but can I reduce it even more? because I only want to use OCR (image text reader).

Bharat Biradar · Answer 6 · Wed May 26 2021 14:20:09 GMT+0800 (China Standard Time)

@adriancsbna Could give me the android part of build.gradle that you used in your app. The same section mentioned by iks-ngb

Bharat Biradar · Answer 7 · Wed May 26 2021 14:27:44 GMT+0800 (China Standard Time)

@MalayAgrawal Maybe it can be reduced. I looked at the size of our example app using devtools and I found this.

You can exclude models by mentioning it in ignoreAssetsPattern but I'm not sure which among these models are used for OCR.

First try by excluding whole model folder and then individually for each older. Let me know if it works.

Adrian Casabuena Sáez · Answer 8 · Thu May 27 2021 03:34:25 GMT+0800 (China Standard Time)

Hi @bharat-biradar here you have it

I managed to reduce the size by adding ignoreAssetsPattern '!mlkit_pose:' but I'm sure I can add more to lower the size further. Do you know what options there are to add there?

buildTypes {
release {
signingConfig signingConfigs.debug
aaptOptions {
ignoreAssetsPattern '!mlkit_pose:'
}
}
debug {
signingConfig signingConfigs.debug
aaptOptions {
ignoreAssetsPattern '!mlkit_pose:'
}
}
}

packagingOptions {
    exclude 'lib/**/libtranslate_jni.so'
    exclude 'lib/**/libdigitalink.so'
    exclude 'lib/**/libxeno_native.so'
    //exclude 'lib/**/libmlkitcommonpipeline.so'
    exclude 'lib/**/libbarhopper_v2.so'
    //exclude 'lib/**/libclassifier_jni.so'
    exclude 'lib/**/libface_detector_v2_jni.so'
    exclude 'lib/**/libtensorflowlite_jni.so'
    exclude 'lib/**/liblanguage_id_jni.so'
}

Malay Agrawal · Answer 9 · Thu May 27 2021 03:40:36 GMT+0800 (China Standard Time)

buildTypes {
       release {
           signingConfig signingConfigs.release
            aaptOptions {
                ignoreAssetsPattern "mlkit_pose"
            }
       }
   }
   packagingOptions {
        exclude 'lib/**/libtranslate_jni.so'
        exclude 'lib/**/libdigitalink.so'
        exclude 'lib/**/libxeno_native.so'
        exclude 'lib/**/libmlkitcommonpipeline.so'
        exclude 'lib/**/libbarhopper_v2.so'
        exclude 'lib/**/libclassifier_jni.so'
        exclude 'lib/**/libface_detector_v2_jni.so'
        exclude 'lib/**/libtensorflowlite_jni.so'
        exclude 'lib/**/liblanguage_id_jni.so'
    }

OCR is working great and app size reduced to 30MB approx

Bharat Biradar · Answer 10 · Thu May 27 2021 12:10:33 GMT+0800 (China Standard Time)

@adriancsbna Try adding models along with mlkit_pose

Tushar Patel · Answer 11 · Thu May 27 2021 12:12:18 GMT+0800 (China Standard Time)

since I'm using this package for scanning barcode to get results which unnecessary lib and models i need to excluded?
Thank you in advance : )

Bharat Biradar · Answer 12 · Thu May 27 2021 17:40:10 GMT+0800 (China Standard Time)

@TusharPatel18 I think you should comment out common pipeline and libbarh... I'm not sure I'm guessing as it looks similar to barcode, in in ignore assets add mlkit_pse and image label. Scroll up to see this implementation

Alex Hartford · Answer 13 · Thu May 27 2021 23:20:43 GMT+0800 (China Standard Time)

since I'm using this package for scanning barcode to get results which unnecessary lib and models i need to excluded?
Thank you in advance : )

One of my apps only uses it for barcode scanning, the following is how I have it configured:

Click to expand

packagingOptions {
    exclude 'lib/**/libtranslate_jni.so'
    exclude 'lib/**/libdigitalink.so'
    exclude 'lib/**/libxeno_native.so'
    exclude 'lib/**/libmlkitcommonpipeline.so'
    // exclude 'lib/**/libbarhopper_v2.so' → required for barcode detection
    exclude 'lib/**/libclassifier_jni.so'
    exclude 'lib/**/libface_detector_v2_jni.so'
    exclude 'lib/**/libtensorflowlite_jni.so'
    exclude 'lib/**/liblanguage_id_jni.so'
}

Alex Hartford · Answer 14 · Thu May 27 2021 23:23:37 GMT+0800 (China Standard Time)

I'm not super familiar with the iOS side of things, is there an equivalent solution for iOS? Or is it necessary at all?

Francisco Bernal · Answer 15 · Fri May 28 2021 00:05:33 GMT+0800 (China Standard Time)

not for now, 3rd party dependencies are managed in Xcode using Cocoapods or Swift Package Manager, ours uses Cocoapods. Eventually I will work on a way to only add the scanners you need, but for now in iOS the bundle comes with all.

Ashwin Ramakrishnan · Answer 16 · Sun May 30 2021 00:46:34 GMT+0800 (China Standard Time)

I am working on an app to use face detection only. Is there any more stuff I can add to the ignoreAssetsPattern under aaptOptions to use only face recognition?

My current release app size is 58MB. The app is basically the example app from this repo with everything removed except the face recognition parts.

This is my current aaptOptions

aaptOptions {
                ignoreAssetsPattern '!mlkit_pose:!mlkit_label_default_model:'
            }

Bharat Biradar · Answer 17 · Sun May 30 2021 01:05:32 GMT+0800 (China Standard Time)

@ashwinkey04 Analyze the app size through dev tools and look at the instances under assets section. Try removing those which seem unnecessary. An example screenshot can be obtained above.

Ashwin Ramakrishnan · Answer 18 · Sun May 30 2021 12:52:45 GMT+0800 (China Standard Time)

Thanks a lot @bharat-biradar. I removed a few models without breaking the app and managed to bring the size down to 35 MB for the bulk release apk and 23 MB for v8a. Yet to try and remove more assets as I am expected to make something less than 10 MB. Thanks for the time :)

Bhanuka Isuru · Answer 19 · Tue Jun 01 2021 20:15:40 GMT+0800 (China Standard Time)

@adriancsbna Thanks. how to reduce app size in IOS?

tokyokamera · Answer 20 · Mon Jun 07 2021 12:23:40 GMT+0800 (China Standard Time)

I want to use Japanese Text recognition only. What should I do?

wazini-john · Answer 21 · Thu Jul 01 2021 16:55:33 GMT+0800 (China Standard Time)

@ashwinkey04 What did you exclude for only facedetection ?

wazini-john · Answer 22 · Thu Jul 01 2021 17:03:00 GMT+0800 (China Standard Time)

I'm only using facedetector
My gradle file looks like this

buildTypes {
		release {
			// TODO: Add your own signing config for the release build.
			// Signing with the debug keys for now, so `flutter run --release` works.
			signingConfig signingConfigs.debug
			aaptOptions {
				ignoreAssetsPattern 'mlkit_pose'
				ignoreAssetsPattern '!mlkit_pose:!mlkit_label_default_model:'
				ignoreAssetsPattern '!mlkit_pose:'
			}
		}
		debug {
			aaptOptions {
				ignoreAssetsPattern 'mlkit_pose'
				ignoreAssetsPattern '!mlkit_pose:!mlkit_label_default_model:'
				ignoreAssetsPattern '!mlkit_pose:'
			}
		}
	}
	packagingOptions {
		exclude 'lib/**/libtranslate_jni.so'
		exclude 'lib/**/libdigitalink.so'
		exclude 'lib/**/libxeno_native.so'
		exclude 'lib/**/libmlkitcommonpipeline.so'
		exclude 'lib/**/libbarhopper_v2.so'
		exclude 'lib/**/libclassifier_jni.so'
		exclude 'lib/**/libpredictor_jni.so'
		exclude 'lib/**/libtextclassifier3_jni_tclib.so'
		// exclude 'lib/**/libface_detector_v2_jni.so'
		exclude 'lib/**/libtensorflowlite_jni.so'
		exclude 'lib/**/liblanguage_id_jni.so'
	}

But the size is still huge.. especially the hobbes.tflite,

What else do I exclude ?

Bharat Biradar · Answer 23 · Thu Jul 01 2021 19:58:24 GMT+0800 (China Standard Time)

@wazini-john Try excluding hobbes.tflite it should not be related to face detection IG.

Deepak Lohmod · Answer 24 · Thu Jul 01 2021 19:59:34 GMT+0800 (China Standard Time)

How can i remove hobbes.tflite, I also need only for face detection

Bharat Biradar · Answer 25 · Thu Jul 01 2021 20:08:12 GMT+0800 (China Standard Time)

@deepaklohmod6789 Mention hobbes.tflite in ignoreAssetsPattern along with any other assets you want to ignore. Refer to the example above.

Deepak Lohmod · Answer 26 · Thu Jul 01 2021 20:09:10 GMT+0800 (China Standard Time)

aaptOptions {
                ignoreAssetsPattern 'mlkit_pose'
                ignoreAssetsPattern 'hobbes.tflite'
                ignoreAssetsPattern '!mlkit_pose:!mlkit_label_default_model:'
                ignoreAssetsPattern '!mlkit_pose:'
            }

I did this but still the size is 86.6 mb

Dennis Chege · Answer 27 · Thu Jul 01 2021 21:54:14 GMT+0800 (China Standard Time)

@deepaklohmod6789 run flutter clean then flutter pub get...
combine all options to one string ignoreAssetsPattern '!mlkit_pose:!mlkit_label_default_model:!hobbes.tflite:'

Deepak Lohmod · Answer 28 · Thu Jul 01 2021 23:03:59 GMT+0800 (China Standard Time)

@cngeru thanks a lot, it worked : )

Patrice Müller · Answer 29 · Tue Jul 06 2021 15:44:06 GMT+0800 (China Standard Time)

Is there any technical reason that is holding us back from splitting this plugin into multiple packages? I don't see one use case where I need all the ml-models at once. Or do I oversee something?

Satish · Answer 30 · Tue Jul 06 2021 20:39:01 GMT+0800 (China Standard Time)

@swissonid i completely agree. I think would add more value if we can split the models into different module like - TextRecognise, BarCode Reader etc...

Dennis Chege · Answer 31 · Tue Jul 06 2021 21:44:03 GMT+0800 (China Standard Time)

@SatishKumarRaizada and @swissonid Someone opened an issue regarding this issue here

Francisco Bernal · Answer 32 · Wed Jul 07 2021 00:15:30 GMT+0800 (China Standard Time)

@swissonid, @SatishKumarRaizada: collaboration is always welcome, feel free to send a PR with what you are proposing

Bharat Biradar · Answer 33 · Wed Jul 07 2021 00:43:50 GMT+0800 (China Standard Time)

@swissonid I am checking if we can work with unbundled models instead of model bundels. If it works I think size won't be an issue then.

Patrice Müller · Answer 34 · Wed Jul 07 2021 19:28:27 GMT+0800 (China Standard Time)

@fbernaly I spend yesterday on smaller lib (google_ml_vision )because that the one we use right now. Anyway, the problem is the same. And with a "simple" PR it isn't done. If the vision of this Plugin / Lib is to split it into smaller pieces, that will a fundamental change in the architecture. And gonna take its time.
This is a sketched possible solution. Not Yet finished.

Francisco Bernal · Answer 35 · Thu Jul 08 2021 00:50:51 GMT+0800 (China Standard Time)

Thanks @swissonid : I will look at it. Splitting this plugin into multiple packages will take some time. First we need to have parity between iOS and Android, because right now we have some features implemented in Android but not in iOS.

NightCoder · Answer 36 · Fri Aug 20 2021 04:36:53 GMT+0800 (China Standard Time)

Any ideas to reduce AppSize in iOS. I am only using OCR and app size after including this plugin increases by ~50MB.

Surajsahani28 · Answer 37 · Thu Sep 16 2021 22:24:17 GMT+0800 (China Standard Time)

How we reduce app size in IOS ?

NightCoder · Answer 38 · Fri Sep 17 2021 00:47:21 GMT+0800 (China Standard Time)

@Surajsahani28 I couldn’t find a way to reduce the app size with this plugin..I ended using the cloud ocr ..

hesmnyn · Answer 39 · Wed Sep 22 2021 16:15:40 GMT+0800 (China Standard Time)

not for now, 3rd party dependencies are managed in Xcode using Cocoapods or Swift Package Manager, ours uses Cocoapods. Eventually I will work on a way to only add the scanners you need, but for now in iOS the bundle comes with all.

any update?
@fbernaly

Pablo Lavín · Answer 40 · Fri Oct 01 2021 05:58:52 GMT+0800 (China Standard Time)

If it's not too much to ask for, I would love if anybody can list what
ignoreAssetsPattern
and
packagingoptions

refer to each of the ML Kit assets, or share a link of any docs that talk about that.

Sid Jha · Answer 41 · Fri Oct 08 2021 23:31:06 GMT+0800 (China Standard Time)

For anybody looking to seriously reduce the size, I would suggest going to the plugin implementation.
Comment unnecessary files in the build.grade. Then comment all the files src>nlp / src>vision based on what you need. This is a hit and trial method and you will get few errors just keep commenting down things whose implementation you have commented and in the end, it will compile. In my case, I only needed the translator plugin. It is easier than going through apk size tool again and again.

dependencies {
    // implementation 'com.google.android.gms:play-services-mlkit-barcode-scanning:16.2.1'

    // implementation 'com.google.android.gms:play-services-mlkit-image-labeling:16.0.5'
    // implementation 'com.google.mlkit:image-labeling-custom:16.3.1'
    // implementation 'com.google.mlkit:image-labeling-automl:16.2.1'

    // implementation 'com.google.android.gms:play-services-mlkit-text-recognition:17.0.0'
    // implementation 'com.google.mlkit:language-id:17.0.0'
    // implementation 'com.google.mlkit:digital-ink-recognition:17.0.0'

    // implementation 'com.google.mlkit:pose-detection:17.0.1-beta5'
    // implementation 'com.google.mlkit:pose-detection-accurate:17.0.1-beta5'

     implementation 'com.google.mlkit:translate:16.1.2'
    // implementation 'com.google.mlkit:entity-extraction:16.0.0-beta2'
    // implementation 'com.google.mlkit:smart-reply:16.2.0'

    // implementation 'com.google.mlkit:object-detection:16.2.6'
    // implementation 'com.google.mlkit:object-detection-custom:16.3.3'
    // implementation 'com.google.mlkit:linkfirebase:16.1.1'

    // implementation 'com.google.android.gms:play-services-mlkit-face-detection:16.2.0'
}

ljmatan · Answer 42 · Sun Oct 24 2021 22:01:12 GMT+0800 (China Standard Time)

If anyone is interested in barcode / QR scanning only, you can check out this fork.

Please note: it's not reflective of how you should do native programming properly, but it works with a previous barcode scanner implementation in Flutter.

Panmari · Answer 43 · Wed Oct 27 2021 04:36:29 GMT+0800 (China Standard Time)

Looks like the most fool-proof way to decrease size is to

Create a fork
Do what #26 (comment) suggests and remove all dependencies you don't need
Depend on your fork.

@bharat-biradar do you have a suggestion on how to get this behavior implemented upstream?

Bharat Biradar · Answer 44 · Thu Oct 28 2021 21:07:50 GMT+0800 (China Standard Time)

@panmari A way to use from your fork would be to

Fork the repo
Comment out the API's you don't want to use from the build.gradle (dependencies corresponding to particular API's can be easily identified from the name)
At this stage there should be errors showing in some classes delete them then comment out the their usage instance in MlKitMethodCallHandler.java over here
And this should be done by this point

I have made an example in the barcode_only branch. Hope this helps.

Nils Seip · Answer 45 · Fri Oct 29 2021 17:35:37 GMT+0800 (China Standard Time)

@bharat-biradar Thank you for the idea.

I created a fork for

text identification
language identification

For the iOS side: wouldn't conditional compilation help ? That is the same as deleting the not needed files. But then we still need a solution for the CocoaPods podspec so that only the necessary dependencies are pulled by CocoaPods.

Bharat Biradar · Answer 46 · Mon Nov 01 2021 00:46:21 GMT+0800 (China Standard Time)

@nachtmaar Sorry but I don't know about Ios side of things but please try and if it works, please mention it here.

David Serrano Canales · Answer 47 · Sat Nov 13 2021 01:29:12 GMT+0800 (China Standard Time)

Thanks @nachtmaar for the effort of creating the fork, that was exactly what I was looking for.

Ankush Das · Answer 48 · Sat Feb 12 2022 14:20:53 GMT+0800 (China Standard Time)

i only use pose detection and my aaptOptions is

`aaptOptions {
ignoreAssetsPattern '!mlkit_ocr:!mlkit_label_default_model:'
ignoreAssetsPattern 'hobbes.tflite'

}`

packagingOptions { exclude 'lib/**/libtranslate_jni.so' exclude 'lib/**/libdigitalink.so' exclude 'lib/**/libxeno_native.so' exclude 'lib/**/libmlkitcommonpipeline.so' exclude 'lib/**/libbarhopper_v2.so' exclude 'lib/**/libclassifier_jni.so' exclude 'lib/**/libface_detector_v2_jni.so' exclude 'lib/**/libtensorflowlite_jni.so' // exclude 'lib/**/liblanguage_id_jni.so' → required for language detection }

but it's size still 129 mb please help me to reduce the size.

Francisco Bernal · Answer 49 · Fri Apr 22 2022 01:16:51 GMT+0800 (China Standard Time)

We released a new version of google_ml_kit in which we have split all the plugins in multiple plugins to avoid removing the models for API you are not using, go to our readme and start using the new plugin you need.

More details here:

https://github.com/bharat-biradar/Google-Ml-Kit-plugin/tree/master/packages/google_ml_kit

Visio-Biswaroop · Answer 50 · Sat Apr 08 2023 18:58:16 GMT+0800 (China Standard Time)

Hi, I am using:
dependencies {
...
...
implementation 'com.google.mlkit:translate:17.0.1'
...
}
I want to just use Language X (any one language) to English translation. Is there a way I can optimise MLKit further to reduce my apk size?

KartavyaBagga · Answer 51 · Sun May 07 2023 20:05:10 GMT+0800 (China Standard Time)

buildTypes {
       release {
           signingConfig signingConfigs.release
            aaptOptions {
                ignoreAssetsPattern "mlkit_pose"
            }
       }
   }
   packagingOptions {
        exclude 'lib/**/libtranslate_jni.so'
        exclude 'lib/**/libdigitalink.so'
        exclude 'lib/**/libxeno_native.so'
        exclude 'lib/**/libmlkitcommonpipeline.so'
        exclude 'lib/**/libbarhopper_v2.so'
        exclude 'lib/**/libclassifier_jni.so'
        exclude 'lib/**/libface_detector_v2_jni.so'
        exclude 'lib/**/libtensorflowlite_jni.so'
        exclude 'lib/**/liblanguage_id_jni.so'
    }

OCR is working great and app size reduced to 30MB approx

How to reduce app size further below 30MB