CMSIS-NN CIFAR10 example for STM32F746G-DISCO does not work?

Question

CMSIS-NN CIFAR10 example for STM32F746G-DISCO does not work?

marcin-ch opened this issue 5 years ago · comments

Hello All,

I am following this Image recognition on Arm Cortex-M with CMSIS-NN guide

I have exactly the same hardware:

STM32F746G-DISCO
STM32F4DIS-CAM

the same software installed

Ubuntu 16.04 LTS
Python 2.7.12
Caffe
GNU Tools for Arm Embedded Processors 7-2017-q4-major

and I am able to reproduce all mentioned in the guide steps (including building basic camera app, quantizing the model, converting the model), except the final one! = Deploy transformed model on an Arm Cortex-M processor

The final call is:

#Run this command in cmsisnn_demo folder
mbed compile -m DISCO_F746NG -t GCC_ARM --source . --source ../ML-examples/cmsisnn-cifar10/code/m7 --source ../ML-examples/cmsisnn-cifar10/camera_demo/camera_with_nn/ --source ../CMSIS_5/CMSIS/NN/Include --source ../CMSIS_5/CMSIS/NN/Source --source ../CMSIS_5/CMSIS/DSP/Include --source ../CMSIS_5/CMSIS/DSP/Source --source ../CMSIS_5/CMSIS/Core/Include -j8

The only difference is in

--source ../ML-examples/cmsisnn-cifar10/camera_demo/camera_with_nn/

the original call refers to

--source ../ML-examples/cmsisnn-cifar10/camera_demo/camera_app/

But I have not changed camera_app folder, just used already prepared by ARM folder with camera_with_nn (ARM provides ready-to-go camera apps without NN and with NN, here)

And mbed finishes with info

[mbed] ERROR: "/usr/bin/python" returned error.

(I know, not much descriptive... but I do not see any ERRORs during building, only: multiple definition of or first defined here)

So perhaps any known issues reffering to above?
Has anyone tried to complete this guide?

Hints more than welcome.

marcin-ch · Answer 1 · Thu May 02 2019 21:39:31 GMT+0800 (China Standard Time)

Not sure if I call it a workaround but finally I have this example running.
I just changed the final call to:

#Run this command in cmsisnn_demo folder
mbed compile -m DISCO_F746NG -t GCC_ARM --source . --source ../ML-examples/cmsisnn-cifar10/code/m7 --source ../ML-examples/cmsisnn-cifar10/camera_demo/camera_with_nn/ --source ../CMSIS_5/CMSIS/NN/Include --source ../CMSIS_5/CMSIS/NN/Source --source ../CMSIS_5/CMSIS/DSP/Include --source ../CMSIS_5/CMSIS/Core/Include -j8

it means I just simply cut this one:

--source ../CMSIS_5/CMSIS/DSP/Source

marcin-ch · Answer 2 · Wed May 22 2019 19:46:23 GMT+0800 (China Standard Time)

just to close the issue and prove that I went through all the steps and got working demo 😆

Young Olutosin · Answer 3 · Sat Jul 06 2019 20:56:05 GMT+0800 (China Standard Time)

Hello marcin_ch, I am currently working on this project for my thesis. Please I need help getting it through. Do you have the link to step by step procedures so that I can follow?

marcin-ch · Answer 4 · Wed Jul 10 2019 16:56:06 GMT+0800 (China Standard Time)

Hi @youngolutosin ,
I followed this guide Image recognition on Arm Cortex-M with CMSIS-NN guide as mentioned in my first post here. You can download this guide as PDF file (please see "Download PDF" option).

Young Olutosin · Answer 5 · Fri Jul 12 2019 00:47:46 GMT+0800 (China Standard Time)

I have gone through the steps and I was able to Implement the first basic camera example and it worked. I am stucked at check failed:mdb_status == 0 (2 vs.0) No file or directory. I have edited the mean_file and source file but in caffe/exmaples/cifar10 directory, I don't have mean.binaryproto, cifar10_train_lmdb and cifar10_test_lmdb. Please how did you solve the problem? Thank you in advance.

marcin-ch · Answer 6 · Mon Jul 15 2019 17:58:31 GMT+0800 (China Standard Time)

Hi @youngolutosin ,
first of all, make sure you double-checked Troubleshooting guide

And then have a look at issue I created here with the same question as you. Hope that helps.

Young Olutosin · Answer 7 · Mon Jul 15 2019 23:13:03 GMT+0800 (China Standard Time)

Thank you for your response. I already figured out the problem. My project is working now. Seems the model is sensitive to some pictures. I an working on how to train the model with my own dataset and reduce the class to the specific I wanted. Thank you and I really appreciate. Regards, Young.

…

On Mon, 15 Jul 2019, 12:58 marcin-ch, ***@***.***> wrote: Hi @youngolutosin <https://github.com/youngolutosin> , first of all, make sure you double-checked Troubleshooting guide <https://developer.arm.com/solutions/machine-learning-on-arm/developer-material/how-to-guides/image-recognition-on-arm-cortex-m-with-cmsis-nn/troubleshooting> And then have a look at issue I created here <#14> with the same question as you. Hope that helps. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16?email_source=notifications&email_token=AK2TEGPX375RZYUVYQV4ABTP7RC4TA5CNFSM4HJC4N2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZ5HJNA#issuecomment-511341748>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AK2TEGJZK4272STVYUIOTFDP7RC4TANCNFSM4HJC4N2A> .

marcin-ch · Answer 8 · Tue Jul 16 2019 04:23:43 GMT+0800 (China Standard Time)

@youngolutosin ,
yup, model is not very reliable, but they mentioned about it in the guide:

Note: that this model is very easily affected by changes in light conditions, we have chosen it only for simplicity.

And what may be also helpful for you is the keyword: transfer learning.
In simple words you can use some already existing/trained model for image classification and re-train it using your dataset. It should greatly shorten the amount of time needed to train the model.

Hope that helps and good luck!

shell0108 · Answer 9 · Mon Apr 06 2020 17:43:44 GMT+0800 (China Standard Time)

@marcin-ch @youngolutosin Hello ! I'm doing the same CMSIS_NN CIFAR10 example ,following the same guide ,but I met some problems.

It seems that you have done it successfully, and I will be very grateful if you offered some help!
here is my _issue,

Josh Bindels · Answer 10 · Sun Nov 08 2020 18:05:01 GMT+0800 (China Standard Time)

Hi @youngolutosin @marcin-ch @shell0108 ,

I'm currently working on this project for my thesis as well.

Did you notice a difference when running the model on the board or on a laptop?
When I run accuracy tests using caffe or as a python script on my host machine I'm getting a decent accuracy. But, when I run a similar accuracy test on the board the accuracy is quite low, ~30%.

Young Olutosin · Answer 11 · Sun Nov 08 2020 18:17:17 GMT+0800 (China Standard Time)

Hi, The accuracy of the cifar model after optimization will obviously reduce due to the conversion of the model from float32 into fixed point integer. This technique compresses the model and it affects the performance of the model. You know the idea is to find a balance between speed, accuracy and model size so that the model will fit into the flash and ram respectively. This is very difficult to achieve. I cannot remember the precise amount of drop in accuracy, but yes, the accuracy dropped when I ran it on the board. You can look into STM32 CubeAI software expansion (it supports Keras, TFlowLite, Caffe, ONNX etc). Alot of improvement has been made ever since and you can achieve a better result using this. I hope it helps. Cheers.

…

On Sun, 8 Nov 2020 at 12:05, Josh Bindels ***@***.***> wrote: Hi @youngolutosin <https://github.com/youngolutosin> @marcin-ch <https://github.com/marcin-ch> @shell0108 <https://github.com/shell0108> , I'm currently working on this project for my thesis as well. Did you notice a difference when running the model on the board or on a laptop? When I run accuracy tests using caffe or as a python script on my host machine I'm getting a decent accuracy. But, when I run a similar accuracy test on the board the accuracy is quite low, ~30%. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AK2TEGN32GXWT223NC4JAZTSOZUNTANCNFSM4HJC4N2A> .

-- *Warm Regards,* *Ademola Olutosin Ajibola* *Master Communicative Electronics,* *+37257844919,* *Tallinn University of Technology, School of Information Technology,* *Ehitajate tee 5, 19086, Tallinn, Estonia.*

Josh Bindels · Answer 12 · Sun Nov 08 2020 23:02:06 GMT+0800 (China Standard Time)

This looks very promising! Thank you, I really appreciate it! On Sun, 8 Nov 2020 at 11:17 AM, youngolutosin <notifications@github.com> wrote:

…

Hi, The accuracy of the cifar model after optimization will obviously reduce due to the conversion of the model from float32 into fixed point integer. This technique compresses the model and it affects the performance of the model. You know the idea is to find a balance between speed, accuracy and model size so that the model will fit into the flash and ram respectively. This is very difficult to achieve. I cannot remember the precise amount of drop in accuracy, but yes, the accuracy dropped when I ran it on the board. You can look into STM32 CubeAI software expansion (it supports Keras, TFlowLite, Caffe, ONNX etc). Alot of improvement has been made ever since and you can achieve a better result using this. I hope it helps. Cheers. On Sun, 8 Nov 2020 at 12:05, Josh Bindels ***@***.***> wrote: > Hi @youngolutosin <https://github.com/youngolutosin> @marcin-ch > <https://github.com/marcin-ch> @shell0108 <https://github.com/shell0108> , > > I'm currently working on this project for my thesis as well. > > Did you notice a difference when running the model on the board or on a > laptop? > When I run accuracy tests using caffe or as a python script on my host > machine I'm getting a decent accuracy. But, when I run a similar accuracy > test on the board the accuracy is quite low, ~30%. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > < #16 (comment) >, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AK2TEGN32GXWT223NC4JAZTSOZUNTANCNFSM4HJC4N2A > > . > -- *Warm Regards,* *Ademola Olutosin Ajibola* *Master Communicative Electronics,* *+37257844919,* *Tallinn University of Technology, School of Information Technology,* *Ehitajate tee 5 <https://www.google.com/maps/search/Ehitajate+tee+5?entry=gmail&source=g>, 19086, Tallinn, Estonia.* — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#16 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABZCLFT7W7UK5VPF63UQVXLSOZV3VANCNFSM4HJC4N2A> .

Young Olutosin · Answer 13 · Mon Nov 09 2020 00:33:52 GMT+0800 (China Standard Time)

You are welcome.

…

On Sun, 8 Nov 2020 at 17:02, Josh Bindels ***@***.***> wrote: This looks very promising! Thank you, I really appreciate it! On Sun, 8 Nov 2020 at 11:17 AM, youngolutosin ***@***.***> wrote: > Hi, > > The accuracy of the cifar model after optimization will obviously reduce > due to the conversion of the model from float32 into fixed point integer. > This technique compresses the model and it affects the performance of the > model. > > You know the idea is to find a balance between speed, accuracy and model > size so that the model will fit into the flash and ram respectively. This > is very difficult to achieve. > > I cannot remember the precise amount of drop in accuracy, but yes, the > accuracy dropped when I ran it on the board. > > You can look into STM32 CubeAI software expansion (it supports Keras, > TFlowLite, Caffe, ONNX etc). Alot of improvement has been made ever since > and you can achieve a better result using this. > > I hope it helps. > > Cheers. > > > On Sun, 8 Nov 2020 at 12:05, Josh Bindels ***@***.***> > wrote: > > > Hi @youngolutosin <https://github.com/youngolutosin> @marcin-ch > > <https://github.com/marcin-ch> @shell0108 < https://github.com/shell0108> > , > > > > I'm currently working on this project for my thesis as well. > > > > Did you notice a difference when running the model on the board or on a > > laptop? > > When I run accuracy tests using caffe or as a python script on my host > > machine I'm getting a decent accuracy. But, when I run a similar accuracy > > test on the board the accuracy is quite low, ~30%. > > > > — > > You are receiving this because you were mentioned. > > Reply to this email directly, view it on GitHub > > < > #16 (comment) > >, > > or unsubscribe > > < > https://github.com/notifications/unsubscribe-auth/AK2TEGN32GXWT223NC4JAZTSOZUNTANCNFSM4HJC4N2A > > > > . > > > > > -- > > > > > > *Warm Regards,* > > *Ademola Olutosin Ajibola* > > *Master Communicative Electronics,* > > *+37257844919,* > > > *Tallinn University of Technology, School of Information Technology,* > > *Ehitajate tee 5 > <https://www.google.com/maps/search/Ehitajate+tee+5?entry=gmail&source=g >, > 19086, Tallinn, Estonia.* > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > < #16 (comment) >, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/ABZCLFT7W7UK5VPF63UQVXLSOZV3VANCNFSM4HJC4N2A > > . > — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AK2TEGIYV7JELNDS64RAFT3SO2XHXANCNFSM4HJC4N2A> .

-- *Warm Regards,* *Ademola Olutosin Ajibola* *Master Communicative Electronics,* *+37257844919,* *Tallinn University of Technology, School of Information Technology,* *Ehitajate tee 5, 19086, Tallinn, Estonia.*

Garrett Marcinak · Answer 14 · Tue Jul 20 2021 23:18:50 GMT+0800 (China Standard Time)

Hello @marcin-ch

I know this is a very old project for you, but I am trying to figure out this camera for another project and I am having trouble with the stm32 DCMI. I was wondering if you could upload the generated bin file you get when running the lines:

cd cmsisnn_demo/
mbed compile -m DISCO_F746NG -t GCC_ARM --source . --source ../ML-examples/cmsisnn-cifar10/camera_demo/camera_app/

Although my end goal is ML, I really need to figure out how to use the camera first. I would really appreciate any advice or help, thanks!!

marcin-ch · Answer 15 · Wed Jul 21 2021 19:49:57 GMT+0800 (China Standard Time)

@beyonson

where should this .bin file be located? As you mentioned, it's been a while since I touched this project last time, so I need some guidance.
Is this path below right one?
~/CMSISNN_Webinar/cmsisnn_demo/BUILD/DISCO_F746NG/GCC_ARM
if so, then please see attached, and please notice I have no hardware to check it on my end, so I am sending it AS IS with hope that it is working version.
cmsisnn_demo.zip

Narendra Kumar · Answer 16 · Mon Aug 14 2023 14:35:14 GMT+0800 (China Standard Time)

Hi @youngolutosin ,
I followed this guide Image recognition on Arm Cortex-M with CMSIS-NN guide as mentioned in my first post here. You can download this guide as PDF file (please see "Download PDF" option).

Can you share your project report, I am working image recognition using cortex processor with cmsis library, can you help me in completing the project, I don't have much knowledge on this topic.

marcin-ch · Answer 17 · Tue Aug 15 2023 06:25:32 GMT+0800 (China Standard Time)

@NarendraKumarMadireddy seems this is the one
Image recognition on Arm Cortex-M with CMSIS-NN.pdf
and Youtube video as well which might be helpful
Image recognition on Arm Cortex-M with CMSIS-NN in 5 steps

Young Olutosin · Answer 18 · Tue Aug 15 2023 15:11:59 GMT+0800 (China Standard Time)

It’s been awhile. I really do not have access to this project anymore.

On Tue, 15. Aug 2023 at 01:25, marcin-ch ***@***.***> wrote: @NarendraKumarMadireddy <https://github.com/NarendraKumarMadireddy> seems this is the one Image recognition on Arm Cortex-M with CMSIS-NN.pdf <https://github.com/ARM-software/ML-examples/files/12339969/Image.recognition.on.Arm.Cortex-M.with.CMSIS-NN.pdf> and Youtube video as well which might be helpful Image recognition on Arm Cortex-M with CMSIS-NN in 5 steps <https://www.youtube.com/watch?v=EkYp0glSenE> — Reply to this email directly, view it on GitHub <#16 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AK2TEGL3WTFJLAQMPKQXIQDXVKQWRANCNFSM4HJC4N2A> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- *Regards,* *Olutosin Ademola* *+37253564286,* *DevOps Engineer | ML Engineer* *Tallinn, Estonia.*

Narendra Kumar · Answer 19 · Tue Aug 15 2023 20:29:16 GMT+0800 (China Standard Time)

Hello All,

I am following this Image recognition on Arm Cortex-M with CMSIS-NN guide

I have exactly the same hardware:
STM32F746G-DISCO
STM32F4DIS-CAM
the same software installed
Ubuntu 16.04 LTS
Python 2.7.12
Caffe
GNU Tools for Arm Embedded Processors 7-2017-q4-major
and I am able to reproduce all mentioned in the guide steps (including building basic camera app, quantizing the model, converting the model), except the final one! = Deploy transformed model on an Arm Cortex-M processor

The final call is:
#Run this command in cmsisnn_demo folder
mbed compile -m DISCO_F746NG -t GCC_ARM --source . --source ../ML-examples/cmsisnn-cifar10/code/m7 --source ../ML-examples/cmsisnn-cifar10/camera_demo/camera_with_nn/ --source ../CMSIS_5/CMSIS/NN/Include --source ../CMSIS_5/CMSIS/NN/Source --source ../CMSIS_5/CMSIS/DSP/Include --source ../CMSIS_5/CMSIS/DSP/Source --source ../CMSIS_5/CMSIS/Core/Include -j8
The only difference is in
--source ../ML-examples/cmsisnn-cifar10/camera_demo/camera_with_nn/
the original call refers to
--source ../ML-examples/cmsisnn-cifar10/camera_demo/camera_app/
But I have not changed camera_app folder, just used already prepared by ARM folder with camera_with_nn (ARM provides ready-to-go camera apps without NN and with NN, here)

And mbed finishes with info
[mbed] ERROR: "/usr/bin/python" returned error.
(I know, not much descriptive... but I do not see any ERRORs during building, only: multiple definition of or first defined here)

So perhaps any known issues reffering to above?
Has anyone tried to complete this guide?

Hints more than welcome.

Can you share your project report, I am working on this image recognition using cortex processor with cmsis library, can you help me in completing the project, I don't have much knowledge on this topic.

Can you share the correct detailed step by step process.

Narendra Kumar · Answer 20 · Tue Aug 15 2023 20:30:28 GMT+0800 (China Standard Time)

Hello marcin_ch, I am currently working on this project for my thesis. Please I need help getting it through. Do you have the link to step by step procedures so that I can follow?

Can you share your project report, I am working image recognition using cortex processor with cmsis library, can you help me in completing the project, I don't have much knowledge on this topic.

Can you share me the step by step procedure please 🥺.

Narendra Kumar · Answer 21 · Tue Aug 15 2023 20:31:48 GMT+0800 (China Standard Time)

Hi @youngolutosin @marcin-ch @shell0108 ,

I'm currently working on this project for my thesis as well.

Did you notice a difference when running the model on the board or on a laptop?
When I run accuracy tests using caffe or as a python script on my host machine I'm getting a decent accuracy. But, when I run a similar accuracy test on the board the accuracy is quite low, ~30%.

Can you share your project report, I am working image recognition using cortex processor with cmsis library, can you help me in completing the project, I don't have much knowledge on this topic.

Can you share me the step by step procedure 🥺 please

Young Olutosin · Answer 22 · Tue Aug 15 2023 20:33:50 GMT+0800 (China Standard Time)

I don’t have access to the project anymore. Please sto spamming.

On Tue, 15. Aug 2023 at 15:31, Narendra Kumar ***@***.***> wrote: Hi @youngolutosin <https://github.com/youngolutosin> @marcin-ch <https://github.com/marcin-ch> @shell0108 <https://github.com/shell0108> , I'm currently working on this project for my thesis as well. Did you notice a difference when running the model on the board or on a laptop? When I run accuracy tests using caffe or as a python script on my host machine I'm getting a decent accuracy. But, when I run a similar accuracy test on the board the accuracy is quite low, ~30%. Can you share your project report, I am working image recognition using cortex processor with cmsis library, can you help me in completing the project, I don't have much knowledge on this topic. Can you share me the step by step procedure 🥺 please — Reply to this email directly, view it on GitHub <#16 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AK2TEGLZGHOMYGE5MLC72TDXVNT37ANCNFSM4HJC4N2A> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- *Regards,* *Olutosin Ademola* *+37253564286,* *DevOps Engineer | ML Engineer* *Tallinn, Estonia.*

Narendra Kumar · Answer 23 · Tue Aug 15 2023 20:37:43 GMT+0800 (China Standard Time)

@NarendraKumarMadireddy seems this is the one
Image recognition on Arm Cortex-M with CMSIS-NN.pdf
and Youtube video as well which might be helpful
Image recognition on Arm Cortex-M with CMSIS-NN in 5 steps

Thankyou but can you share me some report or etc , I am new to Linux and caffe

Narendra Kumar · Answer 24 · Tue Aug 15 2023 20:39:09 GMT+0800 (China Standard Time)

I don’t have access to the project anymore. Please sto spamming.

On Tue, 15. Aug 2023 at 15:31, Narendra Kumar @.***>
wrote:

Hi @youngolutosin https://github.com/youngolutosin @marcin-ch
https://github.com/marcin-ch @shell0108 https://github.com/shell0108 ,

I'm currently working on this project for my thesis as well.

Did you notice a difference when running the model on the board or on a
laptop?
When I run accuracy tests using caffe or as a python script on my host
machine I'm getting a decent accuracy. But, when I run a similar accuracy
test on the board the accuracy is quite low, ~30%.

Can you share your project report, I am working image recognition using
cortex processor with cmsis library, can you help me in completing the
project, I don't have much knowledge on this topic.

Can you share me the step by step procedure 🥺 please

—
Reply to this email directly, view it on GitHub
#16 (comment),
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AK2TEGLZGHOMYGE5MLC72TDXVNT37ANCNFSM4HJC4N2A
.
You are receiving this because you were mentioned.Message ID:
@.***>

--

Regards,

Olutosin Ademola

+37253564286,

DevOps Engineer | ML Engineer

Tallinn, Estonia.

Sorry , thankyou

Narendra Kumar · Answer 25 · Fri Aug 18 2023 00:39:24 GMT+0800 (China Standard Time)

Last question, do anyone have the windows version tutorial/PDFs/report of this project?