This code declares a webserver to run on your computer. Open the page on your mobile device to stream IMU data back to the host. Use this data to calculate the device relative position, orientation, etc.
Inertial Measurement Units (IMUs) are used everywhere; in planes, autonomous vehicles, submarines, smart watches, computers ... and your mobile phone. They are general purpose modules constructor add to products everytime position, orientation, speed or motion capture is necessary. As future engineers, you ought to be able to interact with the typical sensor and present results in nice visualisation. From my experience at AKKA Technologies, demonstrating your involvement in projects mixing low level data, physics and software would help you get hired pretty fast !
Some examples using IMUs are :
- All autonomous cars use an IMU, see Apollo 2 (link to specs)
- Open Source Autonomous Driving projects such as Autoware use some (link)
- Robotics manufacturer (example)
- IMUs price go from a 2€ (like a MPU-6050) to 20€ (like a BNO080) to ks€ (see GPS-RTK) !!
But where will we find an IMU for each student, and that you already have at home (#stayhome) ? Thankfully your smartphones have good IMUs, so let's push them at their maximum and find out how good they are. What works for the phones will work for Autonomous Vehicles as well.
We will stream the phone accelerometer and gyroscope data to the computer and experiment with data fusion. Beneath is what high level sensor fusion looks like (source), so keep up !
We will turn the internal accelerometer, gyroscope and magnetometer in an Inertial Measurement Unit as an introduction to robotics. You will be measuring the relative position of your device through sensor fusion and use it to measure distances at home.
L3: 10 Apr 8.30-11.45 (NOTE SAME DAY TWO LECTURES!)
L4: 10 Apr 13.30-15.30
L5: 24 April 13.30-15.30
L6: 15 May 13.30-15.30
L7: 18 May 8.30-11.45
L8: 22 May 13.30-15.30 EXAM
Objective: run the code, handle the data and write test_imu.TestImu.test_mean_data
.
Preparations:
- Join the telegram chat (link on moodle)
- Create a private repository named IN104_Project_Name_Name
- Invite me as contributor (on gitlab, maintainer)
- Clone your repository on your computer
- Add my repo as a new remote
git remote add teacher https://github.com/vtalpaert/phone-imu
- List your remotes for verification
git remote -v
, you should seeorigin
andteacher
- Pull my code
git pull teacher master
- In case your repo was not empty, merge incoming commits
- Create environment using the install instructions below
Deadline April 16th. In imu.py
, change the method run
to :
-
calculate mean over last 100 values (use get_first_data in this case). Tip: try
deque
, the same aslist
but with fixed length>>> import collections >>> help(collections.deque)
-
calculate mean and std of time difference between two samples (with get_first_data as well). Tip: Define a
mean
function for clearer code, it will use the built-inssum
andlen
-
explain the difference between taking the mean every hundred values ('slow average'), and updating the mean at each value ('running average'). Bonus, try other smoothing functions than the mean. See bellow for example.
Example of the plot to include in your report :
To plot this, you can use my first function from draw.py
.
How to submit homework :
- Write some code
- Commit and push to your repo
- Your teammate pulls your modifications
- Your teammate writes some code as well
- Your teammate pushes his code
- You pull his code
- When your code completes the mean of 100 values, comment your last commit on github with a sample of the output and tag me (@vtalpaert)
- Same when you did the mean time difference
Deadline April 23rd.
Control time !
-
Pull and merge the latest changes from my repo
git pull teacher master
. -
Is there an error ? Read carefully the git output, if there is a merge conflict, this is a normal and expected error. You must manually correct the conflicting files before commiting the merge
-
Are there new requirements ?
pip3 install -r requirements.txt
again. -
Write a test for calculating the mean on a data list, put it in
test_imu.TestImu.test_mean_data
-
Speed up your code ! In
imu.py
classIMU
, changethread_update_delay
to0.01
or0.1
. Now measure the new effective time delay using what you did last week. Comment your results on the commits. Does it change something when you useget_first_data
? Withget_last_data
? Why ? -
Now change
client_send_interval
. Change the interval length to 1 millisecond, try 0 as well. What happens ? -
(The refresh rate question does not apply anymore since I removed the separate thread, I you want to go back to this question and the
thread.py
code, usegit checkout b3128338f84efb6c3845586d089fd797e08f7433
thengit checkout -b thread-question
. Don't forget to push this new branch as well if you want me to see it) -
Now we want to run the computer side code as fast as possible, so we will keep
thread_update_delay = 0.01
. Use now onlyclient_send_interval
and measure the mean delta between two data pieces. Include in your report a table with different intervals (between 0 and 50ms), the mean delta and standard deviation -
Add to your report a plot that looks like this:
You can use the second function from the current
draw.py
. It will take care of the mean and std by itself. I used 500 measures for each interval in[100, 50, 20, 15, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
. For me, 20 milliseconds seems the most stable settings. Tip: you need the same exact number of measures at each index (500 values for each interval) for the draw function to work.
Deadline May 5th, 2020.
Control space !
For your convenience, I added a live plotting tool in draw.py
, here is me plotting 100 acceleration raw values in X, Y, and Z. Keep in mind that plotting slows your code down, so you may only call live_plot.draw()
every 10 iterations for example. Tip: use the disk button to save a clean picture, the graph will not update in the meantime.
(To reset the code at this example, use git checkout 783955fceae81eee06b19614b81640a66529daa3
)
We will now try to find the position of the device on one axis. As you can guess, using the acceleration we should in theory be able to calculate the position, but in practice this won't work. Let's experience first hand exactly what is happening with the following instructions, and keep our observations in our final report.
- Intialize the IMU with a position and velocity of 0s. Keep your device on a table or any plane surface.
- Visualize the acceleration, and find your X-axis by moving the phone. How does the acceleration change over time ? Is it centered on zero when immobile ? If you slightly tilt the device, is it centered on another value ? Why is the acceleration non-zero ? (Write the answers in your report)
- Calculate the
delta_t
the difference in two samples. For me it was not exactly myclient_send_interval
value of 20ms, but rather 21.5ms. - In the IMU
run
function, use the acceleration to update the position. If you integrate the acceleration over time, you get a velocity, do it a second time and you get the relative position. I used the formuladelta_p = v * delta_t + 0.5 * a * delta_t ** 2
When I did it, the position would drift linearly after a very short time even if the device was not moving, see my example :By carefully moving the phone in a translation along the X-axis for around 20cm, then back to the initial position and so forth, I got :
Include your own version of this plot in your report.
- The first problem to solve, if that the acceleration is not necessarily centered on zero. Correct the raw value by substracting a base value. This is where your mean function may come in handy. This is loosely equivalent to using a high-pass filter if your mean is recalculated over time (so
filtered = raw - mean
) - Then get rid of the noise in the acceleration with a low pass filter. If you look at the very first image on this page, you see that taking the mean is a low pass (orange vs blue). Tip : you can also update a value using
current_value = 0.1 * observed_value + 0.9 * current_value
for slow changes. - Plot the raw acceleration, high-pass, low-pass and combined low and high pass values. Experiment different parameters in your filters and explain what worked best for you.
- With the two filters on the acceleration, I moved the phone to the left, to the right and back at the start position. I calculated the velocity and position to be :
For cleaning the velocity value, I made the assumption it would (very) slowly go back to zero over time (again, equivalent to a high pass). Furthermore, only specific values of acceleration are taking into account due to the cutoff frequencies of my combined filters. Looking at the orange/velocity and green/position lines, you can verify the velocity is null when the position is at a maximum. This is the kind of plot you must include in your report for this deliverable.
That's it for the homework. Keep in mind the drift is bound to happen whatever you try, so you will need to restart the calculations quite often. Finding the right parameters in your filters is crucial. In the end, the parameters will work for a specific motion (in my case: 0cm, +20cm, -20cm, 0cm in approx 3 seconds). So find one you like and stick to it while tuning your parameters.
Task 1 dimension : basic ruler
Task 2 dimensions : circular movement
Task 3 dimensions : free displacement
Final presentations
Will come soon
- Create a python environment, with Anaconda or Virtual Env. With Anaconda, use for example
conda create -n imu python=3.7
- Source your env. With Anaconda, use
conda activate imu
- Verify you are using Python 3 :
python --version
- Install dependencies
pip install -r requirements.txt
, read the output to check everything went well - Note your local IP address, such as
192.168.1.64
for me. On linux, useifconfig
- Run tests with
python -m unittest
- Run server with
python server.py
. Pro tip: you can stop the script withCtrl-C
- On your device, open the address
192.168.1.64:5000
to visit the server homepage - On some browser, the timestamp has a reduced precision (explanation), on Firefox you need for example to disable
privacy.reduceTimerPrecision
inabout:config
socket.error: [Errno 98] Address already in use
means another flask instance is still running. Restart your terminal or typefuser 5000/tcp -k
(might need a sudo) for linux (source)