Error when downloading firmware: KeyError: 'fs_block_size'
NicoBijl opened this issue · comments
when downloading the firmware for Shelly dimmer 1 (or just SHSW-1) i get the following error from line 86 of shelly_firmware.py. KeyError: 'fs_block_size'
./shelly_firmware.py --download SHDM-1 --output shelly-dm-1-firmware
Verbose logging:
INFO: Shelly firmware binary download tool. Version 0.1
INFO: Downloading firmware binary file for device SHDM-1
INFO: Output file is set to: shelly-dm-1-firmware
DEBUG: Fetching data from URL: http://api.shelly.cloud/files/firmware
DEBUG: Got response 200 for URL: http://api.shelly.cloud/files/firmware
DEBUG: Data JSON received and it looks sane. isok = True
DEBUG: Model SHDM-1 found!
DEBUG: The following files were found in downloaded firmware package
dimmer-1.0/manifest.json
dimmer-1.0/dimmer.bin
dimmer-1.0/esp_init_data_default_v08.bin
dimmer-1.0/fs.bin
dimmer-1.0/rboot.bin
DEBUG: The manifest seems to be named dimmer-1.0/manifest.json
INFO: Found platform dimmer in firmware package
DEBUG: Iterating over firmware parts...
DEBUG: Searching for part rboot.bin in firmware package
DEBUG: The following files were found in downloaded firmware package
dimmer-1.0/manifest.json
dimmer-1.0/dimmer.bin
dimmer-1.0/esp_init_data_default_v08.bin
dimmer-1.0/fs.bin
dimmer-1.0/rboot.bin
DEBUG: The file for part rboot.bin seems to be named dimmer-1.0/rboot.bin
DEBUG: Part data verification requested
DEBUG: The following checksums were calculated:
Data 2dfef208220da2cf0ad424f5a3cb560537222d23
Manifest 2dfef208220da2cf0ad424f5a3cb560537222d23
DEBUG: Checksums match. Success!
DEBUG: Found part boot:
Start address: 0x0
Size: 0x910
Data: e902fff0f0051040000010400806000000000000000000000000000000000000...
DEBUG: Found part boot_cfg:
Start address: 0x7000
Size: 0x1000
Data: ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff...
DEBUG: Searching for part fs.bin in firmware package
DEBUG: The following files were found in downloaded firmware package
dimmer-1.0/manifest.json
dimmer-1.0/dimmer.bin
dimmer-1.0/esp_init_data_default_v08.bin
dimmer-1.0/fs.bin
dimmer-1.0/rboot.bin
DEBUG: The file for part fs.bin seems to be named dimmer-1.0/fs.bin
DEBUG: Part data verification requested
DEBUG: The following checksums were calculated:
Data 978ca57768de940b6ba2645cde812b3ac2b440cb
Manifest 978ca57768de940b6ba2645cde812b3ac2b440cb
DEBUG: Checksums match. Success!
DEBUG: Found part fs:
Start address: 0xbb000
Size: 0x40000
Data: 018001000280020002000200020002000000030003000300030003000300ffff...
Traceback (most recent call last):
File "/Users/nico/Projects/personal/shelly-firmware/./shelly_firmware.py", line 293, in <module>
main()
File "/Users/nico/Projects/personal/shelly-firmware/./shelly_firmware.py", line 284, in main
download_and_build_firmware(firmware_url, args.output)
File "/Users/nico/Projects/personal/shelly-firmware/./shelly_firmware.py", line 117, in download_and_build_firmware
build_firmware(fw_pkg.content, output_file)
File "/Users/nico/Projects/personal/shelly-firmware/./shelly_firmware.py", line 86, in build_firmware
'block_size': part['fs_block_size'],
KeyError: 'fs_block_size'
Any ideas?
I too have this error.
Did you ever made the process with success?
I have the same issue with a SHSW-PM.
(shelly-firmware-sQFpiWbz) pi@raspberrypi:~/shelly/shelly-firmware $ ./shelly_firmware.py -d SHSW-PM -o shelly1pm
INFO: Shelly firmware binary download tool. Version 0.1
INFO: Downloading firmware binary file for device SHSW-PM
INFO: Output file is set to: shelly1pm
INFO: Found platform switch1pm in firmware package
Traceback (most recent call last):
File "./shelly_firmware.py", line 299, in
main()
File "./shelly_firmware.py", line 290, in main
download_and_build_firmware(firmware_url, args.output)
File "./shelly_firmware.py", line 121, in download_and_build_firmware
build_firmware(fw_pkg.content, output_file)
File "./shelly_firmware.py", line 90, in build_firmware
'block_size': part['fs_block_size'],
KeyError: 'fs_block_size'
Same for the Shelly 2.5
$ ./shelly_firmware.py --download SHSW-25
INFO: Shelly firmware binary download tool. Version 0.1
INFO: Downloading firmware binary file for device SHSW-25
INFO: Output file is set to: firmware.bin
DEBUG: Fetching data from URL: http://api.shelly.cloud/files/firmware
DEBUG: Got response 200 for URL: http://api.shelly.cloud/files/firmware
DEBUG: Data JSON received and it looks sane. isok = True
DEBUG: Model SHSW-25 found!
DEBUG: The following files were found in downloaded firmware package
switch25-1.0/manifest.json
switch25-1.0/esp_init_data_default_v08.bin
switch25-1.0/fs.bin
switch25-1.0/rboot.bin
switch25-1.0/switch25.bin
DEBUG: The manifest seems to be named switch25-1.0/manifest.json
INFO: Found platform switch25 in firmware package
DEBUG: Iterating over firmware parts...
DEBUG: Searching for part rboot.bin in firmware package
DEBUG: The following files were found in downloaded firmware package
switch25-1.0/manifest.json
switch25-1.0/esp_init_data_default_v08.bin
switch25-1.0/fs.bin
switch25-1.0/rboot.bin
switch25-1.0/switch25.bin
DEBUG: The file for part rboot.bin seems to be named switch25-1.0/rboot.bin
DEBUG: Part data verification requested
DEBUG: The following checksums were calculated:
Data c0996f11597ee2a88af5efaa727cff7a09f8ac16
Manifest c0996f11597ee2a88af5efaa727cff7a09f8ac16
DEBUG: Checksums match. Success!
DEBUG: Found part boot:
Start address: 0x0
Size: 0x910
Data: e902fff0e8051040000010400006000000000000000000000000000000000000...
DEBUG: Found part boot_cfg:
Start address: 0x1000
Size: 0x1000
Data: ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff...
DEBUG: Searching for part fs.bin in firmware package
DEBUG: The following files were found in downloaded firmware package
switch25-1.0/manifest.json
switch25-1.0/esp_init_data_default_v08.bin
switch25-1.0/fs.bin
switch25-1.0/rboot.bin
switch25-1.0/switch25.bin
DEBUG: The file for part fs.bin seems to be named switch25-1.0/fs.bin
DEBUG: Part data verification requested
DEBUG: The following checksums were calculated:
Data d23405ad94c1d71281971a9efd6621d5f54740ef
Manifest d23405ad94c1d71281971a9efd6621d5f54740ef
DEBUG: Checksums match. Success!
DEBUG: Found part fs:
Start address: 0xbb000
Size: 0x40000
Data: 018001000280020002000200020002000000030003000300030003000300ffff...
Traceback (most recent call last):
File "./shelly_firmware.py", line 299, in <module>
main()
File "./shelly_firmware.py", line 290, in main
download_and_build_firmware(firmware_url, args.output)
File "./shelly_firmware.py", line 121, in download_and_build_firmware
build_firmware(fw_pkg.content, output_file)
File "./shelly_firmware.py", line 90, in build_firmware
'block_size': part['fs_block_size'],
KeyError: 'fs_block_size'
The manifest metadata seems to have changed:
I have a zip file for an older Shelly 2.5 firmware (1.9.2) and it looks like this:
Shelly 2.5 1.9.2 manifest
{
"build_id": "20201128-102046/v1.9.2@e83f7025",
"build_timestamp": "2020-11-28T10:20:46Z",
"name": "switch25",
"parts": {
"boot": {
"addr": 0,
"cs_sha1": "c0996f11597ee2a88af5efaa727cff7a09f8ac16",
"size": 2320,
"src": "rboot.bin",
"update": false
},
"boot_cfg": {
"addr": 4096,
"fill": 255,
"size": 4096
},
"fs": {
"addr": 765952,
"addr1": 1814528,
"cs_sha1": "19239cc4b4806ac4d0772b5882e0822000e70837",
"fs_block_size": 4096,
"fs_erase_size": 4096,
"fs_page_size": 256,
"fs_size": 262144,
"size": 262144,
"src": "fs.bin",
"type": "fs"
},
"fw": {
"addr": 32768,
"addr1": 1081344,
"cs_sha1": "fea5a408450c98bd356cf9c3bd4cbae59b4d2e28",
"size": 617968,
"src": "switch25.bin"
},
"rf_cal_data": {
"addr": 2076672,
"fill": 255,
"size": 4096
},
"sys_params": {
"addr": 2080768,
"cs_sha1": "ff105e66b313201f5b40a2e02b7f00db24022088",
"size": 128,
"src": "esp_init_data_default_v08.bin",
"type": "sys_params"
}
},
"platform": "esp8266",
"version": "1.0"
}Shelly 2.5 manifest 1.11.0
{
"build_id": "20210702-121359/v1.11.0-rc2-g7766530",
"build_timestamp": "2021-07-02T12:13:59Z",
"name": "switch25",
"parts": {
"boot": {
"addr": 0,
"cs_sha1": "c0996f11597ee2a88af5efaa727cff7a09f8ac16",
"cs_sha256": "95272bce7c38561f9e89c05053d62cf35a000fcc191dc412ebb02a0c8f040b00",
"size": 2320,
"src": "rboot.bin",
"update": false
},
"boot_cfg": {
"addr": 4096,
"fill": 255,
"size": 4096
},
"fs": {
"addr": 765952,
"addr1": 1814528,
"cs_sha1": "d4f337163f2b23f5a7f0f2db9a6ef2d264a32065",
"cs_sha256": "63e62b6e74563f25c99a92af60306f81c9df7479830c3a60d01e1f2041c26428",
"fs_size": 262144,
"size": 262144,
"src": "fs.bin",
"type": "fs"
},
"fw": {
"addr": 32768,
"addr1": 1081344,
"cs_sha1": "0fa663af7b16516e03ecae7f4116a91b75ab0363",
"cs_sha256": "836704a6dac4b6d48cad041e7a5e9ec41e334359aab05dd44cd6a44819868624",
"size": 627936,
"src": "switch25.bin"
},
"rf_cal_data": {
"addr": 2076672,
"fill": 255,
"size": 4096
},
"sys_params": {
"addr": 2080768,
"cs_sha1": "ff105e66b313201f5b40a2e02b7f00db24022088",
"cs_sha256": "b2218087cf938ce665b26ac049f7d146677c70fe2909205a4d0e6a58aef0e4b3",
"size": 128,
"src": "esp_init_data_default_v08.bin",
"type": "sys_params3"
}
},
"platform": "esp8266",
"version": "1.0"
}
In fact the changed format seem to be the problem/solution.
I just changed the code in line 89ff to:
fs_params = {
'block_size': 4096,
'page_size': 256,
'total_size': part['fs_size'],
'erase_size': 4096
}
Of course, this is hardcoding parameters, but it solves the problem until a better way is found ;-)
Unfortunately the created firmware has the known 'index page' problem too.
Apologies for not having the spare time to look into this for so long..
Thanks @Ramblurr for spotting the changes in manifest!
Just updated this piece of code in order to get the total/block/page/erase size from unspiffs8 tool output instead of extracting them from manifest.
I'm yet to test this on a real Shelly device, so feel free to post any issues you may find.