Issues with mapping on a empty flash chip

Question

Issues with mapping on a empty flash chip

jacobn1 opened this issue 5 years ago · comments

Hi,

I'm attempting to port this code into a embedded system with an initially clean flash chip (all sectors 0xFF).

Calling init followed by resume does not create a map file written to the chip (verified by looking at my SPI trace to the chip).
Internal Map file set to:
NandMap 0x2000db10 56 dhara_map
journal 0x2000db10 48 dhara_journal
nand 0x0003f1f8 0x2000db10 4 const dhara_nand*
page_buf 0x2000cb10 0x2000db14 4 uint8_t*
log2_ppc 0x04 0x2000db18 1 unsigned char
epoch 0x00 0x2000db19 1 unsigned char
flags 0x00 0x2000db1a 1 unsigned char
bb_current 0x00000000 0x2000db1c 4 unsigned int
bb_last 0x0000003f 0x2000db20 4 unsigned int
tail_sync 0x00000000 0x2000db24 4 unsigned int
tail 0x00000000 0x2000db28 4 unsigned int
head 0x00000000 0x2000db2c 4 unsigned int
root 0xffffffff 0x2000db30 4 unsigned int
recover_next 0xffffffff 0x2000db34 4 unsigned int
recover_root 0xffffffff 0x2000db38 4 unsigned int
recover_meta 0xffffffff 0x2000db3c 4 unsigned int
gc_ratio 0x01 0x2000db40 1 unsigned char
count 0x00000000 0x2000db44 4 unsigned int

The first time I write to the flash via dhara_map_write(...)
the internal map file changes to
NandMap 0x2000db10 56 dhara_map
journal 0x2000db10 48 dhara_journal
nand 0x0003f1f8 0x2000db10 4 const dhara_nand*
page_buf 0x2000cb10 0x2000db14 4 uint8_t*
log2_ppc 0x04 0x2000db18 1 unsigned char
epoch 0x00 0x2000db19 1 unsigned char
flags 0x01 0x2000db1a 1 unsigned char
bb_current 0x00000000 0x2000db1c 4 unsigned int
bb_last 0x0000003f 0x2000db20 4 unsigned int
tail_sync 0x00000000 0x2000db24 4 unsigned int
tail 0x00000000 0x2000db28 4 unsigned int
head 0x00000001 0x2000db2c 4 unsigned int
root 0x00000000 0x2000db30 4 unsigned int
recover_next 0xffffffff 0x2000db34 4 unsigned int
recover_root 0xffffffff 0x2000db38 4 unsigned int
recover_meta 0xffffffff 0x2000db3c 4 unsigned int
gc_ratio 0x01 0x2000db40 1 unsigned char
count 0x00000001 0x2000db44 4 unsigned int

At this point if I do not call sync, no map file is saved in flash and i can not access the data over reboots (however I can find and access the data before reboot).
If I call sync the map file changes to the following and is written to flash

NandMap 0x2000db10 56 dhara_map
journal 0x2000db10 48 dhara_journal
nand 0x0003f1f8 0x2000db10 4 const dhara_nand*
page_buf 0x2000cb10 0x2000db14 4 uint8_t*
log2_ppc 0x04 0x2000db18 1 unsigned char
epoch 0x00 0x2000db19 1 unsigned char
flags 0x00 0x2000db1a 1 unsigned char
bb_current 0x00000000 0x2000db1c 4 unsigned int
bb_last 0x0000003f 0x2000db20 4 unsigned int
tail_sync 0x0000000e 0x2000db24 4 unsigned int
tail 0x0000000e 0x2000db28 4 unsigned int
head 0x00000010 0x2000db2c 4 unsigned int
root 0x0000000e 0x2000db30 4 unsigned int
recover_next 0xffffffff 0x2000db34 4 unsigned int
recover_root 0xffffffff 0x2000db38 4 unsigned int
recover_meta 0xffffffff 0x2000db3c 4 unsigned int
gc_ratio 0x01 0x2000db40 1 unsigned char
count 0x00000001 0x2000db44 4 unsigned int

At this point if I call read into the same logical sector 0, dhara_map_find returns with DHARA_ERROR_NOT_FOUND. So the map appears to be losing the data immediately.

I feel like i may be missing an important step. is there any documentation or examples or tutorials for accessing Dhara? Any ideas as to whats going wrong? I've been stuck on this problem for a few days now.

Thanks so much for your help and the awesome FTL.
Nate

jacobn1 · Answer 1 · Fri Aug 23 2019 05:31:29 GMT+0800 (China Standard Time)

FYI:
My flash chip uses the following sizes and is a MT29F8 8Gb chip.

#define FLASH_BYTES_PER_PAGE 4096
#define FLASH_EXTRA_BYTES_OFFSET FLASH_BYTES_PER_PAGE
#define FLASH_EXTRA_BYTES_PER_PAGE 256
#define FLASH_PAGES_PER_BLOCK 64
#define FLASH_BLOCKS_PER_PLANE 2048
#define FLASH_NUMBER_OF_BLOCKS 4096

#define FLASH_BYTES_PER_PAGE_LOG2 12
#define FLASH_PAGES_PER_BLOCK_LOG2 6

const struct dhara_nand MTF29F_NandSize = {FLASH_BYTES_PER_PAGE_LOG2, FLASH_PAGES_PER_BLOCK_LOG2, FLASH_NUMBER_OF_BLOCKS};

Daniel Beer · Answer 2 · Fri Aug 23 2019 07:07:12 GMT+0800 (China Standard Time)

On Thu, Aug 22, 2019 at 02:14:17PM -0700, jacobn1 wrote: I'm attempting to port this code into a embedded system with an initially clean flash chip (all sectors 0xFF). Calling init followed by resume does not create a map file written to the chip (verified by looking at my SPI trace to the chip). Internal Map file set to:

That's correct -- resume is a read-only operation.

The first time I write to the flash via dhara_map_write(...) the internal map file changes to

...

At this point if I do not call sync, no map file is saved in flash and i can not access the data over reboots (however I can find and access the data before reboot). If I call sync the map file changes to the following and is written to flash

That's also correct so far. Nothing is guaranteed to be readable on the next recovery unless you've called sync to flush metadata.

At this point if I call read into the same logical sector 0, dhara_map_find returns with DHARA_ERROR_NOT_FOUND. So the map appears to be losing the data immediately. I feel like i may be missing an important step. is there any documentation or examples or tutorials for accessing Dhara? Any ideas as to whats going wrong? I've been stuck on this problem for a few days now. Thanks so much for your help and the awesome FTL.

It doesn't sound like you're using the library incorrectly, but are you absolutely sure that your flash access functions are working as required? You could also try modifying one of the tests included in the source package (e.g. the map test) to mimic what you're doing, and perhaps change the sim.c parameters to match the layout of your flash chip and see if the problem still occurs -- if it still happens with the simulator, then that's likely a bug in dhara. Cheers, Daniel

…

-- Daniel Beer <dlbeer@gmail.com> http://dlbeer.co.nz/ PGP: BA6E 0B26 1F89 246C E3F3 C910 1E58 C43A 160A 553B

jacobn1 · Answer 3 · Sat Aug 24 2019 02:20:04 GMT+0800 (China Standard Time)

Thanks Daniel!

I haven't had a chance to try out the sim yet ... however I think I see the root of the issue. if I'm completely off track let me know and I'll start looking at your sim code (I usually love my embedded debugger :) ).

It looks like the root of the problem is in map.c : trace_path
in trace path it performs a search for the depth over DHARA_RADIX_DEPTH which defines to (sizeof(dhara_sector_t) << 3) or 32 due to dhara_sector_t being 4 bytes long
since my log2_ppc is set to 0x04 when i wrote my first page of data, dhara coped it across the entire row (filling up 0x0-0xE with the same data then putting the metadata into 0x0F).

There is no exit in trace_path during the search, so it will continue through the entire 32 depth checks ... which fails due to my flash only having 16.

so my question is: why doesn't trace_path exit once it finds the correct location instead of checking all 32?
also where did the number 32 come from? shouldn't it use log2_ppc as instead or am i missing something?

Thanks!
Nate

jacobn1 · Answer 4 · Sat Aug 24 2019 02:46:05 GMT+0800 (China Standard Time)

FYI: my meta data being saved after my 1 time page write is this:
44 68 61 00 0D 00 00 00 00 00 00 00 3F 00 00 00 01 00 00 00 01 00 00 00

Daniel Beer · Answer 5 · Sat Aug 24 2019 04:48:11 GMT+0800 (China Standard Time)

On Fri, Aug 23, 2019 at 11:20:05AM -0700, jacobn1 wrote: I haven't had a chance to try out the sim yet ... however I think I see the root of the issue. if I'm completely off track let me know and I'll start looking at your sim code (I usually love my embedded debugger :) ). It looks like the root of the problem is in map.c : trace_path in trace path it performs a search for the depth over DHARA_RADIX_DEPTH which defines to (sizeof(dhara_sector_t) << 3) or 32 due to dhara_sector_t being 4 bytes long since my log2_ppc is set to 0x04 when i wrote my first page of data, dhara coped it across the entire row (filling up 0x0-0xE with the same data then putting the metadata into 0x0F). There is no exit in trace_path during the search, so it will continue through the entire 32 depth checks ... which fails due to my flash only having 16. so my question is: why doesn't trace_path exit once it finds the correct location instead of checking all 32? also where did the number 32 come from? shouldn't it use log2_ppc as instead or am i missing something?

No, DHARA_RADIX_DEPTH has nothing to do with the flash layout -- it's the number of levels in the logical tree (the one that maps logical sectors to physical pages). The section of metadata you quoted looks reasonable if you've just written logical sector 1. I would suggest testing your NAND driver first (and in particular random-access reads -- note that Dhara will not always request a read from the beginning of a flash page). Cheers, Daniel

…

-- Daniel Beer <dlbeer@gmail.com> http://dlbeer.co.nz/ PGP: BA6E 0B26 1F89 246C E3F3 C910 1E58 C43A 160A 553B

jacobn1 · Answer 6 · Sat Aug 24 2019 07:27:45 GMT+0800 (China Standard Time)

OK digging deeper into my NAND flash read code the offset was not being obeyed (apparently the MT29F) does not abide by the offset if in continuous read mode that is setup by default with very poor documentation.

Thanks for all your support and feel free to close this bug! :)
Nate

Chris Hemingway · Answer 7 · Sun Dec 22 2019 23:35:10 GMT+0800 (China Standard Time)

@jacobn1 I am running into the same problem I think, how did you resolve this issue? Did you change to use the READ PAGE CACHE RANDOM command?