Proper way to read from a W25Q128JV flash, using Adafruit_SPIFlash

[Alessandro Francesconi]

Hello, I would like to ask for some support as I'm experiencing strange behaviors when reading a file.

I'm working with a hardware based on Arduino Zero board (SAMD21) and Winbond W25Q128JV flash memory. Software-side, I'm using littlefs v2.8 together with Adafruit_SPIFlash library for low-level operations.

I've started setting the lfs_config structure, looking at the W25Q architecture described in the official DS: https://www.winbond.com/resource-files/w25q128jv_dtr%20revc%2003272018%20plus.pdf

lfs_cfg.read  = lfs_bd_read;
lfs_cfg.prog  = lfs_bd_write;
lfs_cfg.erase = lfs_bd_erase;
lfs_cfg.sync  = lfs_bd_sync;
lfs_cfg.read_size = 256;
lfs_cfg.prog_size = 256;
lfs_cfg.block_size = 1024 * 64;
lfs_cfg.cache_size = 256;
lfs_cfg.block_count = 256;
lfs_cfg.lookahead_size = 16;
lfs_cfg.block_cycles = 500;

The lfs_bd_<xxx> functions, as said, use the SPIFlash library:

static Adafruit_FlashTransport_SPI flashTransport(CHIP_SELECT, SPI);
static Adafruit_SPIFlash flash(&flashTransport);

int lfs_bd_read(const struct lfs_config* cfg, lfs_block_t block, lfs_off_t off, void* buffer, lfs_size_t size) {
  return flash.readBuffer(block * cfg->block_size + off, (uint8_t*)buffer, size) != size ? LFS_ERR_IO : LFS_ERR_OK;
}
int lfs_bd_write(const struct lfs_config* cfg, lfs_block_t block, lfs_off_t off, const void* buffer, lfs_size_t size) {
  return flash.writeBuffer(block * cfg->block_size + off, (const uint8_t*)buffer, size) != size ? LFS_ERR_IO : LFS_ERR_OK;
}
int lfs_bd_erase(const struct lfs_config* cfg, lfs_block_t block) {
  return flash.eraseBlock(block) ? LFS_ERR_OK : LFS_ERR_IO;
}
int lfs_bd_sync(const struct lfs_config* cfg) {
  return flash.syncDevice() ? LFS_ERR_OK : LFS_ERR_IO;
}

The operations like file system creation, mounting, writing/reading files seemed to work so far. But after some deeper tests, I've found a strange behavior of my board when reading from a file.

For example, if I need to read exactly 64 Bytes from a file, I simply do this:

size_t dataSize = 64;
void* data = malloc(dataSize);
int rd = lfs_file_read(&lfs, file, data, dataSize);

The operation always returns the right number of readed bytes and the content of data is as expect. But, I've found that my board start behaving strangely when dataSize is different than 64... example, 65. Simplifying, I have interrupts that stop working as expect. Note that the actual size of the file is larger than 65.

This difference when using 64 (a power of two) vs. 65 size makes me think that the internal read operation might be wrong. Maybe I misunderstood how the block-read operation works? Hence my questions:

• Are the values assigned in lfs_cfg correct for W25Q128JV flash memory?
• Am I using the SPIFlash APIs in the proper way into lfs_bd_xxx functions?

I'm sorry if I gave no details on the "strange behaviors", so far. I hope the info are enough to have a quick review of my configuration.

[Clemens Schartmüller]

Hi,

I am also working on a similar thing with the same flash. I dont see any issue in your lfs_cfg.
However, when using Adafruit_SPIFlash and/or Adafruit_SPIFlash_Base and/or Adafruit_LittleFS I've also seen odd behavior that seems very similar (e.g. being able to write buffers of 512 bytes but not 513 etc.).

I've had success though with using raw lfs_ commands though, when implementing the "user provided" read/prog/erase functions via the Adafruit_FlashTransport_SPI implementation. Make sure to always do a readystatus-check before any operation and a WEL=1 cmd before a prog-call, though

The only issue here (for me) is the significant performance-spike after a block is full, which is probably due to the block allocator. (where a write-call then costs somewhere in the neighborhood of 200-250ms, while a normal 256 byte write-call costs just 1-2ms). I will hopefully solve this with caching and none-blocking readystatus-checks

[Christopher Haster]

One concern is does Adafruit_SPIFlash support progs/reads in multiples of prog_size/read_size? If you have a read_size of 256 bytes, littlefs might request a read of 512 bytes for example.

But realistically this only happens if you request a read that large or cache_size is that large. Reading 64 bytes when read_size is 256 bytes shouldn't really trigger this situation.

The power-of-two issue is definitely suspect, but I don't see anything wrong on littlefs's side. Maybe there's an issue in the driver? Are the write SPI commands being issued? Is there a mixup of bits vs bytes anywhere? I would try to log the block device transactions and find the problematic one, that might offer more clues to what is going wrong.

The only issue here (for me) is the significant performance-spike after a block is full

If it's any consolation, this is a known issue that is being worked on: #783

[Clemens Schartmüller]

One concern is does Adafruit_SPIFlash support progs/reads in multiples of prog_size/read_size? If you have a read_size of 256 bytes, littlefs might request a read of 512 bytes for example.

But realistically this only happens if you request a read that large or cache_size is that large. Reading 64 bytes when read_size is 256 bytes shouldn't really trigger this situation.

The power-of-two issue is definitely suspect, but I don't see anything wrong on littlefs's side. Maybe there's an issue in the driver? Are the write SPI commands being issued? Is there a mixup of bits vs bytes anywhere? I would try to log the block device transactions and find the problematic one, that might offer more clues to what is going wrong.

The only issue here (for me) is the significant performance-spike after a block is full

If it's any consolation, this is a known issue that is being worked on: #783

My investigations have shown that Adafruit_SPIFlash does some wonky address calculations, busywaits and buffering itself, why I avoided using it after some testing. But yes, it forces 256 byte-writes (which is the max write size for the flash chip) as far as I have seen.

Thanks for the hint to #783 (I guess this is still offtopic here) - I will check out lfs_fs_gc next.

[Alessandro Francesconi]

Make sure to always do a readystatus-check before any operation and a WEL=1 cmd before a prog-call, though

Hi @CSC-Sendance , what do you mean in this line? Are they both recommendations to be addressed into lfs_bd_xxx functions?
In particular:

• "always do a readystatus-check before any operation" ... how? Should I call flash.syncDevice() ? Isn't it automatically called by LittleFS by means of lfs_bd_sync, when needed?
• "WEL=1 cmd before a prog-call": what is it?

[Clemens Schartmüller]

Make sure to always do a readystatus-check before any operation and a WEL=1 cmd before a prog-call, though

Hi @CSC-Sendance , what do you mean in this line? Are they both recommendations to be addressed into lfs_bd_xxx functions? In particular:

• "always do a readystatus-check before any operation" ... how? Should I call flash.syncDevice() ? Isn't it automatically called by LittleFS by means of lfs_bd_sync, when needed?
• "WEL=1 cmd before a prog-call": what is it?

Hi,

These were hints in case you want to use the Adafruit_FlashTransport_SPI classes directly, without the additional Adafruit_SPIFlash layer on top. If you still use the latter, Adafruit_SPIFlash already does (blocking-)wait for the flash's BUSY-flags to clear (with its method waitUntilReady()) and also sets the "Write Enable Latch" to 1 (writeEnable()).

Here is a sample implementation of the user-defined read/prog/erase/sync methods" for littlefs when using Adafruit_FlashTransport_SPI directly (writeEanble()andwaitUntilReady() are the same as in [Adafruit_SPIFlashBase.cpp`](https://github.com/adafruit/Adafruit_SPIFlash/blob/master/src/Adafruit_SPIFlashBase.cpp)).

static int _internal_flash_read(const struct lfs_config *c, lfs_block_t block, lfs_off_t off, void *buffer, lfs_size_t size)
{
    (void)c;
    waitUntilReady();
    uint32_t addr = lba2addr(block) + off;
    int ret = flashTransport.readMemory(addr, (uint8_t *)buffer, size) ? 0 : LFS_ERR_CORRUPT;
    return ret;
}

uint8_t singleProgBuf[EXTFLASH_PROGSIZE];
static int _internal_flash_prog(const struct lfs_config *c, lfs_block_t block, lfs_off_t off, const void *buffer, lfs_size_t size)
{
    size_t remaining = size;
    uint32_t addr = lba2addr(block) + off;
    uint32_t wr_offset = 0;
    while (remaining)
    {
        waitUntilReady();
        writeEnable();
        uint32_t wr_bytes = min(EXTFLASH_PROGSIZE, remaining); // write max of 256 bytes at once
        memcpy(singleProgBuf, buffer + wr_offset, wr_bytes);
        int ret = flashTransport.writeMemory(addr + wr_offset, (const uint8_t *)singleProgBuf, wr_bytes) ? 0 : LFS_ERR_CORRUPT;
        if (ret)
        {
            return LFS_ERR_CORRUPT;
        }
        wr_offset += wr_bytes;
        remaining -= wr_bytes;
    }
    return 0;
}

static int _internal_flash_erase(const struct lfs_config *c, lfs_block_t block)
{
    (void)c;
    uint32_t addr = lba2addr(block);
    waitUntilReady();
    writeEnable();
    int ret = flashTransport.eraseCommand(SFLASH_CMD_ERASE_SECTOR, addr); // ERASE command has to fit the littlefs-block size (i.e. sector erase for 4kB block, full 64kB-block erase for a 64kb block
    return ret ? 0 : LFS_ERR_CORRUPT;
}

static int _internal_flash_sync(const struct lfs_config *c)
{
    (void)c;
    return 0; // we always sync directly with no cache
}