Giters

ikwzm / udmabuf

User space mappable dma buffer device driver for Linux.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Setting dma-coherent on devices created through /dev/u-dma-buf-mgr

lucasbrasilino opened this issue · comments

commented

Hi.

I'm using u-dma-buf on a Zynq MPSoC Ultrascale+ board (ZCU102). I'm trying to make the allocated DMA buffer to be cache coherent while using Exclusive access (atomic operations).
My design connects using the HPC0 port, and I'm driving AxCACHE/AxUSER/AxPROT/AxLOCK signals. My application allocates DMA buffer on demand, at startup, using the /dev/u-dma-buf-mgr. When buffer is created, I see dma_coherent is 0 (using /sys/... file).
Is there a way to set dma_coherent to 1 upon buffer allocation ? I'm not using device tree because looks like it already creates buffers of a pre-defined size.

For now, I was able to enable write coherency: when the PL writes to memory (using AWCACHE[3:0] = 4'b1100), PS access data immediately. When PS writes data, PL reads the previous value written by the PL, not by the PS.

thanks.

commented

Thank you for the issue.

At the moment, u-dma-buf is configured by u-dma-buf-mgr only by device name and buffer capacity. Originally u-dma-buf was designed with the device tree in mind. Settings by other methods are still incomplete.

Thank you for your feedback.

commented

By the way, you said that your design uses HPC0. Do you use "Broadcasting Inner Shareable"? "Zynq UltraScale MPSoC Cache Coherency" has the following description.

https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18842098/Zynq+UltraScale+MPSoC+Cache+Coherency

5.2 Broadcasting Inner Shareable
This method alters a register of MPSoC to enable inner shareable transactions to be broadcast. The brdc_inner bit of the lpd_apu register in the LPD_SLCR module must be written while the APU is in reset. The requirement to alter the register while the APU is in reset can be accomplished in multiple manners.

For Linux, cache coherency cannot be guaranteed unless ZynqMP is configured as described in "Broadcasting Inner Shareable". The dma-coherent property is just for the Linux kernel to check if DMA guarantees cache coherence in hardware. It does not configure DMA in hardware to guarantee cache coherency.

commented

Yes, I did enable broadcasting inner shareable. BTW, I'm using exclusive access (atomic memory updates - AxLOCK = 1'b1) and trying to make them coherent. The behavior I see is quite different from the described in the AXI specs.

Since the main problem is when PS atomically updates cached memory and PL can't see the updates, I also tried to enable the Enable_snoops bit from CCI's Snoop_Control_Register_S3 as in here: https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18842098/Zynq+UltraScale+MPSoC+Cache+Coherency#ZynqUltraScaleMPSoCCacheCoherency-6Snooping

The other way around works fine: when PL updates cached memory, PS can see the updates (coherent).

I've posted more details in Xilinx forum here: https://bit.ly/3tng81y

Thanks for the help.

commented

I've posted more details in Xilinx forum here: https://bit.ly/3tng81y

I accessed the URL provided, but it was "Message Not Found".