Doubts with DMA coherency mode and quirk-mmap

[ALIBERA]

Hi,
First thanks for your fantastic work maintaining this driver!
I have a doubt about DMA coherency mode. Does this mode maintains the coherence by hardware?

In the u-dma-buf.c linux driver, in the mmap function i see that SYNC_MODE_DMACOHERENT sets vma->vm_page_prot = _PGPROT_DMACOHERENT(vma->vm_page_prot), which is exactly the same setting than in the SYNC_MODE_WRITECOMBINE vma->vm_page_prot = _PGPROT_DMACOHERENT(vma->vm_page_prot), as:
#define _PGPROT_WRITECOMBINE(vm_page_prot) pgprot_writecombine(vm_page_prot)
#define _PGPROT_DMACOHERENT(vm_page_prot) pgprot_writecombine(vm_page_prot)

I have not clear why is it needed to have these two modes enabled in the mmap() linux driver´s function. Pls could you help me to understand this mode?

I also have a doubt about quirk-mmap. You say that The Linux Kernel mainline turns off caching when doing mmap() for architectures such as ARM and ARM64 where cache aliasing problems can occur. Does it mean that if you are using a hardware coherence architecture that is using ARM64 CPUs, then dma_mmap_coherent() makes the memory uncached even if it is a hardware coherence architecture? This is a big issue, as many of the CPUs nowdays are ARM64.

[KAWAZOME Ichiro]

Thanks for the issue.

I have a doubt about DMA coherency mode. Does this mode maintains the coherence by hardware?

DMA coherent mode is the mode used in arm around the time of Linux Kernel 4, and it has little meaning today.
u-dma-buf only supports this legacy mode for backward compatibility.

I also have a doubt about quirk-mmap. You say that The Linux Kernel mainline turns off caching when doing mmap() for architectures such as ARM and ARM64 where cache aliasing problems can occur. Does it mean that if you are using a hardware coherence architecture that is using ARM64 CPUs, then dma_mmap_coherent() makes the memory uncached even if it is a hardware coherence architecture? This is a big issue, as many of the CPUs nowdays are ARM64.

The current Linux dma_mmap_coherent() will only cache memory on hardware coherence architectures.

dma_mmap_coherent() calls dma_direct_mmap(), which in turn uses dma_pgprot() to determine cache availability as follows

int dma_direct_mmap(struct device *dev, struct vm_area_struct *vma,
		void *cpu_addr, dma_addr_t dma_addr, size_t size,
		unsigned long attrs)
{
	unsigned long user_count = vma_pages(vma);
	unsigned long count = PAGE_ALIGN(size) >> PAGE_SHIFT;
	unsigned long pfn = PHYS_PFN(dma_to_phys(dev, dma_addr));
	int ret = -ENXIO;

	vma->vm_page_prot = dma_pgprot(dev, vma->vm_page_prot, attrs);

	if (dma_mmap_from_dev_coherent(dev, vma, cpu_addr, size, &ret))
		return ret;
	if (dma_mmap_from_global_coherent(vma, cpu_addr, size, &ret))
		return ret;

	if (vma->vm_pgoff >= count || user_count > count - vma->vm_pgoff)
		return -ENXIO;
	return remap_pfn_range(vma, vma->vm_start, pfn + vma->vm_pgoff,
			user_count << PAGE_SHIFT, vma->vm_page_prot);
}

dma_pgprot() is as follows

#ifdef CONFIG_MMU
/*
 * Return the page attributes used for mapping dma_alloc_* memory, either in
 * kernel space if remapping is needed, or to userspace through dma_mmap_*.
 */
pgprot_t dma_pgprot(struct device *dev, pgprot_t prot, unsigned long attrs)
{
	if (force_dma_unencrypted(dev))
		prot = pgprot_decrypted(prot);
	if (dev_is_dma_coherent(dev))
		return prot;
#ifdef CONFIG_ARCH_HAS_DMA_WRITE_COMBINE
	if (attrs & DMA_ATTR_WRITE_COMBINE)
		return pgprot_writecombine(prot);
#endif
	return pgprot_dmacoherent(prot);
}
#endif /* CONFIG_MMU */

Note that the macro dev_is_dma_coherent() is used here.
dma_pgprot() does nothing if dev_is_dma_coherent() is true.
dma_pgprot() returns the return value of pgprot_dmacoherent() if dev_is_dma_coherent() is false.
And pgprot_dmacoherent() returns an architecture-dependent value.
If the architecture is ARM64, pgprot_dmacoherent() returns the same value as pgprot_writecombine().

On the other hand, the macro dev_is_dma_coherent() is also used to determine if a hardware coherence architecture is present.
One of the functions controlling the cache, dma_sync_single_for_device(), calls dma_direct_sync_single_for_device(), which is is as follows.

static inline void dma_direct_sync_single_for_device(struct device *dev,
		dma_addr_t addr, size_t size, enum dma_data_direction dir)
{
	phys_addr_t paddr = dma_to_phys(dev, addr);

	if (unlikely(is_swiotlb_buffer(dev, paddr)))
		swiotlb_sync_single_for_device(dev, paddr, size, dir);

	if (!dev_is_dma_coherent(dev))
		arch_sync_dma_for_device(paddr, size, dir);
}

Again, the macro dev_is_dma_coherent() is used.
dma_direct_sync_single_for_device() does nothing if dev_is_dma_coherent() is true.
dma_direct_sync_single_for_device() performs architecture-dependent cache operations if dev_is_dma_coherent() is false.
If the architecture is ARM64, arch_sync_dma_for_device() will flush the cache.

That is, if dev_is_dma_coherent() is true, dma_mmap_coherent() enables the cache but does not perform any software cache operations (i.e., it must be a hardware coherence architecture).
Conversely, if dev_is_dma_coherent() is false, dma_mmap_coherent() will disable the cache.

This creates a bit of a problem.
This is the case for architectures that are not hardware coherence architectures, but do not have cache aliasing problems.
Since there is no cache aliasing problem, you want to turn on the cache during mmap(), but since it is not a hardware coherence architecture, the cache is forced to be turned off.

For example, the Cortex-A9, one of the ARM implementations, and the Cortex-A53, one of the ARM64 implementations, clearly state in their documentation that their Data Cache implementation is PIPT (Physically Indexed Physically Tagged).
PIPT does not cause cache aliasing problems.
In other words, in these implementation examples, the cache should be turned on during mmap() since there is no cache aliasing problem with respect to the DMA buffer.
However, if the architecture is not hardware coherence, the cache is forced to be turned off.

To address this issue, u-dma-buf has a quirk-mmap mode.
When quirk-mmap is enabled, u-dma-buf does not use dma_mmap_coherent(), but uses its own mmap().
This custom mmap() sets the cache independently of the return value of dev_is_dma_coherent().

[ALIBERA]

Thanks a lot for your great explanation!

I still have some doubts:

1. If the user sets for example SYNC_MODE_NONCACHED, then vm_page_prot is set to “not cached”. If you have a hardware coherence architecture, then you do not need to use quirk-mmap mode, so you execute dma_mmap_coherent() function. In the dma_mmap_coherent() function, dma_pgprot() returns prot if it is a hardware coherent architecture, but just imagine that this value was previously set to non-cached by error. I understand that the user should not use any of the case methods of udmabuf_device_file_mmap() if the architecture is hardware coherent and he added “dma-coherent” in the device tree, but maybe it could be included a return error value if the user does an error selecting any of the case combinations and adding “dma-coherent” in the device tree.

Also, if the user sets SYNC_MODE_NONCACHED and does not use quirk-mmap mode, then dma_pgprot() in the dma_mmap_coherent() will select pgprot_writecombine(prot) or pgprot_dmacoherent(prot), overwriting pgprot_noncached(vm_page_prot) selected in the SYNC_MODE_NONCACHED mode . Probably I am missing something, pls could you clarify to me?

2. Concerning quirk-mmap, you say that this mode uses its own mmap(), but I see that it only calls to udmabuf_mmap_vma_open(vma). I do not find where this custom mmap() sets the cache independently of the return value of dev_is_dma_coherent(). Could you indicate me where it is done?

Thanks,
Alberto

[KAWAZOME Ichiro]

1. If the user sets for example SYNC_MODE_NONCACHED, ...

As you pointed out, currently there remain many unknown settings for sync-mode when not in quirk-mmap mode.
If you wish to configure cache settings by sync-mode, please use quirk-mmap mode as much as possible.

2. Concerning quirk-mmap, you sa that this mode uses its own mmap(), but ...

Note that during udmabuf_mmap_vma_open(), udmabuf_mmap_vm_ops is overwritten in vma->vm_ops.
And the .fault of udmabuf_mmap_vm_ops is set to udmabuf_mmap_vma_fault().
This is a technique called on-demand paging.
This allows the equivalent of mmap() to be performed without changing the value of vma->vm_page_prot.

[ALIBERA]

I have it all clear now! thanks for your patience and your explanations!
Best Regards,
Alberto

[KAWAZOME Ichiro]

Congratulations on your issue being resolved.
I guess I'm closing this issue.