Giters

edsrzf / mmap-go

A portable mmap package for Go

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Can multiple golang binaries use the same memory mapped file?

shiskeyoffles opened this issue · comments

commented

My hope is I can share some data between two Golang binaries

Cannot use APIs since it has to be high-performance. I could use local REDIS but I'm looking for more performance and I'm hoping share memory might be the answer

  1. One binary read/writes
  2. Other binary only reads
commented

It should be possible for the Unix implementation. I'm less sure about the Windows implementation.

The Unix implementation specifies the MAP_SHARED flag, which means that if two processes map the same file into memory, they'll reference the same memory, and be able to see updates. You will need to synchronize access as usual, which could be a little trickier than usual since the two processes cannot share things like channels or mutexes. I believe atomics should still work normally.

From a very quick scan of Windows docs, it sounds like the two processes would need to specify the same lpName argument to CreateFileMapping in order to reference the same memory. Currently this package passes nil for lpName, so this isn't possible.

I haven't specifically built with this use case in mind, so am not familiar with all the details. If anyone would like to make improvements to documentation or support for these use cases, I'd be open to contributions.

commented

Thanks @edsrzf !

I have another question and it has to do with my lack of in-depth understanding of how Memory Mapped files work.

I understand that writing to a memory-mapped file does not immediately write to the physical disk unless it is "flushed".

Say using binary 1 I write data into memory mapped file

How soon will it be available for binary 2 for reading?

Is it

  1. Immediate like a database
  2. Or does binary take noticeable time to load? In which case this would be a deal breaker for the current scenario 😃

This leads me to another question, would both the binaries be actually reading from the same memory addresses, or will they have their own copy of the file in memory?

commented

I believe MAP_SHARED means that the two processes reference the same physical memory, but probably through different virtual addresses. There's only one copy. This would mean updates are visible "immediately" (again, with proper memory synchronization).

Whether or not changes are flushed to storage is a separate question.