[knem-devel] "unexisting region cookie" error

Brice Goglin Brice.Goglin at inria.fr
Tue Jul 12 17:38:14 CEST 2011

Le 12/07/2011 17:20, bin Wang a écrit :
> hello Brice,
> one more question.
> What's the max memory size that could be registered into KNEM kernel
> module? I didn't find any info on the webpage, so I assume that
> there is no limitation. As long as userspace application can get 
> memory allocation from OS successfully, KNEM is able to register
> it for RDMA.

Yes, there is no such limit.

Note that the application can get memory allocation from the OS
successfully (virtual memory) but some failure may occur later when the
OS actually allocates the corresponding physical memory. If you allocate
virtual memory without touching it, it may not be physically allocated
immediately by the operating system. In this case, registering a memory
region will allocate the relevant physical memory. If you have some
memory pressure on your machine, the actual physical memory allocation
failure may thus happen during knem memory registration (the create
ioctl will return -1).

> Besides, KNEM just registers the memory addresses, not the contents 
> in the memory buffer, which means that once a reading process gets a
> multiple-use cookie, it could always read from that buffer even if 
> the buffer has already been updated after registration. I should
> guarantee in my own code that the buffer should not be updated before 
> its data has been copied by some other reading processes.
> Is my understanding correct?

Yes. It's very similar to RDMA semantics (but we have nothing like
fences or any other synchronization primitives).


More information about the knem-devel mailing list