[knem-devel] how to write INT value with KNEM "atomically"

Brice Goglin Brice.Goglin at inria.fr
Thu Sep 29 14:28:58 CEST 2011

Le 29/09/2011 14:15, George Bosilca a écrit :
>> Totally random example: Open-MX doesn't do anything to make
>> kernel-written data actually visible to user-space. It just adds a write
>> memory barrier to ensure that the notification flag is written at the
>> end. But there's no flush and even no return from syscall in most
>> receive cases.
> But then on what KNEM is different? If you don't have a memory barrier/fence before raising the flag to signal the end of the copy, then upon return from the knem function the user data might be wrong. It look like we already have to have a memory barrier at the end of all the iovec copies in order to have consistent behavior.

You can only compare Open-MX and KNEM when you use this new model of
doing a KNEM put with both the data and the notification flag at the
same time. KNEM would need a barrier/fence just like Open-MX already does.
We never needed this in KNEM earlier because we never used KNEM to
transmit the notification flag together with the data. We always waited
for the KNEM copy ioctl to finish (returning from syscalls does a
barrier/fence) and then we send an explicit notification message to
other side if needed.


More information about the knem-devel mailing list