[knem-devel] how to write INT value with KNEM "atomically"

Brice Goglin Brice.Goglin at inria.fr
Mon Sep 26 13:39:26 CEST 2011


Thanks George!

By the way, would anybody be interested in having a new KNEM copy flag
saying:
    "add a memory barrier between iovec writes during a single copy"
The idea would be that if you post a KNEM request to copy buffer A +
flag B (two iovecs) at the same time, you have the guarantee that A is
done when B gets actually copied.

Brice




Le 26/09/2011 13:35, George Bosilca a écrit :
> Valentin,
>
> You can extend your knem-based communication framework with a shared
> memory region. You do the large data transfers with knem, everything
> else will go through the shared memory region (coordination messages
> as well as short data transfers). You can order the memory writes
> using the memory flush instructions, and this work globally, including
> the areas copied using knem.
>
>   George.
>
>
>
> On Sep 26, 2011, at 5:04, valentin <valentin.petrov at itseez.com
> <mailto:valentin.petrov at itseez.com>> wrote:
>
>> Ok. In fact, that's the answer i'd expect. Nevertheless, thanks a lot
>> for you help!
>>
>> 26.09.2011 12:31, Brice Goglin пишет:
>>> Hello Valentin,
>>> KNEM writes data using memcpy-like functions. The way those
>>> functions actually move bytes depend a lot on the kernel version,
>>> architecture and whether we're in the beginning/middle/end of the
>>> actual copy. It may copy from 1 to 16bytes at once, and maybe
>>> 32bytes with AVX. So it's very hard to us to have any idea of what's
>>> going on underneath (without loosing performance). I am not even
>>> sure that memcpy is guaranteed to copy in order (endianness would
>>> also be a problem for INTs anyway). Memory barriers may be needed.
>>> So I am afraid I cannot help you much here.
>>>
>>> You could ask KNEM to write your INT + a special flag byte for
>>> notification. When using vectorial memory regions, KNEM writes
>>> iovecs in order, but again I am not sure whether memory barriers may
>>> be needed to guarantee that you're target process will actually see
>>> those writes in order.
>>>
>>> Brice
>>>
>>>
>>>
>>>
>>>
>>> Le 26/09/2011 10:17, valentin a écrit :
>>>> Hello All,
>>>> i'm trying to use KNEM to perform a one-sided put operation. A
>>>> process A puts an integer value to Process B. Process B waits
>>>> actively while the int value at the appropriate address is changed
>>>> and uses this value after that. My problem is that KNEM writes data
>>>> by bytes (as far as i see it now). So the Process B continues to
>>>> work (leaves active wait) once the first byte in the int variable
>>>> is changed, despite the fact that the whole new value has not been
>>>> entirely written  yet. So, is there some way to enable KNEM to
>>>> perform such kind of task, i.e. to write data corresponding to the
>>>> predefined types (like INT) "atomically"?
>>>> I would greatly appreciate any help!
>>>>
>>>> -- 
>>>> BR, Valentin Petrov
>>>>
>>>>
>>>> _______________________________________________
>>>> knem-devel mailing list
>>>> knem-devel at lists.gforge.inria.fr <mailto:knem-devel at lists.gforge.inria.fr>
>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/knem-devel
>>>
>>
>> -- 
>> BR, Valentin Petrov
>> _______________________________________________
>> knem-devel mailing list
>> knem-devel at lists.gforge.inria.fr
>> <mailto:knem-devel at lists.gforge.inria.fr>
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/knem-devel

-------------- section suivante --------------
Une pi�ce jointe HTML a �t� nettoy�e...
URL: <http://lists.gforge.inria.fr/pipermail/knem-devel/attachments/20110926/a02203ef/attachment.htm>


More information about the knem-devel mailing list