[knem-devel] (no subject)

Brice Goglin Brice.Goglin at inria.fr
Fri Feb 4 17:43:33 CET 2011


Le 04/02/2011 17:15, neuerrechner at gmx.de a écrit :
> knem 0.9.5
>  Driver ABI=0xd
>  Flags: forcing 0x0, ignoring 0x0
>  DMAEngine: KernelSupported Enabled NoChannelAvailable
>  Debug: NotBuilt
>  Requests Submitted          : 12240
>  Requests Processed/DMA      : 0
>  Requests Processed/Thread   : 0
>  Requests Processed/PinLocal : 0
>  Requests Failed/NoMemory    : 0
>  Requests Failed/ReadCmd     : 0
>  Requests Failed/FindRegion  : 0
>  Requests Failed/Pin         : 0
>  Requests Failed/MemcpyToUser: 0
>  Requests Failed/MemcpyPinned: 0
>  Requests Failed/DMACopy     : 0
>  Dmacpy Cleanup Timeout      : 0
>
> So it seems no requests have been processed. 
> What could be the problem here and why does it state "NoChannelAvailable"? 
>   

Hello Florian,

All your requests have been processed. As long as they don't fail (which
means they appear in "Failed/something"), all submitted requests are
processed fine. But there is no counter for requests that are processed
in the standard manner (no DMA, no offload to a thread, no local
pinning). I'll see if I can clarify this.

Are you sure that I/OAT DMA is enabled in your hardware? Sometimes it is
disabled in the BIOS. If it's enabled, you will see something like this
in lspci:

00:16.0 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 22)
00:16.1 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 22)

right after these "standard lines":

00:14.2 PIC: Intel Corporation 5520/5500/X58 I/O Hub Control Status and RAS Registers (rev 22)
00:14.3 PIC: Intel Corporation 5520/5500/X58 I/O Hub Throttle Registers (rev 22)

Anyway, I/OAT isn't very efficient compared to a standard memory copy in
recent processors. So it's probably better to not use it anyway. That's
why Open MPI doesn't tell KNEM to use I/OAT by default.

Brice




More information about the knem-devel mailing list