You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
PR #218 added the ability to dispatch a kernel on host when CudaUVM is used, by creating a team policy on host. However, CudaUVM is still a device memory space, with behind-the-scenes sync between host and device. In particular, this means that a View with memory space HostSpace cannot be assigned to a view with memory space CudaUVM, and viceversa. This can have bad consequences for codes that use host mirrors, and are templated on the memory space. For instance, the following won't work:
since the host mirror still has CudaUVM as mem space (b/c uvm is accessible from host), but CudaUVM is not assignable to HostSpace. On the other hand, CudaHostPinnedSpace is both assignable and accessible from device and host. The snippet above would work with CudaHostPinnedSpace in place of CudaUVM.
However, notice that nothing in the team policy requires to know what memory space the views that will be accessed will be on. In fact, the concept of "view" is orthogonal to that of a team policy. All that the team policy needs to know is the execution space where the kernel will be run on. We should therefore remove the requirement that CudaUVM is used, and allow the user to create a policy on host regardless. It is up to the user to ensure that the views used in the kernel are accessible on host.
The drawback would be that Kokkos does not offer a configuration option to make CudaHostPinnedSpace the default mem space for the Cuda execution space (while it does so for CudaUVM). Therefore, any usage of ExecSpace::memory_space would yield something different from the host pinned space.
The text was updated successfully, but these errors were encountered:
PR #218 added the ability to dispatch a kernel on host when CudaUVM is used, by creating a team policy on host. However, CudaUVM is still a device memory space, with behind-the-scenes sync between host and device. In particular, this means that a View with memory space
HostSpace
cannot be assigned to a view with memory spaceCudaUVM
, and viceversa. This can have bad consequences for codes that use host mirrors, and are templated on the memory space. For instance, the following won't work:since the host mirror still has CudaUVM as mem space (b/c uvm is accessible from host), but CudaUVM is not assignable to HostSpace. On the other hand, CudaHostPinnedSpace is both assignable and accessible from device and host. The snippet above would work with
CudaHostPinnedSpace
in place ofCudaUVM
.However, notice that nothing in the team policy requires to know what memory space the views that will be accessed will be on. In fact, the concept of "view" is orthogonal to that of a team policy. All that the team policy needs to know is the execution space where the kernel will be run on. We should therefore remove the requirement that CudaUVM is used, and allow the user to create a policy on host regardless. It is up to the user to ensure that the views used in the kernel are accessible on host.
The drawback would be that Kokkos does not offer a configuration option to make
CudaHostPinnedSpace
the default mem space for theCuda
execution space (while it does so forCudaUVM
). Therefore, any usage ofExecSpace::memory_space
would yield something different from the host pinned space.The text was updated successfully, but these errors were encountered: