dankwiki

CUBAR: Difference between revisions

← Older edit

Dank (talk | contribs)

2 user groups
11,325 edits
No edit summary
 
(13 intermediate revisions by the same user not shown)
Line 1: Line 1:
[[File:TheG8.png|right|thumb|CUDA'd-up beyond all repair]]
[[File:TheG8.png|right|thumb|CUDA'd-up beyond all repair]]
[[CUDA]] (and [http://en.wikipedia.org/wiki/GPGPU General-Purpose Graphics Processing Unit] programming in general) is rapidly becoming a mainstay of high-performance computing. As CUDA and OpenCL move off of the workstation, and into the server -- off of the console, and into the cluster -- the security of these systems will become critical parts of the associated [http://en.wikipedia.org/wiki/Trusted_computing_base trusted computing base]. Even ignoring the issue of multiuser security, the properties of isolation and (to a lesser extent) confidentiality are important for debugging, profiling and reproducibility. I've authored the <s>cudafucker</s> CUBAR set of tools to investigate the security properties -- primarily the means and parameters of memory protection, and the division of protection between soft- and hardware -- of [[CUDA]] on NVIDIA hardware since the G80 architecture.
[[CUDA]] (and [http://en.wikipedia.org/wiki/GPGPU General-Purpose Graphics Processing Unit] programming in general) is rapidly becoming a mainstay of high-performance computing. As CUDA and OpenCL move off of the workstation, and into the server -- off of the console, and into the cluster -- the security of these systems will become critical parts of the associated [http://en.wikipedia.org/wiki/Trusted_computing_base trusted computing base]. Even ignoring the issue of multiuser security, the properties of isolation and (to a lesser extent) confidentiality are important for debugging, profiling and reproducibility. I've authored the CUBAR set of tools to investigate the security properties -- primarily the means and parameters of memory protection, and the division of protection between soft- and hardware -- of [[CUDA]] on NVIDIA hardware since the G80 architecture. This was essential information in the attempt to implement an open source CUDA software stack (see [[libcudest]]).


See our 2010 paper, "[[Media:Cubar2010.pdf|My Other Computer is Your GPU]]."
==Questions==
==Questions==
===Memory details===
===Memory details===
Line 38: Line 39:
===Protection===
===Protection===
* What mechanisms, if any, exist to protect memory? At what granularities (of address and access) do they operate?
* What mechanisms, if any, exist to protect memory? At what granularities (of address and access) do they operate?
** What about registers? CUDA doesn't do traditional context switches due to register preallocation.
* How is memory protection split across hardware, kernelspace, and userspace?
* How is memory protection split across hardware, kernelspace, and userspace?
** Any userspace protection can, of course, be trivially subverted
** Any userspace protection can, of course, be trivially subverted
Line 44: Line 46:
* What memories, if any, are scrubbed between kernels' execution?
* What memories, if any, are scrubbed between kernels' execution?
* How many different regions can be tracked? How many contexts? What behavior exists at these limits?
* How many different regions can be tracked? How many contexts? What behavior exists at these limits?
===Variation===
===Variation===
* Have these mechanisms changed over various hardware?
* Have these mechanisms changed over various hardware?
Line 72: Line 75:


==Code==
==Code==
* GitHub project: [http://github.com/dankamongmen/wdp/tree/master/cs8803ss-project/ dankamongmen:wdp/cs8803ss-project]
* GitHub project: [http://github.com/dankamongmen/cubar dankamongmen:cubar]
===Building===
===Building===
Clone the [[git]] repository. Define (either via shell export or within a new file Makefile.local):
Clone the [[git]] repository. Define (either via shell export or within a new file Makefile.local):
Line 79: Line 82:


==Tools==
==Tools==
[[File:Cudash-hyperboxl.png|right|thumb|cudash in use]]
===CUBAR tools===
===CUBAR tools===
Only those tools which launch CUDA kernels use the Runtime API. They might be converted to the Driver API.
Only those tools which launch CUDA kernels use the Runtime API. They might be converted to the Driver API.
Line 92: Line 96:
* <tt>cudaranger</tt> - Attempts to perform selectable memory accesses across the specified virtual address range of a device. Calculates the number of non-zero words.
* <tt>cudaranger</tt> - Attempts to perform selectable memory accesses across the specified virtual address range of a device. Calculates the number of non-zero words.
It is necessary that <tt>cudadump</tt> spawn <tt>cudaranger</tt> rather than launching the kernels itself; a kernel failure results in all libcuda calls returning an early error (CUDA_ERROR_DEINITIALIZED (4) or CUDA_ERROR_LAUNCH_FAILURE (700)) for the remainder of a process's life (ie, creating a new context usually also fails '''FIXME: provide proof''').
It is necessary that <tt>cudadump</tt> spawn <tt>cudaranger</tt> rather than launching the kernels itself; a kernel failure results in all libcuda calls returning an early error (CUDA_ERROR_DEINITIALIZED (4) or CUDA_ERROR_LAUNCH_FAILURE (700)) for the remainder of a process's life (ie, creating a new context usually also fails '''FIXME: provide proof''').
====cudash====
The CUDA Shell (currently making use of the runtime api for kernel launches) facilitates stateful, invasive examination of the CUDA memory/process system. It has online help via the "help" command.
* ''FIXME'': document commands
===Other tools===
===Other tools===
* [http://nouveau.freedesktop.org/wiki/REnouveau REnouveau], the Reverse-Engineering-for-nouveau tool
* [http://nouveau.freedesktop.org/wiki/REnouveau REnouveau], the Reverse-Engineering-for-nouveau tool
Line 107: Line 114:
** but that's [http://www.eecs.berkeley.edu/~demmel/ James Demmel] (aka a Real American Badass) for you!
** but that's [http://www.eecs.berkeley.edu/~demmel/ James Demmel] (aka a Real American Badass) for you!
* "[http://www.eecg.toronto.edu/~myrto/gpuarch-ispass2010.pdf Demystifying the GPU Through Benchmarking]" continues with this theme.
* "[http://www.eecg.toronto.edu/~myrto/gpuarch-ispass2010.pdf Demystifying the GPU Through Benchmarking]" continues with this theme.
* LWN's State of Nouveau articles ([http://lwn.net/Articles/269558 Part 1] and [http://lwn.net/Articles/270830/ Part 2]
* LWN's State of Nouveau articles ([http://lwn.net/Articles/269558 Part 1] and [http://lwn.net/Articles/270830/ Part 2])
* The [http://nouveau.freedesktop.org/wiki/IrcChatLogs Nouveau Development Companion]
* The [http://nouveau.freedesktop.org/wiki/IrcChatLogs Nouveau Development Companion]
* This Japanese [http://pc.watch.impress.co.jp/docs/2008/0617/kaigai446.htm article] has some great diagrams
* This Japanese [http://pc.watch.impress.co.jp/docs/2008/0617/kaigai446.htm article] has some great diagrams
* [[libcudest]], my effort to reverse engineer and reimplement libcuda.so
[[CATEGORY: GPGPU]]
[[CATEGORY: Projects]]
Retrieved from "https://nick-black.com/dankwiki/index.php?title=CUBAR"