About Kernel Documentation Linux Kernel Contact Linux Resources Linux Blog

Documentation / accelerators / ocxl.rst




Custom Search

Based on kernel version 4.16.1. Page generated on 2018-04-09 11:52 EST.

1	========================================================
2	OpenCAPI (Open Coherent Accelerator Processor Interface)
3	========================================================
4	
5	OpenCAPI is an interface between processors and accelerators. It aims
6	at being low-latency and high-bandwidth. The specification is
7	developed by the `OpenCAPI Consortium <http://opencapi.org/>`_.
8	
9	It allows an accelerator (which could be a FPGA, ASICs, ...) to access
10	the host memory coherently, using virtual addresses. An OpenCAPI
11	device can also host its own memory, that can be accessed from the
12	host.
13	
14	OpenCAPI is known in linux as 'ocxl', as the open, processor-agnostic
15	evolution of 'cxl' (the driver for the IBM CAPI interface for
16	powerpc), which was named that way to avoid confusion with the ISDN
17	CAPI subsystem.
18	
19	
20	High-level view
21	===============
22	
23	OpenCAPI defines a Data Link Layer (DL) and Transaction Layer (TL), to
24	be implemented on top of a physical link. Any processor or device
25	implementing the DL and TL can start sharing memory.
26	
27	::
28	
29	  +-----------+                         +-------------+
30	  |           |                         |             |
31	  |           |                         | Accelerated |
32	  | Processor |                         |  Function   |
33	  |           |  +--------+             |    Unit     |  +--------+
34	  |           |--| Memory |             |    (AFU)    |--| Memory |
35	  |           |  +--------+             |             |  +--------+
36	  +-----------+                         +-------------+
37	       |                                       |
38	  +-----------+                         +-------------+
39	  |    TL     |                         |    TLX      |
40	  +-----------+                         +-------------+
41	       |                                       |
42	  +-----------+                         +-------------+
43	  |    DL     |                         |    DLX      |
44	  +-----------+                         +-------------+
45	       |                                       |
46	       |                   PHY                 |
47	       +---------------------------------------+
48	
49	
50	
51	Device discovery
52	================
53	
54	OpenCAPI relies on a PCI-like configuration space, implemented on the
55	device. So the host can discover AFUs by querying the config space.
56	
57	OpenCAPI devices in Linux are treated like PCI devices (with a few
58	caveats). The firmware is expected to abstract the hardware as if it
59	was a PCI link. A lot of the existing PCI infrastructure is reused:
60	devices are scanned and BARs are assigned during the standard PCI
61	enumeration. Commands like 'lspci' can therefore be used to see what
62	devices are available.
63	
64	The configuration space defines the AFU(s) that can be found on the
65	physical adapter, such as its name, how many memory contexts it can
66	work with, the size of its MMIO areas, ...
67	
68	
69	
70	MMIO
71	====
72	
73	OpenCAPI defines two MMIO areas for each AFU:
74	
75	* the global MMIO area, with registers pertinent to the whole AFU.
76	* a per-process MMIO area, which has a fixed size for each context.
77	
78	
79	
80	AFU interrupts
81	==============
82	
83	OpenCAPI includes the possibility for an AFU to send an interrupt to a
84	host process. It is done through a 'intrp_req' defined in the
85	Transaction Layer, specifying a 64-bit object handle which defines the
86	interrupt.
87	
88	The driver allows a process to allocate an interrupt and obtain its
89	64-bit object handle, that can be passed to the AFU.
90	
91	
92	
93	char devices
94	============
95	
96	The driver creates one char device per AFU found on the physical
97	device. A physical device may have multiple functions and each
98	function can have multiple AFUs. At the time of this writing though,
99	it has only been tested with devices exporting only one AFU.
100	
101	Char devices can be found in /dev/ocxl/ and are named as:
102	/dev/ocxl/<AFU name>.<location>.<index>
103	
104	where <AFU name> is a max 20-character long name, as found in the
105	config space of the AFU.
106	<location> is added by the driver and can help distinguish devices
107	when a system has more than one instance of the same OpenCAPI device.
108	<index> is also to help distinguish AFUs in the unlikely case where a
109	device carries multiple copies of the same AFU.
110	
111	
112	
113	Sysfs class
114	===========
115	
116	An ocxl class is added for the devices representing the AFUs. See
117	/sys/class/ocxl. The layout is described in
118	Documentation/ABI/testing/sysfs-class-ocxl
119	
120	
121	
122	User API
123	========
124	
125	open
126	----
127	
128	Based on the AFU definition found in the config space, an AFU may
129	support working with more than one memory context, in which case the
130	associated char device may be opened multiple times by different
131	processes.
132	
133	
134	ioctl
135	-----
136	
137	OCXL_IOCTL_ATTACH:
138	
139	  Attach the memory context of the calling process to the AFU so that
140	  the AFU can access its memory.
141	
142	OCXL_IOCTL_IRQ_ALLOC:
143	
144	  Allocate an AFU interrupt and return an identifier.
145	
146	OCXL_IOCTL_IRQ_FREE:
147	
148	  Free a previously allocated AFU interrupt.
149	
150	OCXL_IOCTL_IRQ_SET_FD:
151	
152	  Associate an event fd to an AFU interrupt so that the user process
153	  can be notified when the AFU sends an interrupt.
154	
155	OCXL_IOCTL_GET_METADATA:
156	
157	  Obtains configuration information from the card, such at the size of
158	  MMIO areas, the AFU version, and the PASID for the current context.
159	
160	
161	mmap
162	----
163	
164	A process can mmap the per-process MMIO area for interactions with the
165	AFU.
Hide Line Numbers
About Kernel Documentation Linux Kernel Contact Linux Resources Linux Blog

Information is copyright its respective author. All material is available from the Linux Kernel Source distributed under a GPL License. This page is provided as a free service by mjmwired.net.