system/devices/cxl.rst

14 with CXL Host Bridges, which have CXL Root Ports which may be directly
26   - Configuration space access
27   - BAR mapped memory accesses used for registers and mailboxes.
28   - MSI/MSI-X
29   - AER
30   - DOE mailboxes
31   - IDE
32   - Many other PCI express defined interfaces..
36   - Equivalent of accessing DRAM / NVDIMMs. Any access / feature
46 --------------------
65 ------------------------------
71 as Root Complex Integrated End Points. CXL rev 2.0 looks a lot
76 ----------------------
109   may be a mapping to a single Root Port (RP) or across a set of
112 CXL Root Ports (CXL RP)
114 A CXL Root Port serves the same purpose as a PCIe Root Port.
139 by creating an upstream port (cxl-upstream) and a number of
140 downstream ports on the internal switch bus (cxl-downstream).
142 CXL Memory Devices - Type 3
151 ---------------------
162   |<------------------SYSTEM PHYSICAL ADDRESS MAP (1)----------------->|
179               | PCI/CXL Root Bus 0c |    | PCI/CXL Root Bus 0d |
188    (3)|  Root Port 0  | | Root Port 1 | | Root Port 2| | Root Port 3 |
215 (2) **Two CXL Host Bridges**. Each of these has 2 CXL Root Ports and
237 (3) **Four CXL Root Ports.** In this case the CXL Type 3 devices are
247   |<------------------SYSTEM PHYSICAL ADDRESS MAP (1)----------------->|
264               | PCI/CXL Root Bus 0c |    | PCI/CXL Root Bus 0d |
272       |  Root Port 0  |
282             ---------------------------------------------------
302 ---------------------
305   qemu-system-x86_64 -M q35,cxl=on -m 4G,maxmem=8G,slots=8 -smp 4 \
307   -object memory-backend-file,id=cxl-mem1,share=on,mem-path=/tmp/cxltest.raw,size=256M \
308   -object memory-backend-file,id=cxl-lsa1,share=on,mem-path=/tmp/lsa.raw,size=256M \
309   -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \
310   -device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2 \
311   -device cxl-type3,bus=root_port13,persistent-memdev=cxl-mem1,lsa=cxl-lsa1,id=cxl-pmem0,sn=0x1 \
312   -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G
316   qemu-system-x86_64 -M q35,cxl=on -m 4G,maxmem=8G,slots=8 -smp 4 \
318   -object memory-backend-ram,id=vmem0,share=on,size=256M \
319   -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \
320   -device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2 \
321   -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,id=cxl-vmem0 \
322   -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G
326   qemu-system-x86_64 -M q35,cxl=on -m 4G,maxmem=8G,slots=8 -smp 4 \
328   -object memory-backend-ram,id=vmem0,share=on,size=256M \
329   -object memory-backend-file,id=cxl-lsa0,share=on,mem-path=/tmp/lsa.raw,size=256M \
330   -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \
331   -device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2 \
332   -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,lsa=cxl-lsa0,id=cxl-vmem0 \
333   -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G
336 interleave across 2 CXL host bridges.  Each host bridge has 2 CXL Root Ports, with
339   qemu-system-x86_64 -M q35,cxl=on -m 4G,maxmem=8G,slots=8 -smp 4 \
341   -object memory-backend-file,id=cxl-mem1,share=on,mem-path=/tmp/cxltest.raw,size=256M \
342   -object memory-backend-file,id=cxl-mem2,share=on,mem-path=/tmp/cxltest2.raw,size=256M \
343   -object memory-backend-file,id=cxl-mem3,share=on,mem-path=/tmp/cxltest3.raw,size=256M \
344   -object memory-backend-file,id=cxl-mem4,share=on,mem-path=/tmp/cxltest4.raw,size=256M \
345   -object memory-backend-file,id=cxl-lsa1,share=on,mem-path=/tmp/lsa.raw,size=256M \
346   -object memory-backend-file,id=cxl-lsa2,share=on,mem-path=/tmp/lsa2.raw,size=256M \
347   -object memory-backend-file,id=cxl-lsa3,share=on,mem-path=/tmp/lsa3.raw,size=256M \
348   -object memory-backend-file,id=cxl-lsa4,share=on,mem-path=/tmp/lsa4.raw,size=256M \
349   -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \
350   -device pxb-cxl,bus_nr=222,bus=pcie.0,id=cxl.2 \
351   -device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2 \
352   -device cxl-type3,bus=root_port13,persistent-memdev=cxl-mem1,lsa=cxl-lsa1,id=cxl-pmem0,sn=0x1 \
353   -device cxl-rp,port=1,bus=cxl.1,id=root_port14,chassis=0,slot=3 \
354   -device cxl-type3,bus=root_port14,persistent-memdev=cxl-mem2,lsa=cxl-lsa2,id=cxl-pmem1,sn=0x2 \
355   -device cxl-rp,port=0,bus=cxl.2,id=root_port15,chassis=0,slot=5 \
356   -device cxl-type3,bus=root_port15,persistent-memdev=cxl-mem3,lsa=cxl-lsa3,id=cxl-pmem2,sn=0x3 \
357   -device cxl-rp,port=1,bus=cxl.2,id=root_port16,chassis=0,slot=6 \
358   -device cxl-type3,bus=root_port16,persistent-memdev=cxl-mem4,lsa=cxl-lsa4,id=cxl-pmem3,sn=0x4 \
359 …-M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.targets.1=cxl.2,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-gran…
363   qemu-system-x86_64 -M q35,cxl=on -m 4G,maxmem=8G,slots=8 -smp 4 \
365   -object memory-backend-file,id=cxl-mem0,share=on,mem-path=/tmp/cxltest.raw,size=256M \
366   -object memory-backend-file,id=cxl-mem1,share=on,mem-path=/tmp/cxltest1.raw,size=256M \
367   -object memory-backend-file,id=cxl-mem2,share=on,mem-path=/tmp/cxltest2.raw,size=256M \
368   -object memory-backend-file,id=cxl-mem3,share=on,mem-path=/tmp/cxltest3.raw,size=256M \
369   -object memory-backend-file,id=cxl-lsa0,share=on,mem-path=/tmp/lsa0.raw,size=256M \
370   -object memory-backend-file,id=cxl-lsa1,share=on,mem-path=/tmp/lsa1.raw,size=256M \
371   -object memory-backend-file,id=cxl-lsa2,share=on,mem-path=/tmp/lsa2.raw,size=256M \
372   -object memory-backend-file,id=cxl-lsa3,share=on,mem-path=/tmp/lsa3.raw,size=256M \
373   -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \
374   -device cxl-rp,port=0,bus=cxl.1,id=root_port0,chassis=0,slot=0 \
375   -device cxl-rp,port=1,bus=cxl.1,id=root_port1,chassis=0,slot=1 \
376   -device cxl-upstream,bus=root_port0,id=us0 \
377   -device cxl-downstream,port=0,bus=us0,id=swport0,chassis=0,slot=4 \
378   -device cxl-type3,bus=swport0,persistent-memdev=cxl-mem0,lsa=cxl-lsa0,id=cxl-pmem0,sn=0x1 \
379   -device cxl-downstream,port=1,bus=us0,id=swport1,chassis=0,slot=5 \
380   -device cxl-type3,bus=swport1,persistent-memdev=cxl-mem1,lsa=cxl-lsa1,id=cxl-pmem1,sn=0x2 \
381   -device cxl-downstream,port=2,bus=us0,id=swport2,chassis=0,slot=6 \
382   -device cxl-type3,bus=swport2,persistent-memdev=cxl-mem2,lsa=cxl-lsa2,id=cxl-pmem2,sn=0x3 \
383   -device cxl-downstream,port=3,bus=us0,id=swport3,chassis=0,slot=7 \
384   -device cxl-type3,bus=swport3,persistent-memdev=cxl-mem3,lsa=cxl-lsa3,id=cxl-pmem3,sn=0x4 \
385   -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=4k
388 ------------
391 [persistent-memdev] attributes. [memdev] will default to a persistent memory
393 with [persistent-memdev].
396 ----------------------------
410 ----------
412  - Consortium website for specifications etc:
414  - Compute Express Link (CXL) Specification, Revision 3.1, August 2023