system/devices/cxl.rst

6 software running on a QEMU emulated host or to the internals of
19 in how the devices are emulated in QEMU. In most cases the various
26   - Configuration space access
27   - BAR mapped memory accesses used for registers and mailboxes.
28   - MSI/MSI-X
29   - AER
30   - DOE mailboxes
31   - IDE
32   - Many other PCI express defined interfaces..
36   - Equivalent of accessing DRAM / NVDIMMs. Any access / feature
40 * Cache operations. The are mostly irrelevant to QEMU emulation as
41   QEMU is not emulating a coherency protocol. Any emulation related
46 --------------------
64 Scope of CXL emulation in QEMU
65 ------------------------------
75 CXL System components
76 ----------------------
77 A CXL system is made up a Host with a number of 'standard components'
78 the control and capabilities of which are discoverable by system software
87 This information is available to system software, when making
92 Note: QTG 0 is the only one currently supported in QEMU.
99 space is described to system software via a CXL Host Bridge
139 by creating an upstream port (cxl-upstream) and a number of
140 downstream ports on the internal switch bus (cxl-downstream).
142 CXL Memory Devices - Type 3
145 by a generic operating system driver. They have HDM decoders
151 ---------------------
153 are emulated in QEMU, let us consider a memory read in a fully configured
154 CXL topology.  Note that system software is responsible for configuration
155 of all components with the exception of the CFMWs. System software is
158 for system RAM.
160 Example system topology. x marks the match in each decoder level::
162   |<------------------SYSTEM PHYSICAL ADDRESS MAP (1)----------------->|
209     ranges of the system physical address map.  Each CFMW has
227     CXL Type 3 0 (as part of a 2 way interleave at the system level
229     HDM4 is used to enable system wide 4 way interleave across all
247   |<------------------SYSTEM PHYSICAL ADDRESS MAP (1)----------------->|
282             ---------------------------------------------------
302 ---------------------
305   qemu-system-x86_64 -M q35,cxl=on -m 4G,maxmem=8G,slots=8 -smp 4 \
307   -object memory-backend-file,id=cxl-mem1,share=on,mem-path=/tmp/cxltest.raw,size=256M \
308   -object memory-backend-file,id=cxl-lsa1,share=on,mem-path=/tmp/lsa.raw,size=256M \
309   -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \
310   -device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2 \
311   -device cxl-type3,bus=root_port13,persistent-memdev=cxl-mem1,lsa=cxl-lsa1,id=cxl-pmem0,sn=0x1 \
312   -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G
316   qemu-system-x86_64 -M q35,cxl=on -m 4G,maxmem=8G,slots=8 -smp 4 \
318   -object memory-backend-ram,id=vmem0,share=on,size=256M \
319   -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \
320   -device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2 \
321   -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,id=cxl-vmem0 \
322   -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G
326   qemu-system-x86_64 -M q35,cxl=on -m 4G,maxmem=8G,slots=8 -smp 4 \
328   -object memory-backend-ram,id=vmem0,share=on,size=256M \
329   -object memory-backend-file,id=cxl-lsa0,share=on,mem-path=/tmp/lsa.raw,size=256M \
330   -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \
331   -device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2 \
332   -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,lsa=cxl-lsa0,id=cxl-vmem0 \
333   -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G
339   qemu-system-x86_64 -M q35,cxl=on -m 4G,maxmem=8G,slots=8 -smp 4 \
341   -object memory-backend-file,id=cxl-mem1,share=on,mem-path=/tmp/cxltest.raw,size=256M \
342   -object memory-backend-file,id=cxl-mem2,share=on,mem-path=/tmp/cxltest2.raw,size=256M \
343   -object memory-backend-file,id=cxl-mem3,share=on,mem-path=/tmp/cxltest3.raw,size=256M \
344   -object memory-backend-file,id=cxl-mem4,share=on,mem-path=/tmp/cxltest4.raw,size=256M \
345   -object memory-backend-file,id=cxl-lsa1,share=on,mem-path=/tmp/lsa.raw,size=256M \
346   -object memory-backend-file,id=cxl-lsa2,share=on,mem-path=/tmp/lsa2.raw,size=256M \
347   -object memory-backend-file,id=cxl-lsa3,share=on,mem-path=/tmp/lsa3.raw,size=256M \
348   -object memory-backend-file,id=cxl-lsa4,share=on,mem-path=/tmp/lsa4.raw,size=256M \
349   -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \
350   -device pxb-cxl,bus_nr=222,bus=pcie.0,id=cxl.2 \
351   -device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2 \
352   -device cxl-type3,bus=root_port13,persistent-memdev=cxl-mem1,lsa=cxl-lsa1,id=cxl-pmem0,sn=0x1 \
353   -device cxl-rp,port=1,bus=cxl.1,id=root_port14,chassis=0,slot=3 \
354   -device cxl-type3,bus=root_port14,persistent-memdev=cxl-mem2,lsa=cxl-lsa2,id=cxl-pmem1,sn=0x2 \
355   -device cxl-rp,port=0,bus=cxl.2,id=root_port15,chassis=0,slot=5 \
356   -device cxl-type3,bus=root_port15,persistent-memdev=cxl-mem3,lsa=cxl-lsa3,id=cxl-pmem2,sn=0x3 \
357   -device cxl-rp,port=1,bus=cxl.2,id=root_port16,chassis=0,slot=6 \
358   -device cxl-type3,bus=root_port16,persistent-memdev=cxl-mem4,lsa=cxl-lsa4,id=cxl-pmem3,sn=0x4 \
359 …-M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.targets.1=cxl.2,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-gran…
363   qemu-system-x86_64 -M q35,cxl=on -m 4G,maxmem=8G,slots=8 -smp 4 \
365   -object memory-backend-file,id=cxl-mem0,share=on,mem-path=/tmp/cxltest.raw,size=256M \
366   -object memory-backend-file,id=cxl-mem1,share=on,mem-path=/tmp/cxltest1.raw,size=256M \
367   -object memory-backend-file,id=cxl-mem2,share=on,mem-path=/tmp/cxltest2.raw,size=256M \
368   -object memory-backend-file,id=cxl-mem3,share=on,mem-path=/tmp/cxltest3.raw,size=256M \
369   -object memory-backend-file,id=cxl-lsa0,share=on,mem-path=/tmp/lsa0.raw,size=256M \
370   -object memory-backend-file,id=cxl-lsa1,share=on,mem-path=/tmp/lsa1.raw,size=256M \
371   -object memory-backend-file,id=cxl-lsa2,share=on,mem-path=/tmp/lsa2.raw,size=256M \
372   -object memory-backend-file,id=cxl-lsa3,share=on,mem-path=/tmp/lsa3.raw,size=256M \
373   -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \
374   -device cxl-rp,port=0,bus=cxl.1,id=root_port0,chassis=0,slot=0 \
375   -device cxl-rp,port=1,bus=cxl.1,id=root_port1,chassis=0,slot=1 \
376   -device cxl-upstream,bus=root_port0,id=us0 \
377   -device cxl-downstream,port=0,bus=us0,id=swport0,chassis=0,slot=4 \
378   -device cxl-type3,bus=swport0,persistent-memdev=cxl-mem0,lsa=cxl-lsa0,id=cxl-pmem0,sn=0x1 \
379   -device cxl-downstream,port=1,bus=us0,id=swport1,chassis=0,slot=5 \
380   -device cxl-type3,bus=swport1,persistent-memdev=cxl-mem1,lsa=cxl-lsa1,id=cxl-pmem1,sn=0x2 \
381   -device cxl-downstream,port=2,bus=us0,id=swport2,chassis=0,slot=6 \
382   -device cxl-type3,bus=swport2,persistent-memdev=cxl-mem2,lsa=cxl-lsa2,id=cxl-pmem2,sn=0x3 \
383   -device cxl-downstream,port=3,bus=us0,id=swport3,chassis=0,slot=7 \
384   -device cxl-type3,bus=swport3,persistent-memdev=cxl-mem3,lsa=cxl-lsa3,id=cxl-pmem3,sn=0x4 \
385   -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=4k
388 ------------
391 [persistent-memdev] attributes. [memdev] will default to a persistent memory
393 with [persistent-memdev].
396 ----------------------------
410 ----------
412  - Consortium website for specifications etc:
414  - Compute Express Link (CXL) Specification, Revision 3.1, August 2023