Lines Matching +full:non +full:- +full:overlap +full:- +full:time
1 .. SPDX-License-Identifier: GPL-2.0
9 :Authors: - Fenghua Yu <fenghua.yu@intel.com>
10 - Tony Luck <tony.luck@intel.com>
11 - Vikas Shivappa <vikas.shivappa@intel.com>
38 # mount -t resctrl resctrl [-o cdp[,cdpl2][,mba_MBps][,debug]] /sys/fs/resctrl
57 pseudo-locking is a unique way of using cache control to "pin" or
59 "Cache Pseudo-Locking".
96 own settings for cache use which can over-ride
128 Corresponding region is pseudo-locked. No
131 Indicates if non-contiguous 1s value in CBM is supported.
136 Non-contiguous 1s value in CBM is supported.
155 non-linear. This field is purely informational
166 "per-thread":
216 5 Reads to slow memory in the non-local NUMA domain
218 3 Non-temporal writes to non-local NUMA domain
219 2 Non-temporal writes to local NUMA domain
220 1 Reads to memory in the non-local NUMA domain
262 counter can be considered for re-use.
275 mask f7 has non-consecutive 1-bits
332 When the resource group is in pseudo-locked mode this file will
334 pseudo-locked region.
345 Each resource has its own line and format - see below for details.
356 cache pseudo-locked region is created by first writing
357 "pseudo-locksetup" to the "mode" file before writing the cache
358 pseudo-locked region's schemata to the resource group's "schemata"
359 file. On successful pseudo-locked region creation the mode will
360 automatically change to "pseudo-locked".
384 -------------------------
389 1) If the task is a member of a non-default group, then the schemata
399 -------------------------
400 1) If a task is a member of a MON group, or non-default CTRL_MON group
421 are evicted and re-used while the occupancy in the new group rises as
436 max_threshold_occupancy - generic concepts
437 ------------------------------------------
442 occupancy has gone down. If there is a time when system has a lot of
443 limbo RMIDs but which are not ready to be used, user may see an -EBUSY
449 Schemata files - general concepts
450 ---------------------------------
456 ---------
468 ---------------------
475 0x3, 0x6 and 0xC are legal 4-bit masks with two bits set, but 0x5, 0x9
477 if non-contiguous 1s value is supported. On a system with a 20-bit mask
542 ----------------------------------------------------------------
548 ------------------------------------------------------------------
556 ------------------------
569 ------------------------------------------
577 ---------------------------------------------
585 ---------------------------------------
604 ---------------------------------
619 --------------------------------------------------
639 --------------------------------------------------------------------
658 Cache Pseudo-Locking
661 application can fill. Cache pseudo-locking builds on the fact that a
662 CPU can still read and write data pre-allocated outside its current
663 allocated area on a cache hit. With cache pseudo-locking, data can be
666 pseudo-locked memory is made accessible to user space where an
670 The creation of a cache pseudo-locked region is triggered by a request
672 to be pseudo-locked. The cache pseudo-locked region is created as follows:
674 - Create a CAT allocation CLOSNEW with a CBM matching the schemata
675 from the user of the cache region that will contain the pseudo-locked
676 memory. This region must not overlap with any current CAT allocation/CLOS
677 on the system and no future overlap with this cache region is allowed
678 while the pseudo-locked region exists.
679 - Create a contiguous region of memory of the same size as the cache
681 - Flush the cache, disable hardware prefetchers, disable preemption.
682 - Make CLOSNEW the active CLOS and touch the allocated memory to load
684 - Set the previous CLOS as active.
685 - At this point the closid CLOSNEW can be released - the cache
686 pseudo-locked region is protected as long as its CBM does not appear in
687 any CAT allocation. Even though the cache pseudo-locked region will from
689 any CLOS will be able to access the memory in the pseudo-locked region since
691 - The contiguous region of memory loaded into the cache is exposed to
692 user-space as a character device.
694 Cache pseudo-locking increases the probability that data will remain
698 “locked” data from cache. Power management C-states may shrink or
699 power off cache. Deeper C-states will automatically be restricted on
700 pseudo-locked region creation.
702 It is required that an application using a pseudo-locked region runs
704 with the cache on which the pseudo-locked region resides. A sanity check
705 within the code will not allow an application to map pseudo-locked memory
707 pseudo-locked region resides. The sanity check is only done during the
711 Pseudo-locking is accomplished in two stages:
714 of cache that should be dedicated to pseudo-locking. At this time an
717 2) During the second stage a user-space application maps (mmap()) the
718 pseudo-locked memory into its address space.
720 Cache Pseudo-Locking Interface
721 ------------------------------
722 A pseudo-locked region is created using the resctrl interface as follows:
725 2) Change the new resource group's mode to "pseudo-locksetup" by writing
726 "pseudo-locksetup" to the "mode" file.
727 3) Write the schemata of the pseudo-locked region to the "schemata" file. All
731 On successful pseudo-locked region creation the "mode" file will contain
732 "pseudo-locked" and a new character device with the same name as the resource
734 by user space in order to obtain access to the pseudo-locked memory region.
736 An example of cache pseudo-locked region creation and usage can be found below.
738 Cache Pseudo-Locking Debugging Interface
739 ----------------------------------------
740 The pseudo-locking debugging interface is enabled by default (if
744 location is present in the cache. The pseudo-locking debugging interface uses
746 the pseudo-locked region:
750 example below). In this test the pseudo-locked region is traversed at
758 When a pseudo-locked region is created a new debugfs directory is created for
760 write-only file, pseudo_lock_measure, is present in this directory. The
761 measurement of the pseudo-locked region depends on the number written to this
782 In this example a pseudo-locked region named "newlock" was created. Here is
816 In this example a pseudo-locked region named "newlock" was created on the L2
829 # _-----=> irqs-off
830 # / _----=> need-resched
831 # | / _---=> hardirq/softirq
832 # || / _--=> preempt-depth
834 # TASK-PID CPU# |||| TIMESTAMP FUNCTION
836 pseudo_lock_mea-1672 [002] .... 3132.860500: pseudo_lock_l2: hits=4097 miss=0
849 # mount -t resctrl resctrl /sys/fs/resctrl
866 allocations can overlap or not. The allocations specifies the maximum
882 Again two sockets, but this time with a more realistic 20-bit mask.
884 Two real time tasks pid=1234 running on processor 0 and pid=5678 running on
885 processor 1 on socket 0 on a 2-socket and dual core machine. To avoid noisy
886 neighbors, each of the two real-time tasks exclusively occupies one quarter
890 # mount -t resctrl resctrl /sys/fs/resctrl
899 Next we make a resource group for our first real time task and give
906 Finally we move our first real time task into this resource group. We
913 # taskset -cp 1 1234
915 Ditto for the second real time task (with the remaining 25% of cache)::
920 # taskset -cp 2 5678
926 For our first real time task this would request 20% memory b/w on socket 0.
929 # echo -e "L3:0=f8000;1=fffff\nMB:0=20;1=100" > p0/schemata
931 For our second real time task this would request an other 20% memory b/w
935 # echo -e "L3:0=f8000;1=fffff\nMB:0=20;1=100" > p0/schemata
939 A single socket system which has real-time tasks running on core 4-7 and
940 non real-time workload assigned to core 0-3. The real-time tasks share text
946 # mount -t resctrl resctrl /sys/fs/resctrl
955 Next we make a resource group for our real time cores and give it access
963 Finally we move core 4-7 over to the new group and make sure that the
965 also get 50% of memory bandwidth assuming that the cores 4-7 are SMT
966 siblings and only the real time threads are scheduled on the cores 4-7.
976 to overlap with that allocation.
979 system with two L2 cache instances that can be configured with an 8-bit
984 # mount -t resctrl resctrl /sys/fs/resctrl/
994 fail because of the overlap with the schemata of the default group::
1001 -sh: echo: write error: Invalid argument
1005 To ensure that there is no overlap with another resource group the default
1018 A new resource group will on creation not overlap with an exclusive resource
1033 A resource group cannot be forced to overlap with an exclusive resource group::
1036 -sh: echo: write error: Invalid argument
1040 Example of Cache Pseudo-Locking
1042 Lock portion of L2 cache from cache id 1 using CBM 0x3. Pseudo-locked
1047 # mount -t resctrl resctrl /sys/fs/resctrl/
1050 Ensure that there are bits available that can be pseudo-locked, since only
1051 unused bits can be pseudo-locked the bits to be pseudo-locked needs to be
1060 Create a new resource group that will be associated with the pseudo-locked
1061 region, indicate that it will be used for a pseudo-locked region, and
1062 configure the requested pseudo-locked region capacity bitmask::
1065 # echo pseudo-locksetup > newlock/mode
1068 On success the resource group's mode will change to pseudo-locked, the
1069 bit_usage will reflect the pseudo-locked region, and the character device
1070 exposing the pseudo-locked region will exist::
1073 pseudo-locked
1076 # ls -l /dev/pseudo_lock/newlock
1077 crw------- 1 root root 243, 0 Apr 3 05:01 /dev/pseudo_lock/newlock
1082 * Example code to access one page of pseudo-locked cache region
1095 * cores associated with the pseudo-locked region. Here the cpu
1132 /* Application interacts with pseudo-locked memory @mapping */
1146 ----------------------------
1154 1. Read the cbmmasks from each directory or the per-resource "bit_usage"
1185 $ flock -s /sys/fs/resctrl/ find /sys/fs/resctrl
1189 $ cat create-dir.sh
1191 mask = function-of(output.txt)
1195 $ flock /sys/fs/resctrl/ ./create-dir.sh
1214 exit(-1);
1226 exit(-1);
1238 exit(-1);
1247 if (fd == -1) {
1249 exit(-1);
1263 ----------------------
1270 ------------------------------------------------------------------------
1274 # mount -t resctrl resctrl /sys/fs/resctrl
1314 --------------------------------------------
1317 # mount -t resctrl resctrl /sys/fs/resctrl
1334 ---------------------------------------------------------------------
1345 # mount -t resctrl resctrl /sys/fs/resctrl
1368 Example 4 (Monitor real time tasks)
1369 -----------------------------------
1371 A single socket system which has real time tasks running on cores 4-7
1372 and non real time tasks on other cpus. We want to monitor the cache
1373 occupancy of the real time threads on these cores.
1376 # mount -t resctrl resctrl /sys/fs/resctrl
1380 Move the cpus 4-7 over to p1::
1393 -----------------------------------------------------------------
1408 +---------------+---------------+---------------+-----------------+
1410 +---------------+---------------+---------------+-----------------+
1412 +---------------+---------------+---------------+-----------------+
1414 +---------------+---------------+---------------+-----------------+
1416 +---------------+---------------+---------------+-----------------+
1418 +---------------+---------------+---------------+-----------------+
1420 +---------------+---------------+---------------+-----------------+
1422 +---------------+---------------+---------------+-----------------+
1424 +---------------+---------------+---------------+-----------------+
1426 +---------------+---------------+---------------+-----------------+
1428 +---------------+---------------+---------------+-----------------+
1430 +---------------+---------------+---------------+-----------------+
1432 +---------------+---------------+---------------+-----------------+
1434 +---------------+---------------+---------------+-----------------+
1436 +---------------+---------------+---------------+-----------------+
1438 +---------------+---------------+---------------+-----------------+
1440 +---------------+---------------+---------------+-----------------+
1442 +---------------+---------------+---------------+-----------------+
1444 +---------------+---------------+---------------+-----------------+
1446 +---------------+---------------+---------------+-----------------+
1448 +---------------+---------------+---------------+-----------------+
1450 +---------------+---------------+---------------+-----------------+
1452 +---------------+---------------+---------------+-----------------+
1454 +---------------+---------------+---------------+-----------------+
1456 +---------------+---------------+---------------+-----------------+
1458 +---------------+---------------+---------------+-----------------+
1460 +---------------+---------------+---------------+-----------------+
1462 +---------------+---------------+---------------+-----------------+
1464 +---------------+---------------+---------------+-----------------+
1472 …958/https://www.intel.com/content/www/us/en/processors/xeon/scalable/xeon-scalable-spec-update.html
1474 2. Erratum BDF102 in Intel Xeon E5-2600 v4 Processor Product Family Specification Update:
1475 …w.intel.com/content/dam/www/public/us/en/documents/specification-updates/xeon-e5-v4-spec-update.pdf
1478 …are.intel.com/content/www/us/en/develop/articles/intel-resource-director-technology-rdt-reference-…