Documentation/filesystems/xfs-delayed-logging-design.txt

4 Introduction to Re-logging in XFS
8 such as inodes and dquots, are logged in logical format where the details
9 logged are made up of the changes to in-core structures rather than on-disk
18 modifications to a single object to be carried in the log at any given time.
23 changes in the new transaction that is written to the log.
26 written to disk after change D, we would see in the log the following series
39 In other words, each time an object is relogged, the new transaction contains
40 the aggregation of all the previous changes currently held only in the log.
42 This relogging technique also allows objects to be moved forward in the log so
44 moving forward.  This can be seen in the table above by the changing
46 direct encoding of the location in the log of the transaction.
54 keeps relogging the inode and btree buffers as they get modified in each
55 removal operation. This keeps them moving forward in the log as the operation
71 doing aggregation of transactions in memory - batching them, if you like - to
80 that can be made to the filesystem at any point in time - if all the log
92 multiple times before they are committed to disk in the log buffers. If we
94 transactions A through D are committed to disk in the same log buffer.
98 contains all the changes from the previous changes. In other words, we have one
99 necessary copy in the log buffer, and three stale copies that are simply
101 objects, these "stale objects" can be over 90% of the space used in the log
106 From a conceptual point of view, XFS is already doing relogging in memory (where
109 infrastructure to keep track of logical changes in memory prior to physically
110 formatting the changes in a transaction to the log buffer. Hence we cannot avoid
111 accumulating stale objects in the log buffers.
114 changes to objects in memory outside the log buffer infrastructure. Because of
117 tracked in the current infrastructure. The big problem is how to accumulate
118 them and get them to the log in a consistent, recoverable manner.
124 metadata changes from the size and number of log buffers available. In other
126 written to the log at any point in time, there may be a much greater amount
127 being accumulated in memory. Hence the potential for loss of metadata on a
131 will result in a consistent filesystem. What it does mean is that as far as the
139 of accumulating changes in memory for some period before writing them to the
140 log is used effectively in many filesystems including ext3 and ext4. Hence
141 no time is spent in this document trying to convince the reader that the
143 such implementing it in XFS is purely an exercise in software engineering.
145 The fundamental requirements for delayed logging in XFS are simple:
178 vector array that points to the changed regions in the item. The log write code
180 transaction commit while the item is locked in the transaction. Instead of
186 the changes in a format that is compatible with the log buffer writing code.
189 resulting in a vector that is transactionally consistent and can be accessed
194 formatting method and the delayed logging formatting can be seen in the
230 are in the item, so we'd need a new encapsulation method for regions in the log
233 region headers in the formatting stage, which is problematic as there is per
239 handled in exactly the same manner as the existing log vectors are handled.
241 relogged in memory.
246 Now that we can record transactional changes in memory in a form that allows
248 them so that they can be written to the log at some later point in time.  The
251 exist once the object has been included in a transaction.
255 and as such are stored in the Active Item List (AIL) which is a LSN-ordered
258 that is in the AIL can be relogged, which causes the object to be pinned again
259 and then moved forward in the AIL when the log buffer IO completes for that
262 Essentially, this shows that an item that is in the AIL can still be modified
265 can we store state in any field that is protected by the AIL lock. Hence the
266 committed item tracking needs it's own locks, lists and state fields in the log
272 in transaction commit order, so when an object is relogged it is removed from
273 it's place in the list and re-inserted at the tail. This is entirely arbitrary
274 and done to make it easy for debugging - the last items in the list are the
276 transactional integrity (as discussed in the next section) so the ordering is
283 all the items in the CIL must be written into the log via the log buffers.
284 We need to write these items in the order that they exist in the CIL, and they
287 log replay - all the changes in all the objects in a given transaction must
289 a transaction is not replayed because it is not complete in the log, then
292 To fulfill this requirement, we need to write the entire CIL in a single log
297 be at least one complete transaction in the log at any given time. If a
300 only complete previous transaction in the log. This will result in a recovery
308 bigger with a lot more items in it. The worst case effect of this is that we
331 checkpoints to be written into the log buffers in the case of log force heavy
333 requires that we strictly order the commit records in the log so that
343 checkpoint context so that the log item can be released. In diagrammatic form,
397 and unpin) in the log vector chain and then free the log vector chain and
402 it. The fact that we walk the log items (in the CIL) just to chain the log
409 vectors in one checkpoint transaction. I'd guess this is a "measure and
411 is in the dev tree....
419 committed to the log. In the rare case that a dependent operation occurs (e.g.
430 As discussed in the checkpoint section, delayed logging uses per-checkpoint
461 need to check that all the prior contexts in the committing list are also
463 synchronisation in the log force code so that we don't need to wait anywhere
470 and push if required. Indeed, placing the current sequence checkpoint flush in
483 still need to reserve the space in the log for the checkpoint.
485 A typical transaction reserves enough space in the log for the worst case space
488 etc. as well as the actual space for all the changed metadata in the
491 of log vectors in the transaction).
497 10,000 inodes, we have about 1.5MB of metadata to write in 20,000 vectors. Each
498 vector is 12 bytes, so the total to be logged is approximately 1.75MB. In
500 each, so we in 1.5MB of directory buffers we'd have roughly 400 buffers and a
508 reservation by tracking the space currently used by the object in the CIL and
509 then calculating the increase or decrease in space used as the object is
530 space available in the log if we are to use static reservations, and that is
535 items in the CIL and using this to dynamically calculate the amount of log
538 the difference in space required is removed from the transaction that causes
540 available in their reservation for this as they have already reserved the
542 will always be less than or equal to the maximal amount in the reservation.
558 If the transaction subsystem goes idle while we still have items in the CIL,
561 allow the idle log to be covered (effectively marked clean) in exactly the same
573 buffers. Hence items that are relogged in the log buffers will have a pin count
574 for every outstanding transaction they were dirtied in. When each of these
581 completion relationship. Every time an object is relogged in the CIL it goes
589 insertion into the CIL, unpin on checkpoint completion". In other words, the
591 pin the object the first time it is inserted into the CIL - if it is already in
601 the fact pinning the item is dependent on whether the item is present in the
617 points in the design - the three important ones are:
626 the amount of space available in the log for their reservations. The practical
627 limit here is in the order of several hundred concurrent transactions for a
628 128MB log, which means that it is generally one per CPU in a machine.
634 are in progress). Ultimately a two pass algorithm where the formatting is done
640 want every other CPU in the machine spinning on the CIL lock. Given that
644 main reason for choosing a sleeping lock even though nothing in either the
698 		Record modifications in log item
700 		Pin item in memory
726 independent of steps 8-9. An item can be locked in steps 1-6 or steps 8-9
728 at the same time. If the log item is in the AIL or between steps 6 and 7
743 		Record modifications in log item
745 		Pin item in memory if not pinned in CIL
781 logging methods are in the middle of the life cycle - they still have the same
782 beginning and end and execution constraints. The only differences are in the