Merge tag 'docs-7.1' of git://git.kernel.org/pub/scm/linux/kernel/git/docs/linux

Pull documentation updates from Jonathan Corbet:
"A busier cycle than I had expected for docs, including:

- Tran

Merge tag 'docs-7.1' of git://git.kernel.org/pub/scm/linux/kernel/git/docs/linux

Pull documentation updates from Jonathan Corbet:
"A busier cycle than I had expected for docs, including:

- Translations: some overdue updates to the Japanese translations,
Chinese translations for some of the Rust documentation, and the
beginnings of a Portuguese translation.

- New documents covering CPU isolation, managed interrupts, debugging
Python gbb scripts, and more.

- More tooling work from Mauro, reducing docs-build warnings, adding
self tests, improving man-page output, bringing in a proper C
tokenizer to replace (some of) the mess of kernel-doc regexes, and
more.

- Update and synchronize changes.rst and scripts/ver_linux, and put
both into alphabetical order.

... and a long list of documentation updates, typo fixes, and general
improvements"

* tag 'docs-7.1' of git://git.kernel.org/pub/scm/linux/kernel/git/docs/linux: (162 commits)
Documentation: core-api: real-time: correct spelling
doc: Add CPU Isolation documentation
Documentation: Add managed interrupts
Documentation: seq_file: drop 2.6 reference
docs/zh_CN: update rust/index.rst translation
docs/zh_CN: update rust/quick-start.rst translation
docs/zh_CN: update rust/coding-guidelines.rst translation
docs/zh_CN: update rust/arch-support.rst translation
docs/zh_CN: sync process/2.Process.rst with English version
docs/zh_CN: fix an inconsistent statement in dev-tools/testing-overview
tracing: Documentation: Update histogram-design.rst for fn() handling
docs: sysctl: Add documentation for /proc/sys/xen/
Docs: hid: intel-ish-hid: make long URL usable
Documentation/kernel-parameters: fix architecture alignment for pt, nopt, and nobypass
sched/doc: Update yield_task description in sched-design-CFS
Documentation/rtla: Convert links to RST format
docs: fix typos and duplicated words across documentation
docs: fix typo in zoran driver documentation
docs: add an Assisted-by mention to submitting-patches.rst
Revert "scripts/checkpatch: add Assisted-by: tag validation"
...

show more ...

docs: c_lex.py: store logger on its data

By having the logger stored there, any code using CTokenizer can
log messages there.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed

docs: c_lex.py: store logger on its data

By having the logger stored there, any code using CTokenizer can
log messages there.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <467979dc18149e4b2a7113c178e0cb07919632f2.1774256269.git.mchehab+huawei@kernel.org>

show more ...

Merge branch 'mauro' into docs-mw

This series comes after:
https://lore.kernel.org/linux-doc/cover.1773770483.git.mchehab+huawei@kernel.org/

It basically contains patches I submitted before on

Merge branch 'mauro' into docs-mw

This series comes after:
https://lore.kernel.org/linux-doc/cover.1773770483.git.mchehab+huawei@kernel.org/

It basically contains patches I submitted before on a 40+ patch series,
but were less relevant, plus a couple of other minor fixes:

- patch 1 improves one of the CTokenizer unit test, fixing some
potential issues on it;
- patches 2 and 3 contain some improvement/fixes for Sphinx
Python autodoc extension. They basically document c_lex.py;

- The remaining patches:
- create a new class for kernel-doc config;
- fix some internal representations of KdocItem;
- add unit tests for KernelDoc() parser class;
- add support to output KdocItem in YAML, which is a
machine-readable output for all documented kAPI.

None of the patches should affect man or html output.

show more ...

docs: kdoc: add c_lex to generated documentation

Do some fixes at groups() description for it to be parsed by
Sphinx and add it to the documentation.

Signed-off-by: Mauro Carvalho Chehab <mchehab+h

docs: kdoc: add c_lex to generated documentation

Do some fixes at groups() description for it to be parsed by
Sphinx and add it to the documentation.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <799178cf30dd4022fdb1d029ba998a458e037b52.1773823995.git.mchehab+huawei@kernel.org>

show more ...

Merge branch 'mauro' into docs-mw

Mauro says:

This patch series change how kdoc parser handles macro replacements.

Instead of heavily relying on regular expressions that can sometimes
be very comp

Merge branch 'mauro' into docs-mw

Mauro says:

This patch series change how kdoc parser handles macro replacements.

Instead of heavily relying on regular expressions that can sometimes
be very complex, it uses a C lexical tokenizer. This ensures that
BEGIN/END blocks on functions and structs are properly handled,
even when nested.

Checking before/after the patch series, for both man pages and
rst only had:
- whitespace differences;
- struct_group macros now are shown as inner anonimous structs
as it should be.

Also, I didn't notice any relevant change on the documentation build
time. With that regards, right now, every time a CMatch replacement
rule takes in place, it does:

for each transform:
- tokenizes the source code;
- handle CMatch;
- convert tokens back to a string.

A possible optimization would be to do, instead:

- tokenizes source code;
- for each transform handle CMatch;
- convert tokens back to a string.

For now, I opted not do do it, because:

- too much changes on a single row;
- docs build time is taking ~3:30 minutes, which is
about the same time it ws taken before the changes;
- there is a very dirty hack inside function_xforms:
(KernRe(r"_noprof"), ""). This is meant to change
function prototypes instead of function arguments.

So, if ok for you, I would prefer to merge this one first. We can later
optimize kdoc_parser to avoid multiple token <-> string conversions.

-

One important aspect of this series is that it introduces unittests
for kernel-doc. I used it a lot during the development of this series,
to ensure that the changes I was doing were producing the expected
results. Tests are on two separate files that can be executed directly.

Alternatively, there is a run.py script that runs all of them (and
any other python script named tools/unittests/test_*.py"):

$ tools/unittests/run.py
test_cmatch:
TestSearch:
test_search_acquires_multiple: OK
test_search_acquires_nested_paren: OK
test_search_acquires_simple: OK
test_search_must_hold: OK
test_search_must_hold_shared: OK
test_search_no_false_positive: OK
test_search_no_function: OK
test_search_no_macro_remains: OK
TestSubMultipleMacros:
test_acquires_multiple: OK
test_acquires_nested_paren: OK
test_acquires_simple: OK
test_mixed_macros: OK
test_must_hold: OK
test_must_hold_shared: OK
test_no_false_positive: OK
test_no_function: OK
test_no_macro_remains: OK
TestSubSimple:
test_rise_early_greedy: OK
test_rise_multiple_greedy: OK
test_strip_multiple_acquires: OK
test_sub_count_parameter: OK
test_sub_mixed_placeholders: OK
test_sub_multiple_placeholders: OK
test_sub_no_placeholder: OK
test_sub_single_placeholder: OK
test_sub_with_capture: OK
test_sub_zero_placeholder: OK
TestSubWithLocalXforms:
test_functions_with_acquires_and_releases: OK
test_raw_struct_group: OK
test_raw_struct_group_tagged: OK
test_struct_group: OK
test_struct_group_attr: OK
test_struct_group_tagged_with_private: OK
test_struct_kcov: OK
test_vars_stackdepot: OK

test_tokenizer:
TestPublicPrivate:
test_balanced_inner_private: OK
test_balanced_non_greddy_private: OK
test_balanced_private: OK
test_no private: OK
test_unbalanced_inner_private: OK
test_unbalanced_private: OK
test_unbalanced_struct_group_tagged_with_private: OK
test_unbalanced_two_struct_group_tagged_first_with_private: OK
test_unbalanced_without_end_of_line: OK
TestTokenizer:
test_basic_tokens: OK
test_depth_counters: OK
test_mismatch_error: OK

Ran 47 tests

show more ...

docs: c_lex: setup a logger to report tokenizer issues

Report file that has issues detected via CMatch and CTokenizer.

This is done by setting up a logger that will be overriden by
kdoc_parser, whe

docs: c_lex: setup a logger to report tokenizer issues

Report file that has issues detected via CMatch and CTokenizer.

This is done by setting up a logger that will be overriden by
kdoc_parser, when used on it.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <903ad83ae176196a50444e66177a4f5bcdef5199.1773770483.git.mchehab+huawei@kernel.org>

show more ...

docs: c_lex: properly implement a sub() method for CMatch

Implement a sub() method to do what it is expected, parsing
backref arguments like \0, \1, \2, ...

Signed-off-by: Mauro Carvalho Chehab <mc

docs: c_lex: properly implement a sub() method for CMatch

Implement a sub() method to do what it is expected, parsing
backref arguments like \0, \1, \2, ...

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <dbc45b86db18783289d94cfdbba4b72792c47929.1773770483.git.mchehab+huawei@kernel.org>

show more ...

docs: kdoc: create a CMatch to match nested C blocks

The NextMatch code is complex, and will become even more complex
if we add there support for arguments.

Now that we have a tokenizer, we can use

docs: kdoc: create a CMatch to match nested C blocks

The NextMatch code is complex, and will become even more complex
if we add there support for arguments.

Now that we have a tokenizer, we can use a better solution,
easier to be understood.

Yet, to improve performance, it is better to make it use a
previously tokenized code, changing its ABI.

So, reimplement NextMatch using the CTokener class. Once it
is done, we can drop NestedMatch.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <fa818ea164216b17520b588e3f12b81499b76dd7.1773770483.git.mchehab+huawei@kernel.org>

show more ...

docs: add a C tokenizer to be used by kernel-doc

Handling C code purely using regular expressions doesn't work well.

Add a C tokenizer to help doing it the right way.

The tokenizer was written usi

docs: add a C tokenizer to be used by kernel-doc

Handling C code purely using regular expressions doesn't work well.

Add a C tokenizer to help doing it the right way.

The tokenizer was written using as basis the Python re documentation
tokenizer example from:
https://docs.python.org/3/library/re.html#writing-a-tokenizer

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <39787bb8022e10c65df40c746077f7f66d07ffed.1773770483.git.mchehab+huawei@kernel.org>

show more ...