| Server IP : 82.148.16.210 / Your IP : 216.73.216.59 Web Server : nginx/1.29.5 System : Linux mail.sarafai.ru 6.1.0-43-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.162-1 (2026-02-08) x86_64 User : root ( 0) PHP Version : 7.4.33 Disable Function : pcntl_alarm,pcntl_fork,pcntl_waitpid,pcntl_wait,pcntl_wifexited,pcntl_wifstopped,pcntl_wifsignaled,pcntl_wifcontinued,pcntl_wexitstatus,pcntl_wtermsig,pcntl_wstopsig,pcntl_signal,pcntl_signal_get_handler,pcntl_signal_dispatch,pcntl_get_last_error,pcntl_strerror,pcntl_sigprocmask,pcntl_sigwaitinfo,pcntl_sigtimedwait,pcntl_exec,pcntl_getpriority,pcntl_setpriority,pcntl_async_signals,pcntl_unshare, MySQL : OFF | cURL : ON | WGET : ON | Perl : ON | Python : ON | Sudo : ON | Pkexec : OFF Directory : /usr/share/doc/python3-docutils/docs/dev/ |
Upload File : |
<?xml version="1.0" encoding="utf-8" ?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" /> <title>Python Source Reader</title> <meta name="author" content="David Goodger" /> <meta name="date" content="2022-04-02" /> <meta name="copyright" content="This document has been placed in the public domain." /> <link rel="stylesheet" href="../../css/html4css1.css" type="text/css" /> </head> <body> <div class="header"> <a class="reference external" href="https://docutils.sourceforge.io">Docutils</a> | <a class="reference external" href="../index.html">Overview</a> | <a class="reference external" href="../index.html#project-fundamentals">About</a> | <a class="reference external" href="../index.html#user">Users</a> | <a class="reference external" href="../index.html#ref">Reference</a> | <a class="reference external" href="../index.html#howto">Developers</a> <hr class="header"/> </div> <div class="document" id="python-source-reader"> <h1 class="title">Python Source Reader</h1> <table class="docinfo" frame="void" rules="none"> <col class="docinfo-name" /> <col class="docinfo-content" /> <tbody valign="top"> <tr><th class="docinfo-name">Author:</th> <td>David Goodger</td></tr> <tr><th class="docinfo-name">Contact:</th> <td><a class="first last reference external" href="mailto:docutils-develop@lists.sourceforge.net">docutils-develop@lists.sourceforge.net</a></td></tr> <tr><th class="docinfo-name">Revision:</th> <td>9051</td></tr> <tr><th class="docinfo-name">Date:</th> <td>2022-04-02</td></tr> <tr><th class="docinfo-name">Copyright:</th> <td>This document has been placed in the public domain.</td></tr> </tbody> </table> <!-- Minimal menu bar for inclusion in documentation sources in ``docutils/docs/*/`` sub-diretories. Attention: this is not a standalone document. --> <p>This document explores issues around extracting and processing docstrings from Python modules.</p> <p>For definitive element hierarchy details, see the "Python Plaintext Document Interface DTD" XML document type definition, <a class="reference external" href="pysource.dtd">pysource.dtd</a> (which modifies the generic <a class="reference external" href="../ref/docutils.dtd">docutils.dtd</a>). Descriptions below list 'DTD elements' (XML 'generic identifiers' or tag names) corresponding to syntax constructs.</p> <div class="contents topic" id="contents"> <p class="topic-title">Contents</p> <ul class="simple"> <li><a class="reference internal" href="#model" id="toc-entry-1">Model</a></li> <li><a class="reference internal" href="#docstring-extractor" id="toc-entry-2">Docstring Extractor</a></li> <li><a class="reference internal" href="#interpreted-text" id="toc-entry-3">Interpreted Text</a></li> </ul> </div> <div class="section" id="model"> <h1><a class="toc-backref" href="#toc-entry-1">Model</a></h1> <p>The Python Source Reader ("PySource") model that's evolving in my mind goes something like this:</p> <ol class="arabic"> <li><p class="first">Extract the docstring/namespace <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a> tree from the module(s) and/or package(s).</p> <table class="docutils footnote" frame="void" id="footnote-1" rules="none"> <colgroup><col class="label" /><col /></colgroup> <tbody valign="top"> <tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[1]</a></td><td><p class="first last">See <a class="reference internal" href="#docstring-extractor">Docstring Extractor</a> below.</p> </td></tr> </tbody> </table> </li> <li><p class="first">Run the parser on each docstring in turn, producing a forest of doctrees (per nodes.py).</p> </li> <li><p class="first">Join the docstring trees together into a single tree, running transforms:</p> <ul class="simple"> <li>merge hyperlinks</li> <li>merge namespaces</li> <li>create various sections like "Module Attributes", "Functions", "Classes", "Class Attributes", etc.; see <a class="reference external" href="pysource.dtd">pysource.dtd</a></li> <li>convert the above special sections to ordinary doctree nodes</li> </ul> </li> <li><p class="first">Run transforms on the combined doctree. Examples: resolving cross-references/hyperlinks (including interpreted text on Python identifiers); footnote auto-numbering; first field list -> bibliographic elements.</p> <p>(Or should step 4's transforms come before step 3?)</p> </li> <li><p class="first">Pass the resulting unified tree to the writer/builder.</p> </li> </ol> <p>I've had trouble reconciling the roles of input parser and output writer with the idea of modes ("readers" or "directors"). Does the mode govern the transformation of the input, the output, or both? Perhaps the mode should be split into two.</p> <p>For example, say the source of our input is a Python module. Our "input mode" should be the "Python Source Reader". It discovers (from <tt class="docutils literal">__docformat__</tt>) that the input parser is "reStructuredText". If we want HTML, we'll specify the "HTML" output formatter. But there's a piece missing. What <em>kind</em> or <em>style</em> of HTML output do we want? PyDoc-style, LibRefMan style, etc. (many people will want to specify and control their own style). Is the output style specific to a particular output format (XML, HTML, etc.)? Is the style specific to the input mode? Or can/should they be independent?</p> <p>I envision interaction between the input parser, an "input mode" , and the output formatter. The same intermediate data format would be used between each of these, being transformed as it progresses.</p> </div> <div class="section" id="docstring-extractor"> <h1><a class="toc-backref" href="#toc-entry-2">Docstring Extractor</a></h1> <p>We need code that scans a parsed Python module, and returns an ordered tree containing the names, docstrings (including attribute and additional docstrings), and additional info (in parentheses below) of all of the following objects:</p> <ul class="simple"> <li>packages</li> <li>modules</li> <li>module attributes (+ values)</li> <li>classes (+ inheritance)</li> <li>class attributes (+ values)</li> <li>instance attributes (+ values)</li> <li>methods (+ formal parameters & defaults)</li> <li>functions (+ formal parameters & defaults)</li> </ul> <p>(Extract comments too? For example, comments at the start of a module would be a good place for bibliographic field lists.)</p> <p>In order to evaluate interpreted text cross-references, namespaces for each of the above will also be required.</p> <p>See python-dev/docstring-develop thread "AST mining", started on 2001-08-14.</p> </div> <div class="section" id="interpreted-text"> <h1><a class="toc-backref" href="#toc-entry-3">Interpreted Text</a></h1> <p>DTD elements: package, module, class, method, function, module_attribute, class_attribute, instance_attribute, variable, parameter, type, exception_class, warning_class.</p> <p>To classify identifiers explicitly, the role is given along with the identifier in either prefix or suffix form:</p> <pre class="literal-block"> Use :method:`Keeper.storedata` to store the object's data in `Keeper.data`:instance_attribute:. </pre> <p>The role may be one of 'package', 'module', 'class', 'method', 'function', 'module_attribute', 'class_attribute', 'instance_attribute', 'variable', 'parameter', 'type', 'exception_class', 'exception', 'warning_class', or 'warning'. Other roles may be defined.</p> <!-- Local Variables: mode: indented-text indent-tabs-mode: nil fill-column: 70 End: --> </div> </div> </body> </html>