| Server IP : 82.148.16.210 / Your IP : 216.73.216.19 Web Server : nginx/1.29.5 System : Linux mail.sarafai.ru 6.1.0-43-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.162-1 (2026-02-08) x86_64 User : root ( 0) PHP Version : 7.4.33 Disable Function : pcntl_alarm,pcntl_fork,pcntl_waitpid,pcntl_wait,pcntl_wifexited,pcntl_wifstopped,pcntl_wifsignaled,pcntl_wifcontinued,pcntl_wexitstatus,pcntl_wtermsig,pcntl_wstopsig,pcntl_signal,pcntl_signal_get_handler,pcntl_signal_dispatch,pcntl_get_last_error,pcntl_strerror,pcntl_sigprocmask,pcntl_sigwaitinfo,pcntl_sigtimedwait,pcntl_exec,pcntl_getpriority,pcntl_setpriority,pcntl_async_signals,pcntl_unshare, MySQL : OFF | cURL : ON | WGET : ON | Perl : ON | Python : ON | Sudo : ON | Pkexec : OFF Directory : /proc/746/root/usr/share/doc/python3-docutils/docs/dev/ |
Upload File : |
<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />
<title>Docutils Testing</title>
<meta name="authors" content="Lea Wiemann <LeWiemann@gmail.com> David Goodger <goodger@python.org> Docutils developers <docutils-developers@lists.sourceforge.net>" />
<meta name="date" content="2022-04-02" />
<meta name="copyright" content="This document has been placed in the public domain." />
<link rel="stylesheet" href="../../css/html4css1.css" type="text/css" />
</head>
<body>
<div class="header">
<a class="reference external" href="https://docutils.sourceforge.io">Docutils</a> | <a class="reference external" href="../index.html">Overview</a> | <a class="reference external" href="../index.html#project-fundamentals">About</a> | <a class="reference external" href="../index.html#user">Users</a> | <a class="reference external" href="../index.html#ref">Reference</a> | <a class="reference external" href="../index.html#howto">Developers</a>
<hr class="header"/>
</div>
<div class="document" id="docutils-testing">
<h1 class="title"><a class="reference external" href="https://docutils.sourceforge.io/">Docutils</a> Testing</h1>
<table class="docinfo" frame="void" rules="none">
<col class="docinfo-name" />
<col class="docinfo-content" />
<tbody valign="top">
<tr><th class="docinfo-name">Authors:</th>
<td>Lea Wiemann <LeWiemann@gmail.com>
<br />David Goodger <goodger@python.org>
<br />Docutils developers <docutils-developers@lists.sourceforge.net></td></tr>
<tr><th class="docinfo-name">Revision:</th>
<td>9051</td></tr>
<tr><th class="docinfo-name">Date:</th>
<td>2022-04-02</td></tr>
<tr><th class="docinfo-name">Copyright:</th>
<td>This document has been placed in the public domain.</td></tr>
</tbody>
</table>
<!-- Minimal menu bar for inclusion in documentation sources
in ``docutils/docs/*/`` sub-diretories.
Attention: this is not a standalone document. -->
<div class="contents topic" id="contents">
<p class="topic-title">Contents</p>
<ul class="simple">
<li><a class="reference internal" href="#running-the-test-suite" id="toc-entry-1">Running the Test Suite</a><ul>
<li><a class="reference internal" href="#testing-across-multiple-python-versions" id="toc-entry-2">Testing across multiple Python versions</a></li>
</ul>
</li>
<li><a class="reference internal" href="#unit-tests" id="toc-entry-3">Unit Tests</a><ul>
<li><a class="reference internal" href="#writing-new-tests" id="toc-entry-4">Writing New Tests</a></li>
</ul>
</li>
<li><a class="reference internal" href="#functional-tests" id="toc-entry-5">Functional Tests</a><ul>
<li><a class="reference internal" href="#directory-structure" id="toc-entry-6">Directory Structure</a></li>
<li><a class="reference internal" href="#the-testing-process" id="toc-entry-7">The Testing Process</a></li>
<li><a class="reference internal" href="#creating-new-tests" id="toc-entry-8">Creating New Tests</a></li>
<li><a class="reference internal" href="#the-default-configuration-file" id="toc-entry-9">The Default Configuration File</a></li>
</ul>
</li>
</ul>
</div>
<p>When adding new functionality (or fixing bugs), be sure to add test
cases to the test suite. Practise test-first programming; it's fun,
it's addictive, and it works!</p>
<p>This document describes how to run the Docutils test suite, how the
tests are organized and how to add new tests or modify existing tests.</p>
<div class="section" id="running-the-test-suite">
<h1><a class="toc-backref" href="#toc-entry-1">Running the Test Suite</a></h1>
<p>Before checking in any changes, run the entire Docutils test suite to
be sure that you haven't broken anything. From a shell do <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a>:</p>
<pre class="literal-block">
cd docutils/test
python -u alltests.py
</pre>
<p>Before <a class="reference external" href="policies.html#check-ins">checking in</a> changes to the Docutils core, run the tests on
all <a class="reference external" href="../../README.html#requirements">supported Python versions</a> (see below for details).
In a pinch, the edge cases should cover most of it.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">Due to incompatible customization of the standard <a class="reference external" href="https://docs.python.org/3/library/unittest.html">unittest</a>
framework, the test suite does not work with popular test frameworks
like <a class="reference external" href="https://pypi.org/project/pytest/">pytest</a> or <a class="reference external" href="https://pypi.org/project/nose3/">nose</a>.</p>
<!-- cf. https://sourceforge.net/p/docutils/feature-requests/81/ -->
</div>
<table class="docutils footnote" frame="void" id="footnote-1" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[1]</a></td><td>When using the <a class="reference external" href="https://docs.python.org/3/using/windows.html#python-launcher-for-windows">Python launcher for Windows</a>, make sure to
specify a Python version, e.g., <tt class="docutils literal">py <span class="pre">-3.9</span> <span class="pre">-u</span> alltests.py</tt> for
Python 3.9.<!-- cf. https://sourceforge.net/p/docutils/bugs/434/ -->
</td></tr>
</tbody>
</table>
<div class="section" id="testing-across-multiple-python-versions">
<span id="python-versions"></span><h2><a class="toc-backref" href="#toc-entry-2">Testing across multiple Python versions</a></h2>
<p>A Docutils release has a commitment to support a minimum Python version <a class="footnote-reference" href="#footnote-2" id="footnote-reference-2">[2]</a>
and beyond. Before a release is cut, tests must pass in all
<a class="reference external" href="../../README.html#requirements">supported versions</a>.</p>
<p>You can use <a class="reference external" href="https://pypi.org/project/tox/">tox</a> to test with all supported versions in one go.
From the shell:</p>
<pre class="literal-block">
cd docutils
tox
</pre>
<p>To test a specific version, use the <tt class="docutils literal">pyNN</tt> environment. For example:</p>
<pre class="literal-block">
tox -e py37
</pre>
<p><a class="reference external" href="https://github.com/yyuu/pyenv">pyenv</a> can be installed and configured (see <a class="reference external" href="https://github.com/yyuu/pyenv#installation">installing pyenv</a>) to
get multiple Python versions:</p>
<pre class="literal-block">
# assuming your system runs 3.9.x
pyenv install 3.7.12
pyenv install 3.8.12
pyenv install 3.10.1
pyenv global system 3.7.12 3.8.12 3.10.1
# reset your shims
rm -rf ~/.pyenv/shims && pyenv rehash
</pre>
<p>This will give you <tt class="docutils literal">python3.7</tt> through <tt class="docutils literal">python3.10</tt>.
Then run:</p>
<pre class="literal-block">
python3.7 -u alltests.py
[...]
python3.10 -u alltests.py
</pre>
<table class="docutils footnote" frame="void" id="footnote-2" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-2">[2]</a></td><td>Good resources covering the differences between Python versions
are the <cite>What's New</cite> documents (<a class="reference external" href="https://docs.python.org/3/whatsnew/3.10.html">What's New in Python 3.10</a> and
similar).</td></tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="unit-tests">
<h1><a class="toc-backref" href="#toc-entry-3">Unit Tests</a></h1>
<p>Unit tests test single functions or modules (i.e. whitebox testing).</p>
<p>If you are implementing a new feature, be sure to write a test case
covering its functionality. It happens very frequently that your
implementation (or even only a part of it) doesn't work with an older
(or even newer) Python version, and the only reliable way to detect
those cases is using tests.</p>
<p>Often, it's easier to write the test first and then implement the
functionality required to make the test pass.</p>
<div class="section" id="writing-new-tests">
<h2><a class="toc-backref" href="#toc-entry-4">Writing New Tests</a></h2>
<p>When writing new tests, it very often helps to see how a similar test
is implemented. For example, the files in the
<tt class="docutils literal">test_parsers/test_rst/</tt> directory all look very similar. So when
adding a test, you don't have to reinvent the wheel.</p>
<p>If there is no similar test, you can write a new test from scratch
using Python's <tt class="docutils literal">unittest</tt> module. For an example, please have a
look at the following imaginary <tt class="docutils literal">test_square.py</tt>:</p>
<pre class="literal-block">
#! /usr/bin/env python
# $Id: testing.txt 9051 2022-04-02 21:59:06Z milde $
# Author: Your Name <your_email_address@example.org>
# Copyright: This module has been placed in the public domain.
"""
Test module for docutils.square.
"""
import unittest
import docutils.square
class SquareTest(unittest.TestCase):
def test_square(self):
self.assertEqual(docutils.square.square(0), 0)
self.assertEqual(docutils.square.square(5), 25)
self.assertEqual(docutils.square.square(7), 49)
def test_square_root(self):
self.assertEqual(docutils.square.sqrt(49), 7)
self.assertEqual(docutils.square.sqrt(0), 0)
self.assertRaises(docutils.square.SquareRootError,
docutils.square.sqrt, 20)
if __name__ == '__main__':
unittest.main()
</pre>
<p>For more details on how to write tests, please refer to the
documentation of the <tt class="docutils literal">unittest</tt> module.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p>Unit tests and functional test should generally set</p>
<pre class="literal-block">
settings_overrides['_disable_config'] = True
</pre>
<p class="last">in order to be independent on the users local configuration.</p>
</div>
</div>
</div>
<div class="section" id="functional-tests">
<span id="functional"></span><h1><a class="toc-backref" href="#toc-entry-5">Functional Tests</a></h1>
<p>The directory <tt class="docutils literal">test/functional/</tt> contains data for functional tests.</p>
<p>Performing functional testing means testing the Docutils system as a
whole (i.e. blackbox testing).</p>
<div class="section" id="directory-structure">
<h2><a class="toc-backref" href="#toc-entry-6">Directory Structure</a></h2>
<ul class="simple">
<li><tt class="docutils literal">functional/</tt> The main data directory.<ul>
<li><tt class="docutils literal">input/</tt> The input files.<ul>
<li><tt class="docutils literal">some_test.txt</tt>, for example.</li>
</ul>
</li>
<li><tt class="docutils literal">output/</tt> The actual output.<ul>
<li><tt class="docutils literal">some_test.html</tt>, for example.</li>
</ul>
</li>
<li><tt class="docutils literal">expected/</tt> The expected output.<ul>
<li><tt class="docutils literal">some_test.html</tt>, for example.</li>
</ul>
</li>
<li><tt class="docutils literal">tests/</tt> The config files for processing the input files.<ul>
<li><tt class="docutils literal">some_test.py</tt>, for example.</li>
<li><tt class="docutils literal">_default.py</tt>, the <a class="reference internal" href="#default-configuration-file">default configuration file</a>.</li>
</ul>
</li>
</ul>
</li>
</ul>
</div>
<div class="section" id="the-testing-process">
<h2><a class="toc-backref" href="#toc-entry-7">The Testing Process</a></h2>
<p>When running <tt class="docutils literal">test_functional.py</tt>, all config files in
<tt class="docutils literal">functional/tests/</tt> are processed. (Config files whose names begin
with an underscore are ignored.) The current working directory is
always Docutils' main test directory (<tt class="docutils literal">test/</tt>).</p>
<p>For example, <tt class="docutils literal">functional/tests/some_test.py</tt> could read like this:</p>
<pre class="literal-block">
# Source and destination file names.
test_source = "some_test.txt"
test_destination = "some_test.html"
# Keyword parameters passed to publish_file.
reader_name = "standalone"
parser_name = "rst"
writer_name = "html"
settings_overrides['output-encoding'] = 'utf-8'
# Relative to main ``test/`` directory.
settings_overrides['stylesheet_path'] = '../docutils/writers/html4css1/html4css1.css'
</pre>
<p>The two variables <tt class="docutils literal">test_source</tt> and <tt class="docutils literal">test_destination</tt> contain the
input file name (relative to <tt class="docutils literal">functional/input/</tt>) and the output
file name (relative to <tt class="docutils literal">functional/output/</tt> and
<tt class="docutils literal">functional/expected/</tt>). Note that the file names can be chosen
arbitrarily. However, the file names in <tt class="docutils literal">functional/output/</tt> <em>must</em>
match the file names in <tt class="docutils literal">functional/expected/</tt>.</p>
<p>If defined, <tt class="docutils literal">_test_more</tt> must be a function with the following
signature:</p>
<pre class="literal-block">
def _test_more(expected_dir, output_dir, test_case, parameters):
</pre>
<p>This function is called from the test case to perform tests beyond the
simple comparison of expected and actual output files.</p>
<p><tt class="docutils literal">test_source</tt> and <tt class="docutils literal">test_destination</tt> are removed from the
namespace, as are all variables whose names begin with an underscore
("_"). The remaining names are passed as keyword arguments to
<tt class="docutils literal">docutils.core.publish_file</tt>, so you can set reader, parser, writer
and anything else you want to configure. Note that
<tt class="docutils literal">settings_overrides</tt> is already initialized as a dictionary <em>before</em>
the execution of the config file.</p>
</div>
<div class="section" id="creating-new-tests">
<h2><a class="toc-backref" href="#toc-entry-8">Creating New Tests</a></h2>
<p>In order to create a new test, put the input test file into
<tt class="docutils literal">functional/input/</tt>. Then create a config file in
<tt class="docutils literal">functional/tests/</tt> which sets at least input and output file names,
reader, parser and writer.</p>
<p>Now run <tt class="docutils literal">test_functional.py</tt>. The test will fail, of course,
because you do not have an expected output yet. However, an output
file will have been generated in <tt class="docutils literal">functional/output/</tt>. Check this
output file for validity <a class="footnote-reference" href="#footnote-3" id="footnote-reference-3">[3]</a> and correctness. Then copy the file to
<tt class="docutils literal">functional/expected/</tt>.</p>
<p>If you rerun <tt class="docutils literal">test_functional.py</tt> now, it should pass.</p>
<p>If you run <tt class="docutils literal">test_functional.py</tt> later and the actual output doesn't
match the expected output anymore, the test will fail.</p>
<p>If this is the case and you made an intentional change, check the
actual output for validity and correctness, copy it to
<tt class="docutils literal">functional/expected/</tt> (overwriting the old expected output), and
commit the change.</p>
<table class="docutils footnote" frame="void" id="footnote-3" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-3">[3]</a></td><td>The validity of <cite>Docutils XML</cite> can be tested with
<tt class="docutils literal">xmllint <span class="pre"><document-referencing-local-Docutils-DTD>.xml</span> <span class="pre">--valid</span> <span class="pre">--noout</span></tt>.<!-- note: the ``- -dtdvalid`` and ``- -nonet`` options did not help override
a reference to the PUBLIC "docutils.dtd" if there is a local version
on the system (e.g. /usr/share/xml/docutils/docutils.dtd in Debian). -->
</td></tr>
</tbody>
</table>
</div>
<div class="section" id="the-default-configuration-file">
<span id="default-configuration-file"></span><h2><a class="toc-backref" href="#toc-entry-9">The Default Configuration File</a></h2>
<p>The file <tt class="docutils literal">functional/tests/_default.py</tt> contains default settings.
It is executed just before the actual configuration files, which has
the same effect as if the contents of <tt class="docutils literal">_default.py</tt> were prepended
to every configuration file.</p>
</div>
</div>
</div>
</body>
</html>