Sybil Documentation Testing

Sybil validates code examples embedded in documentation and docstrings by parsing and executing them as part of normal test runs.

Official Documentation: https://sybil.readthedocs.io/en/latest/

Installation

pip install sybil[pytest]

Pytest Integration

Configure in conftest.py. See pytest integration docs.

from sybil import Sybil
from sybil.parsers.markdown.codeblock import PythonCodeBlockParser
from sybil.parsers.markdown.skip import SkipParser

pytest_collect_file = Sybil(
    parsers=[
        SkipParser(),
        PythonCodeBlockParser(),
    ],
    patterns=["*.md", "**/*.py"],
).pytest()

Sybil Parameters

See API reference.

Document Types

See API reference.

• Default: Parse entire file
• PythonDocument: Import .py file as module, names available in namespace
• PythonDocStringDocument: Parse only docstrings from .py files

from sybil.document import PythonDocStringDocument

pytest_collect_file = Sybil(
    parsers=[...],
    patterns=["src/**/*.py"],
    document_types={".py": PythonDocStringDocument},
).pytest()

Fixtures

import pytest
from sybil import Sybil


@pytest.fixture
def my_fixture():
    return {"key": "value"}


pytest_collect_file = Sybil(
    parsers=[...],
    patterns=["*.md"],
    fixtures=["my_fixture"],  # Available in document namespace
).pytest()

Setup/Teardown

def sybil_setup(namespace):
    namespace["helper"] = lambda x: x * 2


def sybil_teardown(namespace):
    pass  # Cleanup if needed


pytest_collect_file = Sybil(
    parsers=[...],
    setup=sybil_setup,
    teardown=sybil_teardown,
).pytest()

Disable pytest's Built-in Doctest

Add to pyproject.toml to prevent conflicts:

[tool.pytest.ini_options]
addopts = "-p no:doctest"

Markdown Parsers

See Markdown parsers docs.

from sybil.parsers.markdown.codeblock import PythonCodeBlockParser, CodeBlockParser
from sybil.parsers.markdown.skip import SkipParser
from sybil.parsers.markdown.clear import ClearNamespaceParser

Skip Directives

See skip directive docs.

SkipParser must come before other parsers to handle skip directives.

````markdown <!-- skip: next -->

# This example is skipped

````

<!-- skip: next "reason for skipping" -->

# Skipped and reported as skipped test with reason

<!-- skip: start -->

# Multiple examples

# All skipped

<!-- skip: end -->

<!-- skip: next if(condition_var) -->

# Conditionally skipped based on namespace variable

````

Invisible Code Blocks

Setup code that doesn't render in documentation. See invisible code blocks docs.

<!-- invisible-code-block: python
setup_var = "hidden setup"
-->
````

## Clear Namespace

Reset the document namespace for isolation. See [clear namespace docs](https://sybil.readthedocs.io/en/latest/markdown.html#clearing-the-namespace).

<!-- clear-namespace -->

## Custom Evaluators

See [evaluators API](https://sybil.readthedocs.io/en/latest/api.html#evaluators).

from sybil.parsers.markdown.codeblock import CodeBlockParser from sybil.evaluators.python import PythonEvaluator

Custom evaluator with future imports

evaluator = PythonEvaluator(future_imports=["annotations"])

parser = CodeBlockParser(language="python", evaluator=evaluator)

## Running Sybil Tests

Sybil tests are collected like regular pytest tests. To run only Sybil tests:

Run tests from specific directory containing documented code

pytest src/mypackage/ -v

Exclude regular tests, only run documentation examples

pytest docs/ -v

To exclude Sybil tests from regular test runs, use pytest's `--ignore` flag or configure `addopts` in `pyproject.toml`.

## Example: Complete conftest.py

"""Sybil configuration for docstring testing."""

from sybil import Sybil from sybil.document import PythonDocStringDocument from sybil.evaluators.python import PythonEvaluator from sybil.parsers.markdown.codeblock import CodeBlockParser from sybil.parsers.markdown.skip import SkipParser

import mypackage

def sybil_setup(namespace): """Pre-populate namespace for all examples.""" namespace["pkg"] = mypackage

def sybil_teardown(namespace): """Cleanup after document.""" pass

pytestcollectfile = Sybil( parsers=[ SkipParser(), CodeBlockParser(language="python", evaluator=PythonEvaluator()), CodeBlockParser(language="py", evaluator=PythonEvaluator()), ], patterns=["src/mypackage/**/*.py"], documenttypes={".py": PythonDocStringDocument}, setup=sybilsetup, teardown=sybilteardown, excludes=[ "/tests/", "/internal/", ], ).pytest()