idf_ci.idf_pytest package

Submodules

idf_ci.idf_pytest.models.EMULATOR_MARKER_SERVICES: Dict[str, str] = {'espemu': 'idf,espemu', 'qemu': 'idf,qemu'}

Markers of test cases that run the firmware on a host emulator instead of real hardware, mapped to the pytest-embedded services they require.

Cases carrying one of these markers are host tests, and their emulator name is appended to the target in PytestCase.caseid to keep them apart from the real-target runs of the same test.

class idf_ci.idf_pytest.models.GroupKey(target_selector, env_selector, runner_tags)

Bases: NamedTuple

env_selector: str

Alias for field number 1

classmethod from_case(case: PytestCase)
runner_tags: Tuple[str, ...]

Alias for field number 2

target_selector: str

Alias for field number 0

class idf_ci.idf_pytest.models.GroupedPytestCases(cases: List[PytestCase], *, additional_dict: Dict[GroupKey, Dict[str, Any]] | None = None)

Bases: object

Groups pytest cases by target and environment markers.

additional_dict: Dict[GroupKey, Dict[str, Any]]
property grouped_cases: Dict[GroupKey, List[PytestCase]]

Groups test cases by target and environment markers.

Returns:

Dictionary of GroupKey to list of PytestCases

output_as_github_ci() → str

Generates a JSON string for GitHub Actions CI matrix strategy.

Example output:

{
    "include": [
        {
            "targets": "esp32,esp32",
            "env_markers": "generic and flash_4mb",
            "runner_tags": ["self-hosted", "esp32_2", "generic", "flash_4mb"],
            "nodes": "nodeid1 nodeid2"
        },
        {
            "targets": "esp32,esp32s2",
            "env_markers": "generic and two_duts",
            "runner_tags": ["self-hosted", "esp32", "esp32s2", "generic", "two_duts"],
            "nodes": "nodeid1 nodeid2"
        },
        {
            "targets": "esp32",
            "env_markers": "generic",
            "runner_tags": ["self-hosted", "esp32", "generic"],
            "nodes": "nodeid1"
        }
    ]
}

Example usage:

strategy:
    matrix: ${{fromJson( this_output_as_env_var )}}
Returns:

JSON string suitable for GitHub Actions matrix strategy

output_as_string() → str

Generates a human-readable string representation of grouped test cases.

Returns:

String representation of grouped test cases

class idf_ci.idf_pytest.models.PytestApp(path: str, target: str, config: str)

Bases: object

Represents a pytest app.

property build_dir: str

Returns the build directory for the app.

Note

Matches the build_dir (by default build_@t_@w) in the idf-build-apps config file.

Returns:

The build directory for the app.

class idf_ci.idf_pytest.models.PytestCase(apps: List[PytestApp], item: Function)

Bases: object

Represents a pytest test case.

KNOWN_ENV_MARKERS: ClassVar[Set[str]] = {}
property all_markers: Set[str]
property caseid: str
property configs: List[str]
property emulator_marker: str | None

Get the emulator the test case runs on, if any.

Returns:

The emulator marker name, or None if the case runs on real hardware or on linux

Raises:

ValueError – If the case is marked with more than one emulator marker

property env_markers: Set[str]
property env_selector: str
classmethod from_item(item: Function) → PytestCase | None

Create a PytestCase from a pytest item.

Parameters:

item – Pytest function item

Returns:

PytestCase instance or None if targets not defined

classmethod get_param(item: Function, key: str, default: Any = None) → Any

Get parameter value from pytest item.

Parameters:

  • item – Pytest function item

  • key – Parameter key to retrieve

  • default – Default value if key not found

Returns:

Parameter value or default

get_skip_reason_if_not_built(app_dirs: List[str] | None = None) → str | None

Check if all binaries of the test case are built in the app lists.

Parameters:

app_dirs – App folder paths to check

Returns:

Skip reason string if not all binaries are built, None otherwise

property is_host_test: bool
property is_single_dut: bool
property name: str
property path: str
property runner_tags: Tuple[str, ...]
skipped_targets() → Dict[str, str]

Get targets that are marked with skip markers for current test case.

Returns:

Dictionary of targets with skip reason

property target_selector: str
property targets: List[str]
idf_ci.idf_pytest.models.get_emulator_marker(markers: Iterable[str], *, name: str | None = None) → str | None

Get the emulator marker out of the given marker names.

Parameters:

  • markers – Marker names to check

  • name – Test case name, used in the error message

Returns:

The emulator marker name, or None if there is no emulator marker

Raises:

ValueError – If more than one emulator marker is applied. A test case runs on one emulator or none, so this is always a mistake in the test file.

class idf_ci.idf_pytest.plugin.IdfPytestPlugin(*, cli_target: str, sdkconfig_name: str | None = None)

Bases: object

build_dir(request: FixtureRequest, app_path: str, target: str | None, config: str | None) → str

Find a valid build directory based on priority rules.

Checks local build directories in the following order:

  1. build_<target>_<config>

  2. build_<target>

  3. build_<config>

  4. build

Parameters:

  • request – Pytest fixture request

  • app_path – Path to the application

  • target – Target being used

  • config – Configuration being used

Returns:

Valid build directory name

Raises:

ValueError – If no valid build directory is found

config(request: FixtureRequest) → str

Fixture that provides the configuration for tests.

Parameters:

request – Pytest fixture request

Returns:

Configuration string, defaults to ‘default’ if not specified

static get_case_by_item(item: Item) → PytestCase | None

Get the test case associated with a pytest item.

Parameters:

item – The pytest test item

Returns:

PytestCase object or None if not found

pytest_collection_modifyitems(config: Config, items: List[Function])

Filter test cases based on target, sdkconfig, and available apps.

Parameters:

  • config – Pytest configuration

  • items – Collected test items

pytest_generate_tests(metafunc) → None
pytest_pycollect_makemodule(module_path: Path)

Handle module collection for pytest, mocking any missing modules.

This hook runs before module collection to prevent errors from missing dependencies by automatically mocking them.

Parameters:

module_path – Path to the module being collected

pytest_report_collectionfinish(items: List[Function]) → None
target(request: FixtureRequest) → str

Fixture that provides the target for tests.

Parameters:

request – Pytest fixture request

Returns:

Target string

Raises:

ValueError – If target parameter is not defined

idf_ci.idf_pytest.plugin.pytest_addoption(parser: Parser)

Add custom command line options for IDF pytest plugin.

Parameters:

parser – Pytest command line parser

idf_ci.idf_pytest.plugin.pytest_configure(config: Config)

Configure the pytest environment for IDF tests.

Parameters:

config – Pytest configuration object

idf_ci.idf_pytest.plugin.pytest_unconfigure(config: Config)

Clean up the IDF pytest plugin when pytest is shutting down.

Parameters:

config – Pytest configuration object

idf_ci.idf_pytest.scripts.get_pytest_cases(*, paths: List[str] | None = None, target: str = 'all', sdkconfig_name: str | None = None, marker_expr: UndefinedType | str | None = __UNDEF__, filter_expr: str | None = None, additional_args: List[str] | None = None) → List[PytestCase]

Collect pytest test cases from specified paths.

Parameters:

  • paths – List of file system paths to collect test cases from

  • target – Filter by targets

  • sdkconfig_name – Filter tests whose apps are built with this sdkconfig name

  • marker_expr – Filter by pytest marker expression -m

  • filter_expr – Filter by pytest filter expression -k

  • additional_args – Additional arguments to pass to pytest

Returns:

List of collected PytestCase objects

Raises:

RuntimeError – If pytest collection fails

Module contents

class idf_ci.idf_pytest.GroupedPytestCases(cases: List[PytestCase], *, additional_dict: Dict[GroupKey, Dict[str, Any]] | None = None)

Bases: object

Groups pytest cases by target and environment markers.

additional_dict: Dict[GroupKey, Dict[str, Any]]
property grouped_cases: Dict[GroupKey, List[PytestCase]]

Groups test cases by target and environment markers.

Returns:

Dictionary of GroupKey to list of PytestCases

output_as_github_ci() → str

Generates a JSON string for GitHub Actions CI matrix strategy.

Example output:

{
    "include": [
        {
            "targets": "esp32,esp32",
            "env_markers": "generic and flash_4mb",
            "runner_tags": ["self-hosted", "esp32_2", "generic", "flash_4mb"],
            "nodes": "nodeid1 nodeid2"
        },
        {
            "targets": "esp32,esp32s2",
            "env_markers": "generic and two_duts",
            "runner_tags": ["self-hosted", "esp32", "esp32s2", "generic", "two_duts"],
            "nodes": "nodeid1 nodeid2"
        },
        {
            "targets": "esp32",
            "env_markers": "generic",
            "runner_tags": ["self-hosted", "esp32", "generic"],
            "nodes": "nodeid1"
        }
    ]
}

Example usage:

strategy:
    matrix: ${{fromJson( this_output_as_env_var )}}
Returns:

JSON string suitable for GitHub Actions matrix strategy

output_as_string() → str

Generates a human-readable string representation of grouped test cases.

Returns:

String representation of grouped test cases

class idf_ci.idf_pytest.IdfPytestPlugin(*, cli_target: str, sdkconfig_name: str | None = None)

Bases: object

build_dir(request: FixtureRequest, app_path: str, target: str | None, config: str | None) → str

Find a valid build directory based on priority rules.

Checks local build directories in the following order:

  1. build_<target>_<config>

  2. build_<target>

  3. build_<config>

  4. build

Parameters:

  • request – Pytest fixture request

  • app_path – Path to the application

  • target – Target being used

  • config – Configuration being used

Returns:

Valid build directory name

Raises:

ValueError – If no valid build directory is found

config(request: FixtureRequest) → str

Fixture that provides the configuration for tests.

Parameters:

request – Pytest fixture request

Returns:

Configuration string, defaults to ‘default’ if not specified

static get_case_by_item(item: Item) → PytestCase | None

Get the test case associated with a pytest item.

Parameters:

item – The pytest test item

Returns:

PytestCase object or None if not found

pytest_collection_modifyitems(config: Config, items: List[Function])

Filter test cases based on target, sdkconfig, and available apps.

Parameters:

  • config – Pytest configuration

  • items – Collected test items

pytest_generate_tests(metafunc) → None
pytest_pycollect_makemodule(module_path: Path)

Handle module collection for pytest, mocking any missing modules.

This hook runs before module collection to prevent errors from missing dependencies by automatically mocking them.

Parameters:

module_path – Path to the module being collected

pytest_report_collectionfinish(items: List[Function]) → None
target(request: FixtureRequest) → str

Fixture that provides the target for tests.

Parameters:

request – Pytest fixture request

Returns:

Target string

Raises:

ValueError – If target parameter is not defined

class idf_ci.idf_pytest.PytestApp(path: str, target: str, config: str)

Bases: object

Represents a pytest app.

property build_dir: str

Returns the build directory for the app.

Note

Matches the build_dir (by default build_@t_@w) in the idf-build-apps config file.

Returns:

The build directory for the app.

class idf_ci.idf_pytest.PytestCase(apps: List[PytestApp], item: Function)

Bases: object

Represents a pytest test case.

KNOWN_ENV_MARKERS: ClassVar[Set[str]] = {}
property all_markers: Set[str]
property caseid: str
property configs: List[str]
property emulator_marker: str | None

Get the emulator the test case runs on, if any.

Returns:

The emulator marker name, or None if the case runs on real hardware or on linux

Raises:

ValueError – If the case is marked with more than one emulator marker

property env_markers: Set[str]
property env_selector: str
classmethod from_item(item: Function) → PytestCase | None

Create a PytestCase from a pytest item.

Parameters:

item – Pytest function item

Returns:

PytestCase instance or None if targets not defined

classmethod get_param(item: Function, key: str, default: Any = None) → Any

Get parameter value from pytest item.

Parameters:

  • item – Pytest function item

  • key – Parameter key to retrieve

  • default – Default value if key not found

Returns:

Parameter value or default

get_skip_reason_if_not_built(app_dirs: List[str] | None = None) → str | None

Check if all binaries of the test case are built in the app lists.

Parameters:

app_dirs – App folder paths to check

Returns:

Skip reason string if not all binaries are built, None otherwise

property is_host_test: bool
property is_single_dut: bool
property name: str
property path: str
property runner_tags: Tuple[str, ...]
skipped_targets() → Dict[str, str]

Get targets that are marked with skip markers for current test case.

Returns:

Dictionary of targets with skip reason

property target_selector: str
property targets: List[str]
idf_ci.idf_pytest.get_pytest_cases(*, paths: List[str] | None = None, target: str = 'all', sdkconfig_name: str | None = None, marker_expr: UndefinedType | str | None = __UNDEF__, filter_expr: str | None = None, additional_args: List[str] | None = None) → List[PytestCase]

Collect pytest test cases from specified paths.

Parameters:

  • paths – List of file system paths to collect test cases from

  • target – Filter by targets

  • sdkconfig_name – Filter tests whose apps are built with this sdkconfig name

  • marker_expr – Filter by pytest marker expression -m

  • filter_expr – Filter by pytest filter expression -k

  • additional_args – Additional arguments to pass to pytest

Returns:

List of collected PytestCase objects

Raises:

RuntimeError – If pytest collection fails