Edit this page

NA-MIC Project Weeks

Back to Projects List

Python dependencies in extensions

Key Investigators

Project Description

Many Slicer extension developers have to deal with the problem of external python dependencies: how to specify them, how and when to install them, and how to validate that the required things are installed. Everyone addresses the problem in a different way, often re-inventing the wheel and also often generating new great ideas. I’d like to collect all the best practices and turn them into a framework that is built into core slicer for extension developers to more easily grab and use. Something like “stick your dependencies in here and the use slicer.util.check_python_dependences and slicer.util.install_python_dependencies. If that turns out to be a bad idea for whatever reason, at least I can collect all the best practices and put them into the extension development documentation.

Objective

Approach and Plan

Progress and Next Steps

Table of existing practices

I’ve broken down the problem of external Python dependency into four components:

The table below includes all 94 extensions that are currently in the Slicer extension index and that have some external python dependencies to deal with, and my best quick guess as to how they approach three of the problems above. Here is a legend to interpret the terms I’ve put in the table:

Extension Checking Triggering Installing
ShapeVariationAnalyzer simple user isolated
SlicerVolBrain simple user user
PerkTutor simple top level simple
Q3DCExtension simple top level simple
QuantitativeReporting simple top level simple
SegmentationReview simple top level simple
Slicer-ABLTemporalBoneSegmentation simple top level simple
Slicer-ASLtoolkit simple top level simple
Slicer-MusculoskeletalAnalysis simple processing simple
Slicer-PET-MUST-segmenter simple top level simple
Slicer-TITAN simple processing simple
SlicerANTsPy simple top level simple
SlicerAnatomyCarve simple top level simple
SlicerArduinoController simple top level simple
SlicerAuto3dgm simple top level simple
SlicerAutomatedDentalTools version processing simple
SlicerAutoscoperM simple processing simple
SlicerBigImage simple processing simple
SlicerBiomech simple processing simple
SlicerBreastUltrasoundAnalysis simple top level simple
SlicerBreast_DCEMRI_FTV simple top level simple
SlicerCADSWholeBodyCTSeg simple processing simple
SlicerCBCTToothSegmentation simple processing simple
SlicerCineTrack simple processing simple
SlicerColoc-Z-Stats simple processing simple
SlicerConnectToSupervisely simple processing simple
SlicerDBSCoalignment simple processing simple
SlicerDICOMwebBrowser simple top level simple
SlicerDMRI simple processing simple
SlicerDebuggingTools simple processing simple
SlicerDensityLungSegmentation simple processing simple
SlicerDentalModelSeg version processing isolated, blocking-prevention
SlicerFreeSurfer simple top level simple
SlicerHDBrainExtraction simple processing simple
SlicerHeadCTDeid simple processing simple
SlicerHeart simple processing simple
SlicerIDCBrowser simple top level display
SlicerIVIMFit simple processing simple
SlicerImageAugmenter simple button simple
SlicerJupyter simple processing simple
SlicerKonfAI version top level blocking-prevention
SlicerLungCTAnalyzer simple processing simple
SlicerMEMOS simple processing simple
SlicerMHubRunner simple top level simple
SlicerMONAIAuto3DSeg version processing blocking-prevention
SlicerMONAIViz simple processing simple
SlicerMOOSE simple button simple
SlicerMassVision simple top level simple
SlicerModalityConverter simple button simple
SlicerMorph simple top level simple
SlicerMorphoDepot simple top level simple
SlicerMultiverSeg simple processing simple
SlicerMuscleMap simple processing simple
SlicerNNInteractive simple top level display, blocking-prevention, isolated
SlicerNNUnet version processing simple
SlicerNetstim simple processing simple
SlicerNeuro simple processing display
SlicerNeuroStrip simple top level simple
SlicerNeuropacs version processing simple
SlicerOpenLIFU simple processing display
SlicerOrbitSurgerySim simple top level display
SlicerPhotogrammetry simple top level simple
SlicerPipelines simple top level simple
SlicerPolycysticKidneySeg simple button simple
SlicerPyTorch simple button simple
SlicerPythonTestRunner simple processing simple
SlicerRVXLiverSegmentation simple top level simple
SlicerRadiomics simple top level simple
SlicerSPECTRecon version top level display
SlicerSandbox simple processing simple
SlicerSegmentHumanBody simple top level simple
SlicerSegmentWithSAM simple top level simple
SlicerSkeletalRepresentation simple top level simple
SlicerSoundControl simple top level simple
SlicerStereotaxia simple processing simple
SlicerSurfaceLearner simple top level simple
SlicerThemes simple button simple
SlicerTissueSegmentation simple top level simple
SlicerTomoSAM simple processing simple
SlicerTorchIO simple processing simple
SlicerTotalSegmentator simple processing simple
SlicerTractParcellation simple button simple
SlicerTrame simple top level simple
SlicerUltrasound simple top level simple
SlicerUniGradICON simple top level simple
Slicerflywheelcaseiterator simple top level simple
TCIABrowser simple top level simple
TOMAAT-Slicer simple top level simple
aigt simple top level simple
opendose3d simple top level display
slicer_flywheel_connect simple top level simple

Next steps

In the table above, the green cells are the items I think are worth revisiting and learning from for this project.

Other things I found while looking through these that I’d like to consider:

Further points that I’d like to follow up on:

uv

Using uv instead of pip could provide a huge speedup and unlock many more possibilities. Mike’s AI generated summary provides some inspiration:

Refined next steps

A plan of attack:

Plan for implementation

Dependency specification

File format

For Slicer extensions, I believe requirements.txt is the right way to specify python dependency requirements.

Why not pyproject.toml?

Python object

A dependency can be represented by a packaging.requirements.Requirement. A list of such things is what we should get when we load a requirements.txt file (or a constraints.txt file).

Example of how to load a requirements file:

from packaging.requirements import Requirement

def load_requirements(path):
    """Load requirements.txt into list of Requirement objects."""
    reqs = []
    with open(path) as f:
        for line in f:
            line = line.strip()
            # Skip comments, empty lines, and pip options (-r, -c, --index-url, etc.)
            if line and not line.startswith("#") and not line.startswith("-"):
                reqs.append(Requirement(line))
    return reqs

Checking

One can do a pip --dry-run to use pip’s way of checking, but then we need to call a subprocess which has some overhead. Unlike installation, dependency checking is an operation that might get called upon frequently. It would be good to do it in pure python. It does get a bit complicated mainly because of the possibility of extras in a Requirement, but it’s not that bad; here is how slicer.util.pip_check might work:

from importlib.metadata import version, requires, PackageNotFoundError
from packaging.requirements import Requirement
from packaging.markers import default_environment


def pip_check(req : Requirement|list[Requirement], _seen=None) -> bool:
    """Check if requirement(s) are satisfied.

    For requirements with extras like package[extra1,extra2]>=1.0, this:
    1. Checks if the base package is installed at an acceptable version
    2. Finds which dependencies are activated by the requested extras
    3. Recursively verifies those dependencies are satisfied

    Markers (e.g., "; sys_platform == 'win32'") are evaluated - if a marker
    doesn't apply to the current environment, the requirement is considered
    satisfied (since it doesn't need to be installed).

    Args:
        req: Either a Requirement object or a list of Requirement objects
        _seen: Internal parameter for tracking circular dependencies

    Returns:
        True if all requirements are satisfied, False otherwise

    Example:
        from packaging.requirements import Requirement

        # Single requirement
        req = Requirement("numpy>=1.20")
        if pip_check(req):
            print("numpy is satisfied")

        # Multiple requirements
        reqs = [
            Requirement("numpy>=1.20"),
            Requirement("pandas[excel]>=2.0"),
        ]
        if pip_check(reqs):
            print("All requirements satisfied")
    """
    if _seen is None:
        _seen = set()

    # Handle list of requirements, sharing _seen across all of them
    if isinstance(req, list):
        return all(pip_check(r, _seen) for r in req)

    # Check if requirement's marker applies to current environment
    # If not, consider it satisfied (doesn't need to be installed here)
    if req.marker is not None:
        env = default_environment()
        if not req.marker.evaluate(env):
            return True

    # Avoid rechecking the same requirement
    key = (req.name.lower(), frozenset(req.extras))
    if key in _seen:
        return True
    _seen.add(key)

    # Check if base package is installed at acceptable version
    try:
        installed = version(req.name)
    except PackageNotFoundError:
        return False
    if installed not in req.specifier:
        return False

    # If no extras then we are done
    if not req.extras:
        return True

    # Find dependencies activated by the requested extras
    dep_strings = requires(req.name) or []
    env = default_environment()
    activated = []

    for dep_str in dep_strings:
        dep = Requirement(dep_str)
        if dep.marker is None:
            continue

        # Check if any requested extra activates this dependency
        for extra in req.extras:
            if dep.marker.evaluate({**env, "extra": extra}):
                # Strip marker before recursive check - we've already determined it applies
                dep_str_no_marker = str(dep).split(';')[0].strip()
                activated.append(Requirement(dep_str_no_marker))
                break  # Don't check other extras for same dep

    # Recursively verify all activated dependencies
    return all(pip_check(dep, _seen) for dep in activated)

Triggering

For the triggering problem I propose an explicit checker function followed by non-top-level imports. Maybe the LazyImportGroup approach can be considered for later, but for now I think something clear and simple is needed. The LazyImportGroup cleverly intercepts your first use of an imported module to trigger installation behind the scenes. It’s elegant, but the magic does reduce transparency for everyday Slicer extension developers, making debugging more difficult. For IDE support and type checking, one can use the TYPE_CHECKING pattern to declare imports at the top of the file, which type checkers see but which doesn’t run at runtime.

Here’s how slicer.util.pip_ensure might work:

from packaging.requirements import Requirement


def pip_ensure(
    requirements: list[Requirement],
    prompt: bool = True,
    requester: str | None = None,
    skip_in_testing: bool = True,
    show_progress: bool = True,
) -> None:
    """Ensure requirements are satisfied, installing if needed.

    Call at the point where dependencies are actually needed (e.g., onApplyButton).

    Args:
        requirements: List of Requirement objects to check/install
        prompt: If True, show confirmation dialog before installing
        requester: Name shown in dialog to identify who is requesting the packages
            (e.g., "TotalSegmentator", "MyFilter", "console script")
        skip_in_testing: If True (default), skip installation when Slicer is running
            in testing mode (slicer.app.testingEnabled()). This prevents tests from
            modifying the Python environment. Set to False if your test explicitly
            needs to verify installation behavior.
        show_progress: If True (default), show progress dialog during installation
            with status updates and collapsible log details. If False, show only
            a busy cursor. Since pip_ensure already shows a confirmation dialog,
            showing progress during installation provides a consistent user experience.

    Raises:
        RuntimeError: If user declines installation or installation fails

    Example:
        reqs = slicer.util.load_requirements(Path(__file__).parent / "requirements.txt")
        slicer.util.pip_ensure(reqs, requester="MyExtension")
        import some_package
    """
    import logging

    missing = [req for req in requirements if not pip_check(req)]

    if not missing:
        return  # all satisfied

    # skip installation in testing mode to avoid modifying the environment
    if skip_in_testing and slicer.app.testingEnabled():
        missing_str = ", ".join(str(req) for req in missing)
        logging.info(f"Testing mode is enabled: skipping pip_ensure for [{missing_str}]")
        return

    if prompt:
        package_list = "\n".join(f"• {req}" for req in missing)
        title = f"{requester} - Install Python Packages" if requester else "Install Python Packages"
        count = len(missing)
        message = (
            f"{count} Python package{'s' if count != 1 else ''} "
            f"need{'s' if count == 1 else ''} to be installed.\n\n"
            f"This will modify Slicer's Python environment. Continue?"
        )
        if not slicer.util.confirmOkCancelDisplay(message, title, detailedText=package_list):
            raise RuntimeError("User declined package installation")

    # Install missing packages with optional progress display
    pip_install_with_progress(
        [str(req) for req in missing],
        show_progress=show_progress,
        requester=requester,
    )

The pip_install_with_progress is explained in the Installing section below. Since pip_ensure already shows a confirmation dialog to the user by default, showing progress during the subsequent installation provides a consistent experience as the default.

Example usage

Say you have a Resources/requirements.txt in your Slicer module and it contains

scikit-image>=0.21

Here’s how you might use pip_ensure to trigger install if needed:

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    import skimage

...

class MyFilterWidget(ScriptedLoadableModuleWidget):

    ...

    def onApplyButton(self):
        reqs = slicer.util.load_requirements(self.resourcePath("requirements.txt")) # say this contains "scikit-image>=0.21", for example
        slicer.util.pip_ensure(reqs, requester="MyFilter")
        import skimage

        filtered = skimage.filters.gaussian(array, sigma=2.0)
        ...

Installing

We will build upon slicer.util.pip_install to help solve two problems:

Blocking prevention is technical. We will build this into slicer.util.pip_install by using a QTimer-based polling approach inspired by SlicerMONAIAuto3DSeg.

Progress display will involve creating a modal dialog containing a progress bar and an expandable details section.

I think stuffing progress display functionality into pip_install does not make sense. pip_install could remain a low-level building block (with the ability to be non-blocking), while pip_install_with_progress could be a high-level utility that always waits for completion while showing a modal dialog. Mixing these in one function would create confusing interactions. For example what should pip_install(blocking=False, show_progress=True) do when the caller expects an immediate return but the progress dialog expects to block until completion? It also feels a bit messy to stuff so much Qt gui code into pip_install itself.

Here is what these two changes end up meaning for Slicer extension developers, if they are succesfully implemented:

Implementation notes

The above plan was consolidated into a set of instructions for Claude to execute upon.

It ran with --dangerously-skip-permissions inside an isolated docker environment where it had only a Slicer source tree and build tree to play with and test its implementation. When it was done, I requested unit tests following the style and philosophy of existing ones for slicer.util, a self-review based on some similar past pull requests, and useful documentation updates.

Discussions led to a few changes to the design:

This PR is the outcome!

Illustrations

Background and References

Prior related work and ideas: