Ctrl+K

Solving Package Environments

Solving Package Environments#

The libmambapy.solver submodule contains a generic API for solving requirements (MatchSpec) into a list of packages (PackageInfo) with no conflicting dependencies. This problem is hard to solve (NP-complete) which is why Mamba uses a SAT solver to do so.

Note

There is currently only one solver available in Mamba: LibSolv. For this reason, the generic interface has not been fully completed and users need to access the submodule libmambapy.solver.libsolv for certain types.

Populating the Package Database#

The first thing needed is a Database of all the packages and their dependencies. Packages are organised in repositories, described by a RepoInfo. This serves to resolve explicit channel requirements or channel priority. As such, the database constructor takes a set of ChannelResolveParams to work with Channel work with Channel data internaly (see the usage section on Channels for more information).

The first way to add a repository is from a list of PackageInfo using DataBase.add_repo_from_packages:

import libmambapy

db = libmambapy.solver.libsolv.Database(
    libmambapy.specs.ChannelResolveParams(channel_alias="https://conda.anaconda.org")
)

repo1 = db.add_repo_from_packages(
    packages=[
        libmambapy.specs.PackageInfo(name="python", version="3.8", ...),
        libmambapy.specs.PackageInfo(name="pip", version="3.9", ...),
        ...,
    ],
    name="myrepo",
)

The second way of loading packages is throuch Conda’s reposoitory index format repodata.json using DataBase.add_repo_from_repodata. This is meant as a convenience and performant alternative to the former method, since these files grow large.

repo2 = db.add_repo_from_repodata(
    path="path/to/repodata.json",
    url="htts://conda.anaconda.org/conda-forge/linux-64",
)

One of the reppository can be set to have a special meaning of “installed repository”. It is used as a reference point in the solver to compute changes. For instance if a package is required but is already available in the installed repo, the solving result will not mention it. The function DataBase.set_installed_repo is used for that purpose.

db.set_installed_repo(repo1)

Binary serialization of the database (Advanced)#

The Database reporitories can be serialized in binary format for faster reloading. To ensure integrity and freshness of the serialized file, metadata about the packages, such as source url and RepodataOrigin, are stored inside the file when calling DataBase.native_serialize_repo . Upon reading, similar parameters are expected as input to DataBase.add_repo_from_native_serialization. If they mistmatch, the loading results in an error.

A typical wokflow first tries to load a repository from such binary cache, and then quietly fallbacks to repodata.json on failure.

Creating a solving request#

All jobs that need to be resolved are added as part of a Request. This includes installing, updating, removing packages, as well as solving cutomization parameters.

Request = libmambapy.solver.Request
MatchSpec = libmambapy.specs.MatchSpec

request = Request(
    jobs=[
        Request.Install(MatchSpec.parse("python>=3.9")),
        Request.Update(MatchSpec.parse("numpy")),
        Request.Remove(MatchSpec.parse("pandas"), clean_dependencies=False),
    ],
    flags=Request.Flags(
        allow_downgrade=True,
        allow_uninstall=True,
    ),
)

Solving the request#

The Request and the Database are the two input parameters needed to solve an environment. This task is achieve with the Solver.solve method.

solver = libmambapy.solver.libsolv.Solver()
outcome = solver.solve(db, request)

The outcome can be of two types, either a Solution listing packages (PackageInfo) and the action to take on them (install, remove…), or an UnSolvable type when no solution exist (because of conflict, missing pacakges…).

Examine the solution#

We can test if a valid solution exists by checking the type of the outcome. The attribute Solution.actions contains the actions to take on the installed repository so that it satisfies the Request requirements.

Solution = libmambapy.solver.Solution

if isinstance(outcome, Solution):
    for action in outcome.actions:
        if isinstance(action, Solution.Upgrade):
            my_upgrade(from_pkg=action.remove, to_pkg=action.install)
        if isinstance(action, Solution.Reinstall):
            ...
        ...

Alternatively, an easy way to compute the update to the environment is to check for install and remove members, since they will populate the relevant fields for all actions:

Solution = libmambapy.solver.Solution

if isinstance(outcome, Solution):
    for action in outcome.actions:
        if hasattr(action, "install"):
            my_download_and_install(action.install)
        # WARN: Do not use `elif` since actions like `Upgrade`
        # are represented as an `install` and `remove` pair.
        if hasattr(action, "remove"):
            my_delete(action.remove)

Understand unsolvable problems#

When a problem as no Solution, it is inherenty hard to come up with an explanation. In the easiest case, a requiered package is missing from the Database. In the most complex, many package dependencies are incompatible without a single culprit. In this case, packages should be rebuild with weaker requirements, or with more build variants. The UnSolvable class attempts to build an explanation.

The UnSolvable.problems is a list of problems, as defined by the solver. It is not easy to understand without linking it to specific MatchSpec and PackageInfo. The method UnSolvable.problems_graph gives a more structured graph of package dependencies and incompatibilities. This graph is the underlying mechanism used in UnSolvable.explain_problems to build a detail unsolvability message.

Contents