Skip to content

Latest commit

 

History

History
376 lines (266 loc) · 14 KB

0000-cargo-mutex-features.md

File metadata and controls

376 lines (266 loc) · 14 KB

NOTE: This is work in progress!

Based on the idea from: rust-lang/cargo#2980 (comment)

Summary

This RFC proposes a way to implement "mutually exclusive" features in Cargo, by introducing the concept of "capabilities" of a feature.

Motivation

Cargo currently supports features, which allow one to use conditional compilation. A common use case is to provide some kind of capability, backed by different implementations, based on the feature selected by the user.

In group of cases, these implementations are mutually exclusive. While custom build.rs scripts can be used to implement this behavior partly, it is not a great experience. Neither from the implementor's side, nor from the user's side.

Examples

One example is the crate k8s-openapi. The crate compiles for different Kubernetes versions. However, at most one API version may be active.

To achieve this, the Cargo.toml currently contains features (v1_11 to v1_19), and has a build.rs, which uses custom code to validate this.

Another example is the crate stm32f7xx-hal (and its sibling crates), which require exactly one hardware configuration to be enabled. Again, the Cargo.toml contains a list of features, one for each hardware configuration. It also introduces a meta feature named device-selected, which gets set to simplify checks in the code if a platform was selected.

Guide-level explanation

In addition to an identifier, features can specify one or more capabilities that they provide or require, when the feature is enabled.

Basic example

The following example shows the basic idea:

[features]
json = []
yaml = []
json_foo = [ "foo_json" ]
json_bar = [ "bar_json" ]

[features.json]
requires = ["json_backend"]

[features.json_foo]
provides = ["json_backend"]

[features.json_bar]
provides = ["json_backend"]

In the above example there are two standard features json and yaml, which can be activated as needed. The json feature additionally declares, that it requires the capability json_backend. json_backend is a simple identifier, which is local to the crate. The two additional features json_foo and json_bar both provide the functionality of a JSON backend. Due to some limitation, they are mutually exclusive, and the user has to choose one.

If the user requests the feature json, but does not select a feature, providing the capability json_backend, then cargo will fail the build:

$ cargo build --features json
error: Package `example v0.0.0 (/path)` requires the capability `json_backed`, but no feature which could provide
the capability is enabled. Possible candiates are:
  - json_foo
  - json_bar

At the same time, if a user select more than one feature that provides json_backend, cargo will fail the build as well:

$ cargo build --features json,json_foo,json_bar
error: Package `example v0.0.0 (/path)` has multiple features enabled that provide the mutually exclusive capability
`json_backend`. Enabled features providing this capability are:
  - json_foo
  - json_bar

Another example

In the area of embedded development, it may be interested to let the user provide buffer sizes during compile time to tweak the memory footprint of an application. The following example shows how this can be achieved using features and capabilities:

[package]
requires = ["buffer-size"]

[features]
default = ["1k"]
1k=[]
2k=[]
4k=[]

[features.1k]
provides = ["buffer-size"]

[features.2k]
provides = ["buffer-size"]

[features.4k]
provides = ["buffer-size"]

Using the requires field, the build requests the user to select a feature providing the capability buffer-size, and at the same time, defaults to the feature 1k, which will provide an implementation of a 1k buffer.

The user has the ability to choose a different buffer size, using the different features. At the same time, cargo will ensure that exactly one provider of buffer_size is enabled.

Reference-level explanation

Add as cargo feature

The use of capabilities would need to be added to the list of cargo features.

Cargo.toml

The model of the cargo file needs to extended, to allow for additional feature information. Specifically:

Field for "requires"

Add a field requires: Vec<String> to Package, which lists the capabilities required by this crate.

FeatureMap and FeatureValue

Currently features in a Cargo file is expected to be a map of String -> String, which is translated into a FeatureMap, which is defined as BTreeMap<InternedString, Vec<FeatureValue>>.

The type FeatureValue is currently defined as:

pub enum FeatureValue {
    Feature(InternedString),
    Crate(InternedString),
    CrateFeature(InternedString, InternedString),
}

Instead of having Vec<FeatureValue> as the map's value, we would add a FeatureInformation (or other name) struct:

pub struct FeatureInformation {
    capabilities: Vec<InternedString>,
    enables: Vec<FeatureValue>,
}

Which would be represented in the Cargo.toml as:

[dependencies]
bar = "*"

[features]
#foo = [ "bar", "bar/feature", "baz"] # should still work
foo = { capabilities = ["cap1"], enables = ["bar", "bar/feature", "baz"] }

# alternatively
[features.foo]
capabilities = ["cap1"]
enables = ["bar", "bar/feature", "baz"]

Of course the existing format (commented out in the above example) should still work. However, it doesn't support the use of capabilities.

Aside from moving the string array to the enables field, I would not change its format. This should help people migrate to the new format, as you only need to shift the values to the new field.

Add additional validation

When validating, cargo builds up a list of capabilites: Map<String, Vec<&FeatureInformation>>, listing the found capabilities, and their providing features. Now it validates:

  • that the number of features providing a capability less then 2
  • that for each required capability (for package and feature) the number of features is exactly 1

Conditional compilation

It would be possible that no selected feature provides a certain capability. Thus it makes sense to allow conditional compilation also using capabilities (in addition to #[cfg(feature="foo")]).

Conditional compilation can use the attribute capability, which is handled the same way feature is, just with capabilities:

#[cfg(capability="json_provider")]
fn get_provider() -> Option<Provider> {
    Some(json_provider::new())
}

#[cfg(not(capability="json_provider"))]
fn get_provider() -> Option<Provider> {
    None
}

Capability identifier/name

The capability identifier should be limited to: ASCII letters, digits, _, and -. A future extension (see below) could be a "global capability", which could be scoped by adding a prefix like global:.

Drawbacks

An argument to not do this, could be that using build.rs, you can already achieve similar results. Without the need to introduce additional concepts. And while this is partially true, it is not a very user-friendly experience.

Additionally, if different people create different ways of processing situations like this in Rust code, contained in build.rs, it may be complicated over time as different concepts and ways of handling similar situations evolve.

Also, is it problematic (or even impossible) to crate proper tooling on logic contained in build.rs files.

Rationale and alternatives

NOTE: to be written …

  • Why is this design the best in the space of possible designs?
  • What other designs have been considered and what is the rationale for not choosing them?
  • What is the impact of not doing this?

This design solves the original use case, of providing mutually exclusive features. It does this by using a clearly defined piece of information, the "capability". It also re-uses the existing feature definition and format (in Cargo.toml) as far as possible. So it isn't completely new, just an extension.

Additionally, it leverages the capability concept to allow conditional compilation and "requiring" of a capability, which plays along with the original use case.

Leveraging the concept at a global level is left as a future task (see below).

As the feature definition (FeatureInformation) could easily be amended now with additional information (like "conflicts", "deprecated", …) it feels as a future proof, yet simple extension to the current information model.

However, the current proposal stops at this point and leaves out implementing additional capability requirements (also see Prior art).

Cargo.toml feature definition

An alternate definition of capabilities for features in the cargo file could be to amend the FeatureValue enum:

pub enum FeatureValue {
    Feature(InternedString),
    Crate(InternedString),
    CrateFeature(InternedString, InternedString),
    Capability(InternedString),
}

In Cargo.toml this could be represented as:

[dependencies]
bar = "*"

[features]
foo = ["provides:bar", "bar", "bar/feature", "baz"]
       ^               ^      ^              ^
       |               |      |              |--- Feature
       |               |      |
       |               |      |--- CrateFeature
       |               |
       |               |--- Crate
       |
       |--- Capability

However, this would require to change the way FeatureValue is parsed from a string. This would mean putting more logic into the specialized format. I think that the value/format of the features is already overloaded. So it makes sense to provide some actual fields, like the dependencies have as well.

Prior art

Package manages like RPM, deb, …

Packages managers like RPM and others, already have the concept of capabilities. It is possible to define dependencies on specific other packages, or on "capabilities", which may be provided by different packages.

For example, an RPM of sendmail may declare that it provides the capability of MTA (mail transfer agent). http://rpmfind.net/linux/RPM/fedora/32/x86_64/s/sendmail-8.15.2-43.fc32.x86_64.html. At the same time, other packages can provide the same capability: http://rpmfind.net/linux/rpm2html/search.php?query=MTA

A crate can be seen a "software package" and Cargo as a package manager.

A key difference to packages managers like RPM is, that RPM doesn't consider a capability "mutually exclusive". This can however be achieved by the use of additional rules, like "Conflicts". Also do such package managers support weak dependencies, like "Suggest" or "Recommends". However, adding such concepts makes things more complex, harder to implement and harder to understand for the user.

OSGi

The Java programming language itself did not have module concept before Java 9. However, OSGi did provide a "bundle" concept on top of Java before that. Dependencies between bundles can be declared explicitly by referencing another bundle, or using generic capabilities. See: https://blog.osgi.org/2015/12/using-requirements-and-capabilities.html

"Bundles" in OSGi would map to "crates" in Rust.

While the capabilities concepts have been re-used in OSGi package managers (like P2 or BND), OSGi itself enforces the capabilities during runtime. Which Rust/Cargo would not do.

For OSGi it is also possible to use much more complex definitions, providing actual values in the capabilities, and enforcing requirements using LDAP queries. While this allows one to crate rather complex constructs, it also makes things harder to understand and maintain. Both from an implementation as well from a user perspective.

If such complex scenerios are required, this proposal still allows for the use of custom code in the build.rs script to implement more complex requirements.

Unresolved questions

In the future there should definitely a second look at the use of "global capabilities" (see below).

Future possibilities

Global example

The focus of this RFC is on crate-local capabilities. However, it would be reasonable to expand this also on a global level.

A common example in the embedded space is the "panic handler" or "allocator", which must be selected, but must also be unique on the whole application.

Currently, people simply comment out alternatives (https://github.com/rust-embedded/cortex-m-quickstart/blob/master/Cargo.toml).

For example, if it would be possible to declare global capabilities as well, different crates could offer different panic handler implementations. Cargo could then ensure that exactly one is part of the build tree.

Consider the application having the following Cargo.toml

requires = ["global:panic-handler"]

[dependencies]
panic-rtt-target = { version = "0.1.1", features = ["cortex-m"] }
panic-halt = "0.2.0"
panic-itm = "0.4.1"

[features]
panic-by-rtt = ["panic-rtt-target"]
panic-by-halt = ["panic-halt"]
panic-by-itm = ["panic-itm"]
panic-by-mock = []

[features.panic-by-mock]
provides = ["global:panic-handler"]

Additionally, assume the dependencies panic-rtt-target, panic-halt and panic-itm each have a Cargo.toml containing:

provides = ["global:panic-handler"]

A provides declaration on the global section defines that a crate will always provide this capability.

With this configuration, Cargo will now ensure that exactly one implementation of a panic-handler is enabled in the build. It can also provide guidance on features to enable or disable.