Rust at 0x

Our goals for the CI are:

• 🤖 Automate as much as possible
• 🧐 Provide QA metrics like coverage
• 🏎️ Keep PR acceptance time under 5 min.

CI Setup

[Insert screencap from CircleCI]

Linters

rustfmt, clippy, codechecks

We enable lot's of lints in rustfmt.toml and enable many clippy rules in lints.rs.

Fast builds

Build containers with precompiles, sccache + cache. Parallel builds using artifacts.

Checking for nostd

WebAssembly builds

wasm-gc, twiggy

Code Coverage

https://users.rust-lang.org/t/howto-generating-a-branch-coverage-report/8524

'To do' tracking

More tools

cargo-outdated, cargo-audit, cargo-geiger

List of hacks and workarrounds

• We use Cargo.toml's replace to inject a local crate named hex-literal to override the one used by substrate-runtime. The outdated version used by substrate is not compatible with nostd.

• We need to copy lints.rs manually to each lib.rs and main.rs file. The CI checks that this is done correctly. There is currently no way to share a clippy configuration in a Cargo worskpace. Relevant issue.

• There is no structured way to reject includes of std libraries in a nostd build. If these get included things will fail far down the pipeline when linking the WASM blob, with no indication of the origin. To force the build to immediately on usage of std, we build for a target that has no standard library support. In our case we make an ARM Cortex M3 build. This build will fail.

• Cargo does not support --no-default-features when building a workspace. To work arround we loop over the list of packages.

• const [T; N] where T does not implement Copy can not be initialized using a repeat expression [t; N]. To work around we manually repeat t.

• Procedural macros for expressions require three additional crates to be implemented: 1) A regular library containing the macro implementations over proc_macro2::TokenStream, 2) An implementation crate containing proc_macro::TokenStream wrappers with proc-macro = true set. 3) A crate / See the proc-macro-hack docs. The additional library in our case comes from the requirement that macro functions are available in a library for composition.

• Cargo doc markdown does not support Latex formulas. We work arround this by including the Katex javascript library.

• Cargo doc does not support the --html-in-header argument from rustdoc. We work arround this using the RUSTDOC_ARGS environment variable, but this can not be automated using cargo aliases. TODO: Does this actually work?

• The no-std build for ARM Cortex M3 requires nightly, we can not automate this using cargo aliases. We workarround this by manually specifing cargo +nigthly.

• Macros live outside of the namespacing system.

• Code coverage requires nightly and non-incremental build: https://github.com/rust-lang/rust/issues/42524

• The code coverage build requires a large number of compiler flags, and some of them don't always work. https://github.com/rust-lang/rust/issues/63047

The following multi-stage rube-goldberg workarround deserves special mention:

• The feature flag system in cargo is intened to be strictly additive. But the absense of the std flag is a feature. Cargo will readily take unions on flags during build processes, and this leads to failures, in particular when:

• If a dev-dependency depends on some feature flag being set on a package, that feature flag will also be set for the regular build. In particular our benchmarking rig criterion sets the std flag on some of our dependencies, breaking the nostd build. To work arround this, we would like to make dev-dependencies optional, but:

• Dev-dependencies can not be optional. To work arround this, we turn all dev-dependencies into regular optional dependencies behind feature flags like test and bench. We use required-features to make sure these flags are set for test and bench builds. When running a test, we need to do cargo test --features=test. But:

• In a worskpace command, you can not provide feature flags like --features=test. The only flag available is --all-features, which is what we use. To avoid cumbersome commands, we provide aliases for build and test. (See https://github.com/rust-lang/cargo/pull/7507)