FVM versioning and technical debt

[lemmih]

The fvm crate can be difficult to use natively from Rust for a few reasons, including:

• The incompatibility of fvm-2 and fvm-3,
• the lack of semantic versioning,
• primitive types are dated and that apparently shouldn't be used[1],
• many dependencies (like cid) are frozen and cannot easily be upgraded (this is both an annoyance and a security issue).

I'd like to work towards a future where we have a single crate that captures all of the complexity of the Filecoin network, offers a stable interface that other players in the ecosystem can depend on, and actively try to reduce the amount of code that has to be maintained (by de-duplicating code that is shared by fvm-2 and fvm-3).

The Forest team has a lot of Rust experience and we can devote several engineers to work on this full-time. What are you thoughts on this and what are your plans for future versions of the FVM?

[1]: For example, the fvm_shared::crypto::signature::Signature type doesn't support delegated signatures which are required to connect to a filecoin network. The Forest team was told that clients aren't supposed to use the types exported by the FVM.

@jennijuju @arajasek @maciejwitowski

[Stebalien]

I'd love to see work towards this now that the launch chaos has died down. The current situation is... nasty.

I'd be very interested in separate fvm1, fvm2, etc. crates instead of using major versions (#358 is in scope right now). I'll just call these FVM epochs for clarity.

We also definitely need to split up and reduce the scope of fvm_shared:

1. Crate 1: Basic types that cross between the FVM and actors via syscalls. These types need to be versioned along with FVM epochs (i.e., fvm_shared1, fvm_shared2). Types that don't change between epochs shouldn't be included.
   • Until we start allowing wasm actor deployment. At that point, we'll have to freeze the these types. But we're not ready for that yet.
2. Crate 2: Types that cross the boundary between the FVM and the client, like Address. These types need to be shared by all FVM epochs. However:
   • We need to be very careful with these types to avoid consensus breaking changes. See https://github.com/filecoin-project/go-state-types/blob/393e0b6ffa8839295e6c909b23e57b4a2a099ca5/abi/address.go#L13-L28.
   • This should not include types like Signature (that's really a client-only type).

I'd love to see some prototypes and/or design proposals here.

[lemmih]

@Stebalien Could you write a summary of the discussion in #358. I feel we discussed a lot of things but didn't reach any consensus regarding the path forward. Is that your interpretation as well?

In particular, do we agree on the following:

1. All FVM crates should follow semantic versioning (with no chaos exceptions).
2. The complexity of the FVM versions should be hidden (raul called this a "ref-fvm selector"). Such a crate is interconnected with the FVM and should live in the same repository.
3. Type compatibility should not be broken unless necessary.
4. Old FVM versions have to be actively maintained and have their dependencies be kept up-to-date.

[Stebalien]

Current plan (in progress):

1. Move fvm, fvm_sdk, and fvm_shared to a v2 directory, renaming to fvm2*.
2. Pull fvm, fvm_sdk, and fvm_shared from the release/v2 branch into a v1 directory (renaming to fvm1*).
3. Delete the v1 integration tests, only keeping integration tests for the latest version to reduce maintenance burden.
4. Add conformance (test vectors) for old versions. This is what we want to test anyways (for historical versions, at least).