Generic trait methods and new auto traits

I want to wrap up my consideration of the idea of adding new auto traits to Rust with some notes from a conversation I had with Ariel Ben-Yehuda.

You can read these two previous posts for context:

• Changing the rules of Rust
• Follow up to “Changing the rules of Rust”

Our goal in this exercise is to introduce a new auto trait to Rust without breaking existing code which currently assumes all types have a certain behavior. In the second post, I showed a somewhat pernicious example of how difficult it is to make code in the new world compatible with code from the old world, which depends on generic trait methods.

The most common response I received was that there was, effectively, a “breaking change” in transitioning the crate foo from 2021 to 2024, and therefore the error was in the bar crate. Ralf Jung’s comment summarizes this position well:

I think of this not as a firewall but as a desugaring, which is generally how I approach editions: all code in earlier editions is implicitly desugared into the current edition (or, alternatively and probably more realistically, they are all collectively desugared into “cross-edition Rust”).

When you write this in edition 2021:

pub struct Bar;

impl foo::Foo for Bar {
    fn foo<T>(input: T) {
        std::mem::forget(input);
    }
}

It desugars to

pub struct Bar;

impl foo::Foo for Bar {
    fn foo<T: Leak>(input: T) { // `Leak` bound added by desugaring
        std::mem::forget(input);
    }
}

and then obviously this leads to an error since the trait bounds in the impl are more restrictive than in the trait.

He astutely notes the implication of this interpretation:

The problem is that as a consequence, any 2024-edition trait that doesn’t have Leak bounds on the generics in its functions cannot be implemented in 2021-edition code (unless those functions are defaulted, in which case it can at least still use the default). Which seems pretty bad? E.g., when core gets moved to 2024, we’d likely have to add an explicit Leak bound to Iterator::chain/zip/… or we’d break existing impls that overwrite these methods. That’s not looking good…

In other words, this problem appears similar to the problem with the ?Trait approach from the first blog post, except now it is a problem of generic methods rather than associated types: in order to be backward compatible with existing code, stable APIs would need to be more restrictive than they actually should have to be.

I think this interpretation is right: there is a clear “easy answer” here. But it is unsatisfying. Just like the backward compatibility issues with ?Trait, this makes me question whether it would really be a good idea to add a new auto trait like this, because it will leave the language in a bad state in which certain interfaces have arbitrary restrictions for backward compatibility reasons.

Fortunately, there are only a few of these interfaces in std (though other fundamentally valuable crates have more, like serde):

• Iterator, which has many generic combinator methods
• FromIterator and Extend, which each have a method generic over any iterator

These two examples are different: in the first case, the combinator methods have a provided implementation, which users are not actually meant to override. In the second, the methods are required when implementing these traits (though forgetting the argument would be pathological code in both cases). I’m going to discuss a solution, whereby we can backward compatibly relax the generic bounds on these methods to not require the new auto trait when the edition upgrade occurs.

Both of these ideas rely on this important fact: a type T: Leak can also be passed to an interface expecting any T. Thus, interfaces which have the Leak bound (because they are in the old edition) can forward to interfaces which don’t.

Provided methods (Iterator::map et al.)

An attribute would be added which could be added to provided methods, which causes the compiler to generate two versions of the method, each with the same default definition. In one, a Leak bound is added to every type parameter. In the other, it is not. Old editions call the former, whereas new editions call the latter. If a user in an old edition overrides this method, they only override the method in the old edition.

Possibly, the language could support a syntax for calling the old edition method from the new edition. This would be necessary for the strictest definition of backwards compatibility: if it is important that calls to this method dispatch to the overridden version. However, these are all methods which are not meant to be overridden by users, so the project could determine that this is an allowable change of behavior in the edition.

Required methods (FromIterator::from_iter et al.)

For traits with generic required methods, an attribute would be applied to the trait, creating effectively two versions of the trait: one with the bound added to every method generic, and one without. All of the second version implement the first version, but not vice versa. Whenever a user in an old edition implements the trait, they implement the first one. For them, the second version effectively doesn’t exist.

When a user in the new edition tries to prove that a type implements the trait, if implementation is from the old edition, they only get the first version of the trait, not the second. If they need the second version (because they are passing a !Leak type to the interface), they get an error.

If the type that implements the trait is a generic and a user needs to limit themselves only to supporting the second version of the trait, some additional syntax would need to be added to the bound to indicate that.

Under these rules, here’s what would change with the example in the previous post:

In the crate foo, the Foo trait would gain an attribute (let’s call it #[leak_compatible] for now) indicating that it uses this compatibility layer:

#[leak_compatible]
trait Foo {
    fn foo<T>(_: T);
}

In the crate baz, because it calls T::Foo on its !Leak type Baz, an attribute must be added to its bound T: Foo:

struct Baz;

impl !Leak for Baz { }

fn baz<T: #[no_leak] Foo>() {
    T::foo(Baz);
}

Now, when quux tries to call baz::baz with bar::Bar, it gets a compiler error.

Conclusions

There’s obviously some similarity to things the language team is already working on like “RTN,” which allows adding bounds to the anonymous return type of a trait method, though here the situation is somewhat different, because its not the return type, but a generic parameter on the method, that needs an additional bound.

It’s always possible that more holes will be poked in this “firewall” proposal, more interfaces that could not be backward compatibly upgraded to accepting !Leak types without opaque syntax. As the complexity of the compatibility layer grows, one begins to wonder whether or not it is really worth the cost.

This series of posts hasn’t really been about proposing the Rust project do anything - I have complex, contradictory internal feelings about the idea of supporting linear types in a future edition of Rust - it’s just about exploring more seriously the difficulties of making such a fundamental change to the rules of the language. I felt that previous commentary on this issue was too flippant about the idea of making such a change, and didn’t adequately advance the cause of understanding what would be required.