Refactor: move rewrite inside hugr, Rewrite -> Replace implementing new 'Rewrite' trait

[acl-cqc]

• Hugr::apply_rewrite now just "bounces" off a method implemented by the trait. So we could actually remove Hugr::apply_rewrite altogether??
• Renamed Rewrite to Replace, although I expect it to be entirely replaced by SimpleReplace, Replace, etc. etc.
• The old Hugr::apply_rewrite code is now moved into an implementation of the trait method....
• ....which can see internals of Hugr by virtue of being in the new hugr::replace package

An alternative scheme is to use a fixed enum of the allowable operations: https://github.com/CQCL-DEV/hugr/compare/refactor/replace - but I think the trait version is better because:

• while trait implementations outside the hugr module won't be able to do much, the trait at least allows someone to construct "compound" operations containing a sequence of RewriteOp's that we provide, so it's not pointless;
• The trait avoids a big match, as calling the trait method implicitly does that for us, so I think modularity/legibility is improved
• However, as per first point, we could also just drop the trait altogether and have a bunch of "unrelated" methods...

[aborgna-q]

That's quite restrictive. Normally we'd use ? to resurface any internal errors during the rewriting. Those may happen at any point, so this would require doing (potentially costly) dry-checks before running.

[acl-cqc]

Yah this is tricky, and &mut Hugr -> Result<(), ...> may not be the correct signature.

Firstly I note the "are strongly encouraged". So this is not actually a restriction at all. It was the best I could come up with given the open/unsealed trait, but it does make me think that all of the "primitive" rewrites we provide should meet this recommendation. Which....is indeed restrictive, yes.

Ok, so what are the options for what (can) happen if an error is raised. We don't have an abort/commit transaction-like semantics built in, so essentially this is where we are defining one....

1. If an implementation raises an error, we could say the Hugr is in an undefined state. Without any guarantee, the Hugr is now basically useless, which is to say, you've lost your Hugr. IOW the signature should be apply(Hugr) -> Result<Hugr, E>.
   • Recovering from such situation basically means, the client restores the Hugr from a backup. Said client might be able to say, if that fails, I only need to recover this part of the Hugr, and only back up that part.
2. Perhaps we insist that the Hugr, while it may have been mutated, still passes validate()? That could be extremely tricky to verify - you'd have to check failure at every possible point in the rewrite! - and still not terribly useful.
3. Returning a Hugr where we can get back the original "by calling a no-args cleanup method" seems hard - such a cleanup method would not know what the rewrite did, so this will still be restrictive on the rewrite. Hence, I think we can dismiss that in favour of the strictly-more-flexible:
4. We change "strongly encouraged" to "required". As noted, this is very demanding on implementations; they must either (a) make all checks before they begin, (b) record the changes made sufficiently to undo them - perhaps by performing in some order that makes this easy (e.g. if "removing all unconnected nodes" was sufficient to undo), (c) backup the Hugr before they start (perhaps they need only backup a small portion if they know they will only change that part of the Hugr).
   • (b) and (c) might be easier here if, rather than leaving the Hugr unchanged upon failure, the impl instead provided a (perhaps-expensive) function to retrieve it if the client wants. I guess that'd be a signature like apply(Hugr) -> Result<Hugr, (FnOnce<() -> Hugr>, E)>

Comparing 1. (client backups) and 4. (impl undoes), the former has the advantage that clients who know they can't continue if the rewrite fails, can perform the rewrite without incurring the overhead of backing up; whereas the latter has the advantage that surely the implementation knows better than the caller how to undo its own changes. However, there are more options...

5. Perhaps what we are searching for is the ability to provide a localized mutable view onto part of the Hugr; if the rewrite fails, then that part is undefined (i.e. lost), but at least we can guarantee the rest is unchanged. Then, clients would know what part of the Hugr to backup if they wanted.
6. Or, perhaps we need a separate trait method validate(&Hugr) -> bool. The contract could then be one of:
   • If the provided Hugr passes validate() before the rewrite is applied, and the rewrite validates on the Hugr, then applying the rewrite either succeeds or fails without changing anything (No guarantees if either Hugr or rewrite fails validation)
   • If the provided Hugr passes validate() before the rewrite is applied, and the rewrite validates on the Hugr, then applying the rewrite succeeds; no guarantees if either Hugr or rewrite fails validation
   • If the provided Hugr passes validate() before the rewrite is applied, and the rewrite validates on the Hugr, then applying the rewrite succeeds; if the Hugr validates but the rewrite does not validate, then applying the rewrite either succeeds or fails-without-changes; if the Hugr does not validate, no guarantees

[aborgna-q]

Transactional semantics is it's own beast. Anything beyond 1. will incur a hit to performance, that we may perhaps prefer the user to have control over.

2. (3) (4) don't seem too practical without lots of extra overhead.

5. May be an option for region-focused rewrites, but not necessarily for every rewrite.
6. The pre-execution predicates are a thing in tket, and I think something we wanted to move away from.

I'd say the general rewrite interface shouldn't require this. I propose:

7. Leave these assurances to each Replace operation. Add Transactional<> wrappers or perhaps specialized versions (e.g. struct TransactionalSimpleReplace(SimpleReplace)) so the users that actually want to pay the cost can choose to.

@cqc-alec and @ss2165 may have opinions, but we could move this to a discussion too.

[acl-cqc]

Ok, I'm trying to go with 6. but do it better here. As per slack, this allows

• rewrites with trivial may_fail_destructively()==false if they can check ahead in apply()
• rewrites with trivial may_fail_destructively()==true and no guarantees on apply()
• A transactional wrapper to turn the second case into the first
• Other rewrites to do anything in between
• Clients to have guarantees if they want them (and pay for them when necessary), but not if they don't

[acl-cqc]

It's possible that after we've implemented all of our built-in primitive Rewrites we can reduce the allowable behaviours here but I'm keen to see how (painful) they turn out before we do that.

[acl-cqc]

Hmmm, perhaps I should state that apply may panic if-and-only-if the Hugr doesn't validate()?

[acl-cqc]

Ok, I think I've enacted as discussed. Associated constants still seem a bit WIP (see rust-lang/rust#92827), but the bulk of the work appears to have now been done on nightly (rust-lang/rust#70256)

[acl-cqc]

An alternative is to return Result<(), Option<E>> where returning Err(None) means, no guarantee can be given about apply. If a rewrite really has to get partway through in order to complete its own validity checks then said alternative would support that. But I think we should leave as Result<(), E> until/unless we find a Rewrite where that's difficult.

(Compound Rewrites is one such case. For a sequence of rewrites, verify might have to clone() the Hugr, and then step through the sequence, applying each rewrite before verifying the next.)

[aborgna-q]

I'd say custom enums with {Recoverable, NonRecoverable} variants rather than Option<E>.
But yeah, let's keep it simple for now.

[acl-cqc]

Ok, I've now merged Alec's SimpleReplace with this, thus applying the same refactoring to his code. I haven't managed to implement verify.

I'm tempted to make verify return a "proof" of ability-to-perform rewrite rather than (), that is:

trait TransactionCommitter { // One would like to use FnOnce here but there are Rust problems
  fn commit(h: &mut Hugr) -> ();
}

trait Rewrite<E> {
  type Committer: TransactionCommitter;
  fn verify(&self, h: &Hugr) -> Result<Self::Committer, E>;
  fn apply(&self, h:&mut Hugr) -> Result<(), E> {
    self.verify(h)?.commit()
  }
}

but I think that can be another PR (as it's somewhat contentious) - I'm hoping this isn't anymore

[acl-cqc]

@cqc-alec all this code was basically moved here, modulo renaming self -> h and r -> self. I'm not sure about this .ok() though - that converts the Result to an Option, and then throws away. Should we '.unwrap'? Happy to make a driveby change in this PR if you agree

[cqc-alec]

Yes happy with the drive-by change.

[acl-cqc]

@cqc-alec and here?

[aborgna-q]

Main comment is replacing the generic parameter with an associated error type in Rewrite.

Merging HEAD may break some things.

[aborgna-q]

I'd say custom enums with {Recoverable, NonRecoverable} variants rather than Option<E>.
But yeah, let's keep it simple for now.