Several incompatibilities with master branch of securesystemslib

[joshuagl]

As part of my work to remove securesystemslib.util.TempFile in secure-systems-lab/securesystemslib#180 I need to be able to test tuf against a locally installed copy of securesystemslib.

It turns out that there are several incompatibilities between tuf and the master branch of securesystemslib (perhaps unsurprisingly, there are 214 commits to ssl since the 0.11.3 release). I've started working through the errors, but wanted to file this so that there's a record of the work and to have a discussion area.

There are at least two existing PRs which address some of the issues:

• Adjust to removals from securesystemslib Adjust to removals from securesystemslib #855
• Use TUF specific RELPATH[S]_SCHEMA formats as they have been removed from securesystemslib Use TUF specific formats as they have been removed from securesystemslib #912

These don't break compatibility with the testing against ssl 0.11.3 that's currently happening in CI, however I'm anticipating that some of the changes that will need to be made will either a) be incompatible or b) require different codepaths for different ssl versions.

I'd welcome thoughts on whether backward compatibility is a goal or whether we just need to request a release of ssl and require that version once the changes are ready?

It would be good to add some CI testing against ssl master as part of resolving this issue, but I haven't figure out how to handle that yet. My local testing is using a venv and pip installing my copy of securesystemslib and tuf, before running the tests against the venv with:
$ coverage run aggregate_tests.py

[lukpueh]

Thanks for pointing this out, @joshuagl. Here are my 2 cents:

The main branch, i.e. develop must always have passing tests.

tuf's tests don't need to be backwards compatible when adopting breaking changes from securesystemslib, such as 4e6942f.

I suggest to prepare these PRs but hold off on merging until securesystemslib bumps the release. For in-toto I submitted a PR, similar to your patch above, as "draft" and added a little disclaimer (see in-toto/in-toto#299).

If this strategy becomes to cumbersome, e.g. we end up having many PRs that adopt breaking changes from securesystemslib and maybe depend on each other, we could add an alternative main branch, e.g. develop-for-sslib, into which we merge PRs, whose tests only need to pass with securesystemslib@master (see #915).

What do you think? And what do others think? @SantiagoTorres, @mnm678?

[joshuagl]

I like this. Fully agreed that all tests must pass on the active development branch, I'll create some draft PRs.

[joshuagl]

Just to round up the pending PRs which address this issue:

• Adjust to removals from securesystemslib #855 (merged)
• Use TUF specific formats as they have been removed from securesystemslib #912 (merged)
• Explicitly encode data passed to securesystemslib.keys #917 (requires an SSL release)

[lukpueh]

Fixed for and with https://github.com/theupdateframework/tuf/releases/tag/v0.12.0. Thanks for your contributions, @joshuagl!