Documentation Overhaul

[awwad]

User documentation has to be reorganized and updated.

The main README.md links to Getting_Started.rst, which among other things links to:

• Quickstart
• CLI
• CLI usage examples
• Tutorial

The problems:

• Each of these is actually a tutorial. They cover CLI client, CLI repo, and advanced repo (repository_tool).
• The CLI client is not covered very well.
• Tutorial.py is actually a detailed repository creation tutorial, so it should be renamed something like 'Advanced Repository Tutorial'
• Advanced client is not covered here (client/updater.py). There are some links and references in Tutorial.py about this, but that's not where they belong since that's really a repository tutorial, and they are wrong / outdated anyway. An unlinked-to tuf/client/README.md provides some instructions related to updater.py, but they need to be reviewed and linked to from Getting Started.
• Tutorial.py's client section is bad / out of date:
  • It talks about a repository example created in tuf/README.md, but that file has no real contents, so we need to delete it.
  • It links to a document (tuf/client_setup_and_repository_example.md) that talks about using basic_client.py, which no longer exists. That document should also be deleted. Any valuable copy should be moved to tuf/client/README.md.
  • Client Setup and Test TUF Locally sections should be merged and clarified. ("To test the repository we've just created, we need to connect to it...")

My current opinion is that we should move to having three tutorials instead of the many docs above:

• QUICKSTART, talking about the CLI client and repo, with these additions:
  • a note that these quickstart instructions will use the basic command-line interface, and that much more flexibility is available in "these advanced tutorials for repository_tool.py and updater.py", with links (to the docs that are currently TUTORIAL.md and tuf/client/README.md).
  • a bit more about the CLI client than it currently has
• Advanced Repository Tutorial -- largely, the contents of TUTORIAL.md
• Advanced Client Tutorial -- what's currently in tuf/client/README.md, updated and expanded a bit

We should also probably move those three docs to docs directory.

Miscellaneous other corrections:

• CLI client instructions should point out that it is necessary to run repo.py --init first, since that's actually what sets up the client directory....
• There's at least one missing cd-to-client-directory instruction in the CLI client instructions.

[awwad]

Note that Lukas has suggested a doctest-style validation of the tutorials, in response to a PR in which I added some regression tests for TUTORIAL.md.

[lukpueh]

UPDATE 2019-11-20:
Most of the concerns raised here are now addressed in #775 (struck through below)

---

I just went through TUTORIAL.md (as updated in #775) and noted a few things (first some general observations and then in the ordered of occurrence, not importance):

• There is a lot of text both between the code snippets and also as comment inside the snippets. Maybe we an make the tutorial more concise? @awwad, a.k.a master of words, to the rescue! I also have a couple of suggestions below, for things to leave out/shorten.

• Let's make all snippets # Continuing from the previous section . . .. (fixed in Fix TUTORIAL.md and add regression testing for future issues #775)

• In Fix TUTORIAL.md and add regression testing for future issues #775 @awwad already started consolidating " and ' for strings. But there is still a wild mix (see Single quotes (') or double quotes (") secure-systems-lab/code-style-guidelines#4 for a related discussion).

• The tutorial is almost exclusively about the server side of TUF. Should we extend it to talk some about the updater? Or replace the tiny client.py snippet with a :Next step: Client"-reference to something like
  tuf/client/README.md?

• Let's update the Python interpreter prompt to something newer than 2.7.3 to show that we are going with the times (or leave the prompt out altogether).
  [TUTORIAL.md#L46] (fixed in Fix TUTORIAL.md and add regression testing for future issues #775) (https://github.com/theupdateframework/tuf/blame/6b4a4b233105f07695fe712bb5a2c9c4fde47cfd/docs/TUTORIAL.md#L46)

• The "my_repo" repository created in the very beginning isn't ever used, instead we create another repository called "repository" shortly thereafter, which is used throughout. Let's only create one repo.
  TUTORIAL.md#L50 (fixed in Fix TUTORIAL.md and add regression testing for future issues #775)

• Let's not discuss the exception raised when a wrong password is entered.
  TUTORIAL.md#L147-L154

• Let's not show how to do the same thing for Ed25519 what we do for RSA keys. A side note or pointer to further docs should be enough. The Ed25519 keys don't fit in the continuity of the tutorial either. (using ed25519 key with Fix TUTORIAL.md and add regression testing for future issues #775)

• The first repository.status()-call output seems truncated.
  TUTORIAL.md#L248-L249~~ (fixed in Fix TUTORIAL.md and add regression testing for future issues #775)

• list_of_targets shows relative paths. I got absolute.
  TUTORIAL.md#L365-L366

• When referring to methods, they should be distinguishable from regular functions (e.g writeall() should be something like repository.writeall())
  TUTORIAL.md#L233~~

• repository.targets.add_target(target4_filepath, custom_file_permissions) gave me an exception
  f7f003d and (merged) Have repository_tool.get_filepaths_in_directory use absolute paths #774 mention some problem there.
  But rebasing did not fix it. IIUC target4_filepath should be relative to the targets dir.
  TUTORIAL.md#L385 (fixed in Fix TUTORIAL.md and add regression testing for future issues #775)

• $ repository.dirty_roles() should be >>> repository.dirty_roles()
  TUTORIAL.md#L505 (fixed in Fix TUTORIAL.md and add regression testing for future issues #775)

• Both Consistent Snapshots and Delegate to Hashed Bins sections don't seem to fit in with the rest of the tutorial (?). It's also not clear what the corresponding code snippets do. @awwad made similar observation in Misleading / missing steps in TUTORIAL for delegated roles #838.
  TUTORIAL.md#L544
  TUTORIAL.md#L574

• There is an unexpected comment about a virtualenv called "tufenv".
  TUTORIAL.md#L658-L660 (fixed in Fix TUTORIAL.md and add regression testing for future issues #775)

[lukpueh]

I also still stand by my idea of using doctest. It's actually something I'd like to play around with, because I also want to use it for in-toto (in-toto/in-toto#276).

@awwad, let me know what you think and if I should help out, e.g. by taking over #775.

[awwad]

That sounds great. :) Yeah. I like the doctest idea. I worried a bit that it might affect the readability or structure of the tutorial, but if it works, that'd be awesome. Also, the suggestions and issues raised here are good. If you can take a shot at taking over 775 and integrating this into that, that'd be appreciated.

[lukpueh]

This is already hinted at in the PR description, I just want to bring it back to our attention:

The client update process as described in tuf/client/README.md is outdated and does not align with what's described in the spec.

[trishankatdatadog]

@lukpueh Sorry, I don't understand the question. Shouldn't the client update process be higher level than what's described in the spec anyway?

[lukpueh]

What question, @trishankatdatadog? I just wanted to add a reminder that we need to update tuf/client/README.md.

It describes the old update process that starts by downloading timestamp.json, whereas in the current spec the update process starts by downloading root.json (also see spec#12).

[trishankatdatadog]

Oh, you mean the pseudocode steps in English in that README, not the actual code itself, got it

[lukpueh]

@mnm678 suggested to remind the tutorial reader of the password, used to encrypt the private key on generation, when loading the private key (see #775 (comment)).

[lukpueh]

#1001 removes incomplete and outdated tuf/client_setup_and_repository_example.md and a reference to it in TUTORIAL.md (kudos to @Silvanoc).

As suggested by @awwad above (see PR description), revised client documentation should go into tuf/client/README.md.

[lukpueh]

Re client update process steps (#808 (comment)):

@MVrachev has kindly updated them in client/updater.py and client/README.py. 🎉

[lukpueh]

None of the sub-issues identified above are relevant in python-tuf 1.0.0 anymore:

• legacy documentation has been removed
• up-to-date reference documentation now on RTD
• tutorial-like usage examples now in docs/examples
  (doctest-style testing as suggested above is no longer needed, as the tutorials are testable Python scripts (e.g. #1685))
• Remaining documentation issues are listed under the documentation label