Documentation Re-org #115

winder · 2023-07-06T21:08:51Z

Summary

There is a lot of good content in the docs, but they are spread out a little too much, and the organization is overly influenced by some limitations of the developer portal website.

This PR is intended to re-organizes and re-write the documentation.

Some of the major changes:

The top level README is now the GettingStarted page.
The Configuration page does not have enough content to justify its existence and has been removed.

Test Plan

N/A

codecov · 2023-07-06T21:40:10Z

Codecov Report

Merging #115 (8f6e343) into master (442791a) will increase coverage by 2.61%.
The diff coverage is 76.23%.

@@            Coverage Diff             @@
##           master     #115      +/-   ##
==========================================
+ Coverage   67.66%   70.27%   +2.61%     
==========================================
  Files          32       36       +4     
  Lines        1976     2493     +517     
==========================================
+ Hits         1337     1752     +415     
- Misses        570      646      +76     
- Partials       69       95      +26

Impacted Files	Coverage Δ
conduit/data/block_export_data.go	`100.00% <ø> (+92.30%)`	⬆️
conduit/metrics/metrics.go	`100.00% <ø> (ø)`
conduit/pipeline/metadata.go	`69.11% <ø> (ø)`
...duit/plugins/exporters/filewriter/file_exporter.go	`81.63% <ø> (-1.06%)`	⬇️
conduit/plugins/exporters/postgresql/util/prune.go	`78.43% <ø> (ø)`
conduit/plugins/importers/algod/metrics.go	`100.00% <ø> (ø)`
...gins/processors/filterprocessor/fields/searcher.go	`77.50% <ø> (ø)`
...ins/processors/filterprocessor/filter_processor.go	`83.82% <ø> (+3.54%)`	⬆️
...plugins/processors/filterprocessor/gen/generate.go	`34.28% <ø> (ø)`
conduit/plugins/processors/noop/noop_processor.go	`64.70% <ø> (+6.81%)`	⬆️
... and 16 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

tzaffi

My usual nit regarding adding a line between the end of each paragraph and the following code block plus a question regarding whether we should resolve the introduced TODO's in this PR.

README.md

tzaffi · 2023-07-07T20:28:44Z

README.md

+Once configured, start Conduit with your data directory:
+```sh


Suggested change

Once configured, start Conduit with your data directory:

```sh

Once configured, start Conduit with your data directory:

```sh

README.md

Co-authored-by: Zeph Grunschlag <tzaffi@users.noreply.github.com>

README.md

algoanne · 2023-07-20T14:29:33Z

docs/tutorials/WritingBlocksToFile.md

-  noop        - noop exporter
-  postgresql  - Exporter for writing data to a postgresql instance.
-```
+First we need to make sure we have Conduit installed. Head over to the installation section of [the README](../../README.md) for more details. For this tutorial we'll assume `conduit` is installed and available on the path.


just noticing that one of our two tutorials recommends going to the installation section of the README for install instructions (this one) and the other one just tells you how to do it from the binary. Should we try to be consistent across all our tutorials?

It looks like "IndexerWriter.md" directs people to the release page for a download.

algoanne · 2023-07-20T14:30:58Z

docs/tutorials/WritingBlocksToFile.md

-we can again use the list command. For example,
+`file_writer` exporter.
+To get more details about each of these individually, and their configuration variables,
+use the list command or check the [README](../../conduit/plugins/exporters/filewriter/). For example:


is this the right link? Maybe it should be to the top page of the plugins? And also maybe we should call it "plugins readme" or something?

Suggested change

use the list command or check the [README](../../conduit/plugins/exporters/filewriter/). For example:

use the list command or check the [plugins README](../../conduit/plugins/). For example:

Updated to reference the plugin TOC. I guess it could have links to each of the plugins instead.

docs/tutorials/WritingBlocksToFile.md

algoanne · 2023-07-20T15:34:49Z

docs/tutorials/WritingBlocksToFile.md

-Now we will fill in the proper fields for these. I've got a local instance of algod running at `127.0.0.1:8080`,
-with an API token of `e36c01fc77e490f23e61899c0c22c6390d0fff1443af2c95d056dc5ce4e61302`. If you need help setting up
+Now we will fill in the algod importer configuration. I've got a local instance of algod running at `127.0.0.1:8080`,
+with an API token of `e36c01fc77e490f23e61899c0c22c6390d0fff1443af2c95d056dc5ce4e61302` and aan admin API token of `2d9ceaf47debb8aff77c62475b8337f6ecef75c2aa8aa02170a8d38b9126b75a`. Find these in the `algod.token` and `algod.admin.token` files from your algod data directory. If you need help setting up
 algod, you can take a look at the [go-algorand docs](https://github.com/algorand/go-algorand#getting-started) or our
 [developer portal](https://developer.algorand.org/).


Hmmm I'm just thinking about how to help people get algod set up properly for conduit. In the other tutorial (Indexer Writer one) we do just walk them through how to do it. I wonder if that snippet could be included in the algod importer plugin documentation page, and then we could point to it from anywhere? Because there are specifics for the importer (in particular the follow mode) that people need to know. Here they wouldn't get that information.

Agreed, this link is not very useful to someone new.

Co-authored-by: algoanne <90275860+algoanne@users.noreply.github.com>

algoanne

I think there's still work to be done, but everything in this PR is a major improvement. We can follow-up with further updates in future PRs.

winder added 3 commits July 6, 2023 16:01

First pass with Anne.

d13b216

Delete Configuration.md

67cb2cc

Finish up the config and plugin introduction.

c064f70

Remove extra word.

7246fdc

winder force-pushed the will/doc-updates branch from 2eda6fa to 7246fdc Compare July 7, 2023 00:10

winder added 2 commits July 6, 2023 20:11

Syntax highlighting.

4daf773

Remove extra word.

e824fe7

tzaffi reviewed Jul 7, 2023

View reviewed changes

winder and others added 2 commits July 10, 2023 11:39

Apply suggestions from code review

a6ffb0c

Co-authored-by: Zeph Grunschlag <tzaffi@users.noreply.github.com>

Second pass with Anne.

5ea8f6b

winder commented Jul 10, 2023

View reviewed changes

README.md Outdated Show resolved Hide resolved

winder added 6 commits July 11, 2023 15:46

Will & Anne pt 3

0cbd31d

Will & Anne pt 4

e2ee331

Consistent naming.

fac238a

More edits with Anne

dcefeb3

WIP

a280f8f

Update WritingBlocksToFile.md

0e67a4f

winder marked this pull request as ready for review July 19, 2023 19:22

winder added 2 commits July 19, 2023 15:23

Remove example.

8062436

Remove completed TODO

ed2cdfb

winder requested review from algoanne and tzaffi July 19, 2023 20:09

algoanne reviewed Jul 20, 2023

View reviewed changes

Apply suggestions from code review

8f6e343

Co-authored-by: algoanne <90275860+algoanne@users.noreply.github.com>

algoanne approved these changes Jul 20, 2023

View reviewed changes

winder merged commit bddfb21 into algorand:master Jul 20, 2023
3 checks passed

winder deleted the will/doc-updates branch July 20, 2023 18:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Documentation Re-org #115

Documentation Re-org #115

winder commented Jul 6, 2023

codecov bot commented Jul 6, 2023 •

edited

Loading

tzaffi left a comment

tzaffi Jul 7, 2023

algoanne Jul 20, 2023

winder Jul 20, 2023

algoanne Jul 20, 2023

winder Jul 20, 2023 •

edited

Loading

winder Jul 20, 2023

algoanne Jul 20, 2023

winder Jul 20, 2023

algoanne left a comment

		Once configured, start Conduit with your data directory:
		```sh

	use the list command or check the [README](../../conduit/plugins/exporters/filewriter/). For example:
	use the list command or check the [plugins README](../../conduit/plugins/). For example:

Documentation Re-org #115

Documentation Re-org #115

Conversation

winder commented Jul 6, 2023

Summary

Test Plan

codecov bot commented Jul 6, 2023 • edited Loading

Codecov Report

tzaffi left a comment

Choose a reason for hiding this comment

tzaffi Jul 7, 2023

Choose a reason for hiding this comment

algoanne Jul 20, 2023

Choose a reason for hiding this comment

winder Jul 20, 2023

Choose a reason for hiding this comment

algoanne Jul 20, 2023

Choose a reason for hiding this comment

winder Jul 20, 2023 • edited Loading

Choose a reason for hiding this comment

winder Jul 20, 2023

Choose a reason for hiding this comment

algoanne Jul 20, 2023

Choose a reason for hiding this comment

winder Jul 20, 2023

Choose a reason for hiding this comment

algoanne left a comment

Choose a reason for hiding this comment

codecov bot commented Jul 6, 2023 •

edited

Loading

winder Jul 20, 2023 •

edited

Loading