Payout leaderboard for KodaDot

[yangwao]

We are all monkeys.
And would be nice to build trust for external contributors,
where external contributors could easier justify
that we do payouts on regular basis and see ppl how they contribute etc
The requirement to run as serverless function, best if we can deploy it in some public way, using autocode.xyz or something like that, should be few lines and cracking graphql and right queries
We'd like to put more emphasis on past records.

Spec

• make file payout_leaderboard.md of this KodaDot repository
• scrape all PRs where they have kusama.subscan.io URL referring to a particular transaction
• identify developers contributors
• parse tx and put their balance received on the address, it should be matching what has developer put in the payout link, the format has changed, so mind checking since we've started using a new PR template
• in file show github handle / total amount received / (=total amount recieved/merged PRs) / number of PRs made / merged PRs / closed PRs / lines added/removed / number of total commits merged / link to last received contribution on subscan
• payout_leaderboard.md should be updated on every PR which was merged with payout to reflect newest numbers.

show github handle	total amount received	(=total amount recieved/merged PRs)	number of PRs made	merged PRs	closed PRs	lines added/removed	number of total commits merged	link to last received contribution on subscan

[jbienesdev]

Hi @yangwao this is an interesting feature. Mind if I start creating a script fot this. I'm gonna use Javascript if it's fine for you. Will ask questions if I have any.

[yangwao]

Hi @yangwao this is an interesting feature. Mind if I start creating a script fot this. I'm gonna use Javascript if it's fine for you. Will ask questions if I have any.

Sure, I'm planning to write '21Q4 article for kodadot so if we could get at least some static snapshot numbers, that would be amazing most probably and happy to mention you in the article! :)

[jbienesdev]

That would be awesome, @yangwao ! Really excited to make this work! 🚀

I started to dabble with autocode and I'm liking it so far. Researched a bit on how to query Subscan's KSM value here: https://docs.api.subscan.io/#introduction. Although I need an api key to avail of their higher quotas. Just want to ask if you have any api keys available or I can go ahead and submit the form, making one myself?

Lastly, maybe you can also help me comment the extrinsic hash to older merged PRs that has a txn, like this PR here #692 . You can use this search string is:pull-request https://kusama.subscan.io/ is:closed repo:kodadot/nft-gallery on the pull requests tab. Older PRs doesn't have an extrinsic hash on the comments. I am actually querying each PRs comments and matching with https://kusama.subscan.io/extrinsic/.

Feel free to suggest anything if you have any. Thanks a lot! 😄

[yangwao]

That would be awesome, @yangwao ! Really excited to make this work! 🚀

I started to dabble with autocode and I'm liking it so far. Researched a bit on how to query Subscan's KSM value here: https://docs.api.subscan.io/#introduction. Although I need an api key to avail of their higher quotas. Just want to ask if you have any api keys available or I can go ahead and submit the form, making one myself?

I don't have anything on hand, let's request new

Lastly, maybe you can also help me comment the extrinsic hash to older merged PRs that has a txn, like this PR here #692 . You can use this search string is:pull-request https://kusama.subscan.io/ is:closed repo:kodadot/nft-gallery on the pull requests tab. Older PRs doesn't have an extrinsic hash on the comments. I am actually querying each PRs comments and matching with https://kusama.subscan.io/extrinsic/.

Tricky I see, let's try to do some fix using my address as most of the payouts we're sent through that and match connection to address written in payout section, what about that?

[jbienesdev]

@yangwao this is noted. I'll try this implementation and will let you know if it works. Thanks!

[yangwao]

@yangwao this is noted. I'll try this implementation and will let you know if it works. Thanks!

Hey, any progress on this? So I can publish article with some chart already:)

[jbienesdev]

@yangwao apologies for the slow progress on this feature as I am on vacation currently. I am still waiting for the api key from Subscan and will wait for their email so that I can continue the development. For the meantime, I am able to query Github PRs with payout addresses correctly and is still on the process of cleaning some of the data coming from Github.

Data that is difficult to scrape is the lines added/removed since Github's api is not returning this information. Maybe we can use any website traversing library to capture these values. https://github.com/kodadot/nft-gallery/graphs/contributors

[yangwao]

waiting for the api key from Subscan

I can ask around as they usually reply within few days.

This sounds awesome!

Data that is difficult to scrape is the lines added/removed since Github's api is not returning this information. Maybe we can use any website traversing library to capture these values. https://github.com/kodadot/nft-gallery/graphs/contributors

I'm thinking that what if we could make it as GitHub Action flow? Like once PR is merged, it could automatically make commit to leaderboard.md for example with updated stats.

How do you "store" data right now?
Does gh returns all data well?
For subscan with more transactions probably you don't want to scrape same data again..

[jbienesdev]

I'm thinking that what if we could make it as GitHub Action flow? Like once PR is merged, it could automatically make commit to leaderboard.md for example with updated stats.

Yes! This is what I am thinking. I can test it out if it's working on a separate repo.

How do you "store" data right now?

I haven't thought about it yet but one thing that came into my mind is maybe use an online database like Supabase to track and cache data in order to avoid calling the same data from Subscan.

What do you think about the plan? Feel free to suggest any other tools that you know of.

[yangwao]

We are planning to use CF KV workers for storage as it's fast and convenient tho https://www.cloudflare.com/en-gb/products/workers-kv/

I'm not familiar with Supabase, checking their pricing, it's not something pay as you go I would prefer, but if that initial version we can have free tier, till then I'm okay with that :)

Generally, I would research something which storage solution is popular with serverless functions, like integrations etc.

[yangwao]

Hey @jbienesdev how it's going? 😇

[yangwao]

Seems issue is stalled :|
If anyone else want to take on this, happy to help.

[petersopko]

Seems issue is stalled :|

If anyone else want to take on this, happy to help.

Let's do it, I'll have approx. 20h for KodaDot this weekend, but I'll most likely need some guidance. Also, I'd pick Python over JS for this if that's okay.

[yangwao]

The requirement to run as serverless function, best if we can deploy it in some public way, using autocode.xyz or something like that, should be few lines and cracking graphql and right queries

Also, I'd pick Python over JS for this if that's okay.

We are looking for something serverless and I guess Typescript would be optimal as higher chance someone knows TS than Python, however, I'm aware that python is more data science.

Optionally Rust would be best and I guess we can run it as Github Action.

but I'll most likely need some guidance.

Most of the questions have been answered here but feel free to ask another one :)

[petersopko]

Okay, after clarification on Discord, since I've kind of misunderstood the hackathon part, I guess I'll just look into some solutions in TS for now and see how will the Hackathon play out.

[yangwao]

oh noticed that we should handle failed transactions.. https://kusama.subscan.io/extrinsic/0x310dbcec733682bfcd604ada1f5bf65fba1fd10b9848933e1e906172963c32c4 )))

[petersopko]

oh noticed that we should handle failed transactions.. https://kusama.subscan.io/extrinsic/0x310dbcec733682bfcd604ada1f5bf65fba1fd10b9848933e1e906172963c32c4 )))

they are accounted for in the table. I checked every transaction and included only the ones which had status "Success". Failed transactions won't ever show up :)

[niklasp]

where can i find that document?

[petersopko]

where can i find that document?

current version here :)