blob get fails and locks up SDK

[tzarebczan]

The Issue

While this is running, the sdk fails to shutdown on first attempt.

C:\Users\thoma\Desktop>lbrynet blob get 17aa0c77c404bed99c84c35b1d3621244a611ef9bb522b8db7f3bacaf64c54098b5fe4c1ccabaf656cf843f916fc5e8a --timeout=5
Traceback (most recent call last):
  File "lbrynet\extras\cli.py", line 322, in <module>
  File "lbrynet\extras\cli.py", line 310, in main
  File "asyncio\base_events.py", line 573, in run_until_complete
  File "lbrynet\extras\cli.py", line 32, in execute_command
  File "site-packages\aiohttp\client.py", line 1005, in __aenter__
  File "site-packages\aiohttp\client.py", line 497, in _request
  File "site-packages\aiohttp\client_reqrep.py", line 857, in start
  File "site-packages\aiohttp\helpers.py", line 585, in __exit__
concurrent.futures._base.TimeoutError
[57460] Failed to execute script cli

System Configuration

• LBRY Daemon version:
• LBRY App version:
• LBRY Installation ID:
• Operating system:

Anything Else

Screenshots

Internal Use

Acceptance Criteria

Definition of Done

• Tested against acceptance criteria
• Tested against the assumptions of user story
• The project builds without errors
• Unit tests are written and passing
• Tests on devices/browsers listed in the issue have passed
• QA performed & issues resolved
• Refactoring completed
• Any configuration or build changes documented
• Documentation updated
• Peer Code Review performed

[belikor]

This bug is still not solved. I only added a note in #3354, so I can investigate further what is causing this error.

[cristi-zz]

Hi! I just (forked) cloned lbry-sdk commit be544d6 and started locally in debug mode.
I can confirm, the bug is still present.

When I try to download something (eg using get) everything works as expected. When I am trying to download a blob using its blob_hash and blob_get method, the daemon outputs lbry.dht.protocol.iterative_find:379: blob peer search bottomed out

So, @lyoshenka I would kindly suggest opening the bug again.

@belikor are you actively working on this bug now? I will start a bit of investigation on my own, but please stop me if you found a solution/are close/etc.

[cristi-zz]

I think I got it but the problem is a bit trickier. Found the actual cause (One has to load some default peers into the DHT node before starting the actual blob download) but the root cause is bigger. Some larger code shifting must be performed, that is a bit beyond my knowledge about lbry-sdk.

There are two paths towards BlobDownloader.download_blob() The right one, that works, is from StreamDownloader that loads some fixed peers into the node before triggering the download. The "wrong" one, that timeouts, is from `blob_exchange.downloader.py.download_blob() that does not do this peer loading step.
Loading some peers (from config object) into the node, solves the problem.

PR #3368 . I would recommend NOT to be accepted as is but suggest some refactoring to proper prime the node for download both from StreamDownloader and other "random" places (like downloader.py.download_blob()) After all, the know how is in the config object that is available.

Maybe a static method in the Node class? (There are some processing needed from Config.fixed_peers to KademliaPeer objects)

[belikor]

@belikor are you actively working on this bug now? I will start a bit of investigation on my own, but please stop me if you found a solution/are close/etc.

I am not working on this at the moment, so feel free to propose your own solution.

I noticed that the script located in scripts/download_blob_from_peer.py is precisely able to download an arbitrary blob (which is then immediately removed). So my intention was to compare the code in this script with the code in daemon.jsonrpc_blob_get, and try to figure out what needs to be fixed in the latter, but I've been busy with other things, so no progress on this specific issue from my side.