bug: whatportis db purge failed

[psyray]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

Seems that whatportis didn't purge correctly his db
Here is the log file with error

celery_1       | Previous database will be updated, do you want to continue? [y/N]: Downloading https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.csv...
celery_1       | Populating database...
celery_1       | Traceback (most recent call last):
celery_1       |   File "/usr/local/bin/whatportis", line 8, in <module>
celery_1       |     sys.exit(run())
celery_1       |   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1157, in __call__
celery_1       |     return self.main(*args, **kwargs)
celery_1       |   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1077, in main
celery_1       |     with self.make_context(prog_name, args, **extra) as ctx:
celery_1       |   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 943, in make_context
celery_1       |     self.parse_args(ctx, args)
celery_1       |   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1408, in parse_args
celery_1       |     value, args = param.handle_parse_result(ctx, opts, args)
celery_1       |   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 2400, in handle_parse_result
celery_1       |     value = self.process_value(ctx, value)
celery_1       |   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 2362, in process_value
celery_1       |     value = self.callback(ctx, self, value)
celery_1       |   File "/usr/local/lib/python3.10/dist-packages/whatportis/cli.py", line 83, in update_db
celery_1       |     populate_db()
celery_1       |   File "/usr/local/lib/python3.10/dist-packages/whatportis/cli.py", line 63, in populate_db
celery_1       |     db.purge()
celery_1       |   File "/usr/local/lib/python3.10/dist-packages/tinydb/database.py", line 255, in __getattr__
celery_1       |     return getattr(self.table(self.default_table_name), name)
celery_1       | AttributeError: 'Table' object has no attribute 'purge'
ce

Expected Behavior

No error and whatportis db should be truncated

Steps To Reproduce

launch celery container

Environment

- reNgine: 2.0
- OS: Debian 11
- Python: 3.10
- Docker Engine: 24.0.1
- Docker Compose: 1.29.2
- Browser: FF 118

Anything else?

Problem comes from TinyDb
According to this changelog, It seems the table.purge has been renamed table.truncate in v4.0.0

Whatportis must be updated to 0.8.2, which contains db.truncate instead of db.purge
https://github.com/ncrocfer/whatportis/blob/59a1718bf7c531f2a5a4e213cad0c047ce9c1c94/whatportis/cli.py#L63C18-L63C18

Anyway after further investigation, it seems whatportis last version is really too old (2 years) and if you upgrade it with the following pip command pip install --upgrade whatportis it downgrades a lot of python libs needed by celery, so It could bring a lot of others problems.
I think you should forget whatportis usage, or pip upgrade whatportis without downgrade dependency or only do a sed to change purge by truncate
I've tested with sed and it works fine
sed -i 's/purge()/truncate()/g' /usr/local/lib/python3.10/dist-packages/whatportis/cli.py
But it's not elegant, it's a quick and dirty fix for python 3.10 users only

Here the full pip upgrade log of whatportis inside the celery container

Requirement already satisfied: whatportis in /usr/local/lib/python3.10/dist-packages (0.8)
Collecting whatportis
  Using cached whatportis-0.8.2-py3-none-any.whl (7.5 kB)
Collecting simplejson==3.17.2 (from whatportis)
  Using cached simplejson-3.17.2.tar.gz (83 kB)
  Preparing metadata (setup.py) ... done
Collecting tinydb==4.4.0 (from whatportis)
  Using cached tinydb-4.4.0-py3-none-any.whl (21 kB)
Collecting requests==2.25.1 (from whatportis)
  Downloading requests-2.25.1-py2.py3-none-any.whl (61 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.2/61.2 kB 1.9 MB/s eta 0:00:00
Collecting prettytable==2.1.0 (from whatportis)
  Downloading prettytable-2.1.0-py3-none-any.whl (22 kB)
Collecting click==8.0.1 (from whatportis)
  Downloading click-8.0.1-py3-none-any.whl (97 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 97.4/97.4 kB 5.0 MB/s eta 0:00:00
Requirement already satisfied: wcwidth in /usr/local/lib/python3.10/dist-packages (from prettytable==2.1.0->whatportis) (0.2.8)
Collecting chardet<5,>=3.0.2 (from requests==2.25.1->whatportis)
  Downloading chardet-4.0.0-py2.py3-none-any.whl (178 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 178.7/178.7 kB 10.7 MB/s eta 0:00:00
Collecting idna<3,>=2.5 (from requests==2.25.1->whatportis)
  Downloading idna-2.10-py2.py3-none-any.whl (58 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.8/58.8 kB 28.4 MB/s eta 0:00:00
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests==2.25.1->whatportis) (1.26.9)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests==2.25.1->whatportis) (2023.7.22)
Building wheels for collected packages: simplejson
  Building wheel for simplejson (setup.py) ... done
  Created wheel for simplejson: filename=simplejson-3.17.2-cp310-cp310-linux_x86_64.whl size=123089 sha256=48c0dc5dbe3391db2e6b04be735b002e4ea07c5ab377d99ab3f125544cd0d22b
  Stored in directory: /root/.cache/pip/wheels/4c/d2/65/e6ec94ca89c44483d8010d15e5fbead72b6411e01d26f90a31
Successfully built simplejson
Installing collected packages: tinydb, simplejson, prettytable, idna, click, chardet, requests, whatportis
  Attempting uninstall: tinydb
    Found existing installation: tinydb 4.8.0
    Uninstalling tinydb-4.8.0:
      Successfully uninstalled tinydb-4.8.0
  Attempting uninstall: simplejson
    Found existing installation: simplejson 3.19.2
    Uninstalling simplejson-3.19.2:
      Successfully uninstalled simplejson-3.19.2
  Attempting uninstall: prettytable
    Found existing installation: prettytable 3.9.0
    Uninstalling prettytable-3.9.0:
      Successfully uninstalled prettytable-3.9.0
  Attempting uninstall: idna
    Found existing installation: idna 3.3
    Uninstalling idna-3.3:
      Successfully uninstalled idna-3.3
  Attempting uninstall: click
    Found existing installation: click 8.1.7
    Uninstalling click-8.1.7:
      Successfully uninstalled click-8.1.7
  Attempting uninstall: chardet
    Found existing installation: chardet 5.0.0
    Uninstalling chardet-5.0.0:
      Successfully uninstalled chardet-5.0.0
  Attempting uninstall: requests
    Found existing installation: requests 2.31.0
    Uninstalling requests-2.31.0:
      Successfully uninstalled requests-2.31.0
  Attempting uninstall: whatportis
    Found existing installation: whatportis 0.8
    Uninstalling whatportis-0.8:
      Successfully uninstalled whatportis-0.8
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
celery 5.3.4 requires click<9.0,>=8.1.2, but you have click 8.0.1 which is incompatible.
censys 2.2.7 requires requests>=2.29.0, but you have requests 2.25.1 which is incompatible.
discord-webhook 1.3.0 requires requests<3.0.0,>=2.28.1, but you have requests 2.25.1 which is incompatible.
Successfully installed chardet-4.0.0 click-8.0.1 idna-2.10 prettytable-2.1.0 requests-2.25.1 simplejson-3.17.2 tinydb-4.4.0 whatportis-0.8.2

[github-actions]

👋 Hi @psyray,
Issues is only for reporting a bug/feature request. Please read documentation before raising an issue https://rengine.wiki
For very limited support, questions, and discussions, please join reNgine Discord channel: https://discord.gg/azv6fzhNCE
Please include all the requested and relevant information when opening a bug report. Improper reports will be closed without any response.