Database refactor

[gsanchietti]

Fixes: NethServer/nethsecurity#1300

This pull request introduces significant changes to the API and VPN container.
The focus is on improving overall security.

Main changes

• Dropped SQLite database, move user accounts and units VPN configuration inside Timescale database under the existing report database: the database is has not been renamed to avoid issues on migration. The vpn container now read and writes directly to the database: this also replaces VPN configuration via CCD files.
• 2FA keys and units credentials are now stored encrypted with AES inside the database
• JWT tokens are now stored in memory to avoid prevent access from filesystem
• On startup the API server automatically migrated all status and configuration files to the database. Only VPN certificates are still kept inside the filesystem
• Any user inside the database can be promoted to administrator: administrators are the only accounts that can manage units, unit groups and users
• Added user authorizations limitations using unit groups: by default a new non-admin user can't access any unit. The user must be associated to a unit group to gain access to units.
• Updated the vpn container: bump Alpine and OpenVPN (now on 2.6.x)
• Combining the options ALLOWED_IPS, PUBLIC_ENDPOINTS and LISTEN_ADDRESS is now possible to restrict the controller access for security reasons: only the /register public endpoint can be public, units can then be modified to push all traffic through the VPN

API changes

• Added new endpoints for managing unit groups, including GET, POST, PUT, and DELETE operations to create, update, list, and delete unit groups.
• Introduced /platform for retrieving NS8 information than can be exposed inside the UI
• Added /auth endpoint to check API server status, this can be used by podman to restart the controller if in faulty state
• Added /auth/<unit_id> endpoint with basic authentication, it can be used to forward authentication from other middleware like traefik, it returns 200 if the user can access the unit
• Add vpn_address and api_port inside the /register response payload: these new fields can be used to configure the unit to push data inside the VPN

Configuration updates

• Replaced LISTEN_ADDRESS with a comma-separated list to support multiple listen addresses: with a minor change inside the /register
• Added support for the new ENCRYPTION_KEY and PLATFORM_INFO environment variables for sensitive data encryption and platform metadata storage.
• Added ALLOWED_IPS option, a comma-separated list of allowed IPs, if empty, all IPs are allowed, default is empty
• Added PUBLIC_ENDPOINTS option, a comma-separated list of public endpoints, that can be accessed even if ALLOWED_IPS is set, default is empty
  If ALLOWED_IPS is set, the public endpoints should allow registration and ingestions from units, a good value should be: /api/ingest,/api/units/register

Breaking changes ⚠️

• The PUT /account API now requires unit groups and can't be used to change the account passoword: the controller UI must ber refactored to follow new rules

Deprecations

• Marked several configuration fields (OpenVPNStatusDir, CredentialsDir, SecretsDir) as deprecated, signaling their removal in future versions.
• Removed unused directories (tokens, credentials).

Dependency Changes

• Replaced dgryski/dgoogauth with pquerna/otp for OTP functionality and added google/uuid for UUID generation.

Grafana authentication integration (this WILL NOT be implemented)

If Grafana authentication is integrated with the controller, all users must be created inside the controller itself, including guest users. To support guest users the following changes are needed:

• add a role inside the user table
• prevent access to the controller for guest users
• allow authentication to /auth controller API for non-admin users
• when creating a guest user, also create a dedicated team inside Grafana (see APIs)
• configure Grafana and Promethes authentication forwarding, see grafana commit and prometheus commit

Changes to other parts (not implemented yet)

• Unit: use the VPN endpoint for data ingestion, retrive VPN server IP and API port directly from /register response
• Controller UI: expose platform info
• NS8 module: Database refactor ns8-nethsecurity-controller#89