Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unable to access web gui with VPN_ENABLED=yes #203

Closed
maxfield-allison opened this issue Oct 13, 2023 · 53 comments
Closed

unable to access web gui with VPN_ENABLED=yes #203

maxfield-allison opened this issue Oct 13, 2023 · 53 comments

Comments

@maxfield-allison
Copy link

After migrating my existing, working configuration from a single node docker host to docker swarm, I am unable to access the web gui of the qbittorrent container if the VPN is enabled. I am able to ping local addresses from exec in the container and netstat shows it is listening on the correct port bound to 0.0.0.0:9023
compose snippet:

# Qbittorrent VPN tv – Bittorrent Downloader (Alternative)
  qbit_tv:
    image: "binhex/arch-qbittorrentvpn"
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /mnt/perf_pool/docker/prod/downloaders/qbit_tv:/config
      - /mnt/nas:/nas     
    ports:
      - "6883:6881"
      - "9023:9023"
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=America/Chicago
      - UMASK=002
      - VPN_ENABLED=yes
      - VPN_PROV=airvpn
      - VPN_CLIENT=openvpn
      - LAN_NETWORK=10.1.10.0/24,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16
      - NAME_SERVERS=1.1.1.1,1.0.0.1
      - DEBUG=true
      - WEBUI_PORT=9023
    networks:
      - traefik_net
    cap_add:
      - NET_ADMIN
    deploy:
      replicas: 1
      placement:
        constraints:
          - node.role==worker
      labels:
        - "traefik.enable=true"
        ## HTTP Routers
        - "traefik.http.routers.qbit_tv-rtr.entrypoints=https"
        - "traefik.http.routers.qbit_tv-rtr.rule=Host(`tvdl.$DOMAINNAME`)" 
        - "traefik.http.routers.qbit_tv-rtr.tls=true"
        ## Middlewares
        - "traefik.http.routers.qbit_tv-rtr.middlewares=chain-oauth@file" 
        ## HTTP Services
        - "traefik.http.routers.qbit_tv-rtr.service=qbit_tv-svc"
        - "traefik.http.services.qbit_tv-svc.loadbalancer.server.port=9023"

Supervisord log snippet

Created by...
___.   .__       .__
\_ |__ |__| ____ |  |__   ____ ___  ___
 | __ \|  |/    \|  |  \_/ __ \\  \/  /
 | \_\ \  |   |  \   Y  \  ___/ >    <
 |___  /__|___|  /___|  /\___  >__/\_ \
     \/        \/     \/     \/      \/
   https://hub.docker.com/u/binhex/

2023-10-13 09:40:52.525418 [info] System information Linux e16d632ae146 6.2.0-34-generic #34-Ubuntu SMP PREEMPT_DYNAMIC Mon Sep  4 13:06:55 UTC 2023 x86_64 GNU/Linux
2023-10-13 09:40:52.562648 [info] PUID defined as '1000'
2023-10-13 09:40:52.619331 [info] PGID defined as '1000'
2023-10-13 09:40:52.670187 [info] UMASK defined as '002'
2023-10-13 09:40:52.708236 [info] Permissions already set for '/config'
2023-10-13 09:40:52.750384 [info] Deleting files in /tmp (non recursive)...
2023-10-13 09:40:52.827517 [info] VPN_ENABLED defined as 'yes'
2023-10-13 09:40:52.862105 [info] VPN_CLIENT defined as 'openvpn'
2023-10-13 09:40:52.901381 [info] VPN_PROV defined as 'airvpn'
2023-10-13 09:40:52.952933 [info] OpenVPN config file (ovpn extension) is located at /config/openvpn/TVDLAirVPN_United-States_UDP-443-Entry3.ovpn
2023-10-13 09:40:53.020755 [info] VPN remote server(s) defined as 'us3.vpn.airdns.org,'
2023-10-13 09:40:53.049826 [info] VPN remote port(s) defined as '443,'
2023-10-13 09:40:53.077700 [info] VPN remote protcol(s) defined as 'udp,'
2023-10-13 09:40:53.116658 [info] VPN_DEVICE_TYPE defined as 'tun0'
2023-10-13 09:40:53.156988 [info] VPN_OPTIONS not defined (via -e VPN_OPTIONS)
2023-10-13 09:40:53.205285 [debug] DNS operational, we can resolve name 'us3.vpn.airdns.org' to address '107.167.244.53'
2023-10-13 09:40:53.249390 [debug] iptables default policies available, setting policy to drop...
modprobe: FATAL: Module ip6_tables not found in directory /lib/modules/6.2.0-34-generic
ip6tables v1.8.9 (legacy): can't initialize ip6tables table `filter': Will be implemented real soon.  I promise ;)
Perhaps ip6tables or your kernel needs to be upgraded.
2023-10-13 09:40:53.301662 [warn] ip6tables default policies not available, skipping ip6tables drops
2023-10-13 09:40:53.380096 [debug] Docker interface defined as eth2
2023-10-13 09:40:53.428022 [info] LAN_NETWORK defined as '10.1.10.0/24,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16'
2023-10-13 09:40:53.463203 [info] NAME_SERVERS defined as '1.1.1.1,1.0.0.1'
2023-10-13 09:40:53.501929 [warn] ENABLE_PRIVOXY not defined (via -e ENABLE_PRIVOXY), defaulting to 'no'
2023-10-13 09:40:53.540772 [info] VPN_INPUT_PORTS not defined (via -e VPN_INPUT_PORTS), skipping allow for custom incoming ports
2023-10-13 09:40:53.575326 [info] VPN_OUTPUT_PORTS not defined (via -e VPN_OUTPUT_PORTS), skipping allow for custom outgoing ports
2023-10-13 09:40:53.609709 [info] WEBUI_PORT defined as '9023'
2023-10-13 09:40:53.796121 [info] Starting Supervisor...
2023-10-13 09:40:54,087 INFO Included extra file "/etc/supervisor/conf.d/qbittorrent.conf" during parsing
2023-10-13 09:40:54,087 INFO Set uid to user 0 succeeded
2023-10-13 09:40:54,092 INFO supervisord started with pid 7
2023-10-13 09:40:55,095 INFO spawned: 'start-script' with pid 205
2023-10-13 09:40:55,098 INFO spawned: 'watchdog-script' with pid 206
2023-10-13 09:40:55,099 INFO reaped unknown pid 8 (exit status 0)
2023-10-13 09:40:55,107 DEBG 'start-script' stdout output:
[info] VPN is enabled, beginning configuration of VPN

2023-10-13 09:40:55,108 INFO success: start-script entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2023-10-13 09:40:55,108 INFO success: watchdog-script entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2023-10-13 09:40:55,117 DEBG 'watchdog-script' stdout output:
[info] qBittorrent config file already exists, skipping copy

2023-10-13 09:40:55,117 DEBG 'watchdog-script' stdout output:
[info] Removing session lock file (if it exists)...

2023-10-13 09:40:55,294 DEBG 'start-script' stdout output:
[debug] Contents of ovpn file /config/openvpn/TVDLAirVPN_United-States_UDP-443-Entry3.ovpn as follows...
2023-10-13 09:40:55,297 DEBG 'start-script' stdout output:
# --------------------------------------------------------
# Air VPN | https://airvpn.org | Saturday 26th of August 2023 03:38:43 PM
# OpenVPN Client Configuration
# AirVPN_United-States_UDP-443-Entry3
# --------------------------------------------------------

client
dev tun
remote us3.vpn.airdns.org 443
resolv-retry infinite
nobind
persist-key
auth-nocache
verb 3
explicit-exit-notify 5
push-peer-info
remote-cert-tls server
comp-lzo no
data-ciphers AES-256-GCM:AES-256-CBC:AES-192-GCM:AES-192-CBC:AES-128-GCM:AES-128-CBC
data-ciphers-fallback AES-256-CBC
proto udp
auth SHA512
block-ipv6
pull-filter ignore "route-ipv6"
pull-filter ignore "ifconfig-ipv6"
<ca>
***OMITTED*** 
2023-10-13 09:40:55,311 DEBG 'start-script' stdout output:
[debug] Environment variables defined as follows
ADDITIONAL_PORTS=
APPLICATION=qbittorrent
BASH=/bin/bash
BASHOPTS=checkwinsize:cmdhist:complete_fullquote:extquote:force_fignore:globasciiranges:hostcomplete:interactive_comments:progcomp:promptvars:sourcepath
BASH_ALIASES=()
BASH_ARGC=()
BASH_ARGV=()
BASH_CMDS=()
BASH_LINENO=([0]="0")
BASH_SOURCE=([0]="/root/start.sh")
BASH_VERSINFO=([0]="5" [1]="1" [2]="16" [3]="1" [4]="release" [5]="x86_64-pc-linux-gnu")
BASH_VERSION='5.1.16(1)-release'

2023-10-13 09:40:55,312 DEBG 'start-script' stdout output:
DEBUG=true
DIRSTACK=()
ENABLE_PRIVOXY=no
EUID=0
GROUPS=()
HOME=/home/nobody
HOSTNAME=e16d632ae146
HOSTTYPE=x86_64
IFS=$' \t\n'
LANG=en_GB.UTF-8
LAN_NETWORK=10.1.10.0/24,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16
MACHTYPE=x86_64-pc-linux-gnu
NAME_SERVERS=1.1.1.1,1.0.0.1
OPTERR=1
OPTIND=1
OSTYPE=linux-gnu
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PGID=1000
PIPESTATUS=([0]="0")
PPID=7
PS4='+ '
PUID=1000
PWD=/
SHELL=/bin/bash
SHELLOPTS=braceexpand:hashall:interactive-comments
SHLVL=1
SUPERVISOR_ENABLED=1
SUPERVISOR_GROUP_NAME=start-script
SUPERVISOR_PROCESS_NAME=start-script
TERM=xterm
TZ=America/Chicago
UID=0
UMASK=002
VPN_CLIENT=openvpn
VPN_CONFIG=/config/openvpn/TVDLAirVPN_United-States_UDP-443-Entry3.ovpn
VPN_DEVICE_TYPE=tun0
VPN_ENABLED=yes
VPN_INPUT_PORTS=
VPN_OPTIONS=
VPN_OUTPUT_PORTS=
VPN_PROV=airvpn
VPN_REMOTE_IP_LIST=107.167.244.53
VPN_REMOTE_PORT=443,
VPN_REMOTE_PROTOCOL=udp,
VPN_REMOTE_SERVER=us3.vpn.airdns.org,
WEBUI_PORT=9023
_='[debug] Environment variables defined as follows'
vpn_ping=

2023-10-13 09:40:55,324 DEBG 'start-script' stdout output:
[info] Adding 1.1.1.1 to /etc/resolv.conf

2023-10-13 09:40:55,328 DEBG 'start-script' stdout output:
[info] Adding 1.0.0.1 to /etc/resolv.conf

2023-10-13 09:40:55,348 DEBG 'start-script' stdout output:
[debug] Show name servers defined for container

2023-10-13 09:40:55,349 DEBG 'start-script' stdout output:
nameserver 1.1.1.1
nameserver 1.0.0.1

2023-10-13 09:40:55,349 DEBG 'start-script' stdout output:
[debug] Show contents of hosts file

2023-10-13 09:40:55,350 DEBG 'start-script' stdout output:
127.0.0.1	localhost
::1	localhost ip6-localhost ip6-loopback
fe00::0	ip6-localnet
ff00::0	ip6-mcastprefix
ff02::1	ip6-allnodes
ff02::2	ip6-allrouters
10.0.1.58	e16d632ae146
107.167.244.53	us3.vpn.airdns.org

2023-10-13 09:40:55,358 DEBG 'start-script' stdout output:
[debug] Docker interface defined as eth2

2023-10-13 09:40:55,363 DEBG 'start-script' stdout output:
[info] Default route for container is 172.18.0.1

2023-10-13 09:40:55,366 DEBG 'start-script' stdout output:
[debug] Docker IP defined as 172.18.0.4

2023-10-13 09:40:55,370 DEBG 'start-script' stdout output:
[debug] Docker netmask defined as 255.255.0.0

2023-10-13 09:40:55,565 DEBG 'start-script' stdout output:
[info] Docker network defined as    172.18.0.0/16

2023-10-13 09:40:55,570 DEBG 'start-script' stdout output:
[info] Adding 10.1.10.0/24 as route via docker eth2

2023-10-13 09:40:55,576 DEBG 'start-script' stdout output:
[info] Adding 10.0.0.0/8 as route via docker eth2

2023-10-13 09:40:55,584 DEBG 'start-script' stdout output:
[info] Adding 172.16.0.0/12 as route via docker eth2

2023-10-13 09:40:55,590 DEBG 'start-script' stdout output:
[info] Adding 192.168.0.0/16 as route via docker eth2

2023-10-13 09:40:55,592 DEBG 'start-script' stdout output:
[info] ip route defined as follows...
--------------------

2023-10-13 09:40:55,594 DEBG 'start-script' stdout output:
default via 172.18.0.1 dev eth2 
10.0.0.0/24 dev eth0 proto kernel scope link src 10.0.0.199 

2023-10-13 09:40:55,594 DEBG 'start-script' stdout output:
10.0.0.0/8 via 172.18.0.1 dev eth2 
10.0.1.0/24 dev eth1 proto kernel scope link src 10.0.1.58 
10.1.10.0/24 via 172.18.0.1 dev eth2 
172.16.0.0/12 via 172.18.0.1 dev eth2 
172.18.0.0/16 dev eth2 proto kernel scope link src 172.18.0.4 
192.168.0.0/16 via 172.18.0.1 dev eth2 
local 10.0.0.199 dev eth0 table local proto kernel scope host src 10.0.0.199 
broadcast 10.0.0.255 dev eth0 table local proto kernel scope link src 10.0.0.199 
local 10.0.1.58 dev eth1 table local proto kernel scope host src 10.0.1.58 
broadcast 10.0.1.255 dev eth1 table local proto kernel scope link src 10.0.1.58 
local 127.0.0.0/8 dev lo table local proto kernel scope host src 127.0.0.1 
local 127.0.0.1 dev lo table local proto kernel scope host src 127.0.0.1 
broadcast 127.255.255.255 dev lo table local proto kernel scope link src 127.0.0.1 
local 172.18.0.4 dev eth2 table local proto kernel scope host src 172.18.0.4 
broadcast 172.18.255.255 dev eth2 table local proto kernel scope link src 172.18.0.4 


2023-10-13 09:40:55,608 DEBG 'start-script' stdout output:
[info] iptable_mangle support detected, adding fwmark for tables

2023-10-13 09:40:55,682 DEBG 'start-script' stdout output:
[info] iptables defined as follows...
--------------------

2023-10-13 09:40:55,685 DEBG 'start-script' stdout output:
-P INPUT DROP
-P FORWARD DROP
-P OUTPUT DROP
-A INPUT -s 107.167.244.53/32 -i eth2 -j ACCEPT
-A INPUT -s 172.18.0.0/16 -d 172.18.0.0/16 -j ACCEPT
-A INPUT -s 107.167.244.53/32 -i eth2 -j ACCEPT
-A INPUT -i eth2 -p tcp -m tcp --dport 9023 -j ACCEPT
-A INPUT -i eth2 -p udp -m udp --dport 9023 -j ACCEPT
-A INPUT -p icmp -m icmp --icmp-type 0 -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -i tun0 -j ACCEPT
-A OUTPUT -d 107.167.244.53/32 -o eth2 -j ACCEPT
-A OUTPUT -s 172.18.0.0/16 -d 172.18.0.0/16 -j ACCEPT
-A OUTPUT -d 107.167.244.53/32 -o eth2 -j ACCEPT
-A OUTPUT -o eth2 -p tcp -m tcp --sport 9023 -j ACCEPT
-A OUTPUT -o eth2 -p udp -m udp --sport 9023 -j ACCEPT
-A OUTPUT -p icmp -m icmp --icmp-type 8 -j ACCEPT
-A OUTPUT -o lo -j ACCEPT
-A OUTPUT -o tun0 -j ACCEPT

2023-10-13 09:40:55,686 DEBG 'start-script' stdout output:
--------------------

2023-10-13 09:40:55,688 DEBG 'start-script' stdout output:
[debug] VPN remote configuration options as follows...
[debug] VPN remote server is defined as 'us3.vpn.airdns.org'
[debug] VPN remote port is defined as '443'
[debug] VPN remote protocol is defined as 'udp'
[debug] VPN remote ip is defined as '107.167.244.53'
[debug] OpenVPN command line:- /usr/bin/openvpn --reneg-sec 0 --mute-replay-warnings --auth-nocache --setenv VPN_PROV 'airvpn' --setenv VPN_CLIENT 'openvpn' --setenv DEBUG 'true' --setenv VPN_DEVICE_TYPE 'tun0' --setenv VPN_ENABLED 'yes' --setenv VPN_REMOTE_SERVER 'us3.vpn.airdns.org' --setenv APPLICATION 'qbittorrent' --script-security 2 --writepid /root/openvpn.pid --remap-usr1 SIGHUP --log-append /dev/stdout --pull-filter ignore 'up' --pull-filter ignore 'down' --pull-filter ignore 'route-ipv6' --pull-filter ignore 'ifconfig-ipv6' --pull-filter ignore 'tun-ipv6' --pull-filter ignore 'dhcp-option DNS6' --pull-filter ignore 'persist-tun' --pull-filter ignore 'reneg-sec' --up /root/openvpnup.sh --up-delay --up-restart --keepalive 10 60 --cd /config/openvpn --config '/config/openvpn/TVDLAirVPN_United-States_UDP-443-Entry3.ovpn' --remote 107.167.244.53 443 udp --remote-random
[info] Starting OpenVPN (non daemonised)...

2023-10-13 09:40:55,699 DEBG 'start-script' stdout output:
2023-10-13 09:40:55 Note: --data-cipher-fallback with cipher 'AES-256-CBC' disables data channel offload.
2023-10-13 09:40:55 OpenVPN 2.6.6 [git:makepkg/c9540130121bfc21+] x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] [DCO] built on Aug 15 2023
2023-10-13 09:40:55 library versions: OpenSSL 3.1.3 19 Sep 2023, LZO 2.10
2023-10-13 09:40:55 DCO version: N/A

2023-10-13 09:40:55,699 DEBG 'start-script' stdout output:
2023-10-13 09:40:55 NOTE: the current --script-security setting may allow this configuration to call user-defined scripts

2023-10-13 09:40:55,702 DEBG 'start-script' stdout output:
2023-10-13 09:40:55 TCP/UDP: Preserving recently used remote address: [AF_INET]107.167.244.53:443
2023-10-13 09:40:55 Socket Buffers: R=[212992->212992] S=[212992->212992]
2023-10-13 09:40:55 UDPv4 link local: (not bound)
2023-10-13 09:40:55 UDPv4 link remote: [AF_INET]107.167.244.53:443

2023-10-13 09:40:55,781 DEBG 'start-script' stdout output:
2023-10-13 09:40:55 TLS: Initial packet from [AF_INET]107.167.244.53:443, sid=2b7cc33e ae8c4950

2023-10-13 09:40:55,781 DEBG 'start-script' stdout output:
2023-10-13 09:40:55 net_route_v4_best_gw query: dst 0.0.0.0
2023-10-13 09:40:55 net_route_v4_best_gw result: via 172.18.0.1 dev eth2

2023-10-13 09:40:55,865 DEBG 'start-script' stdout output:
2023-10-13 09:40:55 VERIFY OK: depth=1, C=IT, ST=IT, L=Perugia, O=airvpn.org, CN=airvpn.org CA, emailAddress=info@airvpn.org

2023-10-13 09:40:55,865 DEBG 'start-script' stdout output:
2023-10-13 09:40:55 VERIFY KU OK
2023-10-13 09:40:55 Validating certificate extended key usage
2023-10-13 09:40:55 ++ Certificate has EKU (str) TLS Web Server Authentication, expects TLS Web Server Authentication
2023-10-13 09:40:55 VERIFY EKU OK
2023-10-13 09:40:55 VERIFY OK: depth=0, C=IT, ST=IT, L=Perugia, O=airvpn.org, CN=Merope, emailAddress=info@airvpn.org

2023-10-13 09:40:55,955 DEBG 'start-script' stdout output:
2023-10-13 09:40:55 Control Channel: TLSv1.3, cipher TLSv1.3 TLS_CHACHA20_POLY1305_SHA256, peer certificate: 4096 bit RSA, signature: RSA-SHA512
2023-10-13 09:40:55 [Merope] Peer Connection Initiated with [AF_INET]107.167.244.53:443
2023-10-13 09:40:55 TLS: move_session: dest=TM_ACTIVE src=TM_INITIAL reinit_src=1

2023-10-13 09:40:55,955 DEBG 'start-script' stdout output:
2023-10-13 09:40:55 TLS: tls_multi_process: initial untrusted session promoted to trusted

2023-10-13 09:40:56,550 DEBG 'start-script' stdout output:
2023-10-13 09:40:56 SENT CONTROL [Merope]: 'PUSH_REQUEST' (status=1)
2023-10-13 09:40:56 PUSH: Received control message: 'PUSH_REPLY,comp-lzo no,redirect-gateway  def1 bypass-dhcp,dhcp-option DNS 10.22.90.1,route-gateway 10.22.90.1,topology subnet,ping 10,ping-restart 60,ifconfig 10.22.90.47 255.255.255.0,peer-id 3,cipher AES-256-GCM'

2023-10-13 09:40:56,550 DEBG 'start-script' stdout output:
2023-10-13 09:40:56 OPTIONS IMPORT: --ifconfig/up options modified
2023-10-13 09:40:56 OPTIONS IMPORT: route options modified
2023-10-13 09:40:56 OPTIONS IMPORT: route-related options modified
2023-10-13 09:40:56 OPTIONS IMPORT: --ip-win32 and/or --dhcp-option options modified
2023-10-13 09:40:56 net_route_v4_best_gw query: dst 0.0.0.0
2023-10-13 09:40:56 net_route_v4_best_gw result: via 172.18.0.1 dev eth2
2023-10-13 09:40:56 ROUTE_GATEWAY 172.18.0.1/255.255.0.0 IFACE=eth2 HWADDR=02:42:ac:12:00:04
2023-10-13 09:40:56 TUN/TAP device tun0 opened
2023-10-13 09:40:56 net_iface_mtu_set: mtu 1500 for tun0

2023-10-13 09:40:56,551 DEBG 'start-script' stdout output:
2023-10-13 09:40:56 net_iface_up: set tun0 up
2023-10-13 09:40:56 net_addr_v4_add: 10.22.90.47/24 dev tun0
2023-10-13 09:40:56 /root/openvpnup.sh tun0 1500 0 10.22.90.47 255.255.255.0 init

2023-10-13 09:40:56,554 DEBG 'start-script' stdout output:
2023-10-13 09:40:56 net_route_v4_add: 107.167.244.53/32 via 172.18.0.1 dev [NULL] table 0 metric -1
2023-10-13 09:40:56 net_route_v4_add: 0.0.0.0/1 via 10.22.90.1 dev [NULL] table 0 metric -1
2023-10-13 09:40:56 net_route_v4_add: 128.0.0.0/1 via 10.22.90.1 dev [NULL] table 0 metric -1

2023-10-13 09:40:56,554 DEBG 'start-script' stdout output:
2023-10-13 09:40:56 Initialization Sequence Completed
2023-10-13 09:40:56 Data Channel: cipher 'AES-256-GCM', peer-id: 3, compression: 'stub'
2023-10-13 09:40:56 Timers: ping 10, ping-restart 60
2023-10-13 09:40:56 Protocol options: explicit-exit-notify 5

2023-10-13 09:40:56,557 DEBG 'start-script' stdout output:
[debug] Waiting for valid VPN gateway IP addresses from tunnel...
[debug] Waiting for valid VPN adapter IP addresses from tunnel...

2023-10-13 09:40:57,570 DEBG 'start-script' stdout output:
[debug] Valid local IP address from tunnel acquired '10.22.90.47'

2023-10-13 09:40:57,570 DEBG 'start-script' stdout output:
[debug] Valid gateway IP address from tunnel acquired ''
[debug] Checking we can resolve name 'www.google.com' to address...

2023-10-13 09:40:57,612 DEBG 'watchdog-script' stdout output:
[debug] Checking we can resolve name 'www.google.com' to address...

2023-10-13 09:40:58,678 DEBG 'start-script' stdout output:
[debug] DNS operational, we can resolve name 'www.google.com' to address '142.251.167.103 142.251.167.104 142.251.167.105 142.251.167.106 142.251.167.147 142.251.167.99'

2023-10-13 09:40:58,678 DEBG 'start-script' stdout output:
[info] Attempting to get external IP using 'http://checkip.amazonaws.com'...

2023-10-13 09:40:58,678 DEBG 'watchdog-script' stdout output:
[debug] DNS operational, we can resolve name 'www.google.com' to address '172.253.62.104 172.253.62.105 172.253.62.103 172.253.62.106 172.253.62.147 172.253.62.99'

2023-10-13 09:40:59,127 DEBG 'start-script' stdout output:
[info] Successfully retrieved external IP address 107.167.244.51

2023-10-13 09:40:59,131 DEBG 'start-script' stdout output:
[info] VPN provider 'airvpn' not supported for automatic port forwarding, skipping incoming port assignment

2023-10-13 09:40:59,186 DEBG 'watchdog-script' stdout output:
[debug] Waiting for iptables chain policies to be in place...

2023-10-13 09:40:59,195 DEBG 'watchdog-script' stdout output:
[debug] iptables chain policies are in place
[info] qBittorrent listening interface IP 0.0.0.0 and VPN provider IP 10.22.90.47 different, marking for reconfigure

2023-10-13 09:40:59,200 DEBG 'watchdog-script' stdout output:
[info] qBittorrent not running

2023-10-13 09:40:59,201 DEBG 'watchdog-script' stdout output:
[info] Removing session lock file (if it exists)...

2023-10-13 09:40:59,243 DEBG 'watchdog-script' stdout output:
[info] Attempting to start qBittorrent...

2023-10-13 09:40:59,289 DEBG 'watchdog-script' stdout output:
[info] qBittorrent process started
[info] Waiting for qBittorrent process to start listening on port 9023...

2023-10-13 09:40:59,402 DEBG 'watchdog-script' stdout output:
[info] qBittorrent process listening on port 9023

2023-10-13 09:40:59,403 DEBG 'watchdog-script' stdout output:
[debug] VPN IP is 10.22.90.47
[debug] qBittorrent IP is 10.22.90.47

2023-10-13 09:41:29,405 DEBG 'watchdog-script' stdout output:
[debug] Checking we can resolve name 'www.google.com' to address...

2023-10-13 09:41:29,558 DEBG 'watchdog-script' stdout output:
[debug] DNS operational, we can resolve name 'www.google.com' to address '142.251.163.99 142.251.163.106 142.251.163.105 142.251.163.103 142.251.163.147 142.251.163.104'

2023-10-13 09:41:29,559 DEBG 'watchdog-script' stdout output:
[debug] Waiting for iptables chain policies to be in place...

2023-10-13 09:41:29,567 DEBG 'watchdog-script' stdout output:
[debug] iptables chain policies are in place

2023-10-13 09:41:29,571 DEBG 'watchdog-script' stdout output:
[debug] VPN IP is 10.22.90.47
[debug] qBittorrent IP is 10.22.90.47

netstat -tuln

[root@e16d632ae146 /]# netstat -tuln
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 10.0.0.199:29158        0.0.0.0:*               LISTEN     
tcp        0      0 127.0.0.1:29158         0.0.0.0:*               LISTEN     
tcp        0      0 10.0.1.58:29158         0.0.0.0:*               LISTEN     
tcp        0      0 127.0.0.11:33771        0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:9023            0.0.0.0:*               LISTEN     
tcp        0      0 172.18.0.4:29158        0.0.0.0:*               LISTEN     
tcp        0      0 10.22.90.47:29158       0.0.0.0:*               LISTEN     
udp        0      0 0.0.0.0:49506           0.0.0.0:*                          
udp        0      0 172.18.0.4:51261        0.0.0.0:*                          
udp        0      0 10.22.90.47:44160       0.0.0.0:*                          
udp        0      0 10.0.1.58:29158         0.0.0.0:*                          
udp        0      0 172.18.0.4:29158        0.0.0.0:*                          
udp        0      0 10.0.0.199:29158        0.0.0.0:*                          
udp        0      0 10.22.90.47:29158       0.0.0.0:*                          
udp        0      0 127.0.0.1:29158         0.0.0.0:*                          
udp        0      0 127.0.0.11:47271        0.0.0.0:* 

iptables -L

[root@e16d632ae146 /]# iptables -L
Chain INPUT (policy DROP)
target     prot opt source               destination         
ACCEPT     all  --  us3.vpn.airdns.org   anywhere            
ACCEPT     all  --  172.18.0.0/16        172.18.0.0/16       
ACCEPT     all  --  us3.vpn.airdns.org   anywhere            
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:swa-1
ACCEPT     udp  --  anywhere             anywhere             udp dpt:swa-1
ACCEPT     icmp --  anywhere             anywhere             icmp echo-reply
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            

Chain FORWARD (policy DROP)
target     prot opt source               destination         

Chain OUTPUT (policy DROP)
target     prot opt source               destination         
ACCEPT     all  --  anywhere             us3.vpn.airdns.org  
ACCEPT     all  --  172.18.0.0/16        172.18.0.0/16       
ACCEPT     all  --  anywhere             us3.vpn.airdns.org  
ACCEPT     tcp  --  anywhere             anywhere             tcp spt:swa-1
ACCEPT     udp  --  anywhere             anywhere             udp spt:swa-1
ACCEPT     icmp --  anywhere             anywhere             icmp echo-request
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere  
@maxfield-allison
Copy link
Author

I have done just about everything I can think of to get this to work. I added the docker dns to my name servers config, I've temporarily set all iptables to flush and allow all but nothing seems to work when the VPN is enabled.

@maxfield-allison
Copy link
Author

adding this container to the macvlan_swarm network I created for home assistant, I can now access its macvlan IP and get to the gui on the webui port.

@maxfield-allison
Copy link
Author

After much troubleshooting, I'm almost 100% certain this is an issue with either the VPN configuration in the container and the iptables and routes used or, an issue with airvpn, all in conjunction with the added weirdness caused by docker swarm. I am continuing my troubleshooting and welcome any input. I'd be happy to share more detail about my troubleshooting steps if someone else is having this issue or would like to lend a hand.

@maxfield-allison
Copy link
Author

I ran tcpdump inside the container and observed that my requests were reaching the container from the docker swarm ingress network as well as my traefik network but while I see SYN packets, I'm not seeing any SYN ACK packets.

[root@3802b728ffb4 /]# tcpdump -i eth0 port 19023
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
16:36:21.072163 IP 10.0.0.2.52740 > 3802b728ffb4.19023: Flags [S], seq 1606104803, win 64240, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
16:36:22.072452 IP 10.0.0.2.52740 > 3802b728ffb4.19023: Flags [S], seq 1606104803, win 64240, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
16:36:22.532311 IP 10.0.0.2.52742 > 3802b728ffb4.19023: Flags [S], seq 718849082, win 64240, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
16:36:23.545581 IP 10.0.0.2.52742 > 3802b728ffb4.19023: Flags [S], seq 718849082, win 64240, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
16:36:24.085572 IP 10.0.0.2.52740 > 3802b728ffb4.19023: Flags [S], seq 1606104803, win 64240, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
16:36:25.549044 IP 10.0.0.2.52742 > 3802b728ffb4.19023: Flags [S], seq 718849082, win 64240, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
^C
6 packets captured
6 packets received by filter
0 packets dropped by kernel
[root@3802b728ffb4 /]# tcpdump -i eth1 port 19023
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), snapshot length 262144 bytes
16:36:47.039056 IP 10.0.1.173.60098 > 3802b728ffb4.19023: Flags [S], seq 603898687, win 64860, options [mss 1410,sackOK,TS val 3352343514 ecr 0,nop,wscale 7], length 0
16:36:48.047907 IP 10.0.1.173.60098 > 3802b728ffb4.19023: Flags [S], seq 603898687, win 64860, options [mss 1410,sackOK,TS val 3352344523 ecr 0,nop,wscale 7], length 0
16:36:50.064068 IP 10.0.1.173.60098 > 3802b728ffb4.19023: Flags [S], seq 603898687, win 64860, options [mss 1410,sackOK,TS val 3352346539 ecr 0,nop,wscale 7], length 0
16:36:54.191910 IP 10.0.1.173.60098 > 3802b728ffb4.19023: Flags [S], seq 603898687, win 64860, options [mss 1410,sackOK,TS val 3352350667 ecr 0,nop,wscale 7], length 0
^C
4 packets captured
4 packets received by filter
0 packets dropped by kernel

@binhex
Copy link
Owner

binhex commented Oct 17, 2023

the issue is that this image is deigned to work with the default docker bridge where the docker network is typically different to your LAN.

the way you have it setup you have an overlap, docker network from your log above is:-

2023-10-13 09:40:55,366 DEBG 'start-script' stdout output:
[debug] Docker IP defined as 172.18.0.4

2023-10-13 09:40:55,370 DEBG 'start-script' stdout output:
[debug] Docker netmask defined as 255.255.0.0

From LAN_NETWORK you have multiple networks defined, one of which is 172.16.0.0/12 which overlaps with the docker network.

So the fix is to ensure that the network assigned to docker is outside of your LAN ranges.

@maxfield-allison
Copy link
Author

Forgive the lack of updated information, I did notice that and removed it from the compose before posting my updates.

  qbit_tv:
    image: "binhex/arch-qbittorrentvpn:latest"
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /mnt/perf_pool/docker/prod/downloaders/qbit_tv:/config
      - /mnt/nas:/nas     
    ports:
      - "6883:6881"
      - "19023:19023"
      - "8118:8118"
      - "6883:6881/udp"
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=America/Chicago
      - UMASK=002
      - VPN_ENABLED=yes
      - VPN_PROV=airvpn
      - VPN_CLIENT=openvpn
      - LAN_NETWORK=10.1.10.0/24,10.1.20.0/24
      - NAME_SERVERS=1.1.1.1,1.0.0.1
      - DEBUG=true
      - WEBUI_PORT=19023
    #dns: 127.0.0.11
    networks:
      - traefik_net
    #  - macvlan_swarm
    cap_add:
      - NET_ADMIN
    deploy:
      replicas: 1
      placement:
        constraints:
          - node.role==worker

Supervisord log
https://pastebin.com/Xh9HXsXY

Thank you for taking a look!

@maxfield-allison
Copy link
Author

maxfield-allison commented Oct 17, 2023

Here is what I think is happening,

the docker network detected by the container is the docker swarm gwbridge on eth2.
routes to my lan networks are created for this interface.

default via 172.18.0.1 dev eth2 
10.0.0.0/24 dev eth0 proto kernel scope link src 10.0.0.243 
2023-10-17 17:04:20,774 DEBG 'start-script' stdout output:
10.0.1.0/24 dev eth1 proto kernel scope link src 10.0.1.217 
10.1.10.0/24 via 172.18.0.1 dev eth2 
10.1.20.0/24 via 172.18.0.1 dev eth2 
172.18.0.0/16 dev eth2 proto kernel scope link src 172.18.0.10 
local 10.0.0.243 dev eth0 table local proto kernel scope host src 10.0.0.243 
broadcast 10.0.0.255 dev eth0 table local proto kernel scope link src 10.0.0.243 
local 10.0.1.217 dev eth1 table local proto kernel scope host src 10.0.1.217 
broadcast 10.0.1.255 dev eth1 table local proto kernel scope link src 10.0.1.217 
local 127.0.0.0/8 dev lo table local proto kernel scope host src 127.0.0.1 
local 127.0.0.1 dev lo table local proto kernel scope host src 127.0.0.1 
broadcast 127.255.255.255 dev lo table local proto kernel scope link src 127.0.0.1 
local 172.18.0.10 dev eth2 table local proto kernel scope host src 172.18.0.10 
broadcast 172.18.255.255 dev eth2 table local proto kernel scope link src 172.18.0.10 

iptables rules are created to allow the webgui port 19023 on eth2

-P INPUT DROP
-P FORWARD DROP
-P OUTPUT DROP
-A INPUT -s 199.249.230.39/32 -i eth2 -j ACCEPT
-A INPUT -s 172.18.0.0/16 -d 172.18.0.0/16 -j ACCEPT
-A INPUT -s 199.249.230.39/32 -i eth2 -j ACCEPT
-A INPUT -i eth2 -p tcp -m tcp --dport 19023 -j ACCEPT
-A INPUT -i eth2 -p udp -m udp --dport 19023 -j ACCEPT
-A INPUT -p icmp -m icmp --icmp-type 0 -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -i tun0 -j ACCEPT
-A OUTPUT -d 199.249.230.39/32 -o eth2 -j ACCEPT
-A OUTPUT -s 172.18.0.0/16 -d 172.18.0.0/16 -j ACCEPT
-A OUTPUT -d 199.249.230.39/32 -o eth2 -j ACCEPT
-A OUTPUT -o eth2 -p tcp -m tcp --sport 19023 -j ACCEPT
-A OUTPUT -o eth2 -p udp -m udp --sport 19023 -j ACCEPT
-A OUTPUT -p icmp -m icmp --icmp-type 8 -j ACCEPT
-A OUTPUT -o lo -j ACCEPT
-A OUTPUT -o tun0 -j ACCEPT

When testing, I set all iptables to accept and flushed, but this caused (I assume) the watchdog to terminate the qbittorrent-nox process meaning that even though I was now unrestricted to the application, it wasn't running anymore to service the request. hence my lack of synack in the tcpdump. Edit: doing some more testing and I now dont think the process stopped due to my fiddling.
Since this is docker swarm, the traffic coming in is via either eth0 or eth1, the swarm ingress and traefik networks respectively.

@maxfield-allison
Copy link
Author

I made progress and can access the webgui!
First, I cleared the iptables and chains, set everything to accept; this was to simplify the situation and focus only on the routing.

# Set default policies to ACCEPT
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT

# Flush (delete) all existing rules
iptables -F

# Delete all user-defined chains
iptables -X

next I nano'd /etc/iproute2/rt_tables and added two new routing tables

100 eth0table
101 eth1table

then, I set rules to direct the traffic to the appropriate tables based on the source address

ip route add 10.0.0.0/24 dev eth0 table eth0table
ip route add default via 172.18.0.1 dev eth2 table eth0table

ip route add 10.0.1.0/24 dev eth1 table eth1table
ip route add default via 172.18.0.1 dev eth2 table eth1table

and finally, I created rules to direct the traffic to the appropriate tables based on the source address

ip rule add from 10.0.0.0/24 table eth0table
ip rule add from 10.0.1.0/24 table eth1table

I'm not very familiar with the underlying routing you're using in the container so this may be overly verbose and simplification is very likely possible. Now, I'm going to restart the container and perform the same configuration without adjusting the iptables.

@maxfield-allison
Copy link
Author

ok, confirmed there are some things that need to happen with the iptables after adding those routes. setting all to accept, I can get to the webgui no problem, drop and i still don't have access. I confirmed that out of the box, iptables ACCEPT doesnt work.

@maxfield-allison
Copy link
Author

maxfield-allison commented Oct 18, 2023

ok and of course, simply adding

iptables -A INPUT -s 10.0.0.0/24 -j ACCEPT
iptables -A INPUT -s 10.0.1.0/24 -j ACCEPT
iptables -A OUTPUT -d 10.0.0.0/24 -j ACCEPT
iptables -A OUTPUT -d 10.0.1.0/24 -j ACCEPT

along with the routes mentioned previously fixes this completely. Now I just need your help, @binhex to understand how this can be integrated into the container to explicitly allow Swarm support or alternatively, what I might be doing wrong that would cause this behavior in the stock container.

@maxfield-allison
Copy link
Author

taking a look at arch-int-vpn

@maxfield-allison
Copy link
Author

I'll preface by saying im certainly not a dev but i can hack and read my way through what you have in https://github.com/binhex/arch-int-vpn/blob/master/run/root/iptable.sh and I think it would be doable to add logic to parse the additional interfaces in order to support swarm and additional docker networks. What I'd be able to put together would definitely be inelegant and hacky, certainly without comprehensive handling of all use cases. but then again, maybe something simple wouldn't be too bad.

now that im reading through this again and looking into my previous messages im confused as to why it isnt working out of the box.

I did take another look into iptables -t mangle -nvL and now i think im seeing whats going on a bit more clearly. I'm not entirely sure what the best solution to the problem would be at this point but i certainly see that my previous troubleshooting was severely missing some key information. The mark added to $WEBUI_PORT seems to be the only thing that might need adjusting.

@maxfield-allison
Copy link
Author

now that I've dug in deeper, I think this can be solved by altering the table that is created for $(webui_port)_qbittorrent to include routes back to any additional networks that aren't the VPN. I see more in iptable.sh that would likely need some tlc for this. but i'll be honest, im tapped out today. Looking forward to your thoughts and comments @binhex and thank you again for taking the time to get back to me earlier.

@binhex
Copy link
Owner

binhex commented Oct 25, 2023

@maxfield-allison OK this was a non trivial change, i have added in multi adapter docker support, please can you test this by pulling down newly built image with tag multi_adapter, if it works ok then let me know and i will merge to master and build all VPN related images

@maxfield-allison
Copy link
Author

maxfield-allison commented Oct 25, 2023

@binhex looks like its still unable to connect with the multi_adapter tag.
seems the table for the port is still only sending through the default docker network

[root@a7f4dd10b230 etc]# iptables -t mangle -nvL
Chain PREROUTING (policy ACCEPT 11514 packets, 8882K bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain INPUT (policy ACCEPT 6516 packets, 8443K bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 4613 packets, 436K bytes)
 pkts bytes target     prot opt in     out     source               destination         
  240 12440 MARK       6    --  *      *       0.0.0.0/0            0.0.0.0/0            tcp spt:19023 MARK set 0x1

Chain POSTROUTING (policy ACCEPT 4629 packets, 438K bytes)
 pkts bytes target     prot opt in     out     source               destination         
[root@a7f4dd10b230 etc]# ip rule show
0:      from all lookup local
32765:  from all fwmark 0x1 lookup 19023_qbittorrent
32766:  from all lookup main
32767:  from all lookup default
[root@a7f4dd10b230 etc]# ip route show table 19023_qbittorrent
default via 172.18.0.1 dev eth2 
[root@a7f4dd10b230 etc]# 

@binhex
Copy link
Owner

binhex commented Oct 26, 2023

OK so i am coding blind here as i don't your setup, i have a single interface so it's tricky to debug. OK so do you think the issue is a routing issue or a iptables blocking issue?, i have checked and re-checked the iptables rules and i think i now have support for multi interfaces so i assume this is now a routing issue, correct?.

Can you try docker exec to the container and try manually adding in a route to see if this fixes it, if so let me know what the command was and i can look at incorporating this into the image.

@binhex
Copy link
Owner

binhex commented Oct 26, 2023

Can you try the following:-

ip route add <ip address of the secondary adapter> via <gateway ip address> table 2100_qbittorrent

@maxfield-allison
Copy link
Author

that worked!

[root@a7f4dd10b230 /]# ip route add 10.0.0.0/24 via 10.0.0.2 table 19023_qbittorrent
[root@a7f4dd10b230 /]# ip route show
0.0.0.0/1 via 10.33.90.1 dev tun0 
default via 172.18.0.1 dev eth2 #docker_gwbridge network in swarm, detected as docker network which in swarm is incorrect
10.0.0.0/24 dev eth0 proto kernel scope link src 10.0.0.64 #docker swarm ingress network
10.0.1.0/24 dev eth1 proto kernel scope link src 10.0.1.13 #manually created traefik_net for traefik reverse proxy, defined in compose stack as external
10.1.10.0/24 via 172.18.0.1 dev eth2 #lan network
10.1.20.0/24 via 172.18.0.1 dev eth2 #lan network
10.33.90.0/24 dev tun0 proto kernel scope link src 10.33.90.225 
128.0.0.0/1 via 10.33.90.1 dev tun0 
172.18.0.0/16 dev eth2 proto kernel scope link src 172.18.0.9 #docker_gwbridge network used by swarm for inter node communication
193.37.254.37 via 172.18.0.1 dev eth2 
[root@a7f4dd10b230 /]# ip route show table 19023_qbittorrent #table created based on webui port defined in env
default via 172.18.0.1 dev eth2 
10.0.0.0/24 via 10.0.0.2 dev eth0 #route added by above command 

@binhex
Copy link
Owner

binhex commented Oct 26, 2023

Awesome sauce!, OK leave that last bit with me and I will create another test tagged image and let you know.

@maxfield-allison
Copy link
Author

Hey @binhex anything else I can do? trying to schedule the rest of my migration to swarm.

@binhex
Copy link
Owner

binhex commented Nov 3, 2023

Sorry. slight delay on this as I have been on holiday and also flat out at work, I shall see if I can get around to this over the weekend.

@maxfield-allison
Copy link
Author

no worries, didn't mean to come across as though im rushing you; I know this is a side thing.

@maxfield-allison
Copy link
Author

maxfield-allison commented Nov 12, 2023

@binhex just a bump to keep this on your radar. Thanks again for your efforts!

@binhex
Copy link
Owner

binhex commented Nov 13, 2023

Sorry, got pulled onto other things, no good deed goes unpunished eh :-), i shall try and refocus onto this shortly.

@maxfield-allison
Copy link
Author

Thanks again, I completely understand!

@maxfield-allison
Copy link
Author

leetle bump, Thanks again!

@binhex
Copy link
Owner

binhex commented Dec 5, 2023

leetle bump, Thanks again!

not forgotten, i have been busy and recently ill, i will see if we can finally get this sorted shortly (next few days).

@maxfield-allison
Copy link
Author

I completely understand. you're a beast! Thanks for updating me.

@maxfield-allison
Copy link
Author

Can I have code update for my birthday? lol, I'm getting by fine with adding the docker network routes manually to the {port}_qbittorrent ip table after container start but it would be nice to not have to manually interfere.

@binhex
Copy link
Owner

binhex commented Dec 15, 2023

happy birthday!

looking at this, when i asked you to put in the manual route you specified a default gateway ip address of 10.0.0.2 but looking down further i see your default gateway ip is 172.18.0.1, which is confusing me somewhat, so do you have two gateways defined?, one for each nic?.

i was kind of expecting something like the following:-
ip route add 10.0.0.0/24 via 172.18.0.1 table 19023_qbittorrent

@maxfield-allison
Copy link
Author

maxfield-allison commented Dec 15, 2023

Swarm contains a default network of 172.18.0.0/24 on the docker_gwbridge network. That is detected as the default network for the contianer. (edit) Docker_gwbridge is basically an abstracted layer that helps overlay networking work in swarm. shouldnt be used by the application in the container, generally speaking. the additional networks are 10.0.0.2/24 for the docker ingress network and 10.0.1.2/24 which is my externally defined traefik network.

@binhex
Copy link
Owner

binhex commented Dec 16, 2023

ok so as i dont have your setup this is hard to work out, so how can i identify the ip address for the gateway on the second nic? (eth0), in your example above that is ip 10.0.0.2, i assume it is shown as the gateway ip for the secondary nic, correct?.

@maxfield-allison
Copy link
Author

maxfield-allison commented Dec 16, 2023 via email

@binhex
Copy link
Owner

binhex commented Dec 17, 2023

so my question to you is what linux utility can i use to identify the gateway ip for the secondary nic?, if there is no way to programmatically get this information then i am unable to automatically set it.

what is the output for netstat -rn does this show the gateway ip you want set?, if so please paste the output.

@maxfield-allison
Copy link
Author

well its not looking good. it seems its only pulling the network with no gateway.

Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         10.22.162.1     128.0.0.0       UG        0 0          0 tun0
0.0.0.0         172.18.0.1      0.0.0.0         UG        0 0          0 eth2
10.0.0.0        0.0.0.0         255.255.255.0   U         0 0          0 eth0
10.0.1.0        0.0.0.0         255.255.255.0   U         0 0          0 eth1

@maxfield-allison
Copy link
Author

at this point it appears to me that the only solutions here are a custom entrypoint script, pulling the info from the docker api (sketchy) or adding an env var.

@binhex
Copy link
Owner

binhex commented Dec 19, 2023

Agreed, I will see what the best place is to insert custom routes

@maxfield-allison
Copy link
Author

Might be less sketchy and simpler if we allow either mounting the docker sock (with warnings) or plugging into a socket proxy now that I think about it. Would allow for much easier network config inside the container I think.

@maxfield-allison
Copy link
Author

maxfield-allison commented Dec 20, 2023

one more shortcut would be to keep the logic for multi adapter and merge it into main, then add an env variable that allows the execution of a script or set of commands after container up. so for my case, I would just add

environment:
  EXEC: "ip route add 10.0.0.0/24 via 10.0.0.2 table 9023_qbittorrent; ip route add 10.0.1.0/24 via 10.0.1.1 table 9023_qbittorrent"

To my compose. but this is still kind of inelegant.

Another possibility: take an environment variable for the network and gateway, then feed those to the iptables scripts if they exist.

@maxfield-allison
Copy link
Author

Any luck?

@HHUBSS
Copy link

HHUBSS commented Feb 9, 2024

I'm having this issue that when I enable VPN, I can't access webgui.

@Chrunchy
Copy link

I'm using a simple single node, but was also having issues loading the webinterface.
Log shows everything as OK, and qbit listening on given port - but i couldn't reach it.

It started working as soon as i reverted my pgid and guid back to 0 (puid=0 and pgid=0).
Before that i tried deploying with my usual docker user, but something does not get configured right when running it with that user.

When starting with custom user, directory '/etc/iproute2/' does not exist

'ip rule show' did not show:
32765: from all fwmark 0x1 lookup 9596_qbittorrent

When starting with root, iproute2 is created and rules got added too.

@Legion495
Copy link

Legion495 commented Feb 17, 2024

I'm having this issue that when I enable VPN, I can't access webgui.

I am running into the same issue.
To be specific I have a Mullvad Wireguard config in use for this.
The container is on a bridge. I disabled the kill switch option and this seems to do it...

I noticed the Webui button is always set to 8080?
Edit Advanced view I am a idiot.

@binhex
Copy link
Owner

binhex commented Feb 18, 2024

at this point it appears to me that the only solutions here are a custom entrypoint script,

@maxfield-allison Yep, this is what I'm going for, a new image has been produced, please pull down tagged image multi_adapter, there is a new env var called ENABLE_STARTUP_SCRIPTS set the value to yes and then plonk your custom script in /config/scripts/ and you will see the output of the script at the startup.

@binhex
Copy link
Owner

binhex commented Feb 18, 2024

I'm having this issue that when I enable VPN, I can't access webgui.

Please don't put comments like this it adds zero value there are LITERALLY dozens of reasons why the web ui won't start, if you want help with your issue then please follow this:- https://github.com/binhex/documentation/blob/master/docker/faq/help.md#other-users

@maxfield-allison
Copy link
Author

maxfield-allison commented Feb 19, 2024

Ok, pulled the new image binhex/arch-qbittorrentvpn:multi_adapter@sha256:9079a21e6837bf26b93be7a5115630a7efff7afa9e199a2bc203bd5c4762310f
struggled for a minute to understand why i saw
ENABLE_STARTUP_SCRIPTS not defined (via -e ENABLE_STARTUP_SCRIPTS), defaulting to 'no'
then once i figure out i needed more coffee, corrected my mistake and...
ENABLE_STARTUP_SCRIPTS defined as 'yes'
and wondered why it still wasnt working, only to find a "scripts" folder in config.

And now that I have the script in the correct folder, it is running,

2024-02-19 08:24:19.217349 [info] ENABLE_STARTUP_SCRIPTS defined as 'yes'
2024-02-19 08:24:19.254925 [info] WEBUI_PORT defined as '9022'
2024-02-19 08:24:19.640532 [info] Executing user script '/config/scripts/ipconfig.sh' in the foreground...
Error: argument "9022_qbittorrent" is wrong: "table" value is invalid

Too early for it to alter the table it needs to adjust

#!/bin/bash
# Add routing rules for qBittorrent
ip route add 10.0.0.0/24 via 10.0.0.2 table 9022_qbittorrent
ip route add 10.0.1.0/24 via 10.0.1.1 table 9022_qbittorrent

@maxfield-allison
Copy link
Author

updated the script to create the table if it doesnt exist:

#!/bin/bash

# Define the table name
TABLE_NAME="9022_qbittorrent"

# Check if the routing table already exists
if ! grep -q "$TABLE_NAME" /etc/iproute2/rt_tables; 
then
    # If it does not exist, add the table to /etc/iproute2/rt_tables
    echo "$TABLE_NAME" | tee -a /etc/iproute2/rt_tables > /dev/null
    echo "Routing table $TABLE_NAME added."
else
    echo "Routing table $TABLE_NAME already exists."
fi

# Add routing rules for qBittorrent
ip route add 10.0.0.0/24 via 10.0.0.2 table $TABLE_NAME
ip route add 10.0.1.0/24 via 10.0.1.1 table $TABLE_NAME

echo "Routing rules added to $TABLE_NAME."

but it looks like it isnt able to access or create it

2024-02-19 09:39:37.512823 [info] Executing user script '/config/scripts/ipconfig.sh' in the foreground...
grep: /etc/iproute2/rt_tables: No such file or directory
tee: /etc/iproute2/rt_tables: No such file or directory
2024-02-19 09:39:37.574967 [info] Starting Supervisor...
2024-02-19 09:39:37,918 INFO Included extra file "/etc/supervisor/conf.d/qbittorrent.conf" during parsing
2024-02-19 09:39:37,918 INFO Set uid to user 0 succeeded

However once the user context is set to 0 and the container is initialized,

[root@2c5a3094e41d scripts]# ./ipconfig.sh
Routing table 9022_qbittorrent already exists.
Routing rules added to 9022_qbittorrent.

We're golden

@maxfield-allison
Copy link
Author

ok I was able to put this together and it has 100% solved the problem for me

#!/bin/bash

# Use WEBUI_PORT environment variable for the table ID and table name
TABLE_ID="${WEBUI_PORT}"
TABLE_NAME="${WEBUI_PORT}_qbittorrent"
echo "Configuring routing table $TABLE_NAME."

# Ensure /etc/iproute2 directory and rt_tables file exist
if [ ! -d /etc/iproute2 ]; then
    mkdir -p /etc/iproute2
fi

if [ ! -f /etc/iproute2/rt_tables ]; then
    touch /etc/iproute2/rt_tables
fi

# Check if the routing table already exists, add if it doesn't
if ! grep -q "$TABLE_NAME" /etc/iproute2/rt_tables; then
    echo "$TABLE_ID $TABLE_NAME" >> /etc/iproute2/rt_tables
    echo "Added routing table $TABLE_NAME."
fi

# Add routing rules for qBittorrent
ip route add 10.0.0.0/24 via 10.0.0.2 table $TABLE_NAME
ip route add 10.0.1.0/24 via 10.0.1.1 table $TABLE_NAME
echo "Routing rules configured for $TABLE_NAME."

For anyone else that wants to use this, all you need to do is determine what the subnet and gateway is for the networks that you need to access the container from. this is easy enough with docker network inspect or portainer. then simply fill in the section #add routing rules for qbittorrent.
For example

[
    {
        "Name": "traefik_net",
        "Id": "xxxxx",
        "Created": "xxxxxxx",
        "Scope": "swarm",
        "Driver": "overlay",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "10.0.1.0/24",
                    "Gateway": "10.0.1.1"

@maxfield-allison
Copy link
Author

image

@binhex
Copy link
Owner

binhex commented Feb 19, 2024

OK wonderful, let's close this issue, for anybody else who comes here with 'unable to access the web ui', please read my comment here:- #203 (comment)

@binhex binhex closed this as completed Feb 19, 2024
@AkaBlur
Copy link

AkaBlur commented Jul 7, 2024

ok I was able to put this together and it has 100% solved the problem for me

#!/bin/bash

# Use WEBUI_PORT environment variable for the table ID and table name
TABLE_ID="${WEBUI_PORT}"
TABLE_NAME="${WEBUI_PORT}_qbittorrent"
echo "Configuring routing table $TABLE_NAME."

# Ensure /etc/iproute2 directory and rt_tables file exist
if [ ! -d /etc/iproute2 ]; then
    mkdir -p /etc/iproute2
fi

if [ ! -f /etc/iproute2/rt_tables ]; then
    touch /etc/iproute2/rt_tables
fi

# Check if the routing table already exists, add if it doesn't
if ! grep -q "$TABLE_NAME" /etc/iproute2/rt_tables; then
    echo "$TABLE_ID $TABLE_NAME" >> /etc/iproute2/rt_tables
    echo "Added routing table $TABLE_NAME."
fi

# Add routing rules for qBittorrent
ip route add 10.0.0.0/24 via 10.0.0.2 table $TABLE_NAME
ip route add 10.0.1.0/24 via 10.0.1.1 table $TABLE_NAME
echo "Routing rules configured for $TABLE_NAME."

For anyone else that wants to use this, all you need to do is determine what the subnet and gateway is for the networks that you need to access the container from. this is easy enough with docker network inspect or portainer. then simply fill in the section #add routing rules for qbittorrent. For example

[
    {
        "Name": "traefik_net",
        "Id": "xxxxx",
        "Created": "xxxxxxx",
        "Scope": "swarm",
        "Driver": "overlay",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "10.0.1.0/24",
                    "Gateway": "10.0.1.1"

@maxfield-allison,
I also tried this solution.
Did you really use grep with -q?

In my case this would never return anything from rt_tables even if the routing table is defined there.

Using option -q produces duplicates inside my table file. This can also generate wrong entries when e.g. the TABLE_ID is set wrong or has no value.

In my case the image is: arch-delugevpn:2.1.1-6-05
Just taking ! grep "$TABLE_NAME" /etc/iproute2/rt_tables did the trick.

@maxfield-allison
Copy link
Author

Yea I really used grep with -q. Plenty of ways to skin a cat.

@azsde
Copy link

azsde commented Aug 1, 2024

@maxfield-allison I do not understand the ip route add 10.0.0.0/24 via 10.0.0.2 table $TABLE_NAME part.

When doing docker network inspect, you have

"Subnet": "10.0.1.0/24", "Gateway": "10.0.1.1"

Thus I understand the second line, but not how you came up with the one I quoted. Can you enlighten me ? :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants