goaccess not running in --real-time-html #737

muagkov · 2017-04-14T18:58:02Z

Hello, I have a problem with "--real-time-html"

when i use command:
root@pa:~# goaccess -f /var/log/apache2/access.log -a -o /var/www/html/index.html --real-time-html
i get in terminal the following:
root@pa:~# goaccess -f /var/log/apache2/access.log -a -o /var/www/html/index.html --real-time-html WebSocket server ready to accept new client connections

but on my index.html i have static page with "Last Updated" when i use a command, not real time page.

so when i use command root@pa:~# goaccess -f /var/log/apache2/access.log -a -o /var/www/html/index.html --real-time-html i have the same result like root@pa:~# goaccess -f /var/log/apache2/access.log -a -o /var/www/html/index.html

Please, help me to make it real time

The text was updated successfully, but these errors were encountered:

allinurl · 2017-04-14T19:01:02Z

Can you please post the output of your browser's console. Thanks.

muagkov · 2017-04-14T21:12:41Z

after some period of time there is an error in browser's console :
WebSocket network error: The operation couldn’t be completed. Operation timed out

(index):1210 WebSocket connection to 'ws://104.199.XXX.XXX:7890/' failed: Error in connection establishment: net::ERR_CONNECTION_TIMED_OUT

allinurl · 2017-04-14T21:45:13Z

Sounds like it's not able to reach the server on 104.199.XXX.XXX:7890. Is this the right IP address where GoAccess is running? Also, since it listens on port 7890, please make sure the port is opened.

If you are opening the report through https, you will need to run goaccess with --ssl-cert=<cert.crt> and --ssl-key=<priv.key>.

allinurl · 2017-04-18T19:18:09Z

Were you able to output real-time?

muagkov · 2017-04-18T19:46:31Z

no, i wasn't :(

is this the right IP address where GoAccess is running?

yes

Also, since it listens on port 7890, please make sure the port is opened.

it is open and i changed to others ports - result the same

If you are opening the report through https, you will need to run goaccess with --ssl-cert=<cert.crt> and --ssl-key=<priv.key>.

I'm not use https

allinurl · 2017-04-18T19:51:26Z

Are you behind some sort of proxy? can you please try telnet'ing to that IP and that port? Also, can you run the following and post the output here?

lsof -i -n -P

sajjadalis · 2017-04-20T21:43:24Z

Hi,

I have same issue. Static html page won't update auto or even after refreshing. I checked console and it shows this error.

WebSocket connection to 'ws://sh***************.com:7890/' failed: Error in connection establishment: net::ERR_CONNECTION_TIMED_OUT

I used this command to generate real time report.
goaccess -f /srv/users/serverpilot/log/sh***************/sh***************_nginx.access.log --log-format=COMBINED -o /srv/users/serverpilot/apps/sh***************/public/report.html --real-time-html

I opened port 7890 with this command.
sudo ufw allow 7890
http://prntscr.com/eyw3f9

But still same issue. I'm using serverpilot.io and cloudflare. Its using nginx in front of apache on server. here is result for lsof -i -n -P

dhclient   507        root    6u  IPv4  10989      0t0  UDP *:68
sshd       710        root    3u  IPv4  34347      0t0  TCP *:22 (LISTEN)
sshd       710        root    4u  IPv6  34349      0t0  TCP *:22 (LISTEN)
sshd      1052        root    3u  IPv4 314476      0t0  TCP 205.***.***.***:22->                          182.185.**.***:12797 (ESTABLISHED)
sp-agent  2003        root    4u  IPv4 289138      0t0  TCP 205.***.***.***:3037                          9->35.186.205.226:443 (ESTABLISHED)
sp-agent  2003        root    6u  IPv4 345957      0t0  TCP 205.***.***.***:3430                          6->104.198.211.92:443 (ESTABLISHED)
sp-stats  2373        root    4u  IPv4 353412      0t0  TCP 205.***.***.***:3462                          8->104.198.211.92:443 (ESTABLISHED)
sshd      2411        root    3u  IPv4 343883      0t0  TCP 205.***.***.***:22->                          182.185.**.***:1068 (ESTABLISHED)
monitor   2493        root    3u  IPv4 128167      0t0  TCP 205.***.***.***:5513                          5->104.154.33.168:2010 (ESTABLISHED)
sshd      2610        root    3u  IPv4 348214      0t0  TCP 205.***.***.***:22->                          182.185.**.***:2418 (ESTABLISHED)
sshd      2873        root    3u  IPv4 353164      0t0  TCP 205.***.***.***:22->                          118.212.132.227:43551 (ESTABLISHED)
sshd      2874        sshd    3u  IPv4 353164      0t0  TCP 205.***.***.***:22->                          118.212.132.227:43551 (ESTABLISHED)
sshd      2893        root    3u  IPv4 353506      0t0  TCP 205.***.***.***:22->                          59.45.175.56:43464 (ESTABLISHED)
sshd      2894        sshd    3u  IPv4 353506      0t0  TCP 205.***.***.***:22->                          59.45.175.56:43464 (ESTABLISHED)
master    3132        root   12u  IPv4  23361      0t0  TCP 127.0.0.1:25 (LISTEN                          )
mysqld    3868       mysql   17u  IPv4  25934      0t0  TCP 127.0.0.1:3306 (LIST                          EN)
httpd     4857        root    3u  IPv4  28033      0t0  TCP 127.0.0.1:81 (LISTEN                          )
nginx     5011        root   17u  IPv4  35685      0t0  TCP *:80 (LISTEN)
nginx     5011        root   18u  IPv6  35686      0t0  TCP *:80 (LISTEN)
nginx    29952    www-data    3u  IPv4 353422      0t0  TCP 205.***.***.***:80->                          162.158.79.165:30112 (ESTABLISHED)
nginx    29952    www-data   15u  IPv4 353007      0t0  TCP 205.***.***.***:80->                          162.158.92.50:25877 (ESTABLISHED)
nginx    29952    www-data   16u  IPv4 353683      0t0  TCP 205.***.***.***:80->                          162.158.88.192:14037 (ESTABLISHED)
nginx    29952    www-data   17u  IPv4  35685      0t0  TCP *:80 (LISTEN)
nginx    29952    www-data   18u  IPv6  35686      0t0  TCP *:80 (LISTEN)
nginx    29952    www-data   19u  IPv4 353144      0t0  TCP 205.***.***.***:80->                          162.158.91.3:33829 (ESTABLISHED)
nginx    29952    www-data   20u  IPv4 353440      0t0  TCP 205.***.***.***:80->                          162.158.91.237:28868 (ESTABLISHED)
nginx    29953    www-data    3u  IPv4 353295      0t0  TCP 205.***.***.***:80->                          172.68.58.45:25656 (ESTABLISHED)
nginx    29953    www-data    8u  IPv4 353080      0t0  TCP 205.***.***.***:80->                          162.158.91.231:20428 (ESTABLISHED)
nginx    29953    www-data   13u  IPv4 353597      0t0  TCP 205.***.***.***:80->                          162.158.91.237:26638 (ESTABLISHED)
nginx    29953    www-data   17u  IPv4  35685      0t0  TCP *:80 (LISTEN)
nginx    29953    www-data   18u  IPv6  35686      0t0  TCP *:80 (LISTEN)
nginx    29953    www-data   19u  IPv4 353695      0t0  TCP 205.***.***.***:80->                          162.158.92.14:18796 (ESTABLISHED)
nginx    29954    www-data    3u  IPv4 353024      0t0  TCP 205.***.***.***:80->                          162.158.92.86:35243 (ESTABLISHED)
nginx    29954    www-data    6u  IPv4 353313      0t0  TCP 205.***.***.***:80->                          162.158.69.127:31977 (ESTABLISHED)
nginx    29954    www-data    7u  IPv4 353211      0t0  TCP 205.***.***.***:80->                          162.158.69.127:24322 (ESTABLISHED)
nginx    29954    www-data    8u  IPv4 353655      0t0  TCP 205.***.***.***:80->                          162.158.91.237:14710 (ESTABLISHED)
nginx    29954    www-data   14u  IPv4 353687      0t0  TCP 205.***.***.***:80->                          162.158.88.60:13097 (ESTABLISHED)
nginx    29954    www-data   15u  IPv4 353691      0t0  TCP 205.***.***.***:80->                          162.158.89.95:19092 (ESTABLISHED)
nginx    29954    www-data   17u  IPv4  35685      0t0  TCP *:80 (LISTEN)
nginx    29954    www-data   18u  IPv6  35686      0t0  TCP *:80 (LISTEN)
nginx    29955    www-data    3u  IPv4 353627      0t0  TCP 205.***.***.***:80->                          162.158.69.127:11895 (ESTABLISHED)
nginx    29955    www-data    6u  IPv4 353639      0t0  TCP 205.***.***.***:80->                          162.158.91.237:19781 (ESTABLISHED)
nginx    29955    www-data    7u  IPv4 353065      0t0  TCP 205.***.***.***:80->                          162.158.89.227:26824 (ESTABLISHED)
nginx    29955    www-data    8u  IPv4 353193      0t0  TCP 205.***.***.***:80->                          162.158.74.133:14209 (ESTABLISHED)
nginx    29955    www-data   16u  IPv4 353651      0t0  TCP 205.***.***.***:80->                          162.158.90.142:28079 (ESTABLISHED)
nginx    29955    www-data   17u  IPv4  35685      0t0  TCP *:80 (LISTEN)
nginx    29955    www-data   18u  IPv6  35686      0t0  TCP *:80 (LISTEN)
httpd    29962    www-data    3u  IPv4  28033      0t0  TCP 127.0.0.1:81 (LISTEN                          )
httpd    29963    www-data    3u  IPv4  28033      0t0  TCP 127.0.0.1:81 (LISTEN                          )
httpd    29964    www-data    3u  IPv4  28033      0t0  TCP 127.0.0.1:81 (LISTEN                          )
httpd    30053    www-data    3u  IPv4  28033      0t0  TCP 127.0.0.1:81 (LISTEN                          )
sshd     32051        root    3u  IPv4 292215      0t0  TCP 205.***.***.***:22->                          182.185.**.***:11091 (ESTABLISHED)
sshd     32060 serverpilot    3u  IPv4 292215      0t0  TCP 205.***.***.***:22->                          182.185.**.***:11091 (ESTABLISHED)
sshd     32274        root    3u  IPv4 297153      0t0  TCP 205.***.***.***:22->                          182.185.**.***:11580 (ESTABLISHED)````

allinurl · 2017-04-20T22:08:19Z

@Key7i Did you run lsof -i -n -P while goaccess was running? It should show up on the list, e.g.,

goaccess 1157 root 30u IPv4 3093 0t0 TCP *:7890 (LISTEN)

Also, if the machine running goaccess is reachable from the outside world through 205.***.***.***, then you can try specifying it in the command line with --ws-url:

goaccess /srv/users/serverpilot/log/site/site_nginx.access.log --log-format=COMBINED -o /srv/users/serverpilot/apps/site/public/report.html --real-time-html --ws-url=ws://205.***.***.***:7890

sajjadalis · 2017-04-20T22:29:00Z

Thanks for quick response.

I used this command and it's working fine now.
goaccess /srv/users/serverpilot/log/site/site_nginx.access.log --log-format=COMBINED -o /srv/users/serverpilot/apps/site/public/report.html --real-time-html --ws-url=ws://205.***.***.***:7890

A stupid question maybe. How can i run this real-time report permanently? And after running this real-time command how can i keep using terminal?

Thanks for this great tool.

allinurl · 2017-04-20T22:45:40Z

Glad that worked :) Usually it's best to use the public IP with --ws-url rather than the hostname just because some weird stuff can occur with the host not pointing to the right machine. For instance,

--ws-url=ws://205.***.***.***:7890

Great question, from v1.2 you can pass --daemonize, goaccess will keep running and you will be able to use the terminal as usual.

@muagkov please take a look at my previous response since @Key7i had the same issue. Feel free to reopen it if needed.

trancef0rmer · 2018-05-18T14:02:07Z

I have the same issue. Tried all written above. nothing helped. Got the error : WebSocket connection to wss://ip:7890/ failed: WebSocket opening handshake timed out.
I'm not using https, no proxy, opened port
cmd: goaccess logfile.log -o public_html/index.html --real-time-report --ws-url=wss://host_ip

What else could I try? thank you

allinurl · 2018-05-18T14:05:46Z

@trancef0rmer If you are using the latest version, you can try without --ws-url

goaccess logfile.log -o public_html/index.html --real-time-html

trancef0rmer · 2018-05-18T14:10:06Z

I'm using v.1.2
with no --ws-url I got only the loading gif in the middle of the page

allinurl · 2018-05-18T14:11:28Z

@trancef0rmer what's the output from your browser's console?

trancef0rmer · 2018-05-18T14:15:28Z

It was Mixed Content. I tried over https.
With no --ws-url, It works just fine ofer http )). Thank you!
But how can I handle error with mixt content over https ?
Should I indicate the cert and the key ?

allinurl · 2018-05-18T14:18:33Z

@trancef0rmer You will need to use --ws-url=wss://host as you had it before and pass a --ssl-cert=<path/cert.crt> --ssl-key=<path/priv.key> or make sure everything runs on http.

trancef0rmer · 2018-05-18T14:34:59Z

It works !!
Thank you allinurl !!

trancef0rmer · 2018-05-18T14:35:40Z

Should we wait for some integration with grafana ? TY

allinurl · 2018-05-18T14:47:10Z

@trancef0rmer Glad that worked :)

Should there be an integration with grafana? What are you looking to do with goaccess and grafana?

trancef0rmer · 2018-05-18T14:51:30Z

Just asked ))
Thank you Gerardo !
GoAccess is really a great thing !

allinurl · 2018-05-18T14:53:36Z

@trancef0rmer np, always happy to help :)

trancef0rmer · 2018-05-18T15:08:20Z

One more question
It is a good idea to copy apache log format to goaccess config ?
In apache I have : "%v:%p %h %l %u %t "%r" %>s %O "%{Referer}i" "%{User-Agent}i""
If so, why I'm getting : Token '+0300]' doesn't match specifier '%h' ?

allinurl · 2018-05-18T15:17:05Z

@trancef0rmer no, goaccess and apache use different log formats. Please take a look at the man page for details on the specifiers.

trancef0rmer · 2018-05-22T08:15:11Z

Hi, @allinurl !
Sorry about bothering you with stupid questions, but I can not get rid of Token '+0300]' doesn't match specifier '%h' error.
This is my log:
mydomain.com:443 187.136.215.148 - - [22/May/2018:11:07:00 +0300] "POST /email/?_task=mail&_action=refresh HTTP/1.1" 200 992 "https://mydomain.com/email/?_task=mail&_mbox=INBOX" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36"
This is my log format:
log-format "%v:%^ %h %^ %^ [%d:%t %^] \"%r\" %s %b "%R" "%u""
time-format %T
date-format %d/%b/%Y
What am I missing ? Thank you !

allinurl · 2018-05-22T11:49:57Z

@trancef0rmer This should do it:

 goaccess access.log --log-format=VCOMBINED

trancef0rmer · 2018-05-22T15:03:36Z

Same thing (

allinurl · 2018-05-22T18:07:50Z

@trancef0rmer for the line you posted above, it does work for me. You can try:

goaccess access.log --log-format=VCOMBINED --no-global-config

Does it work parsing that single line?

trancef0rmer · 2018-05-23T07:29:48Z

thank you @allinurl
I have noticed that is some lines of my log file, virtualhost and port ( %v:%p ) are missing
Apache logs monit like that
142.115.381.157 - - [23/May/2018:08:58:45 +0300] "GET / HTTP/1.1" 301 437 "-" "Monit/5.9"
These lines don't allow me to parse log file.
There is a way to ignore them ?
Thank you

allinurl · 2018-05-23T11:57:07Z

@trancef0rmer You can try using --num-tests=0 to parse the log and try to ignore them.

trancef0rmer · 2018-05-23T13:21:15Z

thanks @allinurl, it works with --num-test-0.
However, I think it's not parsing correctly, because unique visitors panel and referring sites panel are not showing anything, and browsers and OSes are all unknown.

allinurl · 2018-05-23T13:45:37Z

@trancef0rmer can you please attach your access log with a substantial number of lines on it?

trancef0rmer · 2018-05-23T13:53:34Z

@allinurl
https://pastebin.com/raw/rmhx2q6s

trancef0rmer · 2018-05-23T14:02:06Z

I just set this custom
log-format "%v:%^ %h %^ %^ [%d:%t %^] \"%r\" %s %b "%R" "%u"" , instead of CLF with VHOST
and referring, browsers and OS appeared.
I don't get what's wrong with that http://prntscr.com/jlmhlh

allinurl · 2018-05-23T14:34:31Z

This works perfectly fine for me with the log you posted above, please try:

# wget https://pastebin.com/raw/rmhx2q6s
# goaccess rmhx2q6s --log-format=VCOMBINED

trancef0rmer · 2018-05-23T15:09:01Z

@allinurl
Tried. It's working, but in real time html that panel is not working as I expected.
I mean
http://prntscr.com/jlnj48
Maybe I need more log data ?

allinurl · 2018-05-23T15:11:35Z

@trancef0rmer that's right, you only have one day worth of data. In a month and if you are not rotating your log, then your report will look similar to rt.goaccess.io.

trancef0rmer · 2018-05-23T15:15:40Z

@allinurl
That makes sense.
Finally I can enjoy reading my apache logs )
Thank you for help.
Nice tool !

trancef0rmer · 2018-05-28T13:33:02Z

Hello, @allinurl
I am using vlogger for splitting apache logs for each vhost, but the output log of that is missing %v. Is there a way for goaccess to sort logs by vhost only by reading access.log path (which contains vhost name) ? TY

allinurl · 2018-05-28T13:37:32Z

@trancef0rmer Please take a look at this #673

trancef0rmer · 2018-05-28T14:11:21Z

Thanks @allinurl,
I guess it is modifying access.log on the fly, before goaccess is reading it.
The difference is that my virtual host name is present in file path, not in file name, but I will try to modify the awk command. ty

trancef0rmer · 2018-05-28T14:13:34Z

Oh, I see now. It's FILENAME, anyway)

allinurl · 2018-05-28T14:18:02Z

@trancef0rmer you can definitely do it with awk. I believe there's an example in the man page where you can extract a field and place it somewhere else.

allinurl added html report question websocket-server labels Apr 15, 2017

allinurl closed this as completed Apr 20, 2017

BarbzYHOOL mentioned this issue Aug 13, 2017

Can't make browser realtime work, it's always static #865

Closed

goaccess not running in --real-time-html #737

goaccess not running in --real-time-html #737

Comments

muagkov commented Apr 14, 2017

allinurl commented Apr 14, 2017

muagkov commented Apr 14, 2017 • edited by allinurl Loading

allinurl commented Apr 14, 2017

allinurl commented Apr 18, 2017

muagkov commented Apr 18, 2017 • edited Loading

allinurl commented Apr 18, 2017

sajjadalis commented Apr 20, 2017 • edited Loading

allinurl commented Apr 20, 2017 • edited Loading

sajjadalis commented Apr 20, 2017 • edited Loading

allinurl commented Apr 20, 2017

trancef0rmer commented May 18, 2018 • edited by allinurl Loading

allinurl commented May 18, 2018

trancef0rmer commented May 18, 2018

allinurl commented May 18, 2018

trancef0rmer commented May 18, 2018 • edited Loading

allinurl commented May 18, 2018

trancef0rmer commented May 18, 2018

trancef0rmer commented May 18, 2018

allinurl commented May 18, 2018

trancef0rmer commented May 18, 2018

allinurl commented May 18, 2018

trancef0rmer commented May 18, 2018

allinurl commented May 18, 2018

trancef0rmer commented May 22, 2018

allinurl commented May 22, 2018

trancef0rmer commented May 22, 2018

allinurl commented May 22, 2018

trancef0rmer commented May 23, 2018

allinurl commented May 23, 2018

trancef0rmer commented May 23, 2018

allinurl commented May 23, 2018

trancef0rmer commented May 23, 2018

trancef0rmer commented May 23, 2018 • edited Loading

allinurl commented May 23, 2018

trancef0rmer commented May 23, 2018

allinurl commented May 23, 2018 • edited Loading

trancef0rmer commented May 23, 2018

trancef0rmer commented May 28, 2018

allinurl commented May 28, 2018

trancef0rmer commented May 28, 2018

trancef0rmer commented May 28, 2018

allinurl commented May 28, 2018

muagkov commented Apr 14, 2017 •

edited by allinurl

Loading

muagkov commented Apr 18, 2017 •

edited

Loading

sajjadalis commented Apr 20, 2017 •

edited

Loading

allinurl commented Apr 20, 2017 •

edited

Loading

sajjadalis commented Apr 20, 2017 •

edited

Loading

trancef0rmer commented May 18, 2018 •

edited by allinurl

Loading

trancef0rmer commented May 18, 2018 •

edited

Loading

trancef0rmer commented May 23, 2018 •

edited

Loading

allinurl commented May 23, 2018 •

edited

Loading