Broke nginx

[dolgovas]

Describe the bug

Sometimes nginx crashed

To reproduce

Steps to reproduce the behavior:

install nginx

nginx -v
nginx version: nginx/1.25.1 (nginx-plus-r30)

install nginx-plus-module-otel
Version: 30+0.1-1~jammy

enable it and tried to send traces to jaeger.
sometimes happened this

/var/log/nginx/error.log-1701637201.gz:2023/12/04 00:03:45 [alert] 3166959#3166959: worker process 2505010 exited on signal 11 (core dumped)
/var/log/nginx/error.log-1701637201.gz:2023/12/04 00:03:46 [alert] 3166959#3166959: worker process 2504758 exited on signal 11 (core dumped)
/var/log/nginx/error.log-1701637201.gz:2023/12/04 00:03:46 [alert] 3166959#3166959: worker process 1643798 exited on signal 11 (core dumped)
/var/log/nginx/error.log-1701637201.gz:2023/12/04 00:03:49 [alert] 3166959#3166959: worker process 2505016 exited on signal 11 (core dumped)
/var/log/nginx/error.log-1701637201.gz:2023/12/04 00:03:49 [alert] 3166959#3166959: worker process 2504822 exited on signal 11 (core dumped)

gdb shows this

Core was generated by `nginx: worker process                   '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  ngx_list_push (l=0x56350a2eb618) at src/core/ngx_list.c:38
38	src/core/ngx_list.c: No such file or directory.
[Current thread is 1 (Thread 0x7f3842ce0740 (LWP 533524))]
(gdb) bt
#0  ngx_list_push (l=0x56350a2eb618) at src/core/ngx_list.c:38
#1  0x00007f381836cd82 in (anonymous namespace)::setHeader (r=0x56350a2eb5b0, name=..., value=...)
    at ./debian/build-nginx/extra/nginx-otel-0.1/src/http_module.cpp:246
#2  0x00007f381836dc57 in (anonymous namespace)::inject (tc=..., r=0x56350a2eb5b0)
    at ./debian/build-nginx/extra/prebuilt/include/opentelemetry/nostd/string_view.h:39
#3  (anonymous namespace)::onRequestStart (r=0x56350a2eb5b0) at ./debian/build-nginx/extra/nginx-otel-0.1/src/http_module.cpp:341
#4  0x00005634edd48461 in ngx_http_core_rewrite_phase (r=0x56350a2eb5b0, ph=<optimized out>)
    at src/http/ngx_http_core_module.c:1005
#5  0x00005634edd43c1d in ngx_http_core_run_phases (r=0x56350a2eb5b0) at src/http/ngx_http_core_module.c:951
#6  0x00005634edd586cb in ngx_http_process_request_line (rev=0x5634fac59cd0) at src/http/ngx_http_request.c:1343
#7  0x00005634edd24b93 in ngx_epoll_process_events (cycle=<optimized out>, timer=<optimized out>, flags=1)
    at src/event/modules/ngx_epoll_module.c:901
#8  0x00005634edd1b884 in ngx_process_events_and_timers (cycle=cycle@entry=0x5634f4e66fb0) at src/event/ngx_event.c:343
#9  0x00005634edd2100b in ngx_worker_process_cycle (cycle=cycle@entry=0x5634f4e66fb0, data=data@entry=0x2)
    at src/os/unix/ngx_process_cycle.c:751
#10 0x00005634edd181d2 in ngx_spawn_process (cycle=cycle@entry=0x5634f4e66fb0,
    proc=proc@entry=0x5634edd20f70 <ngx_worker_process_cycle>, data=data@entry=0x2,
    name=name@entry=0x5634ede3b9e8 "worker process", respawn=respawn@entry=-4) at src/os/unix/ngx_process.c:199
#11 0x00005634edd18c08 in ngx_start_worker_processes (cycle=cycle@entry=0x5634f4e66fb0, n=10, type=type@entry=-4)
    at src/os/unix/ngx_process_cycle.c:355
#12 0x00005634edd2000a in ngx_master_process_cycle (cycle=0x5634f4e66fb0) at src/os/unix/ngx_process_cycle.c:245
#13 0x00005634edcf1ad9 in main (argc=<optimized out>, argv=<optimized out>) at src/core/nginx.c:384

Expected behavior

A clear and concise description of what you expected to happen.

Your environment

# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04.3 LTS"

uname -srvmpio
Linux 5.15.0-79-generic #86-Ubuntu SMP Mon Jul 10 16:07:21 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

[dplotnikov-f5]

Thank you for reporting the issue! Would you be able to share some more details? In particular can you provide:

• the full Nginx configuration in use
• the crash dump the call stack above was taken from

Also do you use any 3rd party modules? If so, would the issue persists with the 3rd party modules disabled?

[dolgovas]

Thank you!
When will you release a new package for nginx-plus?

[thresheek]

Hi @dolgovas, we plan to release NGINX Plus R31 next Tuesday, December 19th, with the otel module containing this fix.

Let me know if that works for you

Thanks!

[thresheek]

Hi @dolgovas, the fix is available with NGINX Plus R31 released today.