Bug Report: Extra Increment on Generator::iterator After Completion Causes Crash Using Clang15

[faker2048]

Crashed Code

Generator<int> impl() {
    co_yield 1;
    co_yield 2;
    co_yield 3;
}

Generator<int> ints() {
    for (int i : impl()) {
      printf("i = %d\n", i);
      co_yield i;
    }
}

int main() {
    for (auto x : ints()) {
        std::cout << x << std::endl;
    }
    return 0;
}

• 🟢 Expected Output

i = 1
1
i = 2
2
i = 3
3

• 🔴 Actual Output

i = 1
1
i = 2
2
i = 3
3
i = 3
3
terminate called after throwing an instance of 'std::logic_error'
  what():  Can't increment generator end iterator

• Debug info

(base) yxb@DESKTOP-H7IJQ06:~/workspace/async_simple/build$ lldb ./demo_examp
le/test_crash 
(lldb) target create "./demo_example/test_crash"
Current executable set to '/home/yxb/workspace/async_simple/build/demo_example/test_crash' (x86_64).
(lldb) r
Process 2435 launched: '/home/yxb/workspace/async_simple/build/demo_example/test_crash' (x86_64)
i = 1
1
i = 2
2
i = 3
3
i = 3
3
terminate called after throwing an instance of 'std::logic_error'
  what():  Can't increment generator end iterator
Process 2435 stopped
* thread #1, name = 'test_crash', stop reason = signal SIGABRT
    frame #0: 0x00007ffff7adc9fc libc.so.6`__GI___pthread_kill [inlined] __pthread_kill_implementation(no_tid=0, signo=6, threadid=140737348114304) at pthread_kill.c:44:76
(lldb) bt
* thread #1, name = 'test_crash', stop reason = signal SIGABRT
  * frame #0: 0x00007ffff7adc9fc libc.so.6`__GI___pthread_kill [inlined] __pthread_kill_implementation(no_tid=0, signo=6, threadid=140737348114304) at pthread_kill.c:44:76
    frame #1: 0x00007ffff7adc9b0 libc.so.6`__GI___pthread_kill [inlined] __pthread_kill_internal(signo=6, threadid=140737348114304) at pthread_kill.c:78:10
    frame #2: 0x00007ffff7adc9b0 libc.so.6`__GI___pthread_kill(threadid=140737348114304, signo=6) at pthread_kill.c:89:10
    frame #3: 0x00007ffff7a88476 libc.so.6`__GI_raise(sig=6) at raise.c:26:13
    frame #4: 0x00007ffff7a6e7f3 libc.so.6`__GI_abort at abort.c:79:7
    frame #5: 0x00007ffff7e18b9e libstdc++.so.6`___lldb_unnamed_symbol7245 + 96
    frame #6: 0x00007ffff7e2420c libstdc++.so.6`___lldb_unnamed_symbol7659 + 12
    frame #7: 0x00007ffff7e24277 libstdc++.so.6`std::terminate() + 23
    frame #8: 0x00007ffff7e2452b libstdc++.so.6`__cxa_rethrow + 75
    frame #9: 0x000055555555580b test_crash`async_simple::coro::Generator<int, void, void>::promise_type::unhandled_exception(this=<unavailable>) at Generator.h:516:13
    frame #10: 0x0000555555555676 test_crash`ints() at test_crash.cpp:14:16
    frame #11: 0x0000555555555428 test_crash`main [inlined] std::coroutine_handle<async_simple::coro::detail::gen_promise_base<int&&>>::resume(this=<unavailable>) const at coroutine.h:273:9
    frame #12: 0x0000555555555421 test_crash`main [inlined] async_simple::coro::Generator<int, void, void>::iterator::operator++(this=<unavailable>) at Generator.h:567:30
    frame #13: 0x0000555555555419 test_crash`main at test_crash.cpp:22:17
    frame #14: 0x00007ffff7a6fd90 libc.so.6`__libc_start_call_main(main=(test_crash`main [inlined] ints() at test_crash.cpp:21
test_crash`main at test_crash.cpp:22:19), argc=1, argv=0x00007fffffffda68) at libc_start_call_main.h:58:16
    frame #15: 0x00007ffff7a6fe40 libc.so.6`__libc_start_main_impl(main=(test_crash`main [inlined] ints() at test_crash.cpp:21
test_crash`main at test_crash.cpp:22:19), argc=1, argv=0x00007fffffffda68, init=<unavailable>, fini=<unavailable>, rtld_fini=<unavailable>, stack_end=0x00007fffffffda58) at libc-start.c:392:3
    frame #16: 0x00005555555551e5 test_crash`_start + 37
(lldb) q

Step to Reproduce

• This bug can be reproduced in both Ubuntu 22.04 clang15 and macos appleclang15.
• can't reproduce in clang19, or using CMAKE_BUILD_TYPE=Debug to build.

0. Setup enviroment (Ubuntu)

sudo apt install clang-15

1. Setup debug code

git clone https://github.com/faker2048/async_simple.git
cd async_simple
git checkout debug_crash

or you can make this simple change follow this git diff in your own repo: main...faker2048:async_simple:debug_crash

2. Compile and Run

export CC=clang-15 &&  export CXX=clang++-15
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=RelWithDebInfo # Notice: can't reproduce using CMAKE_BUILD_TYPE=Debug
make -j100
./demo_example/test_crash

Anything else

Modify Generator::iterator::operator++() we can see _coro.done() is true when rerun ./demo_example/test_crash

• output

i = 1
1
i = 2
2
i = 3
3
i = 3
3
nullptr ?: 0, done ?: 1
Aborted

Are you willing to submit a PR?

• Yes I am willing to submit a PR!
  （还在学习c++20的协程）

[ChuanqiXu9]

I feel this is a compiler bug. There are several bugs in clang15 related to coroutines. It may be fine to use Lazy. But generator is more complex.

I'll close this issue and update the document later.

[faker2048]

@ChuanqiXu9
umm... I forgot to mention: this issue will result in one of the consequences being that with the latest macos default apple-clang (currently version 15), ctest fails to pass a unit test --async_simple_coro_test.

3: Test command: /Users/yxb/workspace/async_simple/build/async_simple/coro/test/async_simple_coro_test
3: Working Directory: /Users/yxb/workspace/async_simple/build/async_simple/coro/test
3: Test timeout computed to be: 10000000
3: Note: Google Test filter = GeneratorTest.testExample
3: [==========] Running 1 test from 1 test suite.
3: [----------] Global test environment set-up.
3: [----------] 1 test from GeneratorTest
3: [ RUN      ] GeneratorTest.testExample
3: start
3: foo!
3: bar!
3: bar!
3: unknown file: Failure
3: C++ exception with description "Can't increment generator end iterator" thrown in the test body.
3: 
3: [  FAILED  ] GeneratorTest.testExample (0 ms)
3: [----------] 1 test from GeneratorTest (0 ms total)
3: 
3: [----------] Global test environment tear-down
3: [==========] 1 test from 1 test suite ran. (0 ms total)
3: [  PASSED  ] 0 tests.
3: [  FAILED  ] 1 test, listed below:
3: [  FAILED  ] GeneratorTest.testExample
3: 
3:  1 FAILED TEST
3/5 Test #3: run_async_simple_coro_test .......***Failed    0.00 sec
test 4
    Start 4: run_async_simple_executor_test

[ChuanqiXu9]

@ChuanqiXu9 umm... I forgot to mention: this issue will result in one of the consequences being that with the latest macos default apple-clang (currently version 15), ctest fails to pass a unit test --async_simple_coro_test.

3: Test command: /Users/yxb/workspace/async_simple/build/async_simple/coro/test/async_simple_coro_test
3: Working Directory: /Users/yxb/workspace/async_simple/build/async_simple/coro/test
3: Test timeout computed to be: 10000000
3: Note: Google Test filter = GeneratorTest.testExample
3: [==========] Running 1 test from 1 test suite.
3: [----------] Global test environment set-up.
3: [----------] 1 test from GeneratorTest
3: [ RUN      ] GeneratorTest.testExample
3: start
3: foo!
3: bar!
3: bar!
3: unknown file: Failure
3: C++ exception with description "Can't increment generator end iterator" thrown in the test body.
3: 
3: [  FAILED  ] GeneratorTest.testExample (0 ms)
3: [----------] 1 test from GeneratorTest (0 ms total)
3: 
3: [----------] Global test environment tear-down
3: [==========] 1 test from 1 test suite ran. (0 ms total)
3: [  PASSED  ] 0 tests.
3: [  FAILED  ] 1 test, listed below:
3: [  FAILED  ] GeneratorTest.testExample
3: 
3:  1 FAILED TEST
3/5 Test #3: run_async_simple_coro_test .......***Failed    0.00 sec
test 4
    Start 4: run_async_simple_executor_test

Thanks for reporting this. I think we can only comment out these tests (the Generator Test) conditionally if the compiler is apple-clang15. Would you like to contribute that? Since it is not easy for me to test it locally.

[faker2048]

@ChuanqiXu9 umm... I forgot to mention: this issue will result in one of the consequences being that with the latest macos default apple-clang (currently version 15), ctest fails to pass a unit test --async_simple_coro_test.

3: Test command: /Users/yxb/workspace/async_simple/build/async_simple/coro/test/async_simple_coro_test
3: Working Directory: /Users/yxb/workspace/async_simple/build/async_simple/coro/test
3: Test timeout computed to be: 10000000
3: Note: Google Test filter = GeneratorTest.testExample
3: [==========] Running 1 test from 1 test suite.
3: [----------] Global test environment set-up.
3: [----------] 1 test from GeneratorTest
3: [ RUN      ] GeneratorTest.testExample
3: start
3: foo!
3: bar!
3: bar!
3: unknown file: Failure
3: C++ exception with description "Can't increment generator end iterator" thrown in the test body.
3: 
3: [  FAILED  ] GeneratorTest.testExample (0 ms)
3: [----------] 1 test from GeneratorTest (0 ms total)
3: 
3: [----------] Global test environment tear-down
3: [==========] 1 test from 1 test suite ran. (0 ms total)
3: [  PASSED  ] 0 tests.
3: [  FAILED  ] 1 test, listed below:
3: [  FAILED  ] GeneratorTest.testExample
3: 
3:  1 FAILED TEST
3/5 Test #3: run_async_simple_coro_test .......***Failed    0.00 sec
test 4
    Start 4: run_async_simple_executor_test

Thanks for reporting this. I think we can only comment out these tests (the Generator Test) conditionally if the compiler is apple-clang15. Would you like to contribute that? Since it is not easy for me to test it locally.

Sure

[faker2048]

@ChuanqiXu9
I think solution is to conditionally remove GeneratorTest.cpp from Bazel and CMake files.

But should we do more, to prevent users from using Generator.h when compiling with Clang-15, adding like the following to the very top of Generator.h:

#if defined(__clang__)
#if __clang_major__ == 15
#error "Clang 15 is not supported due to some issues."
#endif
#endif

[ChuanqiXu9]

@ChuanqiXu9 I think solution is to conditionally remove GeneratorTest.cpp from Bazel and CMake files.

But should we do more, to prevent users from using Generator.h when compiling with Clang-15, adding like the following to the very top of Generator.h:

#if defined(__clang__)
#if __clang_major__ == 15
#error "Clang 15 is not supported due to some issues."
#endif
#endif

It sounds good. Maybe a #warning is better.