perl SIGSEGVs while sending message using AnyEvent::WebSocket::Client

[l1gi]

Hello,

I am experiencing a strange core dump on FreeBSD-13.0 with perl5-32.1 and p5-AnyEvent-WebSocket-Client-0.44_1 installed.

With 50% of probability my perl app crashes while trying to send() a message to established WS connection. This is the stacktrace:

Signal SEGV at /usr/local/lib/perl5/site_perl/mach/5.32/AnyEvent/Handle.pm line 1005 thread 1.
        AnyEvent::Handle::push_write(AnyEvent::Handle=HASH(0x809620498), "\x{81}\x{fe}\x{e}\x{bd}\x{8b}U\x{d9}\x{99}\x{f0}w\x{a9}\x{f8}\x{f9}4\x{b4}\x{ea}\x{a9}o\x{82}\x{bb}\x{ea}4\x{af}\x{fc}\x{fe}%\x{ac}\x{ea}\x{ef}!\x{99}\x{f2}\x{e7}<\x{b7}\x{fc}\x{d4}d\x{ec}\x{f4}\x{a9}y\x{fb}\x{e1}\x{f9}%\x{ac}\x{e9}\x{fe}&\x{bd}\x{ed}\x{cb}>\x{b5}\x{f0}\x{e5}0\x{86}\x{a8}\x{be}"...) called at /usr/local/lib/perl5/site_perl/AnyEvent/WebSocket/Connection.pm line 236 thread 1
        AnyEvent::WebSocket::Connection::send(AnyEvent::WebSocket::Connection=HASH(0x8096007b0), "{\"params\":[\"aaveupusdt\@kline_15m\",\"xrpupusdt\@kline_15m\",\"ltcu"...) called at lib/trade/Binance.pm line 257 thread 1
        trade::Binance::_ws_request(trade::Binance=HASH(0x804e78d98), "SUBSCRIBE", ARRAY(0x804e92708), 2048584) called at lib/trade/Binance.pm line 452 thread 1
        trade::Binance::q_klines(trade::Binance=HASH(0x804e78d98), HASH(0x804a0b6f0)) called at lib/trade/Tool.pm line 126 thread 1
        trade::Tool::dequeue(trade::Binance=HASH(0x804e78d98), HASH(0x81064bd80)) called at lib/trade/Binance.pm line 359 thread 1
        trade::Binance::__ANON__[lib/trade/Binance.pm:360] called at /usr/local/lib/perl5/site_perl/mach/5.32/AnyEvent/Loop.pm line 325 thread 1
        AnyEvent::Loop::__ANON__[/usr/local/lib/perl5/site_perl/mach/5.32/AnyEvent/Loop.pm:326](ARRAY(0x810426690)) called at /usr/local/lib/perl5/site_perl/mach/5.32/AnyEvent/Loop.pm line 212 thread 1
        AnyEvent::Loop::one_event() called at /usr/local/lib/perl5/site_perl/mach/5.32/AnyEvent/Impl/Perl.pm line 46 thread 1
        AnyEvent::CondVar::Base::_wait(AnyEvent::CondVar=HASH(0x807766480)) called at /usr/local/lib/perl5/site_perl/mach/5.32/AnyEvent.pm line 2031 thread 1
        AnyEvent::CondVar::Base::recv(AnyEvent::CondVar=HASH(0x807766480)) called at lib/trade/Binance.pm line 366 thread 1
        trade::Binance::watch(trade::Binance=HASH(0x804e78d98)) called at ./trade line 76 thread 1
        main::__ANON__[./trade:76](trade::Binance=HASH(0x804e78d98)) called at ./trade line 76 thread 1
        eval {...} called at ./trade line 76 thread 1
Abort trap (core dumped)

gdb doesn't help much but shows me that the issue is somewhere down in libssl library.

$ gdb /usr/bin/perl ./perl.core
...
(No debugging symbols found in /usr/bin/perl)
[New LWP 113512]
[New LWP 105200]
[New LWP 113513]
[New LWP 113514]
Core was generated by `/usr/local/bin/perl ./trade watch --level=info'.
Program terminated with signal SIGSEGV, Segmentation fault.
Address not mapped to object.
#0  0x0000000805ac171c in ?? () from /usr/lib/libssl.so.111
[Current thread is 1 (LWP 113512)]
(gdb) bt
#0  0x0000000805ac171c in ?? () from /usr/lib/libssl.so.111
#1  0x0000000805acd986 in SSL_write () from /usr/lib/libssl.so.111
#2  0x0000000805380d69 in ?? () from /usr/local/lib/perl5/site_perl/mach/5.32/auto/Net/SSLeay/SSLeay.so
#3  0x00000008004fe70c in Perl_pp_entersub () from /usr/local/lib/perl5/5.32/mach/CORE/libperl.so.5.32
#4  0x00000008004f38f6 in Perl_runops_standard () from /usr/local/lib/perl5/5.32/mach/CORE/libperl.so.5.32
#5  0x0000000800458b90 in Perl_call_sv () from /usr/local/lib/perl5/5.32/mach/CORE/libperl.so.5.32
#6  0x0000000800ac8242 in ?? () from /usr/local/lib/perl5/5.32/mach/auto/threads/threads.so
#7  0x0000000800ac7a31 in ?? () from /usr/local/lib/perl5/5.32/mach/auto/threads/threads.so
#8  0x00000008005f182b in ?? () from /lib/libthr.so.3
#9  0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x7fffdfffe000

This is the code which handles the connection:

our $connection = undef;                                                                                               
...
        my $client = AnyEvent::WebSocket::Client->new;                                                                 
                                                                                                                   
        my $ws_ready = AnyEvent->condvar;                                                                              
        $client->connect($uri)->cb(                                                                                    
                sub {                                                                                                  
                        $connection = eval { shift->recv; };                                                           
                        if ($EVAL_ERROR) {                                                                             
                                threads->exit(2);                                                                      
                        }                                                                                              
                                                                                                                       
                        $connection->on(each_message => sub {                                                          
                                        my $con = shift;                                                               
                                        my $msg = shift;                                                               
                                                                                                                       
                                        my $r = eval {$self->_ws_response($msg->body);};                               
                                        if ($EVAL_ERROR) {                                                             
                                                threads->exit(4);                                                      
                                        }                                                                              
                                });                                                                                    
                                                                                                                       
                        $connection->on(finish => sub {                                                                
                                        $connection = undef;                                                           
                                        threads->exit(3);                                                              
                                });                                                                                    
                                                                                                                       
                        $ws_ready->send;                                                                               
                                                                                                                       
                });                                                                                                    
                                                                                                                       
        # wait for being connected                                                                                     
        $ws_ready->recv;

And this is the perl line which 'causes' the SIGSEGV:

        $connection->send($msg);                                                                                       

Am I lucky enough to get any hint what could be wrong, please? Dont' hesitate to ask should you need more info.

Thank you

[plicease]

Building a debug version of Perl + OpenSSL may provide some more insight from the gdb trace. Aside from that I don't think AE::WS::C isn't doing anything special with SSL, the more likely problem will be with one of AnyEvent::TLS, IO::Socket::SSL, Net::SSLeay, or OpenSSL itself. If you can reduce the issue to a small example program that demonstrates the crash that could also help.

[schmorp]

Unless AE::WS::C does something very fishy, this is an interaction between OpenSSL and AnyEvent::Handle, and should be reported (with openssl/AnyEvent and Net::SSLeay versions) to the AnyEvent list (anyevent@lists.schmorp.de). Without a reproducer, this will be hard to track down though, so a reproducer is almost rerquired.