Huge CPU utilization after commits on Oct 11

[bazuchan]

On version 223f292 and before syncookied was using about 40% cpu under 5Gbps syn flood.
Next working version 73e1ac0 (and any newer version) utilizes 100% cpu under same load.

[bazuchan]

We are running syncookied on Centos7 with elrepo 4.8 mainline kernel. Network card is Intel X520 with netmap-aware ixgbe driver. Rust is installed with rust-up script from nightly channel.

[bazuchan]

Perf top with "good" version:

  26.26%  sc14                 [.] sha1_transform_ssse3                                                                                                                               
  20.00%  sc14                 [.] syncookied::packet::handle_reply                                                                                                                   
   7.12%  [kernel]             [k] native_queued_spin_lock_slowpath                                                                                                                   
   6.70%  libc-2.17.so         [.] __memcmp_sse4_1                                                                                                                                    
   5.65%  sc14                 [.] syncookied::run::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::hc125f6adeb87694e                                                     
   2.33%  [kernel]             [k] netmap_poll                                                                                                                                        
   1.62%  sc14                 [.] syncookied::tx::Sender::run                                                                                                                        
   1.28%  [kernel]             [k] ixgbe_netmap_rxsync                                                                                                                                
   1.24%  [kernel]             [k] _raw_spin_lock_irqsave                                                                                                                             
   1.08%  sc14                 [.] <std::net::ip::Ipv4Addr as core::cmp::Ord>::cmp                                                                                                    
   1.08%  sc14                 [.] csum_partial_folded                                                                                                                                
   1.02%  sc14                 [.] std::net::ip::Ipv4Addr::new                                                                                                                        
   0.97%  [kernel]             [k] cpupri_set                                                                                                                                         
   0.95%  [kernel]             [k] ixgbe_netmap_txsync                                                                                                                                
   0.87%  sc14                 [.] csum_partial                                                                                                                                       
   0.75%  sc14                 [.] <pnet::packet::tcp::MutableTcpOptionPacket<'a> as pnet::packet::MutablePacket>::payload_mut                                                        
   0.75%  sc14                 [.] pnet::packet::ethernet::MutableEthernetPacket::set_destination                                                                                     
   0.64%  sc14                 [.] pnet::packet::ethernet::MutableEthernetPacket::set_source                                                                                          
   0.60%  [kernel]             [k] fput                                                                                                                                               
   0.59%  [kernel]             [k] netmap_notify                                                                                                                                      

Perf top with "bad" version:

  38.48%  [kernel]            [k] native_queued_spin_lock_slowpath                                                                                                                    
  12.24%  sc2                 [.] syncookied::ring::Worker::run                                                                                                                       
  11.62%  sc2                 [.] sha1_transform_ssse3                                                                                                                                
   8.97%  [kernel]            [k] netmap_poll                                                                                                                                         
   5.62%  [kernel]            [k] _raw_spin_lock_irqsave                                                                                                                              
   3.97%  [kernel]            [k] netmap_notify                                                                                                                                       
   2.18%  [kernel]            [k] __wake_up_common                                                                                                                                    
   2.12%  [kernel]            [k] fput                                                                                                                                                
   1.39%  sc2                 [.] <pnet::util::MacAddr as core::cmp::PartialEq>::eq                                                                                                   
   0.93%  [kernel]            [k] ixgbe_netmap_rxsync                                                                                                                                 
   0.57%  [kernel]            [k] _raw_spin_unlock_irqrestore                                                                                                                         
   0.56%  [kernel]            [k] netmap_rx_irq                                                                                                                                       
   0.53%  sc2                 [.] csum_partial                                                                                                                                        
   0.51%  sc2                 [.] <std::net::ip::Ipv4Addr as core::cmp::Ord>::cmp                                                                                                     
   0.47%  sc2                 [.] csum_partial_folded                                                                                                                                 
   0.46%  [kernel]            [k] ixgbe_netmap_txsync                                                                                                                                 
   0.42%  [kernel]            [k] do_sys_poll                                                                                                                                         
   0.40%  sc2                 [.] pnet::packet::ethernet::MutableEthernetPacket::set_source                                                                                           
   0.32%  [kernel]            [k] __wake_up                                                                                                                                           
   0.32%  [kernel]            [k] napi_complete_done                                                                                                                                  
   0.27%  [kernel]            [k] native_irq_return_iret                                                                                                                              
   0.27%  sc2                 [.] <pnet::packet::tcp::MutableTcpOptionPacket<'a> as pnet::packet::MutablePacket>::payload_mut                                                         
   0.26%  [kernel]            [k] ixgbe_poll