Can't eliminate RX_LATE packets [RAN650, n77, BW:100 MHz, no DPDK]

[ppisanski-sg]

Hey guys,

I saw similar topics here, however couldn't find a fix. I run setup with RAN650, n77, no DPDK, bandwidth 100MHz, center freq: 3879.99 MHz, E810-XXVDA4T, Xeon(R) D-2187NT cpu, 128 GB RAM. Open5gs and srsRAN are running on the same server.
Setup is in sync, rms for ptp4l does not exceed 4, phc offset fluctuates between -40 and 30.
UE -redmi note 5G - attaches successfully, DL at iperf reaches ~200 Mbps between UE and gNB, UL ~50 Mbps.
However I can't get rid of RX_EARLY and RX_LATE packets.

Also number of dropped packets is quite high:

Network interface:

ethtool -i enp2s0
driver: ice
version: 1.14.11
firmware-version: 4.30 0x8001af27 1.3429.0

Kernel: 5.15.0-1066-realtime

lsb_release -a

No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.4 LTS
Release: 22.04
Codename: jammy

My gnb configuration is following:

amf:
addr: 127.0.0.5 # The address or hostname of the AMF.
bind_addr: 127.0.0.5 # A local IP that the gNB binds to for traffic from the AMF.

cu_up:
gtpu_queue_size: 4096

ru_ofh:
ru_bandwidth_MHz: 100 # RU instantaneous bandwidth.
is_prach_cp_enabled: false
t1a_max_cp_dl: 470
t1a_min_cp_dl: 419
t1a_max_cp_ul: 336
t1a_min_cp_ul: 285
t1a_max_up: 345
t1a_min_up: 294
ta4_max: 200
ta4_min: 0
ignore_ecpri_payload_size: true
is_prach_cp_enabled: false
compr_method_ul: bfp
compr_bitwidth_ul: 9
compr_method_dl: bfp
compr_bitwidth_dl: 9
compr_method_prach: bfp
compr_bitwidth_prach: 9
enable_ul_static_compr_hdr: true
enable_dl_static_compr_hdr: true
iq_scaling: 10

cells:
- network_interface: enp2s0 # Ethernet interface name used to communicate with the RU.
ru_mac_addr: (removed from here) # RU MAC address.
du_mac_addr: (removed from here) # DU MAC address.
vlan_tag_cp: 3 # VLAN tag value for C-Plane.
vlan_tag_up: 3 # VLAN tag value for U-Plane.
prach_port_id: [4, 5] # PRACH eAxC port value.
dl_port_id: [0, 1] # Downlink eAxC port values.
ul_port_id: [0, 1] # Uplink eAxC port values.

cell_cfg:
dl_arfcn: 658666
band: 77
channel_bandwidth_MHz: 100
tdd_ul_dl_cfg:
dl_ul_tx_period: 10
nof_dl_symbols: 6
nof_ul_symbols: 4
nof_dl_slots: 7
nof_ul_slots: 2
ssb:
ssb_period: 20
ssb_block_power_dbm: -21
pusch:
dmrs_additional_position: 2
pdsch:
dmrs_additional_position: 2
mcs_table: qam256

olla_target_bler: 0.05
olla_max_cqi_offset: 2
max_rb_size: 275
max_pdschs_per_slot: 6
max_alloc_attempts: 6
common_scs: 30 # Subcarrier spacing in kHz used for data.
plmn: "00101" # PLMN broadcasted by the gNB.
tac: 1 # Tracking area code (needs to match the core configuration).
pci: 1 # Physical cell ID.
nof_antennas_dl: 2 # Downlink number of antennas.
nof_antennas_ul: 2 # Uplink number of anntennas.
prach:
prach_config_index: 7 # PRACH configuration index.
prach_root_sequence_index: 1 # PRACH root sequence index.
zero_correlation_zone: 0 # Zero correlation zone.
prach_frequency_start: 0 #MB - does not work with anything more than 0

log:
filename: /var/log/gnb.log # Path of the log file.
all_level: info # Logging level applied to all layers.

pcap:
mac_enable: false # Set to true to enable MAC-layer PCAPs.
mac_filename: /tmp/gnb_mac.pcap # Path where the MAC PCAP is stored.
ngap_enable: false # Set to true to enable NGAP PCAPs.
ngap_filename: /tmp/gnb_ngap.pcap # Path where the NGAP PCAP is stored.

metrics:
autostart_stdout_metrics: true

RU configuration:

root@benetelru:~# cat /etc/benetel-rootfs-version

RAN650-1v1.0.4-dda1bf5

root@benetelru:~# cat /etc/ru_config.cfg

mimo_mode=1_2_3_4_4x2
downlink_scaling=6
prach_format=long
compression=static_compressed
lf_prach_compression_enable=true
cplane_per_symbol_workaround=disabled
cuplane_dl_coupling_sectionID=disabled
flexran_prach_workaround=disabled
dl_ul_tuning_special_slot=0xfd00000

Moreover I see a lot of received messages, like:
Received Open Fronthaul message size is '2' bytes and is smaller than the section header size
and dropped messages, like:
Dropped received Open Fronthaul message as no uplink PRACH context was found for slot '177.9' and eAxC '4'
at gnb.log

Will be grateful for any hint.

[aibtw]

The messages like " Dropped received Open Fronthaul message as no uplink PRACH context was found for slot '177.9' and eAxC '4'", I think it is because the RU is configured 4x2 and your DU is configured 2x2.

Not sure about the other issues.

[ppisanski-sg]

When I changed to 4x2@DU, number of RX_EARLY and RX_LATE jumps to the sky:

and throughput drops down to the floor level.