Silent packet loss in PcapSpli... Note

Silent packet loss in PcapSplitter: a file collision bug on TCP session reuse

The author encountered file corruption issues while processing large PCAP files with a tshark-wrapped tool. Initial problems involved tshark errors indicating "Total Block Length" issues with large files. After parallelizing PCAP processing, file corruption became more apparent. A reproduction revealed that PcapSplitter reported more files and packets than actually existed on disk, despite exiting successfully. Two incorrect theories, file descriptor exhaustion and a hardcoded limit in PcapSplitter's concurrent file handling, were investigated and ruled out. The true cause was identified in PcapSplitter's FiveTupleSplitter, specifically concerning TCP session reuse. When a new SYN packet arrived for an already tracked five-tuple, the splitter incorrectly assigned a new file number but generated the same filename using only IP and port. This led to either file truncation or a race condition between two writer handlers. A fix was developed that only suffixes filenames upon actual collision. This fix has been tested and merged into the PcapSplitter library.