The network packet capture probe is designed to capture raw network packets off the wire and bulk load them into Kafka.  Kafka files are then picked up by the PCAP Storm Topology and bulk loaded into HDFS.  Each file is stored in HDFS as a Sequence file.  Once in HDFS, the PCAP service is used to read and the Sequence files and deliver compliant PCAP files via a restful API. There can be multiple probes pushing into the same Kafka topic.  The recommended hardware for the probe is an Intel family of network adapters that are supportable by DPDK.  

For description of components that the probe interacts with see:

PCAP Service

PCAP Topology

 

  • No labels