Protocols

July 2, 2025 · View on GitHub

You can find information on the protocols:

The mapping to the protobuf format is listed in the table below.

Field	Description	NetFlow v5	sFlow	NetFlow v9	IPFIX
Type	Type of flow message	NETFLOW_V5	SFLOW_5	NETFLOW_V9	IPFIX
time_received_ns	Timestamp in nanoseconds of when the message was received	Included	Included	Included	Included
sequence_num	Sequence number of the flow packet	Included	Included	Included	Included
sampling_rate	Sampling rate of the flow	Included	Included	Included	Included
sampler_address	Address of the device that generated the packet	IP source of packet	Agent IP	IP source of packet	IP source of packet
time_flow_start_ns	Time the flow started in nanoseconds	System uptime and first	=TimeReceived	System uptime and FIRST_SWITCHED (22)	flowStartXXX (150, 152, 154, 156)
time_flow_end_ns	Time the flow ended in nanoseconds	System uptime and last	=TimeReceived	System uptime and LAST_SWITCHED (23)	flowEndXXX (151, 153, 155, 157)
bytes	Number of bytes in flow	dOctets	Length of sample	IN_BYTES (1) OUT_BYTES (23)	octetDeltaCount (1) postOctetDeltaCount (23)
packets	Number of packets in flow	dPkts	=1	IN_PKTS (2) OUT_PKTS (24)	packetDeltaCount (2) postPacketDeltaCount (24)
src_addr	Source address (IP)	srcaddr (IPv4 only)	Included	Included	IPV4_SRC_ADDR (8) IPV6_SRC_ADDR (27)
dst_addr	Destination address (IP)	dstaddr (IPv4 only)	Included	Included	IPV4_DST_ADDR (12) IPV6_DST_ADDR (28)
etype	Ethernet type (0x86dd for IPv6...)	IPv4	Included	Included	Included
proto	Protocol (UDP, TCP, ICMP...)	prot	Included	PROTOCOL (4)	protocolIdentifier (4)
src_port	Source port (when UDP/TCP/SCTP)	srcport	Included	L4_SRC_PORT (7)	sourceTransportPort (7)
dst_port	Destination port (when UDP/TCP/SCTP)	dstport	Included	L4_DST_PORT (11)	destinationTransportPort (11)
in_if	Input interface	input	Included	INPUT_SNMP (10)	ingressInterface (10)
out_if	Output interface	output	Included	OUTPUT_SNMP (14)	egressInterface (14)
src_mac	Source mac address		Included	IN_SRC_MAC (56)	sourceMacAddress (56)
dst_mac	Destination mac address		Included	OUT_DST_MAC (57)	postDestinationMacAddress (57)
src_vlan	Source VLAN ID		From ExtendedSwitch	SRC_VLAN (58)	vlanId (58)
dst_vlan	Destination VLAN ID		From ExtendedSwitch	DST_VLAN (59)	postVlanId (59)
vlan_id	802.11q VLAN ID		Included	SRC_VLAN (58)	vlanId (58)
ip_tos	IP Type of Service	tos	Included	SRC_TOS (5)	ipClassOfService (5)
forwarding_status	Forwarding status			FORWARDING_STATUS (89)	forwardingStatus (89)
ip_ttl	IP Time to Live		Included	IPTTL (52)	minimumTTL (52
tcp_flags	TCP flags	tcp_flags	Included	TCP_FLAGS (6)	tcpControlBits (6)
icmp_type	ICMP Type		Included	ICMP_TYPE (32)	icmpTypeXXX (176, 178) icmpTypeCodeXXX (32, 139)
icmp_code	ICMP Code		Included	ICMP_TYPE (32)	icmpCodeXXX (177, 179) icmpTypeCodeXXX (32, 139)
ipv6_flow_label	IPv6 Flow Label		Included	IPV6_FLOW_LABEL (31)	flowLabelIPv6 (31)
fragment_id	IP Fragment ID		Included	IPV4_IDENT (54)	fragmentIdentification (54)
fragment_offset	IP Fragment Offset		Included	FRAGMENT_OFFSET (88)	fragmentOffset (88) and fragmentFlags (197)
src_as	Source AS number	src_as	From ExtendedGateway	SRC_AS (16)	bgpSourceAsNumber (16)
dst_as	Destination AS number	dst_as	From ExtendedGateway	DST_AS (17)	bgpDestinationAsNumber (17)
next_hop	Nexthop address	nexthop	From ExtendedRouter	IPV4_NEXT_HOP (15) IPV6_NEXT_HOP (62)	ipNextHopIPv4Address (15) ipNextHopIPv6Address (62)
next_hop_as	Nexthop AS number		From ExtendedGateway
src_net	Source address mask	src_mask	From ExtendedRouter	SRC_MASK (9) IPV6_SRC_MASK (29)	sourceIPv4PrefixLength (9) sourceIPv6PrefixLength (29)
dst_net	Destination address mask	dst_mask	From ExtendedRouter	DST_MASK (13) IPV6_DST_MASK (30)	destinationIPv4PrefixLength (13) destinationIPv6PrefixLength (30)
bgp_next_hop	BGP Nexthop address		From ExtendedGateway	BGP_IPV4_NEXT_HOP (18) BGP_IPV6_NEXT_HOP (63)	bgpNextHopIPv4Address (18) bgpNextHopIPv6Address (63)
bgp_communities	BGP Communities		From ExtendedGateway
as_path	AS Path		From ExtendedGateway
mpls_ttl	TTL of the MPLS label		Included
mpls_label	MPLS label list		Included

Producers

When using the raw producer, you can access a sample:

$ go run main.go -produce raw -format json

This can be useful if you need to debug received packets or looking to dive into a specific protocol (eg: the sFlow counters).

{
    "type": "sflow",
    "message":
    {
        "version": 5,
        "ip-version": 1,
        "agent-ip": "127.0.0.1",
        "sub-agent-id": 100000,
        "sequence-number": 1234,
        "uptime": 19070720,
        "samples-count": 1,
        "samples":
        [
            {
                "header":
                {
                    "format": 2,
                    "length": 124,
                    "sample-sequence-number": 340,
                    "source-id-type": 0,
                    "source-id-value": 6
                },
                "counter-records-count": 1,
                "records":
                [
                    {
                        "header":
                        {
                            "data-format": 1,
                            "length": 88
                        },
                        "data":
                        {
                            "if-index": 6,
                            "if-type": 6,
                            "if-speed": 0,
                            "if-direction": 0,
                            "if-status": 3,
                            "if-in-octets": 0,
                            "if-in-ucast-pkts": 1000,
                            "if-in-multicast-pkts": 0,
                            "if-in-broadcast-pkts": 0,
                            "if-in-discards": 0,
                            "if-in-errors": 0,
                            "if-in-unknown-protos": 0,
                            "if-out-octets": 0,
                            "if-out-ucast-pkts": 2000,
                            "if-out-multicast-pkts": 0,
                            "if-out-broadcast-pkts": 0,
                            "if-out-discards": 0,
                            "if-out-errors": 0,
                            "if-promiscuous-mode": 0
                        }
                    }
                ]
            }
        ]
    },
    "src": "[::ffff:127.0.0.1]:50001",
    "time_received": "2023-04-15T20:44:42.723694Z"
}

When using the Protobuf producer, you have access to various configuration options. The mapping.yaml file can be used with -mapping=mapping.yaml in the CLI.

It enables features like:

Add protobuf fields
Renaming fields (JSON/text)
Hashing key (for Kafka)
Mapping new values from samples

For example, you can rename:

formatter:
  rename: # only for JSON/text
    src_mac: src_macaddr
    dst_mac: dst_macaddr

Columns and renderers

By default, all the columns above will be printed when using JSON or text. To restrict to a subset of columns, in the mapping file, list the ones you want:

formatter:
  fields:
    - src_addr

There is a support for virtual columns (eg: icmp_name).

Renderers are a special handling of fields:

formatter:
  render:
    src_mac: mac
    dst_mac: mac
    dst_net: none # overrides: render the network as integer instead of prefix based on src/dst addr

You can assign a specific formatter.

Map custom fields

If you are using enterprise fields that you need decoded or if you are looking for specific bytes inside the packet sample.

Data coming from the flows can be added to the protobuf either as an unsigned/signed integer a slice of bytes.

The sflow section allow to extract data from packet samples inside sFlow and inside IPFIX (dataframe). The following layers are available:

0: no offset
3, ipv4, ipv6, arp: network layer, offsets to IP/IPv6 header
4, icmp, icmp6, udp, tcp: transport layer, offsets to TCP/UDP/ICMP header
7: application layer, offsets to the TCP/UDP payload

The data extracted will then be added to either an existing field (see samping rate below), or to a newly defined field.

In order to display them with JSON or text, you need to specify them in fields.

formatter:
  fields:
    - sampling_rate
    - custom_src_port
    - juniper_properties
  protobuf:
    - name: juniper_properties
      index: 1001
      type: varint
      array: true
ipfix:
  mapping:
    - field: 34 # samplingInterval provided within the template
      destination: sampling_rate
      endian: little # special endianness

    - field: 137 # Juniper Properties
      destination: juniper_properties
      penprovided: true # has an enterprise number
      pen: 2636 # Juniper enterprise
netflowv9:
  mapping: []
    # ... similar to above but the enterprise number will not be supported
sflow:
  mapping: # also inside an IPFIX dataFrame
    - layer: "4" # Layer
      offset: 0 # Source port
      length: 16 # 2 bytes
      destination: custom_src_port

Another example if you wish to decode the TTL from the IP:

formatter:
  protobuf: # manual protobuf fields addition
    - name: egress_vrf_id
      index: 40
      type: varint
ipfix:
  mapping:
    - field: 51
      destination: ip_ttl_test
netflowv9:
  mapping:
    - field: 51
      destination: ip_ttl_test
sflow:
  mapping:
    - layer: "ipv4"
      offset: 64
      length: 8
      destination: ip_ttl_test
    - layer: "ipv6"
      offset: 56
      length: 8
      destination: ip_ttl_test