Technical Tip: Diagnosing DHCP on a FortiGate

[Markus_M]

Description

This article helps to troubleshoot a device that is not receiving an IP address or options, as expected.

Scope

FortiGate is the DHCP client and is connected to a router that provides address over DHCP or FortiGate is the DHCP server.
For this example we just switched server and client, so you can see the same MAC addresses 00:66:65:72:36:03 and 00:66:65:72:27:02 in both the dhcpc (DHCP Client) and dhcps (DHCP Server) output.

Solution

If FortiGate is the DHCP client:

#diag debug reset
diag debug application dhcpc -1
diag debug enable

To stop the debug:

#diag debug reset
diag debug disable

Example and truncated output:

timer 0xc023f10(send_discover -> send_discover) will expire in 8 secs
timer 0xc023f10(send_discover -> send_discover) will expire in 7 secs
timer 0xc023f10(send_discover -> send_discover) will expire in 6 secs
timer 0xc023f10(send_discover -> send_discover) will expire in 5 secs
timer 0xc023f10(send_discover -> send_discover) will expire in 4 secs
timer 0xc023f10(send_discover -> send_discover) will expire in 3 secs
timer 0xc023f10(send_discover -> send_discover) will expire in 2 secs
timer 0xc023f10(send_discover -> send_discover) will expire in 1 secs
timer 0xc023f10 expired, take action
Sending discover!
Send a packet out.
add hw header
set dst hw addr as: FF:FF:FF:FF:FF:FF
src hw addr: 00:66:65:72:27:02
add ip udp header
dhcpcd_send_packet,267:result:590, ifinde:4
unregister timer:0xc023f10
register timer func=0x74f1a0 arg=0xc047350 name=send_discover -> send_discover
Allocate a new timer
Registered timer 0xc023700 will expiry in 22 secs
timer 0xc023700(send_discover -> send_discover) will expire in 22 secs
timer 0xc023700(send_discover -> send_discover) will expire in 21 secs
fd 14 can be read now
###############3Receive packet:
len=353
del hw header
ether_type:0800
hw addr from: 00:66:65:72:36:03
del ip udp header
final dhcp message len:311
DHCP Message received.
parse dhcp options
parse dhcp option buffer (71 bytes)
option[53], len:1
option[54], len:4
option[51], len:4
option[1], len:4
option[2], len:4
option[3], len:4
option[6], len:4
option[58], len:4
option[59], len:4
option[224], len:17
DHO_BROADCAST_ADDRESS option is missed
DHCPOFFER received on port2

If FortiGate is the DHCP server:

#diag debug reset
diag debug application dhcps -1
diag debug enable

To stop the debug:

#diag debug reset
diag debug disable

Example and truncated output:

[warn]Backing up leasefile
[warn]finished dumping all leases
[debug]locate_network prhtype(1) pihtype(1)
[debug]find_lease(): leaving function WITHOUT a lease
[note]DHCPDISCOVER from 00:66:65:72:36:03 via port2(ethernet)
[debug]Start dumping IP address range:
[debug]IP Range from 10.0.0.2 to 10.0.7.58
[debug]IP Range from 10.0.7.60 to 10.0.15.254
[debug]found a new lease of ip 10.0.0.1
[debug]added ip 10.0.0.1 mac 00:66:65:72:36:03 in vd root
[debug]packet length 548
[debug]op = 1  htype = 1  hlen = 6  hops = 0
[debug]xid = eb93af5  secs = 0  flags = 80
[debug]ciaddr = 0.0.0.0
[debug]yiaddr = 0.0.0.0
[debug]siaddr = 0.0.0.0
[debug]giaddr = 0.0.0.0
[debug]chaddr = 00:66:65:72:36:03
[debug]filename = 
[debug]server_name = 
[debug]  host-name = "fermion-kvm54"
[debug]  dhcp-message-type = 1
[debug]  dhcp-parameter-request-list = 1,2,3,6,12,15,28,40,42,26,240,241
[debug]  dhcp-max-message-size = 1458
[debug]  dhcp-class-identifier = "FortiGate-VM64-KVM"
[debug]  dhcp-client-identifier = 1:0:66:65:72:36:3
[debug]
[pkt]000: 01 01 06 00 f5 3a b9 0e  00 00 80 00 00 00 00 00
[pkt]010: 00 00 00 00 00 00 00 00  00 00 00 00 00 66 65 72
[pkt]020: 36 03 00 00 00 00 00 00  00 00 00 00 00 00 00 00
[……]
[pkt]200: 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
[pkt]210: 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
[pkt]220: 00 00 00 00
[debug]Sending ICMP echo-request to 10.0.0.1
[note]DHCPOFFER on 10.0.0.1 to 00:66:65:72:36:03 via port2(ethernet)
[pkt]000: 02 01 06 00 f5 3a b9 0e  00 00 80 00 00 00 00 00
[pkt]010: 0a 00 00 01 00 00 00 00  00 00 00 00 00 66 65 72
[pkt]020: 36 03 00 00 00 00 00 00  00 00 00 00 00 00 00 00
[……]
[pkt]120: 40 3b 04 00 08 13 30 e0  11 46 47 56 4d 30 31 30
[pkt]130: 30 30 30 30 35 32 37 34  33 00 ff
[debug]sending on port2(ethernet)
[debug]sending using lpf_dhcpd_send_packet
[debug]locate_network prhtype(1) pihtype(1)
[debug]find_lease(): packet contains preferred client IP, cip.s_addr is 10.0.0.1
[debug]find_lease(): leaving function with lease set
[debug]find_lease(): the lease's IP is 10.0.0.1
[note]DHCPREQUEST for 10.0.0.1 from 00:66:65:72:36:03 via port2(ethernet)
[debug]deled ip 10.0.0.1 mac 00:66:65:72:36:03 in vd root
[debug]added ip 10.0.0.1 mac 00:66:65:72:36:03 in vd root
[debug]packet length 548
[debug]op = 1  htype = 1  hlen = 6  hops = 0
[debug]xid = eb93af5  secs = 0  flags = 80
[debug]ciaddr = 0.0.0.0
[debug]yiaddr = 0.0.0.0
[debug]siaddr = 0.0.0.0
[debug]giaddr = 0.0.0.0
[debug]chaddr = 00:66:65:72:36:03
[debug]filename = 
[debug]server_name = 
[debug]  host-name = "fermion-kvm54"
[debug]  dhcp-requested-address = 10.0.0.1
[debug]  dhcp-message-type = 3
[debug]  dhcp-server-identifier = 10.0.7.59
[debug]  dhcp-parameter-request-list = 1,2,3,6,12,15,28,40,42,26,240,241
[debug]  dhcp-max-message-size = 1458
[debug]  dhcp-class-identifier = "FortiGate-VM64-KVM"
[debug]  dhcp-client-identifier = 1:0:66:65:72:36:3
[debug]
[pkt]000: 01 01 06 00 f5 3a b9 0e  00 00 80 00 00 00 00 00
[pkt]010: 00 00 00 00 00 00 00 00  00 00 00 00 00 66 65 72
[pkt]020: 36 03 00 00 00 00 00 00  00 00 00 00 00 00 00 00
[……]
[pkt]1f0: 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
[pkt]200: 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
[pkt]210: 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
[pkt]220: 00 00 00 00
[note]DHCPACK on 10.0.0.1 to 00:66:65:72:36:03 via port2(ethernet)
[pkt]000: 02 01 06 00 f5 3a b9 0e  00 00 80 00 00 00 00 00
[pkt]010: 0a 00 00 01 00 00 00 00  00 00 00 00 00 66 65 72
[pkt]020: 36 03 00 00 00 00 00 00  00 00 00 00 00 00 00 00
[pkt]030: 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
[……]

You will see in the output the DHCP packets and most interestingly the typical DHCP flow of packets: DHCPDISCOVER > DHCPOFFER > DHCPREQUEST and finally DHCPACK

Also the FortiGate packet capture utility will be useful.
Since DHCP is widely used, there is a similar behavior on other DHCP capable devices.

This should also show the DHCP packets in the same order. In the following wireshark packet capture example you will recognize important fields:

- Client MAC Address: 00:66:65:72:36:03   <--- This address ist the Clients hardware address.
- Transaction ID: 0x82628920                   <--- This is a static ID and is unique to one DHCP flow.
- Your (client) IP address: 10.0.0.1            <--- This is the result of the client trying to acquire an IP address.
There also are DHCP options, containing information about the Gateway/next hop, DNS, NTP and others.

Client MAC address and Transaction ID must not change in this DHCP session.
If that is the case, there is possibly another DHCP server interfering and answering the requests. It could also happen that the one of the devices is not responding properly.

Bootstrap Protocol (Discover)
Message type: Boot Request (1)
Hardware type: Ethernet (0x01)
Hardware address length: 6
Hops: 0
Transaction ID: 0x82628920
Seconds elapsed: 0
Bootp flags: 0x8000, Broadcast flag (Broadcast)
Client IP address: 0.0.0.0
Your (client) IP address: 0.0.0.0
Next server IP address: 0.0.0.0
Relay agent IP address: 0.0.0.0
Client MAC address: 00:66:65:72:36:03 (00:66:65:72:36:03)
Client hardware address padding: 00000000000000000000
Server host name not given
Boot file name not given
Magic cookie: DHCP
Option: (53) DHCP Message Type (Discover)
Option: (57) Maximum DHCP Message Size
Option: (61) Client identifier
Option: (55) Parameter Request List
Option: (12) Host Name
Option: (60) Vendor class identifier
Option: (255) End
   
Bootstrap Protocol (Offer)
Message type: Boot Reply (2)
Hardware type: Ethernet (0x01)
Hardware address length: 6
Hops: 0
Transaction ID: 0x82628920
Seconds elapsed: 0
Bootp flags: 0x8000, Broadcast flag (Broadcast)
Client IP address: 0.0.0.0
Your (client) IP address: 10.0.0.1
Next server IP address: 0.0.0.0
Relay agent IP address: 0.0.0.0
Client MAC address: 00:66:65:72:36:03 (00:66:65:72:36:03)
Client hardware address padding: 00000000000000000000
Server host name not given
Boot file name not given
Magic cookie: DHCP
Option: (53) DHCP Message Type (Offer)
Option: (54) DHCP Server Identifier
Option: (51) IP Address Lease Time
Option: (1) Subnet Mask
Option: (2) Time Offset
Option: (3) Router
Option: (6) Domain Name Server
Option: (58) Renewal Time Value
Option: (59) Rebinding Time Value
Option: (224) Private
Option: (255) End

Bootstrap Protocol (Request)
Message type: Boot Request (1)
Hardware type: Ethernet (0x01)
Hardware address length: 6
Hops: 0
Transaction ID: 0x82628920
Seconds elapsed: 0
Bootp flags: 0x8000, Broadcast flag (Broadcast)
Client IP address: 0.0.0.0
Your (client) IP address: 0.0.0.0
Next server IP address: 0.0.0.0
Relay agent IP address: 0.0.0.0
Client MAC address: 00:66:65:72:36:03 (00:66:65:72:36:03)
Client hardware address padding: 00000000000000000000
Server host name not given
Boot file name not given
Magic cookie: DHCP
Option: (53) DHCP Message Type (Request)
Option: (57) Maximum DHCP Message Size
Option: (61) Client identifier
Option: (50) Requested IP Address
Option: (54) DHCP Server Identifier
Option: (55) Parameter Request List
Option: (12) Host Name
Option: (60) Vendor class identifier
Option: (255) End

Bootstrap Protocol (ACK)
Message type: Boot Reply (2)
Hardware type: Ethernet (0x01)
Hardware address length: 6
Hops: 0
Transaction ID: 0x82628920
Seconds elapsed: 0
Bootp flags: 0x8000, Broadcast flag (Broadcast)
Client IP address: 0.0.0.0
Your (client) IP address: 10.0.0.1
Next server IP address: 0.0.0.0
Relay agent IP address: 0.0.0.0
Client MAC address: 00:66:65:72:36:03 (00:66:65:72:36:03)
Client hardware address padding: 00000000000000000000
Server host name not given
Boot file name not given
Magic cookie: DHCP
Option: (53) DHCP Message Type (ACK)
Option: (54) DHCP Server Identifier
Option: (51) IP Address Lease Time
Option: (1) Subnet Mask
Option: (2) Time Offset
Option: (3) Router
Option: (6) Domain Name Server
Option: (58) Renewal Time Value
Option: (59) Rebinding Time Value
Option: (224) Private
Option: (255) End

If required, you can use built-in sniffer to perform packet capture, to verify packet flow and examine it in wireshark. You can do it via CLI or via GUI. The easiest way is to filter packets based on port 67 or 68:

#diag sniffer packet <interface_name/any> "port 67 or port 68" 6 0 l

For detailed information about DHCP options, see RFC 2132, DHCP Options and BOOTP Vendor Extensions. Crosschecking the options can be interesting if your clients receive an IP as expected, but gateway information etc. is incorrect.

Be also aware that this can be more complicated if you implemented some kind of Network Access Control, like FortiNAC.

Related Article:

Technical Note : Reserving a DHCP IP address for a particular MAC address (IP/MAC binding)