strange behaviour of dhcp relay

[sw2090]

I just ran into this:

We have a Windows DHCP that has a scope for a vlan.

the vlan interface on the FGT100E is set to do dhcp relaying to this Windows DHCP.

the Windows DHCP also has dhcp option 138 set for all scopes it has.

If I now connect a client to this vlan interface and set it to request an ip via dhcp it does get an ip from the Windows DHCP. Also the correct scope is used. Networking works fine so far.

However the DHCP ACK is missing the option 138 completely.

If I do packet sniffing on the FGTs interface where the DHCP Server is connected to I do see the forwared request from my client and I do see the answers including the ACK. This ACK does contain option 138 but the value in the option is not what is configured on the DHCP Server. And as said the client doesn't receive option 138 at all.

Did anyone exprience this or similar and has some hint for me why that happens?

[sw2090]

In Addition: if I configure the vlan interface to act as dhcp server itself, set up a scope and add option 138 everything works fine. Client also gets IP and it also gets option 138.

[CatInHat]

This may occur due to differences in DHCP server handling on Windows and FortiGate. The Windows DHCP server may be changing or ignoring option 138 when forwarding requests through the FortiGate. You need to check the DHCP configuration on both platforms to ensure that the option is correctly configured and transmitted

[AEK]

What's contained in option 138 returned by FG? Is it the IP of FG itself?

Just thinking if this may happen when FG is already a WiFi controller for some FAPs.

[Genobaseball10]

I'd be interested in seeing what value is in the option 138 that you see in the pcap file verses the value that you have configured in the DHCP server. The article I'm linking below is old but it may help out a bit.
https://community.fortinet.com/t5/FortiAP/Technical-Tip-Manage-FortiAP-over-L3-network-with-Windows-...

[sw2090]

in the DHCP ACK returned from the FGT to the client there is no option 138 at all.

In the pcap from the interface the DHCP Server is in on FGT there is option 138 but with more ips that look similar to what FortiNet uses as system DNS  since FOS 7.0. On the DHCP Server there is only one ip in option 138 which is not in option 138 in the pcap file.

Also FGT is no wireless controller and we do not have any FAPs. 

[sw2090 · Accepted solution]

We found the culprit now:

The Windows DHCP Server did have the option 138 configured. It also did have the correct address in there. However the data format was wrong. So Option 138 was submitted as text and not ip address. This resulted in 12 bytes of hex payload in option 138 arriving on the client. That being interpreted as hex results in those three ips we saw.

Reconfiguring option 138 on the Windows dhcp server to be of type ip address with  the correct ip made it work correctly.