Errant ARP Replies from interface with no IP.

[JaysPoorChoices]

Hi all, I'm trying to troubleshoot an odd issue and am worried that I'm missing something/out to lunch. This is a much shortened version of the issue based on this figure http://www.asciidraw.com/#Draw3254957678224072795 with the VxLan turned down for testing. 

Config Notes:

[ul]

port3 is only referenced in sys interface and the software switch.

SoftSwitch1 has l2forward and intra-switch-policy implicit.

VxLan implementation only differs from the cookbook for 5.6.6's native VxLan tunneling, so the encap destination is ike.

Tested with and without l2forward.[/ul]

The Fortigate is seeing ARP requests for 10.0.0.1 at port1 and port3 as expected. Unfortunately it's also sending replies from both ports. So clients see two replies: one with the mac for the interface (port1) that actually has that IP, and one with the mac for port3 (which blackholes any subsequent ip packets). That IP is the default gateway so it's basically blackhole-roulette depending on which reply is seen first and how fast the client caches update.

There are no other issues, VxLan tunnel appears to function perfectly when enabled. Can't reproduce the errant ARP with two Hardware Switches or with a Hardware Switch and a single port interface. 

I'd normally let TAC do their thing but I have a deadline coming soon and their initial response is that it appears to be working as expected and that they're looking for a way to disable ARP replies per-interface. That... doesn't seem right. Is there something I'm missing or don't understand? An interface should not respond to ARP requests for an IP to which it isn't bound or proxy-ing in some way, right?

The reason for the hardware switch (and not just changing to a software switch that includes the VxLan IPsec interface) is partly legacy, partly because it's in production and the backup 100D's are offsite at the moment, partly because there are a lot of references to that interface so I'll need to edit the config and apply it offline during a maintenance window. I do plan on testing that as soon as I can turn prod into dev for an hour. In the meantime I would sincerely appreciate any thoughts, suggestions, or corrections.

[gangadar1234]

can you post your config here..

[JaysPoorChoices]

gangdar1234: Sorry I missed your post, I guess I misconfigured my notification settings.

Issue one was our bad. We had a leftover undocumented VIP whose interface was set to 'any'. The 4.0 legacy docs explicitly say that the Fortigate will proxy arp requests for a VIP on any interface to which it is assigned. That might be obvious after you think about it but we didn't at the time. TAC labbed it out and got back to me the next day.   Issue two was a little more insidious. After removing the VIP we had the opposite problem: The softswitch started eating ARP requests for 10.0.0.1. I ended up posting a last-minute maintenance window where I edited the softswitch out and converted the hardware switch into a software switch that included the VxLan interface. Still don't know why that worked but it did. Seems to be running well but the feedback I got was that the VxLan feature is not commonly used.   The config matched the cookbook except that we had l2forward enabled on the involved interfaces and the physical port was plugged back into the hard-switch.