Questions regarding two ISP's in one device

[echo]

Hello!

I have some questions regarding this topics, maybe someone has solved these already? I want to use both ISP's at the same time (not that one is sleeping and inactive when the other one works) so that I can create backup tunnels and manage the firewall on both external addresses, and I configure link-monitor for switchover. Sometimes two ISP's are vlanned (I configure them so), sometimes they are in physically separate ports. Both interfaces, whether virtual or physical, belong to untrust zone which I use in policies. Routing has been done so that two default routes have equal distance but different priority -- I want that for usual traffic out always the primary will be chosen.

1. The test whether the connection works at all.

Well, I can make a port forward or backup tunnel and test it this way, but this is not any quick test. I have found that my usual approach doesn't work and that is that for ping-source I will set primary or backup external address and then ping something out. It works like that only sometimes! I mean, the ping out doesn't work for backup interface, but backup tunnel with headquarter works when I take down the primary tunnel, eg changing IP-address on the other side temporarily or something like that. In cases where pinging out doesn't work, I also can't manage Fortigate on the backup ISP's external address, but in cases where pinging out works, I can manage it. These two behaviours are somehow connected but how? I haven't noticed that it depends on software version or device model.

My tests and comparisons today confused me even more. Because when I pinged out, I watched "diag sniffer packet" output in another window and there it showed that in both cases -- where the ping actually worked and where it didn't -- the primary ISP's interface name was used!

Am I doing something wrong? I see that failovers actually work and tunnels also fail over, I've performed many tests in different environments, but just that initial test that I want to do when the connection is being set up by backup ISP, I don't yet know how exactly.

2. Is it correct that to ping (or tcp-echo or udp-echo) different servers in link-monitor configuration, then to use more than one server, the IP-addresses have to be separated by a _white space_? Or perhaps a comma? I don't want to test this in live systems and unfortunately the cli-reference assumes I know this.

3. If I add a third ISP into the configuration, how should link-monitor be configured then? Or if I want to monitor two other interfaces and use failover between those, independently of ISP interface failovers?

[echo]

For 2., I finally got a chance to test this with a device on my table and the separation symbol is white space indeed. For example,

config system link-monitor     edit "gwdetect-1"         set server "<next-hop-address-by-traceroute>" "8.8.8.8" "<ISP-DNS-server>"     next end

and similarly for the other one. (By the way, I entered them in another order, but Fortigate changed it so can it be that the order is not important?) I also found this from Handbook 5.2, page 1397.

But for the other one, I thought -- why should I have it similarly? Should I have any IP's at all because for example, the recoverytime parameter shows time when to go back to the first ISP. Why should I ping anything at all using backup ISP? I want to get back to the primary asap anyway and this will happen when ping succeeds through primary ISP. Is there any internal logic that may require having this? Or I am doing it just wrong?

Also, where could tcp-echo or udp-echo be useful? Are they usable in the same arp domain only? I tested tcp-echo and I got a switchover to backup ISP after the time that was set with timeout parameter.

Does anybody know how to traceroute out using the backup ISP's interface from the Fortigate device? So far I know only workarounds for this, eg using some internal server behind the Fortigate, or CLI-enabled switch, and create some special NAT policy or policy route for that.

I want to know the details :) I found some information in the FortiOS Handbook 5.2, page 2192, but that was not what I want to achieve.