Dial-up vpn tunnel fluctuates between multiple clients

The scenario I am trying to configure is pretty simple. I have FGT300E (v6.2.1) as a vpn headend and multiple other devices which would act as remote sites. Since the remote sites could have a NATtd WAN ip, I believe the dial-up vpn config on my headend is the best way to do this. This way I also do not need to make any changes on my headend as new remote sites join. However what I have seen is that when I have multiple clients connecting, the tunnel goes down on the previous one before connecting to the new one. I tried using 'set add-route disable' which bring up all tunnels, but from the headend side I can only reach the 1st remote site. Traceroute shows that all packets to the remote sites get routed to the initial remote tunnels virtual interface (instead of each sites virtual interface). I have unique peer-ids, keepalives, auto negotiates etc enabled too.