the double NAT is what is making this a bit complicated. So I would recommend to switch the modem into 'bridge mode' and put all WAN/ISP configuration onto the FGT. Hopefully, your line is not fast (>= 100 Mbps) and using PPPoE - in this case, the modem hardware needs to do the dial in as the FGT CPU will not be strong enough for this.
So, 2 scenarios:
1- modem is bridging
Now the FGT WAN port gets the public IP address (which is good for many reasons, FortiGuard updates for one). Now you create a VIP on the WAN interface, mapping the public IP to the private LAN IP of the server. No port forwarding needed if you do not use the public IP for anything else. Put the VIP as the destination address object into a policy from 'WAN' to 'internal', set service and/or schedule, done.
2- modem does the dial-in
Now you have double NAT. The VIP part on the FGT side stays the same, but now you translate from the private WAN IP of the FGT to the server's address. In order to get the traffic across the modem, you need to configure a 'DMZ' or 'pass-through' on it so that traffic from the WAN will reach the inside of the modem. Depends on make and model.
"Kernel panic: Aiee, killing interrupt handler!"