MacOS Big Sur Fortclient VPN IPSec issues

Just installed macOS Big Sur and cannot get a connection with Fortinet firewall VPN anymore, while it did work with macOS Catalina.

I tried Forticlient version 6.4 which seems to connect just fine (I get an IP in expected range), but ssh/ping does not work. And also I cannot access a intranet http/https page.

Also tried multiple versions of 6.0.x but they all fail to connect and show "Connection was terminated unexpectedly. Error -104". After that, the keyboard (Macbook 16 inch) even fails to register any pressed buttons. For example I open app "notes" and cannot type anything (with every keystroke a sound is played but nothing is written). The only way to get out of this situation is to click "connect to VPN" in forticlient again and before it gets to the error click "disconnect". Then all works as usual (except the VPN obviously).

Anyone else having these issues?

Update Nov 25th 2020: 

Ok, so after quite a bit of testing by the people who maintain our firewall, we managed to make IPSec VPN work with native Mac OS Big Sur VPN client. I am always amazed by the lack of Fortinet response in this type of issues, as the solution seems pretty simple in the end. Eventually the configuration at fortigate firewall stayed exactly as it was, the only configuration I needed to add locally (with respect to using the FortiClient software) is to add a group name under "Authentication Settings". So to make it work we: