Using AD groups sent by SAML response from IDP to authorize users

Hi, 
So - I have my own IDP i'm testing, and I'm wondering if below scenario is possible with SSL-VPN on Fortigate.

First of all - my IDP is independent from any IAM - so I can connect to AD, Octa, Entra etc. to fetch and authenticate users. While querying for users I also can query which groups they are in. The response IDP sends looks more or less like that:

{
"result": true,
"user": {
"id": 96,
"company_id": 14,
"username": "qweqwe@qweqwe",
"status": "active",
"user_type": 0,
"immutable_id": "NjhiMzI5ZGE5ODkzZTM0MDk5YzdkOGFkNWNiOWM5NDAK",
"created_at": "2024-03-04T15:13:54.102Z",
"updated_at": "2024-03-04T15:13:54.102Z",
"email": "qweqwe@qweqwe",
"additional_details": {
"security_groups": [
"Cert Publishers",
"sf_administrators",
"sf_helpdesk"
]
},
"is_webauthn_active": false,
"webauthn_id": null
}
}

There are three entries in "Security Groups" (highlighted), Now I want the VPN to differentiate between the entries, and use appriopriate policies. For example if group sf_helpdesk is found put users on Network A, and when sf_other that put them in Network B. 

Is that even a thing?