System configuration export in YAML seems invalid

[lst3010]

Hello,

I tried to export the system configuration file from my FortiGate with FortiOS v7.2.7 build1577, today the first time in YAML. When exporting using the WebUI I chose: Scope: Global, Backup to: LocalPC, File format: YAML, Password mask: enabled, Encryption: disabled.

When I then tried to load the yaml file with the PyYAML python library v6.0… without the expected success:

import yaml

with open("[…].conf.yaml", encoding='utf-8') as fd:
    yconf = yaml.safe_load(fd)

  File "[…]\Python311\site-packages\yaml\parser.py", line 438, in parse_block_mapping_key
    raise ParserError("while parsing a block mapping", self.marks[-1],
yaml.parser.ParserError: while parsing a block mapping
  in "[…].conf.yaml", line 2407, column 9
expected <block end>, but found '<scalar>'
  in "[…].conf.yaml", line 2408, column 50

When examining with a text-editor, I found the following at the guilty part of the YAML-file (starting at line 2405):

firewall_addrgrp:
    - host-grp_group-name1:
        uuid: 01234567-89ab-cdef-0123-456789abcdef
        member: "host_name1" "host_name2" "host_name3" "host_name4" "net_name1" "net_name2" "net_name3"

So it appears to me that PyYAML is right in it's complaint and the YAML-export takes some liberties in it's interpretation of the yaml specification… there are several different (and many) places in the config file, where internal lists aren't exported as lists in YAML but in space-separated sequences of strings i.e. scalars in YAML-terminology.

Is there a difference depending on the options chosen or the way the file is extracted from the FortiGate? So can I circumvent this issue or am I stuck with some other parsing strategy?

Best regards

Edit: corrected firmware version number

[hbac]

Hi @lst3010,

It seems to match a bug ID 0976722. It is still being investigated.

Regards, 

[trevorj]

Does this bug still exist in firmware versions above 7.2.7?

[lst3010]

Our FortiGate is now running on FortiOS v7.4.5 build2702 (Mature).

The problem stadly still persist…

>>> with open(fn) as fd:
...     yconf = yaml.safe_load(fd)
...     
... 
Traceback (most recent call last):
  File "<console>", line 2, in <module>
[…snipped a lot…]
  File "c:\program files\python312\Lib\site-packages\yaml\parser.py", line 438, in parse_block_mapping_key
    raise ParserError("while parsing a block mapping", self.marks[-1], yaml.parser.ParserError: while parsing a block mapping
  in "HOSTNAME_7-4_2702_202412100955.conf.yaml", line 1820, column 13
expected <block end>, but found '<scalar>'
  in "HOSTNAME_7-4_2702_202412100955.conf.yaml", line 1822, column 25

When searching for the following regex in the config file, there are a many many hits…

"[^"]+" "[^"]+"

 For example

    system_ha:
        […snipped a lot…]
        priority: 200
        monitor: "port1" "port2" "port17" "wan1"
        ipsec-phase2-proposal: aes256gcm

or

        system_zone:
            - ZONENAME:
                description: "Zone description"
                interface: "INTARFACE-NAME" "VLAN-NAME-1" "VLAN-NAME-2" "VLAN-NAME-3" "VLAN-NAME-4" "VLAN-NAME-5"

or

        firewall_addrgrp:
            - net-group-name:
                uuid: 01234567-89ab-cdef-0123-456789abcdef
                member: "SUBNET-NAME-1" "SUBNET-NAME-2"

or

        firewall_policy:
            - 261:
                uuid: 00112233-4455-6677-8899-aabbccddeeff
                srcintf: "INTERFACE-NAME-1"
                dstintf: "INTERFACE-NAME-2"
                action: accept
                srcaddr: "ADDRESS-RANGE-NAME-1" "ADDRESS-RANGE-NAME-1"
                dstaddr: "NET-NAME-1" "NET-NAME-2"
                schedule: "always"
                service: "RDP" "HTTP" "HTTPS"
                logtraffic: all
                comments: "some comment"

Or was the question regarding the fix specifically for the 7.2.x release branch?

Best regards

[trevorj]

That answers my question. Thank you for posting your findings.

[capn]

Do you know of any update on this, and is it actively being worked on?

And have anyone found a workaround apart from search and replace?

[trevorj]

Have not heard anything and put it on the backlog for now but had some initial thoughts that custom PyYAML tags for the offending data structures would work. Would need to determine the offending config data structures and create custom tag classes for them as well as serialization and deserialization.

The issue seems to be with list data structures as lst3010 mentioned. Fortinet is representing them as:

vdom: "root" "vdom-a" "vdom-b" "vdom-c"

Where they should be represented as:

vdom:
  - root
  - vdom-a
  - vdom-b
  - vdom-c

or

vdom: [root, vdom-a, vdom-b, vdom-c]

There are other data elements besides "vdom" that are doing this as in @lst3010 examples above showing the same for "interface", "srcaddr", "dstaddr" etc. A custom tag for each and would only have to do it once but ideally FortiNet would fix the data structures so that they conform with the YAML standards and parse properly.