FortiGate - using 100G-BiDi (BX SR) optics to Juniper QFX Switches

[RDaemen]

We have recently enabled 100G links on our FortiGate firewalls towards non-Fortinet devices.

As we require to leverage existing OM3/OM4 LC Duplex cabling, the FN-TRAN-QSFP28-BIDI optic was chosen on the FortiGates. The other end is a Juniper QFX5120-48Y switch with the JNP-QSFP-100G-BXSR optic. Both optics are 100GBase-BXSR (sometimes called 100GBase-SR-BiDi) and thus should be able to communicate with eachother.

However, with system default configurations, the link will not come up. Both ends will report good light levels but no active link. There were no clear indicators why the link remained down even though light was good.

After fiddling around with various settings, with some interesting results, I have found that we need to explicitly disable FEC on the interface on the FortiGate (enabled by default). In the end, it resulted in the following settings for the 100G port on the FortiGate for the link to come up:

config system interface
    edit "portXX"
        set speed 100Gfull
        set mediatype sr4
        set forward-error-correction disable
    next
end

The underlying cause for the link to not come up out-of-the-box is:

• FortiGate by default enables FEC with cl91-rs-fec
• Juniper by default does not enable FEC, unless explicitly configured
• Currently, (as of JunOS 23.4-R2-S5 and FortiOS 7.4.9) no matching FEC capabilities appear to exist between Juniper and FortiGate

So with this combination, it is required to explicitly disable the FEC on the FortiGate to get the link up-and-running.

For reference, the Fortigate supports the following FEC options:

• disable
• cl91-rs-fec - Reed-Solomon (FEC CL91). (This is the System Default)
• cl74-fc-fec - Fire-Code (FEC CL74).

Juniper supports the following FEC options:

• unconfigured - No FEC config disables the feature.(This is the System Default)
• hgfec - High Gain Forward Error Correction
• sdfec - Sky-Compatible Soft Decision Forward Error Correction
• sdfec15 - Soft Decision Forward Error Correction with 15 percent overhead
• sdfec25 - Denali-Compatible Soft Decision Forward Error Correction with 25 percent overhead

[HarryTran]

Thanks for sharing this finding.
Yes, there is no FEC compatibility between FortiGate and Juniper devices in this case. FortiGate uses the standard IEEE 802.3 FEC, where Clause 91 Reed-Solomon (cl91-rs-fec) is the most common one for both 25G and 100G Ethernet. That’s why FortiGate enables it by default for 100G ports.
On Juniper side, they don’t enable FEC by default and their supported types (like hgfec, sdfec15, etc.) are from a different family — mostly used for coherent or long-reach optics. Those are not compatible with Ethernet’s RS-FEC type.
So when both sides have different FEC types, the link cannot come up. Disabling FEC on FortiGate side is the only way to make it work in this mixed setup.