Importing Firewall Addresses

[seadave]

I'm testing a migration to 5.4.  I have two 500Ds.  One is in production right now running 5.2.3 and is stable.  I decided I want to do a clean config for 5.4 because doing so exposes old settings you no longer need and forces you to rethink how you have rules setup, etc.  It isn't easy, BUT it would be SO much easier if I could export or at least import a CSV list of my over 500 firewall addresses.

I did a select copy from the Firewall Address Screen copied those to Excel.  This makes it very easy to sort and spot anomalies.  I noticed that in 5.4 you have both "FDQN" and "Wildcard FQDN" so you need to make sure addresses are identified as such and have their "Type" converted.  It is really unfortunate that color and tags are no longer in the 5.4 GUI as those were very useful for Addresses.

Anyway being able to build a clean list of addresses and types, then importing as a CSV via the GUi would be awesome.  I may be able to figure out how to transpose my values using a script, but why create all the extra work when it could be built into the firewall and made easy?  I'm sure there is a perl/regex wizard who has done this, but not my specialty unfortunately. 

[Dave_Hall]

This may work for you - save an unenrypted backup of the old config and load that into a text editor, edit/trim the firewall address section, and enter/merge that into the new config (live or via text editer), perform a diagnose debug config-error-log read after merging/loading the changes. 

config firewall address
    edit "update.microsoft.com"
        set associated-interface "port16"
        set type fqdn
        set fqdn "update.microsoft.com"
    next
    edit "download.windowsupdate.com"
        set associated-interface "port16"
        set type fqdn
        set fqdn "download.windowsupdate.com"
    next
    edit "windowsupdate.microsoft.com"
        set associated-interface "port16"
        set type fqdn
        set fqdn "windowsupdate.microsoft.com"
    next
    edit "msftncsi"
        set associated-interface "port16"
        set type fqdn
        set fqdn "www.msftncsi.com"
    next
    edit "download.microsoft.com"
        set associated-interface "port16"
        set type fqdn
        set fqdn "download.microsoft.com"
    next
    edit "wustat.windows.com"
        set associated-interface "port16"
        set type fqdn
        set fqdn "wustat.windows.com"
    next
    edit "ntservicepack.microsoft.com"
        set associated-interface "port16"
        set type fqdn
        set fqdn "ntservicepack.microsoft.com"
    next
end

[ede_pfau]

...never associate an address object with a port...next time, the port is renamed, other hardware or whatever, poof...

And yes, I've ported addresses, adress groups and other mass definitions from the config file into a *.bcmd text file to bulk import it. Runs smoothly.

[seadave]

ede_pfau wrote:

And yes, I've ported addresses, adress groups and other mass definitions from the config file into a *.bcmd text file to bulk import it. Runs smoothly.

Funny was just thanking your for your script.  Can you elaborate on the above?

[ede_pfau]

Well, what I meant was that while porting I grabbed the original data from the old config file (just the part of the "firewall address" block), skimmed through and then bulk imported that into the new config.

Cleaning up is a whole different process, especially in a "grown" config. The XML-like structure makes sorting etc. clumsy in a regular editor but should make it easier if the editor would handle that format. Scripting guy that I am I'd rather not start off with a screen grab but pick the addresses from the config via grep or the like.

There's another task, importing lists of IP addresses, be it blacklisted addresses or TOR network nodes or whatever. They usually come in thousands, and one better use scripting for this. Then, an address group can only hold nnn addresses (model dependent) so the script has to split the addresses up into groups etc.

For example, I had a list of ~ 7.000 addresses to import into a FG-240D (v5.2). Address group size limit is 10.000 - so I read in the Max Values table. The thing would not import more than 4.937 in one group no matter what. So I had to create multiple address groups and one supergroup to hold these. I'm sure Fortinet doesn't know about this yet...

My script really is rudimentary, and for the Windows cmdline. But it's short, and I hope that means it doesn't have many chances to fail.

[seadave]

Thanks.  I've used that method a bunch of times, but the nice thing about Excel is it allows you to really clean things up before exporting back to the native format.  This list has been growing since 2007!  I can search and group on "microsoft.com" for example to see how many duplicates I might have.  User @ede_pfau has a nice script (https://forum.fortinet.com/tm.aspx?m=102532) that should take CSV text output and transpose it to the Fortinet format with the next, set, and end commands added.  I'm planning to test that in the next few days.

You can highlight all of your entries in the FG Addresses Window to copy and then paste into Excel.  For me, having over 500 entries it really helped me spot some mistakes in how another user had entered some FQDNs.

I think one missing field is whether the domain entered is "trusted" or "hostile".  I used to color code my domains, red for bad, green for good.  I also had them tagged.  Along the way those features were removed.  So I'm now considering prefacing all of my entries with TR (trusted) and H (hostile) and then the type of entry it is:

WD - Wildcard Domain

FD - Fully Qualified Domain

SN - Subnet

GEO - Geography

So TR-WD-domain.com would be an example of what the entry NAME would be for a trusted, wildcard, domain.  I'm thinking it not wise to put the * in the description of the name.

That way they should sort logically in the display and allow me to create rules/groups where I'm not mixing WD and SN for example which from experience tends to break rules I think.  One group for each type applied to specific rules keeps things cleaner.

I agree to keep the port to ANY in most cases and it causes problems down the road.