This document describes the integration process of the ThreatSTOP IP Defense on Juniper SRX MX EX Devices via Web Automation

Overview

This document will provide instructions on integrating ThreatSTOP IP Defense with web automation onto your Juniper SRX, EX, MX line device.

Current Version

Current Version Released
5.01-01 04-01-2020

Install Methods

On Device Install via Web Automation

Installing On-Device with Web Automation, which this document covers, allows you to easily configure settings on our Portal web interface. After configuring the settings on our portal, you will run a command on the device to pull down the configuration. From that point forward your device will sync its configuration updates made on the portal. This document will cover each setting in more detail below.

On Device CLI Install

Installing On-Device via CLI, which this document does not cover, will have you run through a series of prompts asking for setting particular to your install. Please see our Juniper devices (On Device CLI) guide for more details.

Quick settings

Quick settings are provided below for expert installers who have already read through the documentation and understand what they are doing.

# Below are a set of copy & pastable commands to get your system up and running.
root@JUNOS% mkdir -p /var/home && cd /var/home && fetch -p ftp://ftp.threatstop.com/pub/ts-juniper.tar.gz && tar zxvf ts-juniper.tar.gz && cd threatstop

% /bin/sh tsadmin add --type=auto --device_id [Device ID]  --auto_key=[Device Key] **WARNING - THIS IS NOT A VALID COMPLETE COMMAND**

Install Parameters

Please make note of the following settings while going through the installation prompts or setting up options for the semi-automated install.

Setting Value
ThreatSTOP Device ID (TDID) Retrieved from the device settings page
Device Auto Key Retrieved from the device settings page

Compatibility

We have tested compatibility with the following:

Juniper Line Version IP Defense Supported Web Automation
SRX 12.3 Yes Yes †
EX 12.3 Yes Yes †
MX 15.1 Yes No

† Requires the dig network utility, usually found on latest firmware builds. Please ensure you are on the latest recommended firmware for your hardware platform.

Please see the manufacturers recommendation for your specific hardware version.

Installation Overview

This document will go over installation and integration of ThreatSTOP via web automation directly on to your Juniper device. The basic steps are as follows:

  1. Add & Configure device in Admin Portal
  2. Download software
  3. Configure ThreatSTOP service on device
  4. Test configuration / logging.

Upgrade Overview

If you are able to uninstall the previous integration first, that is the preferred method. However for those who need to upgrade in place please follow the instructions below.

In the steps below we’ll pretent we installed into /var/home/threatstop

  1. Backup the current installation directory. tar czvf /tmp/previous_install.tgz /var/home/threatstop.
  2. Untar the contents of the new tar file directly into the installation directory. tar xzvf threatstop-junos_5.01-10.tar.gz -C /var/home which will expand the threatstop folder and contents overwriting currently installed files.
  3. Run /bin/sh ts-upgrade to update the configuration file to work with the latest build.
  4. Run /bin/sh tsadmin show and verify you see your devices settings (device id, block list, port, etc..).
  5. Run /bin/sh tsadmin update to test updating the policy to ensure this completes without error.

Prerequisites

  • Juniper running supported firmware.
  • Juniper added & configured via the admin portal.
  • SSH access to the Juniper.
  • Access to root account.
  • Current active ThreatSTOP account.
  • For web automation you will need to have your time synced

Although not required, we highly recommend you create a backup configuration before installing our service.

ThreatSTOP Communication Overview

Connectivity

To retrieve its configuration and policy, and to upload log data, the device needs the following connectivity:

  • DNS over TCP - Policy service
    • Hostname: ts-dns.threatstop.com
    • IP Range: 192.124.129.0/24
    • Outbound TCP port 53 or 5353
  • DNS over TLS - Configuration service
    • Hostname: ts-ctp.threatstop.com
    • IP Range: 204.68.97.208/28
    • Outbound TCP port 5353
  • FTP - Log service
    • Hostname: logs.threatstop.com
    • IP range: 204.68.99.208/28
    • Outbound TCP port 21 / Inbound TCP port 20 (Active FTP) / Negotiated Outbound (passive FTP).
    • Direct Connection
  • NTP
    • Hostname: time.nist.gov
    • Outbound UDP port 123

Setup

Step 1: Portal device configuration

During this step, you will create a device entry on the Admin Portal. You will select a device type and enter the configuration settings. A minimum configuration only requires a handful of settings but optional, advanced settings are also available.

To create a device entry:

  • Log into the Admin Portal with your ThreatSTOP account
  • Browse to the Device page and click Add Device
  • Select the model:
    • Type: IP Defense
    • Manufacturer: Juniper
    • Model: SRX EX MX
    • Integration Type: On device with Web automation

The Admin Portal will display a form to enter the device settings described below.

  • Nickname: this is a mnemonic name used to identify the device. It can be set to any string (A-Z, 0-9, - and _). If you create multiple device entries, each entry must have a unique nickname. The Nickname will be used to identify the device on the device and in the Reporting user interface.

  • Policy: select a pre-defined policy or a customized policy. It must be an IP Defense Policy.

  • IP Type: Access to the ThreatSTOP services is controlled in part using an ACL allowing the device IP to connect. If your device has a static public IP address (the most common case), select static. If your device has a dynamic public IP address, the ThreatSTOP services can lookup the IP address using a DNS fully-qualified name (FQDN).

  • Public IP address: In static mode, this is the public IP address of the device. It is possible to configure multiple device entries with the same public IP address.

  • Note: An optional field to store a note of your choice about the device - location, identifiers, model…

  • Maximum Policy Size: Option to limit the number of entries in the policy. If the policy becomes larger than this setting, the device will truncate it down to the Maximum Policy Size.
  • Number of Prefix Lists: As of ts-juniper v5.01+ you will be prompted to specify how many prefix lists to use for blocking purposes. This setting drives generation and distribution of targets into multiple prefix lists to be assigned to the same ‘ThreatSTOP’ group. The main benefit here is support larger policies.

Advanced Settings

  • Number of addresses per Prefix List: You will be prompted to specify how many addresses each prefix list can contain before it spills over into a subsequent prefix list. This option is only applicable when multiple block prefix-lists are used. The default of 85,000 as mentioned here but your device’s specific memory concerns may drive the need to adjust this setting. We do not recommend going over 85,000 unless you are sure your device can support this. For example, if this setting is set to 20k per prefix list & a max policy size of 80k is set but the policy contains 100k+ items. The policy will first be truncated at 80k, then distributed into four 20k prefix-lists.

  • FTP Mode: This setting allows you to set passive vs. active (default) ftp.

  • DNS Port: The device uses TCP Port 53 (outbound connections) to retrieve policy data. If this port is blocked or filtered (for example, networks using a DNS Application Layer Gateway), use this setting to switch to TCP Port 5353.

Step 2. Download Software

# Step 1. ssh into the juniper
# Step 2. make sure /var/home directory exists & cd into it.
mkdir -p /var/home && cd /var/home
fetch -p ftp://ftp.threatstop.com/pub/ts-juniper.tar.gz
# Step 3 now we run the download through available on-device MD5 to verify data integrety with the hash provided below.
cat ts-juniper.tar.gz | md5
# Step 4. tar extract the threatstop directory into /var/home/threatstop
tar zxvf ./ts-juniper.tar.gz
# You should now have a threatstop directory in /var/home. This directory should be kept in place.
filename md5 hash
ts-juniper.tar.gz f1f7a86f96466d7103ae22dd071f67ef

Step 3. Configuration

  1. SSH into the Juniper device with an admin account, switch users to root by running su -. You can verify you are root by running whoami.

  2. To begin configuring the device, run the following command:

    % /bin/sh tsadmin add --type=auto --device_id=[Device ID] --auto_key=[Device Key] **WARNING - THIS IS NOT A VALID COMPLETE COMMAND**

  3. The software will retreive settings from our servers.

Step 4. Applying the ThreatSTOP Configuration

Since every firewall configuration is different, ThreatSTOP does not automatically apply the filter it has created. Normally the filter is applied on the input side to all interfaces that receive traffic from the Internet or may send traffic to the Internet. If you have grouped interfaces in a range it is simpler to apply the filter to the entire range at once. In the example to the right there is one external interface (ge-0/0/0.0)

To apply the configuration:

  • SSH into your device.
  • Enter CLI mode.
  • Enter Configuration mode.
  • Set the system’s syslog policy to apply the group named ThreatSTOP set system syslog apply-groups ThreatSTOP.
  • Set the system’s policy options to apply the group named ThreatSTOP set policy options apply-groups ThreatSTOP.
  • Set the system’s firewall policy to apply the group named ThreatSTOP set firewall apply-groups ThreatSTOP.
  • commit the changes.
cli
configure
set system syslog apply-groups ThreatSTOP
set policy-options apply-groups ThreatSTOP
set firewall apply-groups ThreatSTOP
commit

Step 5. Applying the Filter to the Interface

The last step is to enable the filter on your interface.

  • Set the interface and unit to use the inet family and filter the input using the ThreatSTOP group. For example: set interfaces ge-0/0/2 unit 0 family inet filter input ThreatSTOP
  • commit the change.
set interfaces ge-0/0/2 unit 0 family inet filter input ThreatSTOP
commit

If you are using multiple prefix list support and are having trouble applying a large policy, you may have to adjust your max policy size to suit your devices needs.

root@mx104# commit
[edit groups ThreatSTOP firewall]
  'filter ThreatSTOP'
    Maximum term size reached building tlvs for filter 'ThreatSTOP'
commit-check failed
  commit-check failed
error: configuration check-out failed

In this case you can simply edit threatstop.conf and adjust your maxpolicysize setting down and re-run /bin/sh tsadmin update which will again attempt to load a fresh policy.

Step 6. Testing Configuration

To ensure your device is properly blocking and logging traffic, test traffic to a known blocked host from your policy.

root@juniper% cli
# replace tdid_XXXXX with your actual ThreatSTOP Device ID
root@juniper> show log tdid_XXXXX.log
# First see if bad.threatstop.com is in your ThreatSTOP-block prefix-list
root@juniper> show configuration | display set | match ThreatSTOP-block | match 64.87.3.133
# If it is not listed you can test with another /32 in the policy
root@juniper> show configuration | display set | match ThreatSTOP-block | match /32
# now we will attempt to curl 64.87.3.133
root@juniper> exit
root@juniper% tail /var/log/tdid_XXXXX.log
Mar 18 23:23:16 SRX300-00 newsyslog[65212]: logfile turned over due to -R from eventd
# Now ping the address from a device on the inside / trusted network segment
root@juniper% ping 64.87.3.133
root@juniper% tail -f /var/log/tdid_XXXXX.log
<158>1 2019-03-19T00:00:25.791Z SRX000-00 - - - - PFE_FW_SYSLOG_IP: FW: ge-0/0/0.0   D icmp 64.87.3.133 10.0.0.77     0     0 (1 packets)
<158>1 2019-03-19T00:00:28.787Z SRX000-00 - - - - last message repeated 3 times

If you see the log lines indicating a successful block, you’re done.

Congratulations, You’ve just protected your network with a world class security threat feed!

Be sure to read our Portal Guide to get the most out of our advanced reporting features.

High Availability / HA / Cluster installs

In a high availability (HA) cluster environment, the ThreatSTOP software should be installed on all cluster nodes. The basic premise is we install the software on all nodes in the same location & ensure the scheduled tasks exist. This way if any secondary nodes are promoted to act as the primary, they immediately kick in producing updates.

Secondary Node Installation

You should complete & verify the standard installation on the primary node before attempting installation on any secondary nodes. Due to how SRX clusters work, the secondary nodes may not be able to access the Internet. In this scenario follow the steps below to ensure your secondary node is properly configured to act as a primary.

  1. Copy the threatstop directory and contents from primary node to secondary nodes. In the example below we use tar to copy over SSH.
  2. Ensure the scheduled tasks are setup by running /bin/sh /var/home/threatstop/install_cron.sh
  3. Test the installation went well on secondary node(s)
 # In this example 10.0.0.1 will represent the primary node
 # In this example 10.0.0.2 will represent a secondary node
ssh [email protected]
root@primary% cd /var/home
root@primary% tar -cvf - threatstop | ssh [email protected] "cd /var/home && tar -xvf -"
root@primary% ssh [email protected] "/bin/sh /var/home/threatstop/install_cron.sh"
[20:06:30] Adding update of policy address lists to cron
[20:06:31] Ensuring block/allow updates will happen after a reboot
# To test the install was successful you should see at least ts-update listed, possibly ts-update-settings if using web automation
root@primary% ssh [email protected] "crontab -l"
# ThreatSTOP cron
09 */2 * * * /bin/sh /cf/var/home/ts-update 2>/dev/null 2>&1

All other firewall setup should have been copied over as part of the ongoing clustering configuration syncing between nodes.

Using Passive vs Active FTP

Active FTP mode is used by default, to setup the logging using passive FTP mode supply the --advanced flag to the tsadmin command during installation to get some additional prompts normally skipped.

Other Operations

Upgrading

If you are upgrading from an older version of the Juniper software, you must uninstall the previous software completely before installing the latest version.

How to check your version

% /bin/sh tsadmin version
5.01

Command Manual

tsadmin has several sub commands to perform various functions outlined below. Sub commands are interpreted differently by the tsadmin & do not have a -- before them.

  • add The add command install & configures the ThreatSTOP service on the device
    • -a the allow list name including the Account ID
    • -b the block list name including the Account ID
    • -t this is the ThreatSTOP Device ID
    • -f filter This will always need to be set to filter
    • -n number of block prefix lists to use †
    • -c number of addresses (capacity) each prefix lists uses when multiple prefix-lists are used. The max policy size will always take precident.
    • -m This should be set to match the Maximum Policy Size supported by your device
    • –device_id this is the ThreatSTOP Device ID (usually beginning with 'tdid_')
    • –auto_key the secret key used for auto configuration
    • –advanced This optional flag enables special prompts to customize port and ftp modes. (Not compatible with web automation installation)
    • –debug This optional flag enables debugging information
  • remove This command removes the ThreatSTOP integration from the device
  • update This command forces the device to update the policy as well as check for new settings if web automation installation is used
  • show This command shows the configured settings
  • help This command shows usage information
  • version This command shows the current version of installation package

† - Your device must support this functionality. Please check your device / firmware documentation or contact the manufacturer to verify this feature is available to you.

Troubleshooting

To get a list of configured settings run:

root@junos% cd /var/home/threatstop
root@junos% /bin/sh tsadmin show

To get a full list of available command line parameters on the command line type:

root@junos% cd /var/home/threatstop
root@junos% /bin/sh tsadmin help

If you are not able to download the policy, run the following:

admin@device# wget -qO - http://logs.threatstop.com/cgi-bin/validip.pl
Your IP address: 1.2.3.4
Address is in the list of authorized hosts

This will tell you if your public IP has been allowed to access the policy.

Not all traffic is being blocked / everything is being blocked:

The most likely reason is that you have not correctly applied the ThreatSTOP filter (see “Applying the Filter” above). If you need help with verifying the filter configuration please contact ThreatSTOP support.

Other firewall rules do not appear to work:

The most common reason for this is the same as 2) above, namely that the ThreatSTOP filter has been incorrectly set up. If you had additional filters then you may have replaced them with the ThreatSTOP filter if you apply it using the “… filter input ThreatSTOP” command. As noted above, if you already have one of more filter on the interface you should use the ‘input-list’ alternative and put the ThreatSTOP filter before your other filters.

No Logs Uploaded:

  • verify the logging was properly setup:
# replace tdid_XXXXX with your own Device ID
root@JUNOS> show configuration | display set| match syslog
set groups ThreatSTOP system syslog file tdid_XXXXX.log firewall any
set groups ThreatSTOP system syslog file tdid_XXXXX.log archive size 100k
set groups ThreatSTOP system syslog file tdid_XXXXX.log archive transfer-interval 60
set groups ThreatSTOP system syslog file tdid_XXXXX.log archive archive-sites "ftp://[email protected]/logs"

You should also confirm that the file /var/log/tdid_XXXXX.log exists.

The logs are transferred to ThreatSTOP using FTP. If you require passive ftp, rerun the setup with the --advanced flag to see some additional options. You will be able to set the ftp to passive vs. active (default) FTP mode.

  • ALG and SRX devices To allow the device to FTP the logs to us on SRX devices, the FTP Application Layer Gateway (ALG) may or may not need to be enabled. By default, the FTP ALG is enabled. To see the status of the FTP ALG, run: 
    show security alg

If it is not listed, then it is disabled. If it is listed, you can disable it by running:

delete security alg ftp
commit

Additional Information

Version Changelog

5.01   Wed Feb 26 2020
  - Added Multiple block prefix-list support
5.00   Wed Mar 16 2019
  - Added Web Automation installation capability
4.02   Fri Apr 29 2016
  - refactored to unified software build for EX, SRX, MX lines

Uninstalling ThreatSTOP

Remove ThreatSTOP service integration from device

ThreatSTOP can be removed in a non-destructive manner by simply removing the ThreatSTOP Group, and uninstalling the integration software. First, delete the configuration steps you steps performed in “Applying the ThreatSTOP Configuration”, generalized steps included below:

  1. SSH into the device.
  2. Enter CLI mode by entering cli.
  3. Enter Configuration mode by entering configure.
  4. Delete the ThreatSTOP filter from the interface.
  5. Delete interfaces ge-0/0/2 unit 0 family inet filter input ThreatSTOP
  6. Delete the ThreatSTOP syslog entry.
  7. Delete system syslog apply-groups ThreatSTOP
  8. Delete the association between the policy and the ThreatSTOP group.
  9. Delete policy-options apply-groups ThreatSTOP
  10. Delete the firewall configuration’s reference to the ThreatSTOP group.
  11. Delete firewall apply-groups ThreatSTOP
  12. commit the changes.

This will remove the policy enforcement from the network device.

cli
configure
delete interfaces ge-0/0/2 unit 0 family inet filter input ThreatSTOP
delete system syslog apply-groups ThreatSTOP
delete policy-options apply-groups ThreatSTOP
delete firewall apply-groups ThreatSTOP
commit

Next, we run the shortcut /bin/sh tsadmin remove from within the installation directory. This will remove the ThreatSTOP group and scheduled configuration from the device.

root@junos% cd /var/home/threatstop
root@junos% /bin/sh tsadmin remove

Time Sync

We recommend you keep your devices time synced if you intend to use our web automation services. It is required to provide a secure channel of communication to the device.

To sync the clock on your device select an NTP server closest to you and run:

# ntpdate <server>
ntpdate time.nist.gov

If NTP is not an option, you can manually set the date/time by following these directions from Juniper.

Enter operational mode in the CLI. Enter the following command substituting the current time:

# user@Juniper> set date YYYYMMDDHHMM.ss
# For example, the following command sets the date and time to 2019-03-26 10:57:55
user@Juniper> set date 201903261057.55

Backuping up configuration

Starting with V5.0 our software automatically creates a backup of the configuration called ‘pre-threatstop’ on first run. We still recommend you create a backup of your Juniper device’s current configuration in case of any device/version specific failures. Please reference your manufacturers guide for up-to-date information as it may be newer than ours:

  • SSH into the device (if you are not currently SSH’ed in).
  • Enter CLI mode
  • Enter Configuration mode
  • Save the current configuration with an explanatory name, e.g. juniper_x100_config_2020-01-30
  • Exit Configuration and CLI modes by entering exit twice.
root@junos% cli
root@junos> configure
root@junos# save backup_Config_2019-12-31
root@junos# exit
root@junos> exit

We recommend you transfer this backup configuration off device for safe keeping.

Restore previous configuration

We create a backup of the configuration prior to installing our service located /opt/threatstop/pre-threatstop. To restore the configuration you run the following commands:

root@junos% cli
root@junos> configure
root@junos# load override /opt/threatstop/pre-threatstop
root@junos# commit
root@junos# exit
root@junos> exit

If you specifically want to revert back to the configuration just before installing the threatstop service:

root@junos% cd /var/home/threatstop
# first make sure you've uninstalled the service
root@junos% /bin/sh tsadmin remove
# now to revert to the configuration backup the software created just before installing the service run
root@junos% /bin/sh tsadmin revert