Industrial Control Systems
Purpose
Help to better understand ICS networks and ideas on protecting them from cyber attacks.
Discuss common weakness and vulnerablities along with cyber risks for ICS.
Importance of knowing the networks that needs to be protected
Discuss mitigation strategies and defense in depth for more secure ICS environment
Definitions
IT
IT refers to anything related to computing technology.
OT
OT refers to hardware and software used to monitor events, processes, and devics and make adjustments in industrial operations.
Operational Technology (OT) refers to systems used to monitor and control industrial operations.
ICS
Industrial Control Systems (ICS) includes systems used to monitor and control industrial processes.
ICS refers to a broad set of control systems including
SCADA (Supervisory Control and Data Acquistion): geographically spread out (pipelines, electric substations)
DCS (Distributed Control System): might be located at one location only such as Nuclear power station with reactor basement (ground and first floor), cooling tower and field controller (communicating with IO and sending data to control room) (Distributed control)
PCS (Process Control System)
EMS (Energy Management System)
AS (Automation System)
SIS (Safety Instrumented System): Separate system from a DCS created specifically for safety purposes. For example as long as the variables (temperature, pressure and other important variables) are within specfied all is good. If not, SIS will shutdown the systems.
Any other automated control system.
Embedded Systems
Embedded Systems is computer system consisting of hardware and software specifically defined for a specific purpose or dedicated task. (Workstations, laptops and servers are not embedded system).
Embedded system used in ICS
Programmable Logic Controller, Remote Terminal Unit, DCS controllers, Intelligent Electronic Devices, field devics (HART, Foundation Fieldbus, Profibus, Devicenet)
Network/communication equipment (Routers, switches, modems, radios, terminal servers, gateways, firewall and other security appliances)
Others (GPS, time synchronziation, network printers, hand-held configuration devices, test equipment)
Field Controllers
Processors (X86, PowerPC, ARM, MIPS)
Memory
Non-volatile Memory
Flash memory, EEPROM, EPROM, ROM
Firmware (boot code, real time operating system (RTOS), application program)
Volatile Memory (lost after power; much less susceptible to being able to manipulate or take items from)
RAM
Variables, stack, buffers
Input/Output
Discrete, Analog, Fieldbus (4 to 10 milliAmps or 0-10 Volts)
Communication Ports
Serial - RS232, RS422/485, USB, modems, radios
Network - Ethernet radio, ControlNet, LonWorks
User interface
Internal
Status lights, small LCD screens (HMIs), keypads, jumpers, dip switches, switches
External
Browers (allows to see the status, working of the devices), Applications (always check if the applications can be shutdown, is there a business use-case for them?). Remember the smaller the attack surface area the better!
Programs
RTOS (Neutrino & RTOS (QNX), VxWorks, Windows CE)
IEC 61131 program languages - Workbences (CoDeSys (allows the ability to program in anyone of the below languages), ISaGRAF) - Languages
Ladder Logic
Function Block Diagram (FBD)
Sequential Function Chart (SFC)
Structured Text (ST)
Instruction List (IL)
Device Drivers and Device Managers
Ethernet/IP Stacks
RS232/RS-485
Memory Managers
User interfaces
Services (Web server, FTP server, SNMP) (Any business case for these running? If not, turn them off)
Debuggers (data for troubleshooting, are we turning it off after debugging? Often, debuggers are turned-on exposing data and possible vulnerablities)
Programmable Logic Controller
Program Execution
A line of code in a PLC program is called a rung.
PLC program execute from left to right and top to bottom.
Each completion of the program is called a scan.
A PLC will complete many scans in a single second (Scan rate: 50-60 milli-seconds/scan; SCADA system scan rate is approx 2 mins; metering at home (water/energy) is approx 15-30 mins).
Programming Concepts
Each rung executes on an “IF-Then” principle
IF the instruction(s) on the left are true then execute the instructions on the right.
Direct/Normal Open Contact
Direct/Normal Open Output Coil
Reverse/Normally Closed Contact
Placing multiple rungs (branch) on a single rung = OR
Placing multiple inputs on the same rung = AND
Data Flow
ICS collect information about some process or function using a communications infrastructure to send the data back to an operator. The operator reviews the data, typically in a graphical format, assesses the operational status of the process, and tunes the system for optimal performance.
Field Devices are the instruments and sensors that measure process parameters and the actuators that control the process. This is the interface between the ICS and the physical process. These sensors or measuring instruments are often referred to as input devices because they “input” data into the ICS.
Field Controllers are responsible for collecting and processing input and output information, sometimes referred to as I/O. They also send the process data to the human machine interface (HMI) and process control commands from the operators. They are often located close to the field devices.
Servers, HMIs, and engineering workstations take the information from field controllers and display the data in a manner that depicts what is happening in the process. The user interface, usually referred to as the HMI, allows the operator to have a real‐time, or near real‐time, operational view of the process. These three components are linked using networks or communication channels.
Field Devices (Meters, Sensors, Valves, Switches) <——-> Field Controllers (PLC, IED, RTU, Controller, PAC) <———–> HMI (SCADA Server, HMI, Workstations, EMS)
Direct connection or Device level protocols (HART, Foundation Fieldbus, Profibus) <———-> Command and Control Protocols (DNP3, Modbus, Ethernet/IP)
- Field Controllers –> Primary Historian –> Secondary Historian
|—> Configuration Database —> HMI —-> HMI
Protocols (ANSI X3.28, BBC 7200, CDC Type 1/2, Contitel, DCP, DNP, Gedac 7020, ICCP, Landis, Modbus, OPC, ControlNet, DeviceNet, DH+, Profibus, Tejas 3/5, TRW 9550, UCA)
Indusoft (HMI Software?)
Connected Components Workbench
Network Discovery and Mapping
Discovery Process in both, Passive is much more stealthy and Active is aggressive in trying to learn things. In both cases, we are mapping out the environment. Often is the case, when we are presented with a case of understading in-production environment, with no-prior person to enquire from, documentation is little, suggestions to how to handle certain performance issue.
Passive Discovery
What?
What is Passive Discovery?
Using information discovered from local memory of any host, to build a vision of an existing Control system environment.
Practicing safe methods to explore and perform reconnaissance.
Attempt to identify network details without sending network packets.
Why?
Why perform passive network discovery?
Safer practice regarding Control System networks (don’t want to break something).
Can yield information that active discover may not be practical for, such as data found in various files.
Use tools passively
When exploring a Control System network, practice passive techniques when mapping.
Utilities and commands are not neccessarily defined as passive. Using a tool passively is a responsibility of the user.
Daily operation of production Control Systems already create expected traffic. Try not to interfere or manipulate pathways when exploring.
Examples and Effects
Neglect to disable name resolution in commands - resolution queries could alert and IDS unnecessarily.
Scanning your own host, from the same host (to know what it is running?). - Self inflicted scans will preoccupy a host’s network resources and may alert a host-based IDS.
Restarting services without planning (often we try turning off and on again without planning). For example, if a watchdog timer checks for a open-port and restarting doesn’t start the service and the port remain closed. - Watchdog timers (checking for a particular state or change in state) could generate timeout signals, and trigger alarms to an operator. Meaningless errors can appear in logs.
Clearing Cache - Clearing cache will cause bursts of packets to repopulate tables.
Artifacts
Tools
ipconfig
,ip
,ifconfig
netstat -anob
/netstat -pantu
route print
/route -n
iptables
tcpdump
+wireshark
EtherApe
History + Logs
.bash_history
Browser History
Remote Desktop History
var/log/messages
var/log/syslog
Configuration files
crontab -l
/etc/network/interfaces
C:\windows\system32\Drivers\etc\hosts
/etc/resolv.conf
Cache
arp -an
nbstat -c
ipconfig /displaydns
How?
ARP
Linux
arp -a -i eth0
will do the DNS resolution that will send the network packets to the DNS server asking for name resolution (Active scan).
arp -a -i eth0 -n
will not do the DNS resolution (more passive).
Windows
arp -a
EtherApe is a good tool to understand what traffic is being generated
Explore the ARP table
Control systems can participate using ethernet.
Investigating ARP Tables are a great local cache to start with.
Use the arp command to view the table.
Take note of the MAC addresses mapped to IPv4 addresses.
Research discovered vendoes from first 3 bytes of the address (OUI - Organization Unique Identifier) and figure out what vendor is famous for what in control systems? (router/PLC/HMI/firewall/Cameras?).
Why look at the ARP table?
Display a list of remote hosts or devices, with with the host has recently communicated.
See if there are two ARP tables? (which probably means two network interfaces in a host connected to different networks?)
Check the table again later. It may change. If it does, this might be an indicatio nof scheduled tasks. Investigate further.
IP
Check IP addressing
Control systems can also participate using IP.
HMI workstations could be PC operating systems. Learn it’s potiential reach with other IP networks.
IP addressing commands can reveal much more than IP address.
Compare previously discovered MAC addresses mappings.
Why look so closely to IP addressing? - PLC’s, RTU’s and various SCADA devices are often controlled by HMI workstations. Knowing the IP connectivity is important security awareness.
Windows
ipconfig /all
check hostname, IP routing enabled (to see if its a router), subnet, gateway, DHCP/DNS servers?
Linux
ifconfig -a
if we do
ifconfig
, it would only show interfaces that are up and in configured state.if we do
ifconfig -a
, it would show interfaces that are configured/present in an up/down state (we might vlan, vpn, bonded interfaces).
ip a
or ip addr show eth0
DNS
cat /etc/resolv.conf
When a host is set to use a DNS server, generally ALL applications can query it.
HMI software becomes configured with network addresses. If the configurations are populated with names instead of numeric IP addresses, then we will be at the mercy of DNS server.
TCP/UDP Ports
Ports
Review any Listening or Established ports.
Compare TCP and UDP port numbers that maybe associated with Control system vendors.
Control System Port Number Examples
BACNet/IP : UDP 47808
DNP3 : TCP 20000, UDP 20000
Ethernet/IP : TCP 44818, UDP 2222, UDP 44818
ICCP : TCP 102
Modbus : TCP 502
Well-know ports range from
0 - 1023
Registered port ranges from
1024 - 49151
Dynamic port ranges from
49152 - 65535
netstat
What is netstat?
Tool for looking at a host current network sessions and listening ports that are being offered.
Why use netstat?
Determine which local servers are TCP or UDP based.
Search for potiential connections being made with any known Control Systems.
View all currently Established connections taking place with HMI, Controller, Historian or other hosts.
Windows
netstat -ano -p tcp
-a all sockets -n no name resolution -b owning process name -o owning processs ID
Linux
netstat -pantu
-p owning process ID -a all sockets -n no name resolution -t tcp -u udp
Check Local Address, Remote Address, State column (listening (those port numbers are listening), established (the host is talking to some other host check what ports (ICS Ports?)))
Probably, we can figure out what the local machine is used for HMI (connects to several devices and a database)
Check if IP addresses are in the same subnet or different (File server, HMI accessing files from an outside network) helps to figure out different subnet or boundary of different subnets.
Routing Table
What is a routing table?
A local table of IP network destinations that the host is able to reach.
Why look at the routing table?
Identify router/gateway IP addresses.
Identify network destinations.
Identify individual host destinations.
When viewing a route table, learn to notice the IP address ranges. Determine which ones appear public and which one appear private.
Make not of any public IP addresses that may appear in configurations found on Control System networks.
Private IPv4 ranges:
10.0.0.0 - 10.255.255.255 /8
172.16.0.0 - 172.31.255.255 /12
192.168.0.0 - 192.168.255.255 /16
If there’s any public IP printed in route table, if exists try to understand why control system needs to talk to the public IP address.
Windows
route print
Linux
route -n
or netstat -rn
Any gateway entry of
0.0.0.0
specifies local interface that has IP address setup on them.Check if any static IP addresses are setup?
Any host with more than one interfaces can act as a gateway.
Linux: check
/proc/sys/net/ipv4/ip_forward
-0
- Not forwarding -1
- Forwarding.Windows: Registry :
HKEY_LOCAL_MACHINE\System\CurrentControlSet\services\Tcpip\Parameters
Check value
IPEnableRouter
netBIOS
What is netBIOS?
Network Basic Input/Output System (netBIOS) - allows applications on diffrent computers to communicate within a local area network.
Used by Microsoft File and Printer Sharing
How can netBIOS be helpful?
Discovers networks and hosts by looking at netBIOS cache (
nbtstat -c
)Cache contains recently contacted systems.
Check the naming convention of the name. For example: FSWCB1, AD2.
FS/AD might represent FileServer or Active Directory.
Numbers 1,2 might highlight that there could be more than 1 server.
TCPDump/Windump
Captures and analyses common network traffic for the command line.
Uses standard libpcap/winpcap to capture/parse network traffic.
Uses Berkeley Packet Filter (BPF) syntax for creating capture filter expressions.
tcpdump can also be active, so probably do
-n
to avoid doing name resolution.Also, each tool can have a vulnerablities, it’s better to run the tool using a different user
-Z username
.
wireshark
GUI network protocol analyszer and packet sniffer.
Libpcap standard library for opening and capturing network traffic.
Customizable dissectors (modules) for proprietary protocols.
Security Notes:
vulnerablities in wireshark could leave your system at risk of compromise if used on active networks.
Not required to run with root privileges
Long-term traffic monitoring should be done with “tcpdump”
Rule of Thumb: Capture with tcpdump and analyze with Wireshark using a normal user account.
Files and Others
Browser history
Control system facilities may have workstations where various routine operations are performed. If particular personnel are no longer avialable, we can still explore a frequenctly used browser to collect information passively.
Address bar pre-populating with any URLs.
Saved usernames and passwords.
Bookmarks or Favorites, relating to Control Systems addresses.
Keystrokes to open recently closed tabs and windows.
Learn how to explore temporarty cache of the specific browser.
.bash_history
What is .bash_history
file?
History file containing a record of executed commands.
Every user of the has their own history file. It is located in the home directory of each user.
Files starting with a period appear hidden by default.
Why look at .bash_history
file?
Routinely executed commands help identify what tasks are performed at the workstation.
Host addresses and filenames could appear with specfied commands. Such as ssh, wget, ftp, rsync, mail and others.
People make mistakes. It may also contain username passwords.
Use the
history
command to view the contents.Check for any new IP address or any file extensions that might be of interest or any mail commands (employee addresses/file names) or any mysql commands (username, password or database name, or remote host (if not present that means mysql server is locally hosted)).
It may provide info on local hosts directories.
Check if any commands shows any USB/HDD/SDD was connected (any
/media
entries).
Active Discovery
What?
What is Active network discovery?
Send network packets and wait for a response in order to identify host and network targets
Can be extremely noisy and easily detected
Why?
Why use active disovery methods?
Identify targets that cannot be otherwise identified using passive discovery techniques.
Provides specific service, port and version information for a given targets.
Identify vulnerablities of accessible services.
How?
arp-scan
arp-scan -g 10.10.10.2/24
nmap
Designed to allow system administrators and individuals to scan large networks to determine which hosts are up and what services they are offering.
network discovery tool that can be used for identifying the systems currently connected to the network
nmap allows to audit what services are running on the identified hosts.
Can be dangerous to IT, SCADA and PCS systems, ICSs and embedded devices.
What is Nmap?
Open source tool for network mapping and security auditing.
Why use nmap?
much faster than manual discovery.
can scan an entire network quickly, and offers several options to customize a scan and its results.
How does nmap work?
Hosts on the network
Services (ports)
Operating systems etc.
Two-stage process
Host discovery
Port scanning
nmap - Discovery methods
User Datagram protocol (UDP)
unreliable stateless communication
No handshaking
Tranmission Control Protocol (TCP)
Reliable stateful communication
3-way handshake
Internet Control Message protocol (ICMP)
Provides control, troubleshooting, and error messages.
Normally used by ping and trace route commands.
Address resolution protocol (ARP)
Discovers Link Layer addresses of network devices.
Communicates in the bounds of single network.
Three-way handshake
Host Discovery
What is host discovery (HD)?
process of identifying active and interesting hosts on a network.
Why does Nmap do HD?
To significantly reduce the amount of time to complete network scans.
Narrows a set of IP ranges into list of active or interesting hosts to be port scanned.
How does HD work?
Uses combination of ARP, ICMP, TCP SYN, TCP ACK packets to identify active hosts.
Default Host Discovery Settings
LAN sends ARP scan (
-PR
)WAN (privileged) sends TCP ACK packet to Port 80.
(
-PA
) and an ICMP echo request query (-PE
)WAN (unprivileged) sends TCP SYN packet (
-PS
) usingconnect()
system call instead of TCP ACK packet.By default nmap will use arp-response for local network host discovery. If we want to use ICMP, use
--send-ip
-P (Host discovery)
Port Scanning
What is port scanning?
process of identifying the status of interesting ports on hosts that are discovered on a network.
Why does nmap do port scanning?
to identify ports that are open on a host
How does port scanning work?
attempts to communicate with each port with a specified set of ports.
port scans are performed on hosts that were identified as active or interesting during HD.
Nmap Port states
Open: Application on target machine is listening for connections or packets on that port.
Closed: No application listening at the moment
Filtered: Firewall, filter or other network obstacle is blocking the port so that Nmap cannot tell if the port is open or closed. Nmap received no response.
Unfiltered: Port is accessible but nmap not able to determine if open or closed.
Open | Filtered: Unable to determine if open or filtered.
Closed | Filtered: Unable to determine if closed or filtered.
Nmap default port scanning settings. - SYN scan (
-sS
) for privileged users. - Connect scan (-sT
) for unprivileged users.If it starts with
-P
(host discover)-s
is for port scanning.
Timing and Performance options
What are timing and performance options
Settings used to control scanning delays, timeouts, retries and parallelism.
Why use timing and performance options?
Help speed up scanning process
Slow down scan to avoid IDS detection
Timing and performance options
Manual options are available but templates are usually sufficient
Template timings options offer throttling abilities not available using manual options.
Nmap results
Why save your nmap results?
easier to analyze and compare scans results (using ndiff)
Results overflow the console window buffer.
Output options
-oN filename.nmap
: Output results in normal format-oX filename.xml
: Output results in XML format-oG filename.gmap
: Output results in grepable format-oA filname
: Output results in all formats.-v
: Verbose output results
--reason
tells the reason.
OS and Version detection
What is OS and version detection.
Identifies operating system by looking at packet charactertistics.
Identifies the version of a service running on a host.
Why use OS and version detection?
Provides information that could help in the selection of exploits and payloads used against a target
How does OS detection work?
Nmap sends a series of TCP and UDP packets to the remote host and examples every bit in the responses.
Nmap compares the results to its database of known OS fingerprints and prints out the OS details if theres is a match.
How does Service and Version Detection Work?
After TCP and/or UDP ports are discovered, version detection interrogates those ports.
Database of probes for querying various services and match expressions to recognize and parse responses.
Tried to determine application name, version number, hostname, device type, OS family, and misc. information.
Nmap Address Schemes
Target hosts can be specified in many ways
1.2.3.1-254
: All 254 possible IP addresses on this subnet.1.2.3.0/24
: Equivalent to above but signifying a Class C address block.1.2.1-4.1-254
: Ranges are allowed for subnets as well.1.2.0.0/16
: The 16-bit netmask will scan the entire clas B address block.
--exclude
exclude a host/range.-sn
only do host scanning phase
ICS challenges
scans can cause computer system to restart
scans can cause embedded devices to freeze or lose configuration and in some severe cases requires vendor involvement.
Nmap considerations
Use connect scan (
-sT
) to prevent dangling connections.Don’t use OS (
-O
) and version detection (-sV
) (Control system would be running PLCs, RTU)Slow the scan down by reducing the rate at which packets are being generated and sent by Nmap.
Consider using exlusion lists (
--exclude
or--excludefile
)
Nessus Vulnerablity Scanner
Can be dangerous to ICSs.
Plugin modules for various ICS protocols.
Security auditing tool consists of two parts
Server (in charge of the scanning process).
Client (presents the interface to the user).
Nessus ICS Plugins
Areva/Alstom Energey management system
DNP3 Binary Inputs access
DNP3:
Link layer addressing DNP3
Unsolicited Messaging
ICCP
ICCP/COTP protocol
ICCP/COTP
TSAP Addressing
LiveData ICCP Server
Matrikon OPC Explorer
Matrikon OPC Server for ControlLogix
Matrikon OPC Server for Modbus
Modbus/TCP
Coil access
Discrete Input Access Programming
Function Code Access
Network Defense, Detection and Analysis
Identify
Asset and Information inventory
An asset inventory is necessary to understand and manage ICS risk and determine priorities for security defenses. The asset inventory is critical for understanding the potential impact of an intrusion
Know your environment
What?
Needs to be protected (PLC, pump, valves, non-electronics still something physical - how it is protected?)
Protection levels are available (What is available by vendors to protect the systems). How data is gathered from the ground-up?
Inter-connections and dependencies are required (what talks to what?, pump talking to PLC (controlling pump speed or flow) if not it might cause something to fail?)
Why?
Are systems critical (any special use, any special vendor?)
Are assets valuable ($$ and information)(produce gas or oil, electricity?)(Does the information provide insights to business to make decisions?)
Who?
Has responsibility for the asset (Who’s responsible System Admin, SPOC (single point of contact))
How?
Are worst-case scenarios identified if compromised (Do we have any plans in place in terms of outside/inside attacker?)
Are methods available for user access to the asset (Does the person have to visit the control room to access the devices or can be access remotely or via VPN?)
Does the information flow throught the system (where it starts/stops? Goes to firewall? Business IT network?)
Other
Field Devices
Easy to forget in asset inventory - “out of sight, out of mind”.
Field devices may be accessed remotely because it is more convenient or may require that a human being physically visit the remote device. When accessing remotely make sure the communication is secure and the device accessing the field devices is secure.
Security Challenges regarding Field Devices.
No centralized management for older field devices.
May lack security capabilities (maybe serial only, make sure we understand what capabilities they have)
Increased use of portable devices to access field devices (Laptops/Tablets?).
Possible Mitigations
Lock down unneeded services, ports and restrict access (Disable unused ports on the switch).
All devices used to interface with the field devices should be secured and monitored (have anti-virus and properly logged and accounted for).
Think about what devices are present and how they are communicating with central system and how they are controlled?
Least Functionality
Determine necessary ports, protocols, and services (What are the vendor recommendations/talk to the vendors what needs to be open on firewall/router)
Deny all others at the host and firewall
Harden devices (be careful while hardening and test whether everything is working or not; Never test on live system.)
Network access control (What can talk to what or each others? )
Use the data from a scan such as Nmap, to identify unused ports and service and disable all unused ports and services off. This should be done at the host. However, if it cannot be done at the host, use other mitigations, such as a firewall, to block any access to the services or any traffic leaving these hosts on these ports.
Hardening systems using security guidelines or controls will also reduce your attack surface. Work with vendors to determine hardening guidelines/settings for ICS equipment
Least Privileges
Establish user accounts for administrators (separate accounts for engineers, administrators and test that they are able to do their work and perform their responsiblities)
Appropriate use of the escalated privilege function (Check if the user needs esclated privileges and it is logged properly and they use it appropriately (whenever it is really required)).
Review work requirements for necessary access requirements
Role-based access (provide appropriate access for appropriate person).
Tools
GrassMarlin (Retired)
GrassMarlin can be used to identify traffic and systems on ICS network.
GrassMarlin is a passive network mapper dedicated to ICS and SCADA networks in support of network security assessments.
GrassMarlin passively maps, and visually displays, an ICS/SCADA network topology while safely conducting device discovery, accounting, and reporting on these critical cyber‐physical systems.
GrassMarlin gives a snapshot of the ICS network including:
Devices part of the network;
Communications between these devices;
Metadata extracted from these communications.
Reads in Zeek Connection logs, PCAP files and PCAP-NG files or can listen on the wire
Protect
IT-OT Convergence
Does IT/OT talk to each other? (They should be able to work together and help each other and whenever they have problems they talk to each other and solve problems by respecting each other.)
What we can do to improve communication between IT and OT teams (invite them to meetings, talk to them regarding something they are expert in and can help (firewall issues))
Human element
Policies and Procedures specific to ICS
Outline rules with regard to securing ICS (What kind of things we need to secure?)
Computer use policy (helps to understand what’s expected and what’s not)
Make security a priority (everyone should be aware of the ICS security)
Training and awareness
Employees are part of your defense (They are the most important people. Employee errors or unintentional actions often leads upto 50% incidents).
See something, say something (If they see something that is not right, ask them to mention)
Talk about security in staff meetings (something going on in your network, group or unit and training around security)
Lessons learnt from past incidents
User education is important.
Do regular phishing tests (As an OT person, we can take help of IT department to set this up.).
Explain to users the consequences of clicking bad links (Usually people often don’t understand why it is bad to click on links, if they understand they are more careful.)
OPSEC
Operational Security, or OPSEC, is when we protect unclassified information from leaking out via our own actions and behaviors. The goal of Cybersecurity OPSEC is to minimize your digital footprint /information leakage and to minimize the damage when things go bad. In the best of scenarios you might almost drop off the grid completely. Remember that OPSEC does not replace any other security disciplines - it supplements them.
Always be aware of what your company is presenting to the outside world (what your network looks from outside? Do we have FTP/SSH server accessible from internet? )
Do you know what is on your company’s external webpage and social media feeds?
Are vendors using your company for free advertising?
Are your IP address ranges showing up in Shodan ICS? If you give data to vendors, do you know how they are storing it?
The OPSEC process is categorized into 5 questions/steps. One of the first questions is, who would want access to the data in question, what needs protected?
The OPSEC process
What needs to be protected?
Who is the threat?
What are my vulnerablities?
What is the threat level?
How should we combat the threats?
Secure Passwords
Adversaries focus on gaining legitimate credentials to traverse the network
NIST SP 800-63B Guidelines (Digital Identity Guidelines - Authentication and Lifecycle Management)
Fewer complexity rules enforced
Expiration of passwords no longer based on a time schedule (If the passwords are good and strong, maybe no need to change them every time)
Passwords should be screened again lists of dictionaries and common, easily guessed passwords (mention to employees that we will try to guess and crack their passwords and they will create strong passwords)
Allow paste functionality from Password Managers (also store your passwords in a safe secure location)
Industry compliance documents or your organisation policies may differ.
NERC CIP standards (CIP-007-5)
NIST-800-53
Vendor Access
Vendor connections to the ICS Network
One of the most common ways malware and viruses are introduced into ICS environments is the use of media that has been shared or used on systems outside the production environment.
To mitigate that risk consider implementing the following:
Implement a Dedicated workstation to transfer files and patches to trusted devices that is up to date with the latest virus and malware definitions not connected to the ICS network.
Do not allow vendors or 3rd party USB’s in ICS environment (We have no idea who’s USB device it is, where it has been, what it contains?)
Have a device whitelisting application or ability to disable media ports.
Provide security policies to govern use.
Configure your removable media policy to notify your security team of events of when access to USB ports or unapproved media is attempted to be used.
Removable Media
If possible, do not allow personal devices to be used in the ICS network (people charging their phones on ICS network?, malicious USB (USBDucky, OMG Cable and others?))
If this is not possible, provide good security policies to manage the use of personal devices, and use company resources to help implement the policies.
Enterprise device management technology can help ensure that only approved assets can be attached to ICS networks and computers.
Lessons learnt from past incidents
Good network segmentation can prevent malware call backs.
Monitor USB usage especially in the ICS environment (inventory of allowed USB devices, who have them and what they are using it for?).
Secure Authentication
Multi-factor Authentication
Definition: What the user knows (password), what the user has (security token), and/or what the user is (biometric validation).
Single factor authentication increases the attack surface.
Use multi-factor authentication for remote access and critical administrative access.
Can be used with VPN, network device access, administrator access to systems.
Example: Many asset owners use single-factor authentication for remote access. If a user has a vulnerable machine, the attack surface is greatly increased.
Secure VPN access
Limit VPN access to business requirements - vendors, technicians, integrators (who has access to what? If providing access to vendor, terminate VPN as close to edge as possible and provide access to only required systems/segmented network/DMZ. Good idea to define that in vendor contract agreements)
Require company issued and configured systems be used without Admin access (No admin access provided until and unless really required). - If they require admin access or access to a particular resource, work with them to figure out how we can provide that securely. Otherwise, technical users will always figure out a way to achieve it which might result in undocumented access.
VPN security policy should check for patches, a personal firewall, and an antivirus product.
Utilise a jump-box, or a virtual desktop for further network access.
Utilise a second domain controller (Have a separate IT/OT domain controller)
VPN Logs
VPN appliance provides a wealth of logging information regarding the perimeter of your network. This information can be used to monitor the health of the system and potentially detect malicious activity. It is important to:
Find unusual login attempts: Look for unusual situations, such as the company President logging in from a Starbucks in England, when the President is actually in the middle of a safari in Africa.
Monitor failed authentication attempts: All devices or processes that require identity authentication should log and/or alert when an identity validation attempt fails.
Monitor successful authentication attempts from different sources: If available, all devices or processes should log and/or alert when the same user logs in simultaneously from two different source locations.
Monitor successful authentication under duress: For critical systems, consider deploying an authentication mechanism that supports duress codes. This allows a user under duress to log into a system using a secondary credential, but alerts that the access was performed under duress.
Monitor failed access attempts: All devices or processes that manage access control to communications, data, or services should log and/or alert when access is requested that is not allowed.
Monitor successful access attempts: All devices or processes that manage access control to communications, data, or services should log when access is requested and allowed.
Lessons Learned
Virtual Machine Use Case
Incident: VM was configured in an ICS environment with the VM hardware (vmware/hardware machine) located in the ICS DMZ. Management interface provided direct connectivity to the corporate network for ease of use. Further, ICS servers in the VM bridged the DMZ firewall to the ICS network
Lesson: Bridged the corporate protected communications to the VM management interface located in the ICS DMZ. Utilize VMware security guidance to setup VMware systems.
VPN/Password Use case
Incident: A user had a VPN connection and was logged in as administrator. The user’s home PC was dual homed with VPN client and a public interface.
Lesson: Proper configuration of VPN client. Limit VPN access to business requirements. Do not allow users to run as admin.
ICS Network segmentation
The Purdue Enterprise Reference Architecture (PERA) Model is suggested by the DHS Assessment Team as a best practice for segmenting networks.
The PERA model segments industrial control devices into hierarchical “levels” of operations within a facility. Using levels as common terminology breaks down and determines plant wide information flow. Zones establish domains of trust for security access and smaller LANs to shape and manage network traffic.
This model groups levels into the following zones for specific functions:
Enterprise Zone: Levels 4 and 5 handle IT networks, business applications/servers (e.g. email, enterprise resource planning - ERP) as well as intranet.
ICS Demilitarized Zone (IDMZ): This buffer zone provides a barrier between the ICS and Enterprise Zones but allows for data and services to be shared securely. All network traffic from either side of the IDMZ terminates in the IDMZ. No traffic traverses the IDMZ. That is, no traffic directly travels between the Enterprise and ICS Zones.
ICS Zone: Level 3 addresses plant wide applications (e.g., historian, asset management, authentication, patch management), consisting of multiple Cell/Area Zones.
Cell/Area Zone: Levels 0, 1 and 2 manage industrial control devices (e.g., controllers, drives, I/O and HMI) and multi-disciplined control applications (e.g., drive, batch, continuous process, and discrete).
Typical Flat network
Poor asset inventory
Poor boundary protection (HMI’s directly connected to the Internet)
Poorly Secured Remote Access
Recommended Secure Network Architecture
Good Asset Inventory and Data flows (How does data flow and what data flow is important/critical (what must always be available))
Good Boundary Protection
Secured Remote monitoring and Access
Isolation of Safety Instrumented Systems (How are safety systems implemented?)
Firewall Implementation
The firewalls are placed at the front line of defense for each of the various zones. These firewalls provide the trusted path for users and applications to communicate with and between all of the various pieces.
There are two complimentary principles for segmenting networks.
The first principle includes the general functions of a system:
Serve external customers
Handle facility environmental controls
Support IT
Process HR data
Run/supervise ICS process data
Run/Supervise ICS
The second principle is trust level.
What is the sensitivity of the data/system/data path?
Segmentation should be implemented using firewalls or at least routers with access control lists (ACLs). Some considerations for firewalls:
Know your environment
How does data flow?
How is data used? (What does that data mean?)
Who uses the data? (Who is the owner of the data? Mostly historian from ICS persecptive)
Newer next generation firewall support multiple ICS protocols/standards.
Trade off efficency vs. security vs. cost (Every device can provide or hinder efficency or has a cost to it)
Erroneously deployed as a cornerstone of architecture (requires month of planning/architected)
Firewall Rules
Without rules, firewall is basically a router.
Block direct traffic from the control network to the corporate network. All ICS traffic should end at the DMZ.
Every protocol permitted between the control network and the DMZ should be explicitly denied between the DMZ and corporate networks (and vice versa).
ICS networks should not be connected directly to the Internet, even if they are protected by a firewall.
Firewall Logs
Firewalls logs provides insights into security threats and traffic behaviour regarding the perimeter of your network. Information can be used to monitor the health of the system and potentially detect malicious activity. It is important to:
Identify traffic denied at the firewall - e.g. traffic from inside the network that is bouncing off the firewall (what traffic is trying to get out?)
Identify traffic allowed at the firewall
Identify multiple connections from multiple devices in your network to a few target locations
Data Diode
A data diode is a unidirectional gateway intended to move data from a more secure network to a less secure network.
A data diode creates a physically se cure, one-way communication channel from
the control system network to the corporate network. Data diodes can be implemented in hardware, software, or a combination of both. The hardware implementation is the most secure because it is physically impossible to send any messages in the reverse direction.
Data Diode vs. Firewalls
Data Diodes
Behaves like a Proxy Server: converts TCP sessions to UDP
Uni-directional communication: reverse tunneling not possible
May cost more than some firewalls
Fewer rules: rules require less auditing
Transmits only the data: no connection between systems.
Firewalls
Two-way communications: tunneling possible.
Rules require more auditing due to complexity of rule set
Cannot create a one-way communication. UDP is one way. Does not create anything but one way.
Patch Management
BEFORE PATCHING ANY ICSOT SYSTEM (PLC/RTU/HMI) ENSURE YOU HAVE A GOOD BAREMETAL BACKUP OR ABILITY
TO RESTORE THE SYSTEM TO THE CURRENT STATE!
Patches are intended to:
Fix known vulnerablities.
Enhance functionality
Software that needs patching includes
Operating System
ICS Application/hardware
Third-party applications
Patch deployment considerations
Test and validate
Offline systems vs. live systems
Work with vendors for patch applicability.
Patching Considerations
Considerations when deciding to patch systems:
How critical is each system to production?
What complications arise in patching critical infrastructure?
What is the cost of a patch?
What is the cost of not applying a patch?
What is the businesssecurity driver in patching?
Do you have a mitigating control in place if you decide patching is not an option?
Potential Patch Complications
Patching can break other software components
Patching can break 3rd party software components
Updating antivirus definitions can inadvertently stop legitimate processes
Sand box systems are not used directly for production
Balance in waiting to test the patch and applying a patch before it is fully tested
Systems remain vulnerable until they are patched, or mitigating controls are implemented.
Application whitelisting
Advantages
Blocks most current malware
Prevents use of unauthorized applications (have good software inventory. Process environment is very predictable)
Does not require daily definitions updates
Administrator installation and approval of new applications.
Limitations
Approved applications - compromised in supply chain.
Malware that exploits application that run in higher-level execution environments such as Java may not be found.
Disadvantages
Requires performance overhead
Requires regular maintainence
Causes some users to be annoyed
Detect
Identify a cybersecurity event
Intrusion Detection System
ICS environments provide a unique opportunity. Compared to a corporate environment, an ICS environment is a steady state. Once again, you must know your environment. Ask and answer the following questions:
WHAT is normal? (Is this documented?) - You know that host “A” talks to host “B,” but not host “C”…
WHEN does “normal” become abnormal? (indicators that something might be going on?) - Host “A” is now talking to host “C”…WHY?
WHOSE applications and services are on your critical networks?
WHICH protocols are used? - Known IT protocols (DNS traffic, HTTP traffic) - Vendor (Proprietary traffic)
IDS Types
Host: Sensors reside on the host system
Network: What traffic is on your network?
Application: Web application firewall, database, firewall, application protocol IDS.
Log: What is happening at the OS level? or at the application level?
Paper: Who came in?
Anomaly: Any combination of the above.
All methods of intrusion detection involve the gathering and analysis of information from various sources within a computer, network, and enterprise to identify possible threats posed by hackers inside or outside the organization.
IDS/IPS Functions
An IDS is not a cure‐all for network security problems. It is an alerting tool to let you know something has happened. An IDS can:
Provide forewarning
Provide forensics data
Provide “situational awareness”
Provide network troubleshooting
Identify policy abuse.
Placing an IDS outside of the firewall can be helpful for situational awareness and forewarning of activities. The IDS can detect scanning or other precursory attack activities that might be dropped by the firewall. An IDS cannot:
Tell you directly if the system was exploited
Monitor actions taken by the system console
Perform analysis of an event (requires human being to analyse ).
HIDS
Host-based intrusion detection (HIDS) refers to intrusion detection that takes place on a single host system. HIDS involves installing an agent on the local host that monitors and reports on the system configuration and application activity. Some common abilities of HIDS systems include:
Provides the “victims” view
Virus detection/mitigation
Local log analysis
File integrity checking
Policy monitoring
Rootkit detection
Network monitoring from the host viewpoint
Real-time alerting
Active response.
HIDS often have the ability to baseline a host system to detect variations in system configuration. In specific vendor implementations, these HIDS agents also allow connectivity to other security systems. This allows for central management of configuration policy and verification.
HIDS Deployment
HIDS tools are initially deployed in “monitor only” mode. This enables the administrator to create a baseline of the system configuration and activity. Active blocking of applications, system changes, and network activity is limited to only the most egregious activities. The policy can then be tuned based on what is considered “normal activity.” Once a policy is configured, it is then applied and distributed to the hosts. Benefits of central management architecture are:
Can be centrally managed with deployable policies.
Ability to apply changes to many systems at once
Create a “baseline” for known system types/use cases
Central authentication, alerting, and reporting
Central audit logging.
The main two concerns with using any HIDS in an ICS environment are:
Does Operating System even support the use of a HIDS?
Do the hosts have enough hardware capacity to support the HIDS (CPU, memory, network bandwidth, etc.)
Network Intrusion Detection (NIDS)
NIDSs scan traffic from its networks and look for known patterns in traffic (packets).
A NIDS can scan both sides of a conversation and can be reactive by blocking traffic when in IPS mode.
NIDS often does not know if the system is Windows, Linux, or a PLC. From a NIDS perspective traffic is traffic, and it simply reports on what traffic is seen on the network.
NIDS can have a high False-Positive or False-Negative rate based on the information used to generate the signatures.
NIDS are connected to the network via a SPAN/mirror port or a network tap.
When using a SPAN port, the switch sends a copy of all the network packets “seen” on one physical port (or an entire VLAN) to another physical port, where the packets can be captured and/or analyzed.
A networking monitoring tap can be used to collect network packets without having to configure a span port on a switch. Think of a tap as a special T‐connection that can read data from the network, but not inject any data of its own into the network traffic.
IDS Sensor Placement
The placement for IDS sensors is important.
Any change in trust zones should have an IDS/IPS deployed
A data diode should be attached to the historian. The IDS can also be deployed here
All points of presences for the external communications should have an IDS/IPS deployed
An IDS on either side of firewalls allows you to audit your firewall rules.
NIDS Signature vs. Anomaly Detection
Signature |
Anomaly |
---|---|
Ex. Snort, Mcafee |
|
Watches for specific events |
Watches for changes in trends |
Only looks for what it has been told |
Learns from gradual changes |
Can deal with any known threat |
Can deal with unknowns, but any attack is subject to false-negative (Doesn’t know what attacks are, just know it’s change in traffic) |
Unaware of network configuration changes |
Sensitive to changes in network devices |
Highly objective inspection |
Subjective, prone to misinterpretations |
Predictable behavior |
Unpredictable behavior |
Easy to tune manually |
Netflow Anomaly Detection
NetFlow is a network protocol developed by Cisco Systems for collecting IP traffic information. NetFlow has become an industry standard for traffic monitoring and is supported by platforms other than Cisco. Routers and switches that have the NetFlow feature enabled produce UDP data streams that are sent to a NetFlow collector (server) where it can be processed and stored.
Describes a set of packets sharing these characteristics: src, sport, dst, dport, protocol, type of service.
Data include: time, number of bytes, number of packets
Usually sent via UDP or Stream Control Transmission Protocol
Distributed Denial of Service
Massive increase in flows
Trojan Horses
“Well-known” or unexpected services
Firewall Policy Violation
Unexpected inside/outside flow
Example Alerts for Anomaly Detection
Hosts scanning for services:
Are there external hosts poking at more than __ internal addresses?
Are there external hosts poking at more than __ ports on 1 (or more) internal hosts?
Internal infected host scanning/talking to for external hosts:
Is some internal host poking at __ external hosts?
Is some internal host poking at __ internal hosts?
Is some internal host poking at dark space (un-allocated Internet address space)?
Internal hosts talking to “Interesting Net blocks” (pick your favorite countries here)
Are there pokes from __ net blocks that may be of interest?
Are there pokes to __ net blocks that may be of interest?
Increased network traffic:
Distributed Denial of Service (DDOS)
Unexpected high volume - Data mining, egress?
Zeek IDS
Open-source
Allows scripting of monitoring policies
Collect logs for analysis (Non-standard ports, Connections, DNS, FTP, Files, HTTP requests, SSL, SMTP activity).
Analyzers for many protocols including Modbus and DNP3
Unexpected protocol level activity.
Logs can be used by several other security products.
IDS vs. IPS
IDS
Watching/ Passive alerting
IPS
Inline, Passive Alerting, Active Response
SNORT
Snort is an open-source network intrusion detection and prevention system. Snort is widely used and has become the standard for IDS/IPS.
Learning to write Snort rules is useful because most IDS/IPS applications will either use the Snort rule format or provide a way to import Snort rules.
If you are able to understand the data flow in your environment, you will be able to design simple anomalous traffic signatures quickly without regard to the actual details of the protocol used.
Snort rules are composed of a rule header and rule options. There are five types of rule options:
Metadata
Payload detection
Non-payload detection
Post-detection
Thresholding and suppression
We will focus on Metadata and payload detection
alert ip ![10.0.10.20, 10.0.10.30] any <> [10.0.10.15] any (msg:"ALERT - Field Controller interacts with another node"; reference:url,mysite.org/rule1; reference:cve,2018-0000;sid:3000001;priority:1;rev:1;)
action |
alert, log, pass, active, dynamic, or a custom defined type |
protocol |
ip, tcp, udp, icmp, any |
src ip and src port |
See below |
direction |
->, <> direction of the traffic that the rule applies to |
dst ip and dst port |
See below |
Msg |
Used by analyst to quickly identify the signature |
Reference |
Can use a predefined tag for a security web site or use “URL” to include any web site reference in the rules |
Sid |
The signature ID is used by Snort to uniquely identify rules. We recommend using a number > 3,000,000 |
Priority |
Allows the user to set the priority of the rule. Highest - 1, Lowest - 10 |
Snort Preprocessors for ICS
A number of attacks cannot be detected by signature matching alone in the detection engine, so protocol “examine” preprocessors step up to the plate and detect suspicious activity. These preprocessors include packet fragmentation, TCP stateful inspection, portscans, and many other Network/Application protocol‐specific activities.
Others modify packets by normalizing traffic so that the detection engine can accurately match signatures. These preprocessors defeat attacks that attempt to evade Snort’s detection engine by manipulating traffic patterns.
Snort cycles packets through every preprocessor to discover attacks that require more than one preprocessor to detect them. If Snort simply quit checking for the suspicious attributes of a packet after it had set off a preprocessor alert, attackers could use this deficiency to hide traffic from Snort.
Preprocessor parameters are configured and tuned via the snort.conf file. The snort.conf file lets you add or remove preprocessors as you see fit. Of particular interest to the ICS community are the DNP3 and Modbus preprocessors.
ICS Specfic: DNP3/Modbus
Other useful preprocessor: SSH, SSL, Portscan, httpinspect
DNP3 Preprocessor Rule Options
dnp3_func: Matches Function Code inside an Application-Layer request/response header
dnp3_ind: Matches on the Internal Indicators flags in Application Response Header (Similar to TCP flags)
dnp3_obj: Matches on request or response object headers
dnp3_data: Reassembled Application-Layer Fragments.
DNP3 Preprocessor Examples
Here are some examples of the new DNP3 preprocessor rule options:
Alerts on DNP3 Write Request:
alert tcp any any -> any 20000 (msg:"DNP3 Write request"; dnp3_func:write; sid:3000001;)
Alerts on reserved_1 OR reserved_2 being set:
alert tcp any 20000 -> any any (msg:"Reserved DNP3 Indicator set"; dnp3_ind:reserved_1,reserved_2; sid:3000002)
Alerts on Content in Re-assembled Application-Layer Fragment:
alert tcp any any -> any any (msg:"badstuff' in DNP3 message"; dnp3_data; content:"badstuff"; sid:3000003;)
Notice in the third rule, dnp3_data sets the content buffer to the beginning of the Re-assembled Application-Layer Fragment then looks for the content: “badstuff”
Modbus Preprocessor Rule Options
modbus_func: Matches against the Function Code inside of a Modbus Application-Layer request/response header
modbus_unit: Matches against the Unit ID field in a Modbus header
modbus_data: Sets the cursor at the beginning of the Data field in Modbus request/response
Modbus Preprocessor Rule Examples
Alerts on specific Modbus function:
alert tcp any any -> any 502 (msg:"Modbus Write Coils request"; modbus_func:write_multiple_coils; sid:3000004;)
Alerts on unauthorized host
var MODBUS_ADMIN 192.168.1.2
alert tcp !$MODBUS_ADMIN any -> any 502 (msg:"Modbus command to Unit 01 from unauthorized host"; modbus_unit:1; sid:3000005;)
Alerts on Content in modbus data field
``alert tcp any any -> any any (msg:”String ‘badstuff’ in Modbus message”; modbus_data; content:”badstuff”; sid:3000006;).
Example Rule Variables
ipvar HOME_NET [1.2.3.0/24,10.0.10.0/24]
ipvar EXTERNAL_NET [!HOME_NET]
ipvar CANARY 1.2.3.4
ipvar PCS [10.0.10.0/24]
ipvar CORP [1.2.3.0/24]
ipvar HMI [10.0.10.20,10.0.10.30]
ipvar AD 1.2.3.20
ipvar FC 10.0.10.15
ipvar HIST1 [10.0.10.150]
ipvar CONFDB [10.0.10.10]
portvar TAG 2000
portvar TAG_RANGE [2000:2020]
Example Rules
#Field Controller (FC) talking to unknown system
alert ip ![$HMI,$HIST1,$CONFDB] any -> $FC any (msg:“ALERT - Field Controller interacts with unknown node"; sid:4000001; priority:1; rev:1;)
#Configuration Database talks to unexpected system
alert ip [$CONFDB] any -> ![$FC,$HMI,$HIST1] any (msg:“ALERT - Configuration DB Communicate with new system; sid:4000002; priority:1; rev:1;)
# PCS network communication with CORP network, trying to bypass the firewall
alert ip [$PCS,!$HIST1] any -> $CORP any (msg:”PCS network talking to CORP network”; sid:4000003; priority:1; classtype:unknown;)
#Configuration Database updates (auditing tool)
log ip [$CONFDB] any -> [$FC,$HMI,$HIST1] any (msg:“AUDIT - Configuration Updates; sid:4000004; priority:10; rev:1;)
# LOOKING FOR BAD TRAFFIC
# Find traffic involving a canary
alert ip any any <> $CANARY any (msg:”The canary is talking”; sid: 4000005; priority:1; classtype:unknown; tag:session,256,packets;)
#Monitor for the Field Controller talking to the Internet
alert tcp $FC any -> $EXTERNAL_NET any (msg:”PLC talking to the outside world”; sid:4000007; priority:1; flags:S; classtype:bad-unknown;)
# Monitor for AD attempting to connect to the Internet
alert tcp $AD any -> $EXTERNAL_NET any (msg:”AD attempting to talk to the outside world”; sid:4000008; priority:1; flags:S; classtype:bad-unknown;)
#Command shell on HMI
alert ip any any -> $HMI any (msg:”cmd.exe on HMI”; content: “cmd.exe”; sid:4000009; priority:1; classtype:unknown;)
Log Sources and Management
Logging Architecture
A central log server can assist in an incident by providing a chronological list of the events surrounding an incident that give the bigger picture.
Multiple systems/sources can send their data to a central log server where it can be correlated with other information.
Correlating with other logs can sometimes make the difference between recognizing an event for what it is (true or false) and then acting accordingly. The same data can provide valuable information (such as an IDS) to the security analyst.
There are some considerations in centralizing logs: - Properly prioritize the function of log management. Define requirements and goals for log performance and monitoring based on applicable laws, regulations, and existing organizational policies. Then, prioritize goals based on balancing the need to reduce risk with the time and resources necessary to perform log management functions. - Create and maintain a secure log management infrastructure. Identify the needed components and determine how they will interact (e.g., firewall rules, diodes). With the various types of information in one place, the log server becomes a valuable system to target a critical system to protect. It should only run the logging service and be in a highly protected area of your network. - Provide appropriate support for staff with log management responsibilities. All efforts to implement log management will be for naught if the staff members who are tasked with log management responsibilities do not receive adequate training, proper tools, or support to do their jobs effectively. The staff members need to understand what situations are normal, bad, and weird. Providing log management tools, documentation, and technical guidance are all critical for the success of log management staff.
Log sources
firewalls
VPN Servers (maybe part of firewall logs)
Operating Systems (e.g Windows, *nix, Mac)
Proxy Server
Web Servers (e.g. IIS, Apache, NGinx)
Databases (e.g. MS SQL, Oracle, MySQL)
Others (e.g. PLCs, HMIs)
Log Transport
syslog
Defacto standard in IT community
Use UDP/TCP
Data diode can be used
Encryption can be used
Third-party tools maybe necessary for some OS or applications.
Operating System Logs
Operating system logs can be used to monitor the health of the system and detect malicious activity
Windows OS - Security Log - System Log - Third-party agent to send logs to a remote server.
Linux/Unix OS - Syslog transport part of OS - auth.log, messages
Security Audit Logging Web Server Logs
Review daily to determine a baseline
Web server logs will show:
who visited the website
when they visited the website
what they did while viewing the website (including SQL queries)
Where they came from?
Security Audit Logging Database Logs
User logins and logouts
Database system starts, stops and restarts
Various system failures and errors
User privilege changes
Database structure changes (tables that has been deleted/data that has been changed)
Most other DBA actions; and
Select or all database data access (if configured to be so)
Security Information and Event Management
Capabilities
Data aggregation
Correlation
Alerting
compliance
Forensics analysis
Honeypots & Canaries
Decoy systems (sit on your network and try to replicate how your network looks like)
Variant of an IDS
Any traffic seen talking to a Honeypot could be considered malicious
Open-source ICS Honeypots are available: Conpot
Canaries (doesn’t communicate with any other system on your network. If an IDS is watching for ANY traffic to/from the canary, you will get an early warning that something is going on that shouldn’t be).
Respond and Recover
Execute activities taken during and after a cybersecurity event.
The Respond Function supports the ability to contain the impact of a potential cybersecurity event. Examples of outcome Categories within this Function include: Response Planning; Communications; Analysis; Mitigation; and Improvements.
The Recover Function supports timely recovery to normal operations to reduce the impact from a cybersecurity event. Examples of outcome Categories within this Function include: Recovery Planning; Improvements; and Communications.
Incident Respond Phases - Preparation –> Identification –> Containment –> Clean-up and Recovery –> Follow-up
Preparation
Build your team
Plan your response - Secure and alternate methods of communication.
Scribe(s) for each group within the team. - Securable room where you can keep accurate and complete information - access to ALL of the logs and data. - Known, certified clean computer systems to do forensics. - Person with the authority to unplug from the internet (maybe your manager, CEO?)
Define your strategy.
Create documentation
Train your teams and users - A practiced plan
Gather threat intelligence - Feeds & threat reports - Yara rules and indicators of known malware (know whats going on in the world)
Use a checklist for starting point
Compliance and safety officers should review the IR plan.
Incident Response Team
Senior Technical staff
Lead and Forensics Analysts
Scribe(s)
Stakeholders from: - Corporate IT - Control Systems - Subject Matter Experts - Public Relations - Legal Counsel - Law Enforcement (if necessary) - IT and/or financial auditors (optional)
Identification
Starts when incident is detected (snort/log alert?)
Forensics tools
Use the intelligence gathered
Thorough analysis of logs and network traffic
Containment
Find the call back addresses
Stop the information flow leaving the network
Stop the malware from spreading
Clean-up and Recovery
Remediation
Intrusion Clean-up
Affected system back-in service
Follow-up
Incident report
Lessons Learned - Update incident response plan - update threat intelligence - Implement new security initiatives
Network Forensics
Main purpose: Incident response and Law Enforcement
Items to analyse in packet Captures - Pattern matching - match specific values - Conversations - identify all sessions of interest - Exports: export sessions of interest
Tools used in network forensics - Wireshark, Network Miner, Tcpdump/windump, tcpflow, tcpxtract, argus, YARA, others.
YARA
Main purposes: to help identify and classify malware samples
Yara Rules - consists of a set of strings and boolean expressions - can be found in security alerts and bulletins - can be used by different security tools
Protocols
Modbus
Modbus protocol is a master/slave protocol: the master reads and writes slaves’ registers.
Modbus RTU is usually used via RS-485 (serial network): one master is present with one or more slaves. Each slave has an unique 8-bit address.
Modbus data is used to read and write “registers” which are 16-bit long.
Holding register: 16-bit; readable and writable
Input register: 16-bit; readable
Coil (Discrete Output): 1-bit long; readable and writeable
Discrete input (Status Input): 1-bit long; readable