In my previous post, I explored the basics of integrating Ubiquiti Dream Machine Pro logs with Microsoft Sentinel, setting the stage for advanced network monitoring and security analysis. Today, we’re taking a significant leap forward by incorporating the threat detection capabilities of the device, which is using Suricata, an open-source network threat detection tool. This extension is enabled as soon IDS or IPS is enabled in the Unifi Admin interface. But sadly, Ubiquiti is not forwarding the detections to syslog by default.
Setting the Stage for Threat Detection
Suricata offers a robust framework for monitoring network traffic in real-time, identifying potential threats, and logging them for further analysis. Configuring Suricata on the Ubiquiti Dream Machine Pro requires a bit of under-the-hood changes, specifically with one of the device’s configuration files via SSH.
Configuring Suricata on Ubiquiti Dream Machine Pro
The key to enabling Suricata’s threat detection lies in manipulating its configuration file, located at “/usr/share/ubios-udapi-server/ips/config/suricata_ubios_high.yaml”. By accessing this file, users can adjust Suricata to their specific security monitoring needs. The simplest is by using the VI editor.
vi suricata_ubios_high.yamlFor non linux admins:
- Press “INSERT” to enter the edit mode
- Adjust the lines as mentioned in the next section (lines 28-46 and 83-87)
- Press Esc to enter Command mode
- Type :wq to write and quit the file.
%YAML 1.1
---
runmode: workers
include: /run/ips/config/homenet.yaml
include: /run/ips/config/rules.yaml
classification-file: /usr/share/ubios-udapi-server/ips/config/classification.config
reference-config-file: /usr/share/ubios-udapi-server/ips/config/reference.config
threshold-file: /run/ips/config/threshold.config
default-log-dir: /run/utm/
stats:
enabled: no
interval: 60
outputs:
- eve-log:
enabled: no
filetype: unix_dgram #regular|syslog|unix_dgram|unix_stream|redis
filename: eve_stat.json
types:
- stats:
totals: yes # stats for all threads merged together
threads: no # per thread stats
deltas: yes # include delta values
- eve-log:
enabled: yes
filetype: syslog #regular|syslog|unix_dgram|unix_stream|redis
identity: "suricata"
facility: local5
types:
- alert:
payload: no # enable dumping payload in Base64
payload-buffer-size: 4kb # max size of payload buffer to output in eve-log
http-body: no # enable dumping of http fields
tagged-packets: yes
- http:
extended: yes
- tls:
extended: yes
- dns:
query: yes # enable logging of DNS queries
answer: no # enable logging of DNS answers
custom: [a]
- eve-log:
enabled: yes
filetype: unix_dgram #regular|syslog|unix_dgram|unix_stream|redis
filename: eve_alert.json
types:
- alert:
payload: no # enable dumping payload in Base64
payload-buffer-size: 4kb # max size of payload buffer to output in eve-log
http-body: no # enable dumping of http fields
tagged-packets: yes
- http:
extended: yes
- tls:
extended: yes
- dns:
query: yes # enable logging of DNS queries
answer: no # enable logging of DNS answers
custom: [a]
- drop:
alerts: yes # log alerts that caused drops
flows: start # start or all: 'start' logs only a single drop
# per flow direction. All logs each dropped pkt.
- files:
logging:
default-log-level: notice
default-output-filter:
outputs:
- console:
enabled: no
# type: json
- file:
enabled: yes
level: info
filename: /run/ips/suricata.log
# type: json
- syslog:
enabled: yes
facility: local5
format: "[%i] <%d> -- "
# type: json
app-layer:
protocols:
tls:
enabled: yes
detection-ports:
dp: 443
#no-reassemble: yes
# bypass encryption packets in the kernel or in hardware
# ref: https://github.com/OISF/suricata/blob/96b48d71042c282a51c83607eb8e25a1bf1e4bdb/suricata.yaml.in#L885
encrypt-handling: bypass
dcerpc:
enabled: detection-only
ftp:
enabled: detection-only
memcap: 32mb
tftp:
enabled: detection-only
ssh:
enabled: detection-only
smtp:
enabled: yes
mime:
decode-mime: yes
decode-base64: yes
decode-quoted-printable: yes
header-value-depth: 2000
extract-urls: yes
body-md5: no
inspected-tracker:
content-limit: 100000
content-inspect-min-size: 32768
content-inspect-window: 4096
imap:
enabled: no
msn:
enabled: no
smb:
enabled: yes
detection-ports:
dp: 139, 445
smb2:
enabled: no
dns:
#global-memcap: 16mb
#state-memcap: 512kb
#request-flood: 500
tcp:
enabled: yes
detection-ports:
dp: 53
udp:
enabled: yes
detection-ports:
dp: 53
http:
enabled: yes
memcap: 32mb
libhtp:
default-config:
personality: IDS
request-body-limit: 100kb
response-body-limit: 100kb
request-body-minimal-inspect-size: 32kb
request-body-inspect-window: 4kb
response-body-minimal-inspect-size: 40kb
response-body-inspect-window: 16kb
response-body-decompress-layer-limit: 2
http-body-inline: auto
#randomize-inspection-sizes: yes
#randomize-inspection-range: 10
double-decode-path: no
double-decode-query: no
http2:
enabled: yes
modbus:
enabled: no
enip:
enabled: no
dnp3:
enabled: no
nfs:
enabled: no
ikev2:
enabled: no
krb5:
enabled: no
dhcp:
enabled: no
ntp:
enabled: no
snmp:
enabled: no
sip:
enabled: no
rfb:
enabled: no
mqtt:
enabled: no
rdp:
enabled: no
asn1-max-frames: 256
#run-as:
# user: suri
# group: suri
sensor-name: usg-sensor
pid-file: /var/run/suricata.pid
#daemon-directory: "/"
coredump:
max-dump: unlimited
host-mode: auto
max-pending-packets: 1000
#default-packet-size: 1514
legacy:
uricontent: enabled
action-order:
- reject
- pass
- drop
- alert
engine-analysis:
rules-fast-pattern: yes
rules: yes
pcre:
match-limit: 2000
match-limit-recursion: 1000
host-os-policy:
windows: [0.0.0.0/0]
bsd: []
bsd-right: []
old-linux: []
linux: []
old-solaris: []
solaris: []
hpux10: []
hpux11: []
irix: []
macos: []
vista: []
windows2k3: []
defrag:
memcap: 8mb
hash-size: 32768
trackers: 32767 # number of defragmented flows to follow
max-frags: 32767 # number of fragments to keep (higher than trackers)
prealloc: yes
timeout: 60
flow:
memcap: 16mb
hash-size: 65536
prealloc: 500
emergency-recovery: 30
#managers: 1 # default to one flow manager
#recyclers: 1 # default to one flow recycler thread
vlan:
use-for-tracking: true
flow-timeouts:
default:
new: 30
established: 300
closed: 0
bypassed: 100
emergency-new: 10
emergency-established: 100
emergency-closed: 0
emergency-bypassed: 50
tcp:
new: 60
established: 600
closed: 60
bypassed: 100
emergency-new: 5
emergency-established: 100
emergency-closed: 10
emergency-bypassed: 50
udp:
new: 30
established: 300
bypassed: 100
emergency-new: 10
emergency-established: 100
emergency-bypassed: 50
icmp:
new: 30
established: 300
bypassed: 100
emergency-new: 10
emergency-established: 100
emergency-bypassed: 50
stream:
memcap: 16mb
checksum-validation: yes # reject wrong csums
inline: auto # auto will use inline mode in IPS mode, yes or no set it statically
bypass: yes # bypass packets when stream.reassembly.depth is reached
reassembly:
memcap: 8mb
depth: 512kb # reassemble 512kb into a stream
toserver-chunk-size: 1280
toclient-chunk-size: 1280
randomize-chunk-size: yes
host:
hash-size: 4096
prealloc: 500
memcap: 4mb
detect:
profile: low # tend to consume CPU
custom-values:
toclient-groups: 3
toserver-groups: 25
sgh-mpm-context: auto
inspection-recursion-limit: 1000
#delayed-detect: yes
prefilter:
default: mpm
grouping:
#tcp-whitelist: 53, 80, 139, 443, 445, 1433, 3306, 3389, 6666, 6667, 8080
#udp-whitelist: 53, 135, 5060
profiling:
#inspect-logging-threshold: 200
grouping:
dump-to-disk: false
include-rules: false # very verbose
include-mpm-stats: false
mpm-algo: ac
spm-algo: bm
threading:
set-cpu-affinity: yes
cpu-affinity:
- management-cpu-set:
cpu: [ 0 ] # include only these cpus in affinity settings
- receive-cpu-set:
cpu: [ "all" ] # include only these cpus in affinity settings
- worker-cpu-set:
cpu: [ "all" ]
prio:
default: "high"
- verdict-cpu-set:
cpu: [ 1 ]
prio:
default: "high"
detect-thread-ratio: 1.0
nfq:
mode: repeat
repeat-mark: 1
repeat-mask: 1
bypass-mark: 1
bypass-mask: 1
fail-open: yes
include: /run/ips/config/iface.yaml
Eve-log: Dual Output with a Caveat
A pivotal feature for forwarding threat detection logs is Suricata’s “eve-log” functionality, which allows data to be sent to the local syslog daemon. This capability is crucial not only for internal logging purposes but also for leveraging Ubiquiti services that prepare the data for the web interface. Interestingly, “eve-log” can be configured to output data in multiple formats simultaneously, enhancing the versatility of logging options.
However, it’s important to note that while “eve-log” can have multiple outputs, the logging of dropped packets (“drops”) is limited to a single instance. This limitation means that while detections are logged, information on whether these threats are subsequently blocked on follow-up attempts may not be readily available. Users must trust in the configured security measures to effectively mitigate these threats.
Suricata Operational Logs
In addition to threat detection logs, Suricata’s operational logs can also be directed to the local syslog daemon. These logs are invaluable for troubleshooting and provide insights into the inner workings and performance of Suricata within the network infrastructure.
First Result
After applying the necessary configuration changes, the new logs start flowing into Microsoft Sentinel.
The Path to Microsoft Sentinel Integration
However, a challenge arises: the logs, now augmented with Suricata’s threat detection data, are not automatically parsed by the existing “UbiquitiAuditEvent” function in Sentinel and we need to extend it with custom processing to fully leverage the new data. There could even be the discussion if these dedicated logs should not flow into a separate table which would allow for example different retention policy. I decided in this blog to leave it as it is and parse them when using with a own workspace function which I called “UbiquitiSuricataAlert”.
Ubiquiti_CL
| where facility == 21
| extend MessageJson= trim_start(@"DreamMachinePro suricata\[[0-9]*\]: ",message)
| extend SrcMacAddr = tostring(parse_json(MessageJson).src_mac)
| extend SrcIpAddr = tostring(parse_json(MessageJson).src_ip)
| extend SrcPortNumber = tostring(parse_json(MessageJson).src_port)
| extend DstIpAddr = tostring(parse_json(MessageJson).dest_ip)
| extend DstMacAddr = tostring(parse_json(MessageJson).dest_mac)
| extend DstPortNumber = tostring(parse_json(MessageJson).dest_port)
| extend FlowId = tostring(parse_json(MessageJson).flow_id)
| extend NetworkProtocol = tostring(parse_json(MessageJson).proto)
| extend DvcInboundInterface = tostring(parse_json(MessageJson).in_iface)
| extend EventVendor = "Ubiquiti-Suricata"
| extend EventCategory = tostring(parse_json(MessageJson).event_type)
| extend Service = tostring(parse_json(MessageJson).app_protocol)
| extend DvcAction = tostring(parse_json(MessageJson).alert.action)
| extend AlertMetadata = tostring(parse_json(MessageJson).alert.metadata)
| extend AlertSignature = tostring(parse_json(MessageJson).alert.signature)
| extend AlertCategory = tostring(parse_json(MessageJson).alert.category)
| extend DvcIpAddr = host
| project TimeGenerated
, EventVendor
, EventCategory
, AlertMetadata
, AlertSignature
, Service
, FlowId
, DvcInboundInterface
, DvcAction
, SrcMacAddr
, SrcIpAddr
, SrcPortNumber
, DstMacAddr
, DstIpAddr
, DstPortNumber
, NetworkProtocol
, DvcIpAddr
Creating Alerts based on the detections
Now, we have the data prepared and would like to show the detections also as Alerts in Sentinel. For this we need to create an Analytic Rule. On the first page we can define a name and description for the detection. As plenty of different alerts with variable Tactics and Techniques we have should leverage a generic name.
The rule logic step contains the mani possibilities. As we have created a Workspace Function in the step before we can now leverage a very simple query which will just create a detection on each occurence.
In the Alert enrichment section we can map the source and destination IP of the alert as an IP entity.
We can also include specific columns in the alert details which can help our security analysts during investigation. I add the signature and the complete AlertMetadata which is a json object with different information about the alert.
Then as every Suricata alert could be a different detection we can leverage the possibility of Microsoft Sentinel to set the title to the threat signature.
As I want the alert popping up on my dashboard I choose a schedule of 15minutes. Potentially I should choose 5 minutes as the Ubiquiti Dream Machine Pro is by default blocking traffic of a threat detection for 5 minutes. Going down to 5 minutes could help if I would create an automation which then can create a firewall rule for a longer time.
As each row represents one detection I also want that it creates for each result an own alert which ends in a 1:1 relationship.
By clicking on “Test with current data” we can see how many alerts would be created in the past 50 execution. As you can see there are plenty of detection just in my home lab.
The next page in the Analytic rule wizard is in my opinion one of the most important. In my example I would like to create a incident for detections, but if attacks are always from one IP and to the same IP then it belongs to the same attack and can be combined in one incident and handled by a Security Analyst in one step.
I skipped now the Automated response part which in this example could be for example automatically creating firewall rules to block the specific source if external or disabling a switch port for internal IP’s. Showcasing such scenarios will be part of one of my next blogs.
After a while the first Incidents are created in Microsoft Sentinel and as you can see the incident combines multiple alerts and is enriched with important information.
Conclusion
Integrating Suricata’s threat detection capabilities of Ubiquiti Dream Machine Pro with Microsoft Sentinel marks a significant step forward in network security monitoring in my lab.
The journey towards enhanced security is ongoing, and with the right tools and configurations, organizations can better protect their digital assets in an ever-evolving threat environment. Sadly that Ubiquiti is not sending this information by default to the configured Syslog server.
- Microsoft Sentinel ASIM Parser demystified - March 31, 2024
- Enhancing Network Security Insights with IDS/IPS of Ubiquiti Dream Machine Pro and Microsoft Sentinel - March 10, 2024
- Ubiquiti Dream Machine Pro Logs to Microsoft Sentinel - February 6, 2024
2 Comments
Microsoft Roadmap, messagecenter en blogs updates van 12-03-2024 - KbWorks · March 12, 2024 at 08:24
[…] Enhancing Network Security Insights with IDS/IPS of Ubiquiti Dream Machine Pro and Microsoft Sentine… […]
Microsoft Sentinel ASIM Parser demystified - Workplace Ninja's · March 31, 2024 at 18:57
[…] ingest the log data. So, check out my previous blogs about how to ingest Ubiquiti Dreammachine Pro including the Suricata Logs into Sentinel. At the end the two required functions are simple and mainly consist of parsing and […]