Skip to content

Commit e318ee3

Browse files
efd6efd6
committed
x-pack/filebeat/module/juniper: fix handling of jrx structured data
Previously, the leading word was ignored. I all our test cases this was in the form /junos@(\d+\.){5}\d+/. When this value is not present, we lose the first structured data value, so be more careful in assessing whether the first element should be discarded.
1 parent 360b009 commit e318ee3

File tree

4 files changed

+60
-35
lines changed

4 files changed

+60
-35
lines changed

‎CHANGELOG.next.asciidoc

+1-34
Original file line numberDiff line numberDiff line change
@@ -113,6 +113,7 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
113113
- Update mito CEL extension library to v1.5.0. {pull}36146[36146]
114114
- Filter out duplicate paths resolved from matching globs. {issue}36253[36253] {pull}36256[36256]
115115
- Fix handling of TCP/UDP address resolution during metric initialization. {issue}35064[35064] {pull}36287[36287]
116+
- Fix handling of Juniper SRX structured data when there is no leading junos element. {issue}36270[36270] {pull}[]
116117

117118
*Heartbeat*
118119

@@ -142,33 +143,6 @@ automatic splitting at root level, if root level element is an array. {pull}3415
142143
- Enable heartbeat-wide publish timeout setting with run_once. {pull}35721[35721]
143144
- Added default timezone UTC to heartbeat docker images to fix synthetics journeys navigation errors. {pull}36193[36193]
144145

145-
*Heartbeat*
146-
147-
148-
*Heartbeat*
149-
150-
151-
*Heartbeat*
152-
153-
154-
*Heartbeat*
155-
156-
157-
*Auditbeat*
158-
159-
160-
*Filebeat*
161-
162-
163-
*Auditbeat*
164-
165-
166-
*Filebeat*
167-
168-
169-
*Heartbeat*
170-
171-
172146
*Metricbeat*
173147

174148
- in module/windows/perfmon, changed collection method of the second counter value required to create a displayable value {pull}32305[32305]
@@ -200,13 +174,6 @@ automatic splitting at root level, if root level element is an array. {pull}3415
200174

201175
- Fix powershell details regexp to prevent excessive backtracking when processing command invocations. {pull}36178[36178]
202176

203-
*Functionbeat*
204-
205-
206-
*Functionbeat*
207-
208-
209-
210177
*Elastic Logging Plugin*
211178

212179

‎x-pack/filebeat/module/juniper/srx/ingest/pipeline.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ processors:
55
- grok:
66
field: message
77
patterns:
8-
- '^<%{POSINT:syslog_pri}>(\d{1,3}\s)?(?:%{TIMESTAMP_ISO8601:_temp_.raw_date})\s%{SYSLOGHOST:syslog_hostname}\s%{PROG:syslog_program}\s(?:%{POSINT:syslog_pid}|-)?\s%{WORD:log_type}\s\[.+?\s%{GREEDYDATA:log.original}\]$'
8+
- '^<%{POSINT:syslog_pri}>(\d{1,3}\s)?(?:%{TIMESTAMP_ISO8601:_temp_.raw_date})\s%{SYSLOGHOST:syslog_hostname}\s%{PROG:syslog_program}\s(?:%{POSINT:syslog_pid}|-)?\s%{WORD:log_type}\s\[(?:[^=]+\s)?%{GREEDYDATA:log.original}\]$'
99
1010
# split Juniper-SRX fields
1111
- kv:

‎x-pack/filebeat/module/juniper/srx/test/flow.log

+1
Original file line numberDiff line numberDiff line change
@@ -23,3 +23,4 @@
2323
<14>1 2020-07-13T16:43:05.041Z SRX100HM RT_FLOW - RT_FLOW_SESSION_CLOSE [[email protected] reason="idle Timeout" source-address="10.1.1.100" source-port="64720" destination-address="91.228.167.172" destination-port="8883" connection-tag="0" service-name="None" nat-source-address="172.19.34.100" nat-source-port="24519" nat-destination-address="91.228.167.172" nat-destination-port="8883" nat-connection-tag="0" src-nat-rule-type="source rule" src-nat-rule-name="our-nat-rule" dst-nat-rule-type="N/A" dst-nat-rule-name="N/A" protocol-id="6" policy-name="default-permit" source-zone-name="trust" destination-zone-name="untrust" session-id-32="3851" packets-from-client="161" bytes-from-client="9530" packets-from-server="96" bytes-from-server="9670" elapsed-time="23755" application="UNKNOWN" nested-application="UNKNOWN" username="N/A" roles="N/A" packet-incoming-interface="ge-0/0/1.0" encrypted="UNKNOWN" application-category="N/A" application-sub-category="N/A" application-risk="1" application-characteristics="N/A" secure-web-proxy-session-type="NA" peer-session-id="0" peer-source-address="0.0.0.0" peer-source-port="0" peer-destination-address="0.0.0.0" peer-destination-port="0" hostname="NA NA" src-vrf-grp="N/A" dst-vrf-grp="N/A"]
2424
<14>1 2020-07-13T16:12:05.530Z SRX100HM RT_FLOW - RT_FLOW_SESSION_CREATE [[email protected] source-address="10.1.1.100" source-port="49583" destination-address="175.16.199.1" destination-port="53" connection-tag="0" service-name="junos-dns-udp" nat-source-address="172.19.34.100" nat-source-port="30838" nat-destination-address="175.16.199.1" nat-destination-port="53" nat-connection-tag="0" src-nat-rule-type="source rule" src-nat-rule-name="our-nat-rule" dst-nat-rule-type="N/A" dst-nat-rule-name="N/A" protocol-id="17" policy-name="default-permit" source-zone-name="trust" destination-zone-name="untrust" session-id-32="15399" username="N/A" roles="N/A" packet-incoming-interface="ge-0/0/1.0" application="UNKNOWN" nested-application="UNKNOWN" encrypted="UNKNOWN" application-category="N/A" application-sub-category="N/A" application-risk="1" application-characteristics="N/A" src-vrf-grp="N/A" dst-vrf-grp="N/A"]
2525
<14>1 2020-07-13T16:12:05.530Z SRX100HM RT_FLOW - APPTRACK_SESSION_CLOSE [[email protected] reason="Closed by junos-alg" source-address="10.1.1.100" source-port="63381" destination-address="175.16.199.1" destination-port="53" service-name="junos-dns-udp" application="UNKNOWN" nested-application="UNKNOWN" nat-source-address="172.19.34.100" nat-source-port="26764" nat-destination-address="175.16.199.1" nat-destination-port="53" src-nat-rule-name="our-nat-rule" dst-nat-rule-name="N/A" protocol-id="17" policy-name="default-permit" source-zone-name="trust" destination-zone-name="untrust" session-id-32="15361" packets-from-client="1" bytes-from-client="66" packets-from-server="1" bytes-from-server="82" elapsed-time="3" username="N/A" roles="N/A" encrypted="No" profile-name="N/A" rule-name="N/A" routing-instance="default" destination-interface-name="ge-0/0/0.0" uplink-incoming-interface-name="N/A" uplink-tx-bytes="0" uplink-rx-bytes="0" category="N/A" sub-category="N/A" apbr-policy-name="N/A" multipath-rule-name="N/A" src-vrf-grp="N/A" dst-vrf-grp="N/A"]
26+
<14>1 2023-08-08T14:28:00.778-05:00 Route1- RT_FLOW - RT_FLOW_SESSION_DENY [source-address="192.168.1.1" source-port="39017" destination-address="8.8.4.4" destination-port="53" connection-tag="0" service-name="junos-dns-udp" protocol-id="17" icmp-type="0" policy-name="dns_deny_outbound" source-zone-name="trust" destination-zone-name="untrust" application="UNKNOWN" nested-application="UNKNOWN" username="N/A" roles="N/A" packet-incoming-interface="reth0.0" encrypted="No" reason="Denied by policy" session-id="85905209174" application-category="N/A" application-sub-category="N/A" application-risk="-1" application-characteristics="N/A" src-vrf-grp="N/A" dst-vrf-grp="N/A"]

‎x-pack/filebeat/module/juniper/srx/test/flow.log-expected.json

+57
Original file line numberDiff line numberDiff line change
@@ -1867,5 +1867,62 @@
18671867
"forwarded",
18681868
"juniper.srx"
18691869
]
1870+
},
1871+
{
1872+
"@timestamp": "2023-08-08T17:28:00.778-02:00",
1873+
"client.ip": "192.168.1.1",
1874+
"client.port": 39017,
1875+
"destination.ip": "8.8.4.4",
1876+
"destination.port": 53,
1877+
"event.action": "flow_deny",
1878+
"event.category": [
1879+
"network"
1880+
],
1881+
"event.dataset": "juniper.srx",
1882+
"event.kind": "event",
1883+
"event.module": "juniper",
1884+
"event.original": "source-address=\"192.168.1.1\" source-port=\"39017\" destination-address=\"8.8.4.4\" destination-port=\"53\" connection-tag=\"0\" service-name=\"junos-dns-udp\" protocol-id=\"17\" icmp-type=\"0\" policy-name=\"dns_deny_outbound\" source-zone-name=\"trust\" destination-zone-name=\"untrust\" application=\"UNKNOWN\" nested-application=\"UNKNOWN\" username=\"N/A\" roles=\"N/A\" packet-incoming-interface=\"reth0.0\" encrypted=\"No\" reason=\"Denied by policy\" session-id=\"85905209174\" application-category=\"N/A\" application-sub-category=\"N/A\" application-risk=\"-1\" application-characteristics=\"N/A\" src-vrf-grp=\"N/A\" dst-vrf-grp=\"N/A\"",
1885+
"event.outcome": "success",
1886+
"event.risk_score": -1.0,
1887+
"event.severity": 14,
1888+
"event.timezone": "-02:00",
1889+
"event.type": [
1890+
"connection",
1891+
"denied"
1892+
],
1893+
"fileset.name": "srx",
1894+
"input.type": "log",
1895+
"juniper.srx.connection_tag": "0",
1896+
"juniper.srx.encrypted": "No",
1897+
"juniper.srx.icmp_type": "0",
1898+
"juniper.srx.process": "RT_FLOW",
1899+
"juniper.srx.reason": "Denied by policy",
1900+
"juniper.srx.service_name": "junos-dns-udp",
1901+
"juniper.srx.session_id": "85905209174",
1902+
"juniper.srx.tag": "RT_FLOW_SESSION_DENY",
1903+
"log.level": "informational",
1904+
"log.offset": 19862,
1905+
"network.iana_number": "17",
1906+
"observer.egress.zone": "untrust",
1907+
"observer.ingress.interface.name": "reth0.0",
1908+
"observer.ingress.zone": "trust",
1909+
"observer.name": "Route1-",
1910+
"observer.product": "SRX",
1911+
"observer.type": "firewall",
1912+
"observer.vendor": "Juniper",
1913+
"related.ip": [
1914+
"192.168.1.1",
1915+
"8.8.4.4"
1916+
],
1917+
"rule.name": "dns_deny_outbound",
1918+
"server.ip": "8.8.4.4",
1919+
"server.port": 53,
1920+
"service.type": "juniper",
1921+
"source.ip": "192.168.1.1",
1922+
"source.port": 39017,
1923+
"tags": [
1924+
"forwarded",
1925+
"juniper.srx"
1926+
]
18701927
}
18711928
]

0 commit comments

Comments
 (0)