Skip to content

Commit e0b6f01

Browse files
efd6mergify[bot]
efd6
authored and
mergify[bot]
committed
x-pack/filebeat/module/juniper: fix handling of jrx structured data (#36308)
Previously, the leading word was ignored. I all our test cases this was in the form /junos@(\d+\.){5}\d+/. When this value is not present, we lose the first structured data value, so be more careful in assessing whether the first element should be discarded. (cherry picked from commit e66e0cb)
1 parent f6864c8 commit e0b6f01

File tree

4 files changed

+60
-28
lines changed

4 files changed

+60
-28
lines changed

‎CHANGELOG.next.asciidoc

+1-27
Original file line numberDiff line numberDiff line change
@@ -95,6 +95,7 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
9595
- Fix panic when sqs input metrics getter is invoked {pull}36101[36101] {issue}36077[36077]
9696
- Make CEL input's `now` global variable static for evaluation lifetime. {pull}36107[36107]
9797
- Fix handling of TCP/UDP address resolution during metric initialization. {issue}35064[35064] {pull}36287[36287]
98+
- Fix handling of Juniper SRX structured data when there is no leading junos element. {issue}36270[36270] {pull}36308[36308]
9899

99100
*Heartbeat*
100101

@@ -124,33 +125,6 @@ automatic splitting at root level, if root level element is an array. {pull}3415
124125
- Enable heartbeat-wide publish timeout setting with run_once. {pull}35721[35721]
125126
- Added default timezone UTC to heartbeat docker images to fix synthetics journeys navigation errors. {pull}36193[36193]
126127

127-
*Heartbeat*
128-
129-
130-
*Heartbeat*
131-
132-
133-
*Heartbeat*
134-
135-
136-
*Heartbeat*
137-
138-
139-
*Auditbeat*
140-
141-
142-
*Filebeat*
143-
144-
145-
*Auditbeat*
146-
147-
148-
*Filebeat*
149-
150-
151-
*Heartbeat*
152-
153-
154128
*Metricbeat*
155129

156130
- in module/windows/perfmon, changed collection method of the second counter value required to create a displayable value {pull}32305[32305]

‎x-pack/filebeat/module/juniper/srx/ingest/pipeline.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ processors:
55
- grok:
66
field: message
77
patterns:
8-
- '^<%{POSINT:syslog_pri}>(\d{1,3}\s)?(?:%{TIMESTAMP_ISO8601:_temp_.raw_date})\s%{SYSLOGHOST:syslog_hostname}\s%{PROG:syslog_program}\s(?:%{POSINT:syslog_pid}|-)?\s%{WORD:log_type}\s\[.+?\s%{GREEDYDATA:log.original}\]$'
8+
- '^<%{POSINT:syslog_pri}>(\d{1,3}\s)?(?:%{TIMESTAMP_ISO8601:_temp_.raw_date})\s%{SYSLOGHOST:syslog_hostname}\s%{PROG:syslog_program}\s(?:%{POSINT:syslog_pid}|-)?\s%{WORD:log_type}\s\[(?:[^=]+\s)?%{GREEDYDATA:log.original}\]$'
99
1010
# split Juniper-SRX fields
1111
- kv:

‎x-pack/filebeat/module/juniper/srx/test/flow.log

+1
Original file line numberDiff line numberDiff line change
@@ -23,3 +23,4 @@
2323
<14>1 2020-07-13T16:43:05.041Z SRX100HM RT_FLOW - RT_FLOW_SESSION_CLOSE [[email protected] reason="idle Timeout" source-address="10.1.1.100" source-port="64720" destination-address="91.228.167.172" destination-port="8883" connection-tag="0" service-name="None" nat-source-address="172.19.34.100" nat-source-port="24519" nat-destination-address="91.228.167.172" nat-destination-port="8883" nat-connection-tag="0" src-nat-rule-type="source rule" src-nat-rule-name="our-nat-rule" dst-nat-rule-type="N/A" dst-nat-rule-name="N/A" protocol-id="6" policy-name="default-permit" source-zone-name="trust" destination-zone-name="untrust" session-id-32="3851" packets-from-client="161" bytes-from-client="9530" packets-from-server="96" bytes-from-server="9670" elapsed-time="23755" application="UNKNOWN" nested-application="UNKNOWN" username="N/A" roles="N/A" packet-incoming-interface="ge-0/0/1.0" encrypted="UNKNOWN" application-category="N/A" application-sub-category="N/A" application-risk="1" application-characteristics="N/A" secure-web-proxy-session-type="NA" peer-session-id="0" peer-source-address="0.0.0.0" peer-source-port="0" peer-destination-address="0.0.0.0" peer-destination-port="0" hostname="NA NA" src-vrf-grp="N/A" dst-vrf-grp="N/A"]
2424
<14>1 2020-07-13T16:12:05.530Z SRX100HM RT_FLOW - RT_FLOW_SESSION_CREATE [[email protected] source-address="10.1.1.100" source-port="49583" destination-address="175.16.199.1" destination-port="53" connection-tag="0" service-name="junos-dns-udp" nat-source-address="172.19.34.100" nat-source-port="30838" nat-destination-address="175.16.199.1" nat-destination-port="53" nat-connection-tag="0" src-nat-rule-type="source rule" src-nat-rule-name="our-nat-rule" dst-nat-rule-type="N/A" dst-nat-rule-name="N/A" protocol-id="17" policy-name="default-permit" source-zone-name="trust" destination-zone-name="untrust" session-id-32="15399" username="N/A" roles="N/A" packet-incoming-interface="ge-0/0/1.0" application="UNKNOWN" nested-application="UNKNOWN" encrypted="UNKNOWN" application-category="N/A" application-sub-category="N/A" application-risk="1" application-characteristics="N/A" src-vrf-grp="N/A" dst-vrf-grp="N/A"]
2525
<14>1 2020-07-13T16:12:05.530Z SRX100HM RT_FLOW - APPTRACK_SESSION_CLOSE [[email protected] reason="Closed by junos-alg" source-address="10.1.1.100" source-port="63381" destination-address="175.16.199.1" destination-port="53" service-name="junos-dns-udp" application="UNKNOWN" nested-application="UNKNOWN" nat-source-address="172.19.34.100" nat-source-port="26764" nat-destination-address="175.16.199.1" nat-destination-port="53" src-nat-rule-name="our-nat-rule" dst-nat-rule-name="N/A" protocol-id="17" policy-name="default-permit" source-zone-name="trust" destination-zone-name="untrust" session-id-32="15361" packets-from-client="1" bytes-from-client="66" packets-from-server="1" bytes-from-server="82" elapsed-time="3" username="N/A" roles="N/A" encrypted="No" profile-name="N/A" rule-name="N/A" routing-instance="default" destination-interface-name="ge-0/0/0.0" uplink-incoming-interface-name="N/A" uplink-tx-bytes="0" uplink-rx-bytes="0" category="N/A" sub-category="N/A" apbr-policy-name="N/A" multipath-rule-name="N/A" src-vrf-grp="N/A" dst-vrf-grp="N/A"]
26+
<14>1 2023-08-08T14:28:00.778-05:00 Route1- RT_FLOW - RT_FLOW_SESSION_DENY [source-address="192.168.1.1" source-port="39017" destination-address="8.8.4.4" destination-port="53" connection-tag="0" service-name="junos-dns-udp" protocol-id="17" icmp-type="0" policy-name="dns_deny_outbound" source-zone-name="trust" destination-zone-name="untrust" application="UNKNOWN" nested-application="UNKNOWN" username="N/A" roles="N/A" packet-incoming-interface="reth0.0" encrypted="No" reason="Denied by policy" session-id="85905209174" application-category="N/A" application-sub-category="N/A" application-risk="-1" application-characteristics="N/A" src-vrf-grp="N/A" dst-vrf-grp="N/A"]

‎x-pack/filebeat/module/juniper/srx/test/flow.log-expected.json

+57
Original file line numberDiff line numberDiff line change
@@ -1867,5 +1867,62 @@
18671867
"forwarded",
18681868
"juniper.srx"
18691869
]
1870+
},
1871+
{
1872+
"@timestamp": "2023-08-08T17:28:00.778-02:00",
1873+
"client.ip": "192.168.1.1",
1874+
"client.port": 39017,
1875+
"destination.ip": "8.8.4.4",
1876+
"destination.port": 53,
1877+
"event.action": "flow_deny",
1878+
"event.category": [
1879+
"network"
1880+
],
1881+
"event.dataset": "juniper.srx",
1882+
"event.kind": "event",
1883+
"event.module": "juniper",
1884+
"event.original": "source-address=\"192.168.1.1\" source-port=\"39017\" destination-address=\"8.8.4.4\" destination-port=\"53\" connection-tag=\"0\" service-name=\"junos-dns-udp\" protocol-id=\"17\" icmp-type=\"0\" policy-name=\"dns_deny_outbound\" source-zone-name=\"trust\" destination-zone-name=\"untrust\" application=\"UNKNOWN\" nested-application=\"UNKNOWN\" username=\"N/A\" roles=\"N/A\" packet-incoming-interface=\"reth0.0\" encrypted=\"No\" reason=\"Denied by policy\" session-id=\"85905209174\" application-category=\"N/A\" application-sub-category=\"N/A\" application-risk=\"-1\" application-characteristics=\"N/A\" src-vrf-grp=\"N/A\" dst-vrf-grp=\"N/A\"",
1885+
"event.outcome": "success",
1886+
"event.risk_score": -1.0,
1887+
"event.severity": 14,
1888+
"event.timezone": "-02:00",
1889+
"event.type": [
1890+
"connection",
1891+
"denied"
1892+
],
1893+
"fileset.name": "srx",
1894+
"input.type": "log",
1895+
"juniper.srx.connection_tag": "0",
1896+
"juniper.srx.encrypted": "No",
1897+
"juniper.srx.icmp_type": "0",
1898+
"juniper.srx.process": "RT_FLOW",
1899+
"juniper.srx.reason": "Denied by policy",
1900+
"juniper.srx.service_name": "junos-dns-udp",
1901+
"juniper.srx.session_id": "85905209174",
1902+
"juniper.srx.tag": "RT_FLOW_SESSION_DENY",
1903+
"log.level": "informational",
1904+
"log.offset": 19862,
1905+
"network.iana_number": "17",
1906+
"observer.egress.zone": "untrust",
1907+
"observer.ingress.interface.name": "reth0.0",
1908+
"observer.ingress.zone": "trust",
1909+
"observer.name": "Route1-",
1910+
"observer.product": "SRX",
1911+
"observer.type": "firewall",
1912+
"observer.vendor": "Juniper",
1913+
"related.ip": [
1914+
"192.168.1.1",
1915+
"8.8.4.4"
1916+
],
1917+
"rule.name": "dns_deny_outbound",
1918+
"server.ip": "8.8.4.4",
1919+
"server.port": 53,
1920+
"service.type": "juniper",
1921+
"source.ip": "192.168.1.1",
1922+
"source.port": 39017,
1923+
"tags": [
1924+
"forwarded",
1925+
"juniper.srx"
1926+
]
18701927
}
18711928
]

0 commit comments

Comments
 (0)