[cyberarkpas] Collect monitoring data

[chrisberkhout]

Proposed commit message

[cyberarkpas] Collect monitoring data

Has the `audit` data stream collect monitoring data and route it to the
`monitor` data stream.

A new dashboard is added to visualize monitoring information.

Author's Checklist

• Confirm that elastic-json-v1.0.xsl will generate syslog data as expected, or adjust it to do so (the prefix before the JSON data may need to be built manually for monitor data). ➙ It will run successfully and generate JSON without a syslog prefix.

Discussion

Elastic's stylesheet elastic-json-v1.0.xsl will translate audit records into a syslog prefix and a JSON suffix. For monitor records it will run successfully and generate the JSON with no prefix. That is what the monitor data stream expects.

Expand for a procedure to verify how elastic-json-v1.0.xsl processes monitor records.

---

Take the elastic-json-v1.0.xsl file, edit it to use a non-local source for the RFC5424Changes.xsl file:

diff --git a/elastic-json-v1.0.xsl b/elastic-json-v1.0.xsl
index 3c6993f..1ef9992 100644
--- a/elastic-json-v1.0.xsl
+++ b/elastic-json-v1.0.xsl
@@ -1,6 +1,6 @@
 <?xml version="1.0" encoding="UTF-8"?>
 <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
-<xsl:import href="./Syslog/RFC5424Changes.xsl"/>
+<xsl:import href="https://gist.githubusercontent.com/chrisberkhout/53cf44d71cd8ff1cf677fa3445afde78/raw/c33fd09f1c26d9bc5b45e76014e7c81e2949f756/RFC5424Changes.xsl"/>
 <xsl:output method='text' version='1.0' encoding='UTF-8' indent='no'/>
 
 <!-- version control variables -->

Open Free Online XSLT Test Tool and paste the XSL file into the middle box.

In the top box, put the following monitor record, press "Run Transformation" and inspect the generated JSON.

<syslog>
    <monitor_record>
        <Timestamp>Jun 25 10:47:19</Timestamp>
        <IsoTimestamp>2013-06-25T10:47:19Z</IsoTimestamp>
        <Hostname>VAULT</Hostname>
        <Vendor>Cyber-Ark</Vendor>
        <Product>VaultMonitor</Product>
        <Version>9.95.0000</Version>
        <AverageExecutionTime>6</AverageExecutionTime>
        <MaxExecutionTime>343</MaxExecutionTime>
        <AverageQueueTime>1</AverageQueueTime>
        <MaxQueueTime>2</MaxQueueTime>
        <NumberOfParallelTasks>1</NumberOfParallelTasks>
        <MaxParallelTasks>20</MaxParallelTasks>
        <TransactionCount>180</TransactionCount>
        <CPUUsage>0</CPUUsage>
        <MemoryUsage>10</MemoryUsage>
        <DriveFreeSpaceInGB>686</DriveFreeSpaceInGB>
        <DriveTotalSpaceInGB>688</DriveTotalSpaceInGB>
        <SyslogQueueSize>0</SyslogQueueSize>
    </monitor_record>
</syslog>

Try the following audit record to compare, and notice that it does generate a prefix.

<syslog>
    <audit_record>
        <Rfc5424>yes</Rfc5424>
        <Timestamp>Mar 14 05:57:21</Timestamp>
        <IsoTimestamp>2021-03-14T12:57:21Z</IsoTimestamp>
        <Hostname>VAULT</Hostname>
        <Vendor>Cyber-Ark</Vendor>
        <Product>Vault</Product>
        <Version>11.7.0000</Version>
        <MessageID>180</MessageID>
        <Desc>Add User</Desc>
        <Severity>Info</Severity>
        <Issuer>Administrator</Issuer>
        <Action>Add User</Action>
        <SourceUser>PSMP_ADB_asr-cyberark-psm-ssh</SourceUser>
        <TargetUser></TargetUser>
        <Safe></Safe>
        <File></File>
        <Station>67.43.156.15</Station>
        <Location></Location>
        <Category></Category>
        <RequestId></RequestId>
        <Reason></Reason>
        <ExtraDetails></ExtraDetails>
        <Message>Add User</Message>
        <GatewayStation></GatewayStation>
    </audit_record>
</syslog>

---

The dashboard suggested in #11320 comes from jcreameriii/PAS-APM-Dashboard-Package-for-Splunk and does custom data generation for the service status row at the top. I have built a similar dashboard, but using only the monitoring data provided by CyberArk PAS.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

How to test this PR locally

The monitor data stream has pipeline tests.

It's not currently possible to system test monitor because its manifest doesn't have a policy template. New test assertions will be possible when elastic/elastic-package#2109 is implemented.

The system test for audit will ingest monitoring data and route it to logs-cyberarkpas.monitor-*. Running the audit system test with --defer-cleanup 10m allows manual inspection of this.

Related issues

• Closes [CyberArk PAS] Add support for monitoring data #11320

Screenshots

The new dashboard:

[jamiehynds]

Confirm that elastic-json-v1.0.xsl will generate syslog data as expected, or adjust it to do so (the prefix before the JSON data may need to be built manually for monitor data).

@chrisberkhout I believe SendMonitoringMessage needs to be set to Yes within the .xsl file in order for CyberArk to output the monitoring data to us. Maybe worth a version bump to v1.1 too.

[chrisberkhout]

Confirm that elastic-json-v1.0.xsl will generate syslog data as expected, or adjust it to do so (the prefix before the JSON data may need to be built manually for monitor data).

@chrisberkhout I believe SendMonitoringMessage needs to be set to Yes within the .xsl file in order for CyberArk to output the monitoring data to us. Maybe worth a version bump to v1.1 too.

The SendMonitoringMessage=yes option is set in the DBPARM.ini file, so I covered that in the README.

Regarding elastic-json-v1.0.xsl, I looked into it and wrote up some more detail in #11320 (comment). It may be that no changes are needed but I'll need to get access to CyberArk's ./Syslog/RFC5424Changes.xsl file or a running server to confirm.

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[elastic-vault-github-plugin-prod]

🚀 Benchmarks report

Package cyberarkpas 👍(1) 💚(0) 💔(1)

Expand to view

Data stream	Previous EPS	New EPS	Diff (%)	Result
audit	2739.73	1319.26	-1420.47 (-51.85%)	💔

To see the full report comment with /test benchmark fullreport

[efd6]

Nits only, then LGTM

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 0d518ca

History

• 💚 Build #17566 succeeded dd5f0d3
• 💚 Build #17544 succeeded 1da6a3e
• 💔 Build #17543 failed 70d0190
• 💔 Build #17404 failed 6f17377

cc @chrisberkhout

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
93.9% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

[elastic-vault-github-plugin-prod]

Package cyberarkpas - 2.24.0 containing this change is available at https://epr.elastic.co/search?package=cyberarkpas