diff --git a/filter-plugin/logstash-filter-oua-guardium/OuaOverConnectJdbcReadme.md b/filter-plugin/logstash-filter-oua-guardium/OuaOverConnectJdbcReadme.md new file mode 100644 index 000000000..0948a5052 --- /dev/null +++ b/filter-plugin/logstash-filter-oua-guardium/OuaOverConnectJdbcReadme.md @@ -0,0 +1,98 @@ +# Oracle Unified Audit Universal Connector Over JDBC Connect + +## Meet Oracle Unified Audit Over JDBC Connect +* Tested versions: 19 +* Environments: On-prem, RDS in AWS +* Supported inputs: Kafka Input (pull) +* Supported Oracle versions: 18, 19, and 21 +* Supported Guardium versions: + * Guardium Data Protection: 12.1 and above + +kafka-connect is framework for streaming data between Apache Kafka and other systems. +Detailed breakdown: +1. Kafka-connect JDBC Connector: used to pull data from `UNIFIED_AUDIT_TRAIL`. +2. Produce to Kafka: The queried data is then sent (produced) to a Kafka topic. +3. Consume with UC: The data in the Kafka topic is consumed by kafka-input plugin and process by the 'guardium-oua-uc' filter plug-in, + a specific Unified Connector designed for your use case. + +### Requirements +1. This feature currently only supported in environment with CM management 12.1 and kafka cluster +2. Unified auditing must be enabled in an Oracle database that will be monitored by this method +3. Download the Oracle JDBC driver. Download here: + https://download.oracle.com/otn-pub/otn_software/jdbc/234/ojdbc8.jar +4. Currently, this plug-in will work only on IBM Security Guardium Data Protection, not in Guardium Insights + +## Setup + +1. **Create a designated Database User for OUA UC to retrieve audit data with minimal privileges (using DBA help) as follows:** + - Assuming the name for the designated Oracle Unified Audit user with minimal permissions will be "guardium" with password "password" + - Connect to Oracle using sysdba account and execute the following commands: + + ``` + CREATE USER guardium IDENTIFIED BY password; + GRANT CONNECT, RESOURCE to guardium; + GRANT SELECT ANY DICTIONARY TO guardium; + ``` + + - To verify your new user's privileges, connect to the Oracle instance that you planning to monitor using the name and credentials for your designated user and run the following statements: + + ``` + select count(*) from AUDSYS.AUD$UNIFIED; + ``` + + - If there are no errors that means you can use this new user for this UC method + + - Apply the following policy to capture changes to system parameters: + ``` + CREATE AUDIT POLICY system_param_changes ACTIONS ALTER SYSTEM; + AUDIT POLICY system_param_changes; + ``` + + 2. **Exclude Auditing for the DB User Performing JDBC Queries + To avoid 'self-monitoring' of the database user executing JDBC queries, follow these steps to exclude the user from being audited:** + ``` + # Connect as SYSDBA + sqlplus / AS SYSDBA + + # make sure audit_trail is set to none + SHOW PARAMETER audit_trail; + + # log to PDB + ALTER SESSION SET CONTAINER = ; + + # Create a new user with connect and dictionary privilege (for example: AUDITUSER) + + CREATE USER AUDITUSER IDENTIFIED BY guardium; + GRANT CONNECT, RESOURCE to AUDITUSER; + GRANT SELECT ANY DICTIONARY TO AUDITUSER; + + # Check which policy enabled + select distinct * from AUDIT_UNIFIED_ENABLED_POLICIES; + + # Remove audit policies like ALL_ACTIONS that include + NOAUDIT POLICY ALL_ACTIONS; + + # Create ALL_ACTIONS policy if not exists + + CREATE AUDIT POLICY ALL_ACTIONS ACTIONS INSERT, SELECT, UPDATE, DELETE; + SELECT * FROM AUDIT_UNIFIED_POLICIES WHERE POLICY_NAME = ' ALL_ACTIONS'; + + # Add ALL_ACTIONS policy to all user except AUDITUSER + AUDIT POLICY ALL_ACTIONS EXCEPT "AUDITUSER"; + ``` + +### Configuring Universal Connector on Guardium Data Protection +1. ### Creating a Kafka Cluster on Guardium + For information on creating Kafka Clusters, see the [Managing Kafka clusters](https://www.ibm.com/docs/en/gdp/12.x?topic=configuration-managing-kafka-clusters) topic. +2. ### Configuring Universal Connector + For information on configuring the Universal Connector on Guardium using the new flow, see [Managing universal connector configuration](https://www.ibm.com/docs/en/gdp/12.x?topic=connector-managing-universal-configuration) topic. + +### Limitations +- Traffic is not getting captured on the Guardium report after the Oracle DB server reboot - as a temporary workaround, uninstalling and then installing the profile again will work in this case. +- Currently, the following activities are not being captured in the Guardian reports: + - Logon/Logoff + - Startup/Shutdown + - backup/restore + + We are aware of this limitation and are actively working on a resolution, which will be included in the upcoming UC version. + diff --git a/filter-plugin/logstash-filter-oua-guardium/OuaOverPipeReadme.md b/filter-plugin/logstash-filter-oua-guardium/OuaOverPipeReadme.md new file mode 100644 index 000000000..48b83201d --- /dev/null +++ b/filter-plugin/logstash-filter-oua-guardium/OuaOverPipeReadme.md @@ -0,0 +1,126 @@ +# Oracle Unified Audit Universal Connector + +## Meet Oracle Unified Audit + +* Tested versions: 18,19 +* Environments: On-prem, RDS in AWS, Oracle Autonomous Database in OCI + + **Note**: Autonomous Database in OCI is supported only by Guardium Data Protection SqlGuard-12.0p7015_Bundle_May_20_2024 and SqlGuard-11.0p545_Bundle_Jul_09_2024. +* Supported inputs: Oracle Unified Audit (pull) +* Supported Oracle versions: 18, 19, and 21 +* Supported Guardium versions: + * Guardium Data Protection: 11.4 and above + + +## Requirements + +1. Unified auditing must be enabled in an Oracle database that will be monitored by this method +2. Download the Basic Instant client package from Oracle. + **Note:** : In this release,only specific instance clients will be supported from starting v21.1.0.0.0 Download [here](https://download.oracle.com/otn_software/linux/instantclient/211000/oracle-instantclient-basic-21.1.0.0.0-1.x86_64.rpm) +3. Oracle Unified Audit Universal Connector is automatically available with Guardium Data Protection versions 12.x, 11.4 with appliance bundle 11.0p490 or later or Guardium Data Protection version 11.5 with appliance bundle 11.0p540 or later releases. +**Note**: For Guardium Data Protection version 11.4 without appliance bundle 11.0p490 or prior or Guardium Data Protection version 11.5 without appliance bundle 11.0p540 or prior, download the [OUA](https://github.com/IBM/universal-connectors/raw/release-v1.2.0/filter-plugin/logstash-filter-oua-guardium/OracleUnifiedAuditPackage/OracleUnifiedAudit/guardium-oua-uc.zip) plug-in. (Do not unzip the offline-package file throughout the procedure). + +4. A designated user for OUA UC should be created for Oracle database access. An existing user with sysdba privileges can also be used + +5. You must create a secret containing your OUA universal connector password. + - Example: `grdapi universal_connector_keystore_add key=OUA_USER_PASS password=` where `` is the OUA universal connector user’s password for the database. `OUA_USER_PASS` will be used in the plug-in configuration as a variable for the password secret. + +Currently, this plug-in will work only on IBM Security Guardium Data Protection, not Guardium Insights + +## Building + +Update the variables in Makefile for your environment's Java home and Logstash location + +## Setup + +1. Create a designated Database User for OUA UC to retrieve audit data with minimal privileges (using DBA help) as follows: + - Assuming the name for the designated Oracle Unified Audit user with minimal permissions will be "guardium" with password "password" + - Connect to Oracle using sysdba account and execute the following commands: + + - For Oracle Autonomous Database in OCI, run the following commands to create a user with the required grants: + ``` + CREATE USER guardium IDENTIFIED BY password; + GRANT CONNECT to guardium; + GRANT SELECT ANY DICTIONARY to guardium; + GRANT AUDIT_VIEWER to guardium; + GRANT SELECT ON v$INSTANCE to guardium; + GRANT SELECT ON v$DATABASE to guardium; + GRANT SELECT ON v$MYSTAT to guardium; + ``` + + - For other environments including RDS in AWS and Oracle Databases On-Premises run the following commands: + ``` + CREATE USER guardium IDENTIFIED BY password; + GRANT CONNECT, RESOURCE to guardium; + GRANT SELECT ANY DICTIONARY TO guardium; + exec DBMS_NETWORK_ACL_ADMIN.APPEND_HOST_ACE(host => 'localhost', + ace => xs$ace_type(privilege_list => xs$name_list('connect', + 'resolve'), principal_name => 'guardium', principal_type => xs_acl.ptype_db)); + ``` + + - To verify your new user's privileges, connect to the Oracle instance that you planning to monitor using the name and credentials for your designated user and run the following statements: + + ``` + select count(*) from AUDSYS.AUD$UNIFIED; + SELECT UTL_INADDR.get_host_address FROM DUAL; + ``` + + - If there are no errors that means you can use this new user for this UC method + +2. Enable the universal collector feature on the designated Guardium collectors or the stand-alone system. See [here](/docs/Guardium%20Data%20Protection/uc_config_gdp.md). + +3. On the collector, go to Setup > Tools and Views > Configure Universal Connector + +4. Click on the "UPLOAD” button and upload the downloaded `oracle-instantclient-basic-21.1.0.0.0-1.x86_64.rpm`. + +6. Then, upload the `guardium-oua-uc.zip` file. This step is not necessary for Guardium Data Protection v12.0 and later. + + 5. Click on the "+". The Connector Configuration dialog opens. + + - Type any unique name in the **Connector name** field. + + - Paste the content from the "[ouaPipe.conf](https://github.com/IBM/universal-connectors/raw/main/filter-plugin/logstash-filter-oua-guardium/ouaPipe.conf)" in the **Input configuration** field + + ``` + pipe { + type => "oua" + command => "${OUA_BINARY_PATH} -c ${THIRD_PARTY_PATH} -s ${THIRD_PARTY_PATH} -r 100 -t 1000 -p 10 -j /${OUA_USER_PASS}@:/" + add_field => {"SERVER_ADDRESS" => ""} + add_field => {"SERVER_PORT" => ""} } + ``` + + - Paste the content from the "[ouaPipe.conf](https://github.com/IBM/universal-connectors/raw/main/filter-plugin/logstash-filter-oua-guardium/ouaPipe.conf)" in the **Filter configuration** field + + ``` + if [type] == "oua" { + json { + source => "message" + } + mutate { + add_field => {"[HostName]" => "%{SERVER_ADDRESS}" } + add_field => {"[PortNumber]" => "%{SERVER_PORT}" } + } + if "_jsonparsefailure" not in [tags] { + oua_filter {} + } + } + + ``` + + - **NOTE**: The type specified for the filters must be unique among Universal Connectors and be identical in the input and filter configurations + + 6. Click **Save**. Guardium validates the new connector and enables the plug-in. After it is validated, it appears on the **Configure Universal Connector** page. + +**Note**: The following arguments are used in the OUA plug-in configuration. +- -c path : path to instantclient libraries +- -a path : path to tnsnames.ora (directory, not the filename) +- -r rows : number of rows to fetch at a single time (default 100) +- -t timeout : timeout for all operations (connect, execute, fetch, etc) specified in milliseconds (default 300000) +- -p period : period of time between passes specified in seconds (default 300) +- -s path : path to directory where state should be saved +- -j : output audits in JSON format + +## Limitation +- Normally, the "statement type" attribute for the "FULL SQL" entity in Guardium reports shows us whether a full SQL statement is a prepared statement. However, because OUA doesn't give us information about whether a statement is a prepared statement or not, the "Statement type" attribute is not applicable for the OUA universal connector plug-in. +- The "record affected" field is not supported when using Oracle with the universal connector. +- If you restart the database, then any Universal Connector that uses the OUA plugin must also be restarted. diff --git a/filter-plugin/logstash-filter-oua-guardium/README.md b/filter-plugin/logstash-filter-oua-guardium/README.md index 48b83201d..8e4edc7c4 100644 --- a/filter-plugin/logstash-filter-oua-guardium/README.md +++ b/filter-plugin/logstash-filter-oua-guardium/README.md @@ -1,126 +1,9 @@ # Oracle Unified Audit Universal Connector - -## Meet Oracle Unified Audit -* Tested versions: 18,19 -* Environments: On-prem, RDS in AWS, Oracle Autonomous Database in OCI +## Follow this link to set up and use Oracle Unified Audit Universal Connector over Pipe - **Note**: Autonomous Database in OCI is supported only by Guardium Data Protection SqlGuard-12.0p7015_Bundle_May_20_2024 and SqlGuard-11.0p545_Bundle_Jul_09_2024. -* Supported inputs: Oracle Unified Audit (pull) -* Supported Oracle versions: 18, 19, and 21 -* Supported Guardium versions: - * Guardium Data Protection: 11.4 and above - +[OuaOverPipeReadme](./OuaOverPipeReadme.md) -## Requirements +## Follow this link to set up and use Oracle Unified Audit Universal Connector over JDBC Connect -1. Unified auditing must be enabled in an Oracle database that will be monitored by this method -2. Download the Basic Instant client package from Oracle. - **Note:** : In this release,only specific instance clients will be supported from starting v21.1.0.0.0 Download [here](https://download.oracle.com/otn_software/linux/instantclient/211000/oracle-instantclient-basic-21.1.0.0.0-1.x86_64.rpm) -3. Oracle Unified Audit Universal Connector is automatically available with Guardium Data Protection versions 12.x, 11.4 with appliance bundle 11.0p490 or later or Guardium Data Protection version 11.5 with appliance bundle 11.0p540 or later releases. -**Note**: For Guardium Data Protection version 11.4 without appliance bundle 11.0p490 or prior or Guardium Data Protection version 11.5 without appliance bundle 11.0p540 or prior, download the [OUA](https://github.com/IBM/universal-connectors/raw/release-v1.2.0/filter-plugin/logstash-filter-oua-guardium/OracleUnifiedAuditPackage/OracleUnifiedAudit/guardium-oua-uc.zip) plug-in. (Do not unzip the offline-package file throughout the procedure). - -4. A designated user for OUA UC should be created for Oracle database access. An existing user with sysdba privileges can also be used - -5. You must create a secret containing your OUA universal connector password. - - Example: `grdapi universal_connector_keystore_add key=OUA_USER_PASS password=` where `` is the OUA universal connector user’s password for the database. `OUA_USER_PASS` will be used in the plug-in configuration as a variable for the password secret. - -Currently, this plug-in will work only on IBM Security Guardium Data Protection, not Guardium Insights - -## Building - -Update the variables in Makefile for your environment's Java home and Logstash location - -## Setup - -1. Create a designated Database User for OUA UC to retrieve audit data with minimal privileges (using DBA help) as follows: - - Assuming the name for the designated Oracle Unified Audit user with minimal permissions will be "guardium" with password "password" - - Connect to Oracle using sysdba account and execute the following commands: - - - For Oracle Autonomous Database in OCI, run the following commands to create a user with the required grants: - ``` - CREATE USER guardium IDENTIFIED BY password; - GRANT CONNECT to guardium; - GRANT SELECT ANY DICTIONARY to guardium; - GRANT AUDIT_VIEWER to guardium; - GRANT SELECT ON v$INSTANCE to guardium; - GRANT SELECT ON v$DATABASE to guardium; - GRANT SELECT ON v$MYSTAT to guardium; - ``` - - - For other environments including RDS in AWS and Oracle Databases On-Premises run the following commands: - ``` - CREATE USER guardium IDENTIFIED BY password; - GRANT CONNECT, RESOURCE to guardium; - GRANT SELECT ANY DICTIONARY TO guardium; - exec DBMS_NETWORK_ACL_ADMIN.APPEND_HOST_ACE(host => 'localhost', - ace => xs$ace_type(privilege_list => xs$name_list('connect', - 'resolve'), principal_name => 'guardium', principal_type => xs_acl.ptype_db)); - ``` - - - To verify your new user's privileges, connect to the Oracle instance that you planning to monitor using the name and credentials for your designated user and run the following statements: - - ``` - select count(*) from AUDSYS.AUD$UNIFIED; - SELECT UTL_INADDR.get_host_address FROM DUAL; - ``` - - - If there are no errors that means you can use this new user for this UC method - -2. Enable the universal collector feature on the designated Guardium collectors or the stand-alone system. See [here](/docs/Guardium%20Data%20Protection/uc_config_gdp.md). - -3. On the collector, go to Setup > Tools and Views > Configure Universal Connector - -4. Click on the "UPLOAD” button and upload the downloaded `oracle-instantclient-basic-21.1.0.0.0-1.x86_64.rpm`. - -6. Then, upload the `guardium-oua-uc.zip` file. This step is not necessary for Guardium Data Protection v12.0 and later. - - 5. Click on the "+". The Connector Configuration dialog opens. - - - Type any unique name in the **Connector name** field. - - - Paste the content from the "[ouaPipe.conf](https://github.com/IBM/universal-connectors/raw/main/filter-plugin/logstash-filter-oua-guardium/ouaPipe.conf)" in the **Input configuration** field - - ``` - pipe { - type => "oua" - command => "${OUA_BINARY_PATH} -c ${THIRD_PARTY_PATH} -s ${THIRD_PARTY_PATH} -r 100 -t 1000 -p 10 -j /${OUA_USER_PASS}@:/" - add_field => {"SERVER_ADDRESS" => ""} - add_field => {"SERVER_PORT" => ""} } - ``` - - - Paste the content from the "[ouaPipe.conf](https://github.com/IBM/universal-connectors/raw/main/filter-plugin/logstash-filter-oua-guardium/ouaPipe.conf)" in the **Filter configuration** field - - ``` - if [type] == "oua" { - json { - source => "message" - } - mutate { - add_field => {"[HostName]" => "%{SERVER_ADDRESS}" } - add_field => {"[PortNumber]" => "%{SERVER_PORT}" } - } - if "_jsonparsefailure" not in [tags] { - oua_filter {} - } - } - - ``` - - - **NOTE**: The type specified for the filters must be unique among Universal Connectors and be identical in the input and filter configurations - - 6. Click **Save**. Guardium validates the new connector and enables the plug-in. After it is validated, it appears on the **Configure Universal Connector** page. - -**Note**: The following arguments are used in the OUA plug-in configuration. -- -c path : path to instantclient libraries -- -a path : path to tnsnames.ora (directory, not the filename) -- -r rows : number of rows to fetch at a single time (default 100) -- -t timeout : timeout for all operations (connect, execute, fetch, etc) specified in milliseconds (default 300000) -- -p period : period of time between passes specified in seconds (default 300) -- -s path : path to directory where state should be saved -- -j : output audits in JSON format - -## Limitation -- Normally, the "statement type" attribute for the "FULL SQL" entity in Guardium reports shows us whether a full SQL statement is a prepared statement. However, because OUA doesn't give us information about whether a statement is a prepared statement or not, the "Statement type" attribute is not applicable for the OUA universal connector plug-in. -- The "record affected" field is not supported when using Oracle with the universal connector. -- If you restart the database, then any Universal Connector that uses the OUA plugin must also be restarted. +[OuaOverConnectJdbc](./OuaOverConnectJdbcReadme.md) \ No newline at end of file