Feature / Basic authentication provider (#200)

* Add gRPC error code translation for UNAVAILABLE and PERMISSION_DENIED

* Add JWT dependency

* JWT encode / decode classes

* HTTP/1 auth handler with pluggable auth provider interface

* Basic auth provider

* Guest auth provider

* Remove JWT dependency in gateway (it is wrapped in common lib)

* Add a test case for JWT encoding / decoding

* Fix quoting in JWT encoding / decoding

* Auth interceptor to perform authentication for gRPC service (no authorization yet)

* Add auth interceptor to the three main services, and log user info in gRPC server wrap

* Add a stub for the new auth tool

* Utility functions for working with keys and key stores

* Make JksSecretLoader use the new CryptoUtils functions

* Error case tests for crypto helpers

* Split JWS logic in to validator (ro) and processor (rw)

* Some basic functional tests for JWT processing / validation (i.e. JWT auth)

* Finish working on JWT processor + tests

* Additions to config loading framework

* Add authentication config to the config file structures

* Config keys for public / private root auth secrets

* Additions to config loading framework

* Add an auth interceptor for gPRC, using the JwtValidator to test authentication

* Apply auth to the core platform services

* A client auth provider, for supplying auth tokens into gRPC calls from clients

* Add auth-tool step to PlatformTest setup code

* Make API tests for metadata service work with auth tokens

* Fixes to pass auth tokens through from data service to metadata service

* Relay owner token in orchestrator

* Config updates for integration tests

* Config updates for end-to-end tests

* Standard handling for plugin config

* Make IAuthProvider a registered plugin serice type

* Use plugins to set up the auth provider in the gateway

* Fix secret key for metadb integration tests

* Move auth provider interface into common lib

* Move the standard auth plugin into the common library

* Switch IAuthProvider to return UserInfo objects

* Gateway core auth implementation

* Gateway auth update for REST APIs

* Dev local config updates

* Put user ID / name from authentication into the metadata

* Make guest auth provider props camel case

* Add default (guest) auth config to sandbox dist config files

* Fix root signing key for gateway tests

* Set up auth keys in integration jobs for metadb

* Stub out implementation of basic auth

* Set up auth keys in integration jobs for metadb

* Allow a secret key to be passed into the platform test setup

* Add password functions to crypto helpers

* Auth tool tasks for adding users to local user db

* Fix in gateway tests

* Handle bearer auth in auth interceptor, report auth errors with an exception

* Tidier closing in auth interceptor on auth failures

* Translate UNAUTHENTICATED code in rest API response translation

* Handle plugin secrets in plugin manager createService

* Add containsAttr in crypto helpers

* Full impl for basic auth provider

* Make config manager give out the user DB

* Add secret attrs to secret loaders

* Use config manager in plugin manager createService to resolve plugin secrets

* Auth tool fixes

* Give TRAC users to auth providers that need it in the gateway

* Refactored auth setup task name in integration workflow

* SnakeYAML vulnerabilities

* Initial documentation for the authentication system

.github/config/trac-int-mariadb.yaml

@@ -12,6 +12,9 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+config:
+  secret.type: PKCS12
+  secret.url: secrets.pfx
 
 # Environment settings are verified in one of the test cases so they need to match
 # MetadataReadApiTest platformInfo()
@@ -21,6 +24,9 @@ platformInfo:
   deploymentInfo:
     region: UK
 
+authentication:
+  jwtIssuer: http://localhost/
+
 instances:
   meta:
     - scheme: http

.github/config/trac-int-mysql.yaml

@@ -12,6 +12,9 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+config:
+  secret.type: PKCS12
+  secret.url: secrets.pfx
 
 # Environment settings are verified in one of the test cases so they need to match
 # MetadataReadApiTest platformInfo()
@@ -21,6 +24,9 @@ platformInfo:
   deploymentInfo:
     region: UK
 
+authentication:
+  jwtIssuer: http://localhost/
+
 instances:
   meta:
     - scheme: http

.github/config/trac-int-postgresql.yaml

@@ -12,6 +12,9 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+config:
+  secret.type: PKCS12
+  secret.url: secrets.pfx
 
 # Environment settings are verified in one of the test cases so they need to match
 # MetadataReadApiTest platformInfo()
@@ -21,6 +24,9 @@ platformInfo:
   deploymentInfo:
     region: UK
 
+authentication:
+  jwtIssuer: http://localhost/
+
 instances:
   meta:
     - scheme: http

.github/config/trac-int-sqlserver.yaml

@@ -12,6 +12,9 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+config:
+  secret.type: PKCS12
+  secret.url: secrets.pfx
 
 # Environment settings are verified in one of the test cases so they need to match
 # MetadataReadApiTest platformInfo()
@@ -21,6 +24,9 @@ platformInfo:
   deploymentInfo:
     region: UK
 
+authentication:
+  jwtIssuer: http://localhost/
+
 instances:
   meta:
     - scheme: http

.github/workflows/integration.yml

@@ -134,6 +134,7 @@ jobs:
               DB_OPTIONS: '--health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=3',
               BUILD_sql_mysql: true,
               TRAC_CONFIG_FILE: '.github/config/trac-int-mysql.yaml',
+              TRAC_SECRET_KEY: wDeq3x-NjaLL7,
               MYSQL_DATABASE: trac,
               MYSQL_USER: trac_admin,
               MYSQL_PASSWORD: trac_admin,
@@ -145,6 +146,7 @@ jobs:
               DB_OPTIONS: '--health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=3',
               BUILD_sql_mariadb: true,
               TRAC_CONFIG_FILE: '.github/config/trac-int-mariadb.yaml',
+              TRAC_SECRET_KEY: uYhnKwq8+esS,
               MYSQL_DATABASE: trac,
               MYSQL_USER: trac_admin,
               MYSQL_PASSWORD: trac_admin,
@@ -156,6 +158,7 @@ jobs:
               DB_OPTIONS: '--health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5',
               BUILD_sql_postgresql: true,
               TRAC_CONFIG_FILE: '.github/config/trac-int-postgresql.yaml',
+              TRAC_SECRET_KEY: hjXks83bX=wxMr,
               POSTGRES_DB: trac,
               POSTGRES_USER: trac_admin,
               POSTGRES_PASSWORD: trac_admin }
@@ -166,6 +169,7 @@ jobs:
               DB_OPTIONS: '-e "NO_DB_OPTIONS=not_used"',  # docker run -e flag sets an env variable, passing '' causes errors
               BUILD_sql_sqlserver: true,
               TRAC_CONFIG_FILE: '.github/config/trac-int-sqlserver.yaml',
+              TRAC_SECRET_KEY: unHkj>weN2jSl,
               MSSQL_PID: Developer,
               ACCEPT_EULA: Y,
               SA_PASSWORD: "tR4c_aDm!n" }
@@ -200,12 +204,21 @@ jobs:
       - name: Build
         run: ./gradlew trac-svc-meta:build trac-svc-meta:testClasses deploy-metadb:build -x test
 
+      # Auth tool will also create the secrets file if it doesn't exist
+      - name: Prepare Auth Keys
+        run: |
+          ./gradlew auth-tool:run --args="\
+          --config ${{ env.TRAC_CONFIG_FILE }} \
+          --secret-key ${{ env.TRAC_SECRET_KEY }} \
+          --task create_root_auth_key EC 256"
+
       # The name and description of the test tenant are verified in one of the test cases so they need to match
       # MetadataReapApiTest listTenants()
       - name: Prepare Database
         run: |
           ./gradlew deploy-metadb:run --args="\
-          --config ${{ matrix.database.TRAC_CONFIG_FILE }} \
+          --config ${{ env.TRAC_CONFIG_FILE }} \
+          --secret-key ${{ env.TRAC_SECRET_KEY }} \
           --task deploy_schema \
           --task add_tenant ACME_CORP 'Test tenant [ACME_CORP]'"
 

dev/compliance/owasp-false-positives.xml

@@ -71,6 +71,7 @@
         <packageUrl regex="true">^pkg:maven/org\.yaml/snakeyaml@.*$</packageUrl>
         <vulnerabilityName>CVE-2022-38752</vulnerabilityName>
         <vulnerabilityName>CVE-2022-38751</vulnerabilityName>
+        <vulnerabilityName>CVE-2022-41854</vulnerabilityName>
     </suppress>
 
     <!-- This error is fixed in Jackson version 2.14-rc1 -->

dev/config/trac-devlocal-gateway.yaml

@@ -14,6 +14,20 @@
 
 config:
   logging: trac-logging.xml
+  secret.type: PKCS12
+  secret.url: secrets.p12
+
+
+authentication:
+
+  jwtIssuer: http://localhost:8080/
+  jwtExpiry: 7200
+
+  provider:
+    protocol: guest
+    properties:
+      userId: guest
+      userName: Guest User
 
 
 port: 8080

dev/config/trac-devlocal.yaml

@@ -23,6 +23,10 @@ platformInfo:
   production: false
 
 
+authentication:
+  jwtIssuer: http://localhost:8080/
+
+
 instances:
 
   meta:

dist/template/etc/trac-gateway.yaml

@@ -14,6 +14,20 @@
 
 config:
   logging: trac-logging.xml
+  secret.type: PKCS12
+  secret.url: secrets.p12
+
+
+authentication:
+
+  jwtIssuer: http://localhost:8080/
+  jwtExpiry: 7200
+
+  provider:
+    protocol: guest
+    properties:
+      userId: guest
+      userName: Guest User
 
 
 port: 8080

dist/template/etc/trac-platform.yaml

@@ -14,13 +14,19 @@
 
 config:
   logging: trac-logging.xml
+  secret.type: PKCS12
+  secret.url: secrets.p12
 
 
 platformInfo:
   environment: SANDBOX
   production: false
 
 
+authentication:
+  jwtIssuer: http://localhost:8080/
+
+
 instances:
 
   meta:

doc/deployment/authentication.rst

@@ -0,0 +1,139 @@
+
+Authentication
+==============
+
+Authentication in TRAC consists of two main elements:
+
+* JWT tokens, which are created and validated by the platform and hold information about users
+* Authentication providers, which provide the log-on mechanism and are the source of user details
+
+When a user logs on they are authenticated by an authentication provider, once authentication is successful
+their details are retrieved from the provider and coded into a JWT token. The JWT token gives them access to
+the platform for a limited period of time, after which they must be re-authenticated with the provider.
+
+TRAC supports multiple authentication providers and more can be added using the TRAC plugins API.
+The JWT mechanism is owned by the platform, and cannot be extended.
+
+
+Root Signing Key
+----------------
+
+The authentication system requires a root signing key, that is used by the platform to sign
+and validate JWT tokens. In order to set up the key you must have secrets configured, in both
+the platform and gateway config files.
+
+You will also need to add an authentication block in both config files, specifying the issuer
+and expiry times for JWT tokens. If you know the DNS address that TRAC will be served from you
+could use this as the JWT issuer, other options could be the user ID of a service account you
+have set up to run TRAC, or a TRAC reserved identifier such as "trac_system".
+
+.. code-block:: yaml
+
+    config:
+      secret.type: PKCS12
+      secret.url: secrets.p12
+
+    authentication:
+      jwtIssuer: http://localhost:8080/
+      jwtExpiry: 7200
+
+The auth-tool utility can be used to generate the root signing key, it will be written into the
+secret store. The available key types are elliptic curve (EC) or RSA. Elliptic curve keys are
+considered to give better security with better performance at lower key sizes. For this reason
+we recommended EC 256 keys.
+
+.. tab-set::
+
+    .. tab-item:: Linux / macOS
+        :sync: platform_linux
+
+        .. code-block:: shell
+
+            cd /opt/trac/current
+            bin/auth-tool run --task create_root_auth_key EC 256
+
+    .. tab-item:: Windows
+        :sync: platform_windows
+
+        .. code-block:: batch
+
+            cd /d C:\trac\tracdap-sandbox-<version>
+            bin\auth-tool.bat run --task create_root_auth_key EC 256
+
+
+Providers
+---------
+
+You need to configure one provider in the authentication section of the gateway config file.
+
+.. note::
+    The JWT settings are still needed in the authentication section of the gateway config file,
+    do not remove them!
+
+**Guest Provider**
+^^^^^^^^^^^^^^^^^^
+
+The guest provider logs everyone in as guest, without prompting for credentials.
+The user ID and name can be set as properties of the provider.
+
+.. code-block:: yaml
+
+    authentication:
+
+      provider:
+        protocol: guest
+        properties:
+          userId: guest
+          userName: Guest User
+
+
+**Basic Provider**
+^^^^^^^^^^^^^^^^^^
+
+The basic provider uses HTTP basic authentication, which typically causes the browser
+authentication window to appear when users try to access pages in a browser.
+
+To use the basic provider, you must configure the TRAC user database. This is set up in the
+config section of the gateway config file. The provider must also be set up in the authentication
+section with protocol 'basic', it does not require any other properties.
+
+.. code-block:: yaml
+
+    config:
+      users.type: PKCS12
+      users.url: local_users.p12
+      users.key: local_users_key
+
+
+    authentication:
+
+      provider:
+        protocol: basic
+
+Before you can use the TRAC user database, you will need to initialize it and add at least one user.
+The auth-tool utility will let you do this. The add_user command is interactive and will ask for
+details to create a user.
+
+.. tab-set::
+
+    .. tab-item:: Linux / macOS
+        :sync: platform_linux
+
+        .. code-block:: shell
+
+            cd /opt/trac/current
+            bin/auth-tool run --task init_trac_users
+            bin/auth-tool run --task add_user
+
+            bin/auth-tool run --task delete_user <user_id>
+
+    .. tab-item:: Windows
+        :sync: platform_windows
+
+        .. code-block:: batch
+
+            cd /d C:\trac\tracdap-sandbox-<version>
+            bin\auth-tool.bat run --task init_trac_users
+            bin\auth-tool.bat run --task add_user
+
+            bin\auth-tool.bat run --task delete_user <user_id>

doc/deployment/index.rst

@@ -15,3 +15,4 @@ Deployment
     sandbox
     platform
     metadata_store
+    authentication

gradle/versions.gradle

@@ -35,6 +35,7 @@ ext {
     slf4j_version = '1.7.36'
     log4j_version = '2.17.2'
     commons_cli_version = "1.5.0"
+    jwt_version = "4.2.1"
 
 
     // SQL (JDBC) drivers

settings.gradle

@@ -69,6 +69,10 @@ project(":tracdap-gateway").projectDir = file("tracdap-services/tracdap-gateway"
 include 'deploy-metadb'
 project(":deploy-metadb").projectDir = file("tracdap-tools/deploy-metadb")
 
+include 'auth-tool'
+project(":auth-tool").projectDir = file("tracdap-tools/auth-tool")
+
+
 // Plugins
 
 include 'aws-config'