Skip to content

Commit

Permalink
[#549] fix Thrift server AWS auth locally
Browse files Browse the repository at this point in the history
In #502, we made the decision to push IRSA-based authentication as the
default. I think this is still worthwhile, but the shift has to be more
holistic.  There are still several other services that assume the AWS
enviroment variables will be provided via the remote-auth-config secret.
To really change the default to IRSA, we likely want to flow a separate
ticket and fully think through the best approach, and unify the auth
method across all deployments.

For now, we will simply keep the secret-based auth as the default.
  • Loading branch information
ewilkins-csi committed Feb 4, 2025
1 parent 57d26ec commit 0fcead5
Showing 1 changed file with 23 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,19 @@ aissemble-thrift-server-chart:
app:
name: "thrift-server"

deployment:
envFromSecret:
AWS_ACCESS_KEY_ID:
secretName: remote-auth-config
key: AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY:
secretName: remote-auth-config
key: AWS_SECRET_ACCESS_KEY

sparkConf: |
spark.hadoop.fs.s3a.endpoint=http://s3-local:4566
spark.hadoop.fs.s3a.access.key=#[[${env:AWS_ACCESS_KEY_ID}]]#
spark.hadoop.fs.s3a.secret.key=#[[${env:AWS_SECRET_ACCESS_KEY}]]#
spark.hadoop.fs.s3.impl=org.apache.hadoop.fs.s3a.S3AFileSystem
spark.hive.server2.thrift.port=10000
spark.hive.server2.thrift.http.port=10001
Expand Down Expand Up @@ -41,6 +52,18 @@ aissemble-hive-metastore-service-chart:
replicationPassword: hive
password: hive

deployment:
env:
- name: AWS_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
name: remote-auth-config
key: AWS_SECRET_ACCESS_KEY
- name: AWS_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
name: remote-auth-config
key: AWS_ACCESS_KEY_ID
configMap:
metastoreServiceConfig:
properties:
Expand Down

0 comments on commit 0fcead5

Please sign in to comment.