Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for tokens requested via oidc-agent. #17

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
Open

Add support for tokens requested via oidc-agent. #17

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
0a596f3
Add support for tokens requested via oidc-agent.
bbockelm Sep 16, 2020
dc450f3
Cleanup error handling a bit.
bbockelm Sep 16, 2020
2bef9d4
Add ability to request tokens in tests and use alternate endpoint files.
bbockelm Sep 16, 2020
becca7c
Add some sample endpoints.
bbockelm Sep 16, 2020
d27e477
Adjust token endpoints.
bbockelm Sep 16, 2020
2f4344b
Allow env var override of VOMS VO
bbockelm Sep 16, 2020
30ab736
Add manchester endpoint
bbockelm Sep 16, 2020
3d962bb
Address comments from code review
bbockelm Oct 1, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 18 additions & 5 deletions bin/smoke-em.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,21 +26,28 @@ CLEAR_LINE="\e[2K"

SOUND_ENDPOINT_RE="successful (100%)"

ENDPOINTS=$BASE/etc/endpoints

OUTPUT_DESCRIPTION="stdout"
QUIET=0
while getopts "h?s:m:xp:qS:" opt; do
while getopts "h?e:s:m:xo:p:qS:" opt; do
case "$opt" in
h|\?)
echo "$0 -x [-s <addr> [-m <mailer>]] [-p <file>] [-S <file>]"
echo "$0 -x [-s <addr> [-m <mailer>]] [-p <file>] [-S <file>] [-o <name>]"
echo
echo " -e <file> endpoint file (defaults to etc/endpoints)"
echo " -s <addr> send report as an email to <addr>"
echo " -m <mailer> use 'mail' or 'thunderbird' to send email"
echo " -x use extended tests, if supported"
echo " -p <file> use <file> for persistent state"
echo " -q limit output to errors and prompts"
echo " -S <file> skip endpoints listed in <file>"
echo " -o <name> use token auth from oidc-agent account <name>"
exit 0
;;
e)
ENDPOINTS="$OPTARG"
;;
s)
sendEmail="$OPTARG"
OUTPUT_DESCRIPTION="emailing"
Expand All @@ -51,6 +58,9 @@ while getopts "h?s:m:xp:qS:" opt; do
x)
EXTENDED_TESTS=1
;;
o)
OIDC_AGENT_ACCOUNT="$OPTARG"
;;
p)
persistentState="$OPTARG"
;;
Expand Down Expand Up @@ -143,10 +153,9 @@ runTests() {

STARTED_AT=$(date +%s)


TOTAL=$(wc -l $BASE/etc/endpoints|awk '{print $1}')
TOTAL=$(wc -l $ENDPOINTS|awk '{print $1}')
COUNT=0
cat $BASE/etc/endpoints | while read name type workarounds url; do
cat $ENDPOINTS | while read name type workarounds url; do
if [ $EXTENDED_TESTS -eq 1 ]; then
case $type in
dCache|DPM|StoRM)
Expand All @@ -160,6 +169,10 @@ runTests() {
options="-f"
fi

if [ ! -z "$OIDC_AGENT_ACCOUNT" ]; then
options="$options -t wlcg -o $OIDC_AGENT_ACCOUNT"
fi

if [[ "$workarounds" == *L* ]]; then
options="$options -L"
fi
Expand Down
51 changes: 47 additions & 4 deletions bin/smoke-test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ extended=0
tokenType=macaroon
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Unfortunately, githib doesn't allow me to make this comment any earlier]

I think the patch should define WLCG_THIRDPARTY_PRIVATE_URL and WLCG_THIRDPARTY_UPLOAD_BASE_URL (as with dteam, escape and atlas VOs), and update the code to use these default values if the client configures smoke-test to use the wlcg VO.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @paulmillar -

Finally started to look at this and I noticed that something appears to be misconfigured on the prometheus side.

This works:

voms-proxy-init -voms dteam
curl --verbose --connect-timeout 60 -D /tmp/tmp.CNpXBTEwRC \
   -s -f -L  --capath /etc/grid-security/certificates -H 'X-No-Delegate: true' \
   --cacert /tmp/x509up_u1221 -E /tmp/x509up_u1221 -H 'Credential: none' \
   -m30 -X POST -H 'Content-Type: application/macaroon-request' \
   -d '{"caveats": ["activity:DOWNLOAD"], "validity": "PT30M"}' -o/tmp/tmp.FbihlmyRjK \
   https://prometheus.desy.de:2443/VOs/dteam/private-file

This doesn't work:

voms-proxy-init -voms wlcg
curl --verbose --connect-timeout 60 -D /tmp/tmp.CNpXBTEwRC \
   -s -f -L  --capath /etc/grid-security/certificates -H 'X-No-Delegate: true' \
   --cacert /tmp/x509up_u1221 -E /tmp/x509up_u1221 -H 'Credential: none' \
   -m30 -X POST -H 'Content-Type: application/macaroon-request' \
   -d '{"caveats": ["activity:DOWNLOAD"], "validity": "PT30M"}' -o/tmp/tmp.FbihlmyRjK \
   https://prometheus.desy.de:2443/VOs/wlcg/private-file

Only difference is replacing the dteam string with wlcg. Access with DAVIX clients appears OK, so I think X.509 auth is working overall -- but perhaps just an issue with the macaroon generation?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, indeed. The problem was that prometheus was missing the .lsc file for the WLCG VOMS server.

I've installed the missing file, so the problem should be fixed now.

Could you give it another go?

workarounds=""
debugOutput=0
vo=dteam
vo=${SMOKE_TEST_VO:-dteam}
bbockelm marked this conversation as resolved.
Show resolved Hide resolved

withColour() {
RESET="\x1B[0m"
Expand Down Expand Up @@ -528,6 +528,34 @@ requestMacaroon() { # $1 Caveats, $2 URL, $3 variable for macaroon, $4 variable
eval $3="$macaroon"
}

requestWlcgToken() { # $1 oidc-agent account name, $2 audience, $3 variable for token, $4 token for result.
bbockelm marked this conversation as resolved.
Show resolved Hide resolved
# Fallback to failing unless we specify otherwise.
eval $4=1
lastTestFailed=0

echo -n "Requesting token from account $1 for audience $2: "

token=$(oidc-token $1 --aud $2)
bbockelm marked this conversation as resolved.
Show resolved Hide resolved
eval $3="$token"
if [ -z "$token" ]; then
fail "Token request failed"
eval $4=3
return
fi

local body=$(echo -n $token | tr '.' ' ' | awk '{print $2;}' | base64 --decode 2>/dev/null)
debug "Returned token body is $body"
audience="$(echo -n $body | jq -r .aud )"
if [ "$audience" != "$2" ]; then
fail "Issuer returned incorrect audience (desired $2; returned $audience)"
eval $4=2
return
fi

eval $4=0
success
}

requestSciToken() { # $1 Scopes, $2 Issuer URL, $3 variable for token, $4 variable for result
local target_scitoken=$(mktemp)
local scitoken_json=$(mktemp)
Expand Down Expand Up @@ -580,7 +608,7 @@ done
# Check if stdout is sent to a terminal, or redirect to a file.
if [ -t 1 ] ; then withColour; else withoutColour; fi

while getopts "h?t:v:p:r:u:s:fxlLCcd2" opt; do
while getopts "h?t:v:o:p:r:u:s:fxlLCcd2" opt; do
case "$opt" in
h|\?)
echo "$0 [-f] [-x] [-c] [-C] [-L] [-2] [-t <token>] [-d] [-v <vo>] [-p <url>] [-u <url>] [-s <url>] URL"
Expand All @@ -592,13 +620,14 @@ while getopts "h?t:v:p:r:u:s:fxlLCcd2" opt; do
echo " -l Disable location-trusted work-around"
echo " -L Enable location-trusted work-around"
echo " -2 Force client not to use TLS v1.3 as work-around"
echo " -t <token> Use <token> for non-X.509 authn to target"
echo " -t <token> Use <token> for non-X.509 authn to target (valid values: macaroon, scitoken, wlcg)"
bbockelm marked this conversation as resolved.
Show resolved Hide resolved
echo " -d Include additional logging"
echo " -v <vo> Test as member of VO <vo>"
echo " -p <url> Use <url> as the public link for TPC PULL"
echo " -r <url> Use <url> as the VO-private link for TPC PULL"
echo " -u <url> Use <url> as the base for HTTP PUSH"
echo " -s <url> SciToken server"
echo " -o <name> oidc-agent account name (for use with wlcg tokens)"
echo
echo "Defaults:"
echo " -p $THIRDPARTY_UNAUTHENTICATED_URL"
bbockelm marked this conversation as resolved.
Show resolved Hide resolved
Expand Down Expand Up @@ -644,8 +673,11 @@ while getopts "h?t:v:p:r:u:s:fxlLCcd2" opt; do
scitoken)
tokenType=SciToken
;;
wlcg)
tokenType=WlcgToken
;;
*)
fatal "Unknown token type \"$OPTARG\". Must be one of \"macaroon\" or \"scitoken\""
fatal "Unknown token type \"$OPTARG\". Must be one of \"macaroon\", \"wlcg\", or \"scitoken\""
;;
esac
;;
Expand All @@ -655,6 +687,9 @@ while getopts "h?t:v:p:r:u:s:fxlLCcd2" opt; do
v)
vo=$OPTARG
;;
o)
OIDC_AGENT_ACCOUNT=$OPTARG
;;
p)
THIRDPARTY_UNAUTHENTICATED_URL=$OPTARG
;;
bbockelm marked this conversation as resolved.
Show resolved Hide resolved
Expand Down Expand Up @@ -808,6 +843,8 @@ if [ $CURL_HAS_CONNECT_TO -eq 0 ]; then
echo "WARNING: curl does not support --connect-to; not specifying an IP address"
IP_ADDRESS_COUNT=1
ALL_IP_ADDRESSES=$(echo "$ALL_IP_ADDRESSES" | awk '{print $1;}')
echo "WARNING: curl does not support --tls-max; not using"
GOLDEN_ENDPOINT_EXTRA_OPTIONS=""
fi


Expand All @@ -818,6 +855,12 @@ case $tokenType in
unset NON_SUT_TEST
;;

WlcgToken)
NON_SUT_TEST=1
requestWlcgToken "$OIDC_AGENT_ACCOUNT" "https://$HOST_PORT" TARGET_TOKEN tokenFailed
unset NON_SUT_TEST
;;

macaroon)
tokenFailed=1
;;
Expand Down
4 changes: 4 additions & 0 deletions etc/token-endpoints
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
NEBRASKA xrootd-D/HDFS - https://red-gridftp1.unl.edu:1094/user/dteam
CERN-EOSPPS EOS - https://eospps.cern.ch:443/eos/opstest/tpc/https
DESY-PROM-DCA dCache - https://prometheus.desy.de:2443/VOs/dteam
PRAGUELCG2-DPM DPM - https://golias100.farm.particle.cz:443/dpm/farm.particle.cz/home/dteam
6 changes: 6 additions & 0 deletions etc/wlcg-token-endpoints
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
NEBRASKA xrootd-D/HDFS - https://red-gridftp1.unl.edu:1094/user/dteam
CERN-EOSPPS EOS - https://eospps.cern.ch:443/eos/opstest/tpc/https
DESY-PROM-DCA dCache - https://prometheus.desy.de:2443/VOs/wlcg
INFN-T1-STO SToRM - https://xfer.cr.cnaf.infn.it:8443/wlcg
PRAGUELCG2-DPM DPM - https://golias100.farm.particle.cz:443/dpm/farm.particle.cz/home/wlcg
MANCHESTER DPM - https://vm33.in.tier2.hep.manchester.ac.uk/dpm/tier2.hep.manchester.ac.uk/home/wlcg