Missing required field PrepID #351
Comments
|
Lukas, sorry for delay in reply, I was traveling and sick. I can reproduce this problem, but since the task name and data range you used is quite old I bet it is issue with a schema of data which has been migrated. Can you try another, new-ish, task name and data range? |
|
Hi @vkuznet, do you have a bit of bandwidth to help Lukas forward ? |
|
Jean-Roch, I can help Lukas once he will explain exactly what needs to be done.
So far I suggested to try out new-ish workflow since it seems to me that old
one have issue with schema (we introduced PrepID at some point and it seems
to me that task/date-range Lukas is using haven't had the PrepID in WMArchive).
Valentin.
…On 0, vlimant ***@***.***> wrote:
Hi @vkuznet, do you have a bit of bandwidth to help Lukas forward ?
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#351 (comment)
|
|
makes sense, if prepid is explicitely requested, or searched about. |
|
Jean-Roch, it is another way around. Originally stored docs haven't had PrepID
and WMArchive schema neither. Later we introduced PrepID in schema and current
schema requires it. That's why we should either search for new-ish task which
has PrepID in it or we need to look-up back original schema and use it in
queries. For simplicity I asked can we look-up new-ish task/workflow.
…On 0, vlimant ***@***.***> wrote:
makes sense, if prepid is explicitely requested, or searched about.
which is it @llayer ? do you retreive the prepid explicitely, or is the query containing prepid ?
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#351 (comment)
|
|
Hi Valentin, Jean-Roch, I finally managed to test and run a new task for a more recent production, e.g. I made a This runs and I get an output file as described in the wiki. I tried to lookup the log file for one unmerged task that was returned, i.e. I made a spec: and then run This returned: So finally this should be the location of the logfile and I should be able to retrieve it with xrtcp, Do you have any idea what might be the issue here? |
|
Lukas, I'm glad that WMArchive procedure to find records is working now, but I doubt I can help here with xrdcp problem. The storage of log files is beyond WMArchive responsibility. You may ask Alan (@amaltaro ) what is a policy to keep log archive files, but my understanding that they may be available on EOS. |
|
Thank you Valentin, I will contact him! |
|
Dear Alan @amaltaro, for my service task I need to access and download a large number of log files. Is there an easy way to download the logs once I have the location? Many thanks!! |
|
Hi Lucas, Second option would be to fetch these from the site the workflow is running at. However, those are transient files and they are collected by a LogCollect job, which makes a bigger tarball of logArchives and transfer it to CERN (both CASTOR and EOS). Does this script find which logCollect tarball contains your unmerged logArchive? |
|
Hi Alan, thanks for your prompt reply. gives me the tarball of the unmerged log Archive, |
|
Hi Alan @amaltaro, Do you have any quick solution to download the logArchive.tar.gz as described in the Thank you so much! |
|
Lucas, you can find these LogCollect tarballs available in both EOS and CASTOR CERN storage: HTH. Alan. |
|
Thank you Alan, I finally managed to download the first files! |
|
Dear @amaltaro, for my project with @vlimant in the last weeks I collected the and it seems that at least most folders in /store/logs/prod/2017/11/WMAgent/ are empty. Many, many thanks in advance! |
|
Hi @llayer , EOS is supposed to keep logs for only a couple of months, so consider yourself lucky for finding more than a year logs in there. |
|
Hi @amaltaro, many thanks, I am now able to locate the files on castor, but copying with xrdcp returns me an Do I need some special permissions? I also cannot make a stager_qry: I am not familiar with castor, so if you could give me another hint it would be fantastic! Cheers, |
|
Have you tried without the service class option? I haven't used castor for a long time now, so you might need to ask CERN IT via a snow ticket. |
Hi,
I have a problem to run myspark from lxplus.
I work with Jean-Roch and I need to access some error logs.
I tried to reproduce the example in https://github.com/dmwm/WMArchive/wiki/How-to-find-records-on-HDFS-using-pyspark
I logged in from an lxplus node following https://hadoop-user-guide.web.cern.ch/hadoop-user-guide/getstart/client_cvmfs.html, since it is not possible anymore to log in via ssh analytix
However when I run the the example in the twiki:
{"spec":{"task":"/amaltaro_StepChain_ReDigi3_HG1612_WMArchive_161130_192654_9283/DIGI","timerange":[20161130,20161202]}, "fields":[]}
myspark --spec=cond.spec --script=RecordFinder --records-output=records.json
I get an error message that tells me that the PrepID is missing:
I attach the output of myspark below.
It would be great if you could help me to solve this problem.
Many, many thanks in advance,
Best Lukas
The text was updated successfully, but these errors were encountered: