Add reporting for historical cpu utilization

[schoate]

I am looking for some reporting command similar to Slurm's sreport. From the man page, "sreport is used to generate reports of job usage and cluster utilization for Slurm jobs saved to the Slurm Database, slurmdbd". The sreport command allows the user to specify different report types with options.

For this specific request, I would like to be able run a command with the arguments host, a time period (start and end), and user (default to total system cpu utilization), and get output back in json or some parseable format: Cluster|Account|Login|Used.

Please let me know if you need additional detail. Thanks!

[cmoussa1]

Thanks @schoate! flux-accounting currently has flux account view-job-records, which looks at historical utilization on a cluster by storing job records. It will return the jobid, timestamps of the job, number of nodes, and the project (equivalent of WCKey it was run under. If run with no filters, it will return all of the job records it has for every user, but you can filter this down by jobid, username, timestamps, or projects. It will return information like this:

   UserID Username  JobID         T_Submit            T_Run       T_Inactive  Nodes                                                                              
0    1002     1002    102 1605633403.22141 1605635403.22141 1605637403.22141      2  
1    1002     1002    103 1605633403.22206 1605635403.22206 1605637403.22206      2 
2    1002     1002    104 1605633403.22285 1605635403.22286 1605637403.22286      2 
3    1002     1002    105 1605633403.22347 1605635403.22348 1605637403.22348      1 
4    1002     1002    106 1605633403.22416 1605635403.22416 1605637403.22416      1

If you would like to get the total job usage per-user, you can get this individually with flux account view-user <username> or in total with flux account view-bank <bank> --tree. Would either of these be sufficient for what you're looking for?

[schoate]

I think flux account view-job-records is close to what I need for data gathering. I'll start with that and see if that covers the entire use case. Thanks!

Is there any way to get Account / Bank information in the flux account view-job-records output, or would I need to run additional commands for each jobid to determine that?

[cmoussa1]

I think at the moment you might need to run additional commands for each jobid to look at account or bank information associated with the job. What kind of information would you be looking for? Are you thinking of additional filtering options with flux account view-job-records to filter by bank? Or maybe to have an additional Bank column in the output of flux account view-job-records?

[schoate]

An additional Bank column in the output of flux account view-job-records would be enough, I don't think I've seen a use case for filtering by bank yet.

[cmoussa1]

@schoate a Bank column as well as the ability to filter jobs by bank has been added with pull request #533 and should make its way into the next flux-accounting release.

[schoate]

Thanks @cmoussa1 !