Handling logging output from custom apps installed in a TOM

[rachel3834]

My TOM includes custom applications used to analyze data on my Targets, in this case fitting a model to a Target lightcurve.
The app outputs its own logs, normally storing them to disk when run stand alone.
I would like for the TOM to store this logging output so that I can make it available through the UI.
The TOM already uses Python's logging module to capture log streams, which can be viewed in a Cloud deployed-TOM via stern.

• Question 1: I need my app to be able to run in stand alone mode and write to local disk as well as output to the TOM's log stream, if available. What is the best-practice way of writing the app to do achieve this?
• Question 2: My app spawns parallelized subprocesses as part of its operation (one per Target), each of which write their own logs. I would like to store the output from this log in my TOM for UI display, without the user needing to resort to stern. The controller process produces a separate, additional log, which ideally I would also like to be able to view. How do you recommend we achieve this?

[jchate6]

What sort of output (quantity mostly) are you expecting and how would you like it displayed?

[rachel3834]

I'm mostly thinking of text logging, typically a few hundred KB in size. I'd like to be able to click a button next to the display of the graphical output from the app to display the logging output on demand.

…

On Mon, Oct 21, 2024 at 9:37 AM Joey Chatelain ***@***.***> wrote: What sort of output (quantity mostly) are you expecting and how would you like it displayed? — Reply to this email directly, view it on GitHub <#1083 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACPJA3CMFFHL64TC6H6G5TLZ4UUS3AVCNFSM6AAAAABQEZLRBSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMRXGE3TIOBXGE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[jchate6]

I think one strategy would be:

1. Set up a django model to keep track of your analysis runs. This could have foreign keys to a target, or whatever fields you need, but importantly also includes a FileField for log output.
2. Pipe any logs or output for the run into a file that is connected to the run model.
3. have a run detail page that displays (or downloads) the log file.

This would be good for larger logs that are specific to a single use case. capturing general logs or logs that are only a few lines of text should probably be handled in a different way.