Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exit on follow by inspecting the current log line #207

Merged
merged 4 commits into from
Oct 2, 2018
Merged

Exit on follow by inspecting the current log line #207

Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
1582533
Exit on follow by inspecting the current log line
ReDeiPirati Sep 28, 2018
9ad8071
Move list creation, simplify substring matching condition
ReDeiPirati Sep 29, 2018
4fa0e91
Move list as a constant. Check status inline.
ReDeiPirati Oct 1, 2018
16234c5
Fix typo
ReDeiPirati Oct 2, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 19 additions & 1 deletion floyd/cli/experiment.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -182,12 +182,30 @@ def get_log_id(job_id):
return instance_log_id


def check_job_termination_line(log_output_line, termination_list):
"""
Get Job termination by inspecting if any of the termination log lines is detected
in the current log output line.
"""
return any(terminal_line in log_output_line for terminal_line in termination_list)


def follow_logs(instance_log_id, sleep_duration=1):
cur_idx = 0
while True:
job_terminated = False

success = "[success] Finished execution"
failure = "[failed] Task execution failed"
shutdown = "[shutdown] Task execution cancelled"
timeout = "[timeout] Task execution cancelled"
termination_list = [success, failure, shutdown, timeout]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

better move this list creation outside so we are not creating a new list on every function call.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, move this outside the function to the top of this file. Also since these are constants use capitalized variable names for easy identification.

SUCCESS_OUTPUT = "[success] Finished execution"
FAILURE_OUTPUT = "[failed] Task execution failed"
..
TERMINATION_OUTPUT_LIST = [SUCCESS_OUTPUT, FAILURE_OUTPUT, ..]


while not job_terminated:
# Get the logs in a loop and log the new lines
log_file_contents = ResourceClient().get_content(instance_log_id)
print_output = log_file_contents[cur_idx:]
# Get the status of the Job from the current log line
job_terminated = check_job_termination_line(print_output, termination_list)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be simplified as:

job_terminated = print_output in termination_list

Then you can remove the extra function call here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cannot go for job_terminated = print_output in termination_list, because it's conditioning for a perfect match while in this case, I'm performing a substring match.

cur_idx += len(print_output)
sys.stdout.write(print_output)
sleep(sleep_duration)
Expand Down