Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ssh: switch from parallel-ssh to ansible-pylibssh #44

Merged
merged 3 commits into from
Jun 12, 2024

Conversation

pbrezina
Copy link
Contributor

@pbrezina pbrezina commented Feb 9, 2024

Unfortunately, parallel-ssh has become a dead project incompatible with
python-3.12+. Some effort was created by its community to make it
compatible, unfortunately the only maintainer with write access is
unresponsive and there is not much hope for a new release.

Therefore we switch to ansible-pylibssh, which provides low level libssh
python bindings. There is significant performance impact in artificial test
case ("running 'exit 0' over and over again is twice as slow) due to
differences in poll implementation, however the impact is neglitable in
real world scenarios (~20 seconds delay on 350 test cases). This small
slowdown also provides a huge benefit in very small CPU usage 0-1% where
parallel-ssh keeps spinning on 10-11%.

Resolves: #26

@pbrezina
Copy link
Contributor Author

pbrezina commented Feb 9, 2024

This requires a few changes to pylibssh that are not yet released, we are waiting for ansible-pylibssh 1.2.0.

@pbrezina pbrezina force-pushed the pylibssh branch 2 times, most recently from 040b353 to d027f50 Compare June 10, 2024 12:27
@pbrezina pbrezina marked this pull request as ready for review June 10, 2024 12:45
@pbrezina
Copy link
Contributor Author

@justin-stephenson pylibssh 1.2.0 was finally released so this is ready for review.

The CI/Example fails dues to missing sssd-kcm package in the containers, Madhuri is going to fix it.

Copy link
Contributor

@justin-stephenson justin-stephenson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack, works for me.

Some minor comments below which still mention pssh

justin@fedora:~/github/pytest-mh$ grep -irn 'pssh'
pytest_mh/ssh.py:515:        pssh simply calls the command as $shell '$command', e.g.
pytest_mh/ssh.py:610:        pssh simply calls the command as $shell '$command', e.g.

pbrezina added 2 commits June 11, 2024 12:00
Unfortunately, parallel-ssh has become a dead project incompatible
with python-3.12+. Some effort was created by its community to make
it compatible, unfortunately the only maintainer with write access
is unresponsive and there is not much hope for a new release.

Therefore we switch to ansible-pylibssh, which provides low level
libssh python bindings. There is significant performance impact
in artificial test case ("running 'exit 0' over and over again is twice
as slow) due to differences in poll implementation, however the impact
is neglitable in real world scenarios (~20 seconds delay on 350 test
cases). This small slowdown also provides a huge benefit in very small
CPU usage 0-1% where parallel-ssh keeps spinning on 10-11%.
  pytest_mh/utils/tc.py:88:90: E226 missing whitespace around arithmetic operator
  pytest_mh/utils/tc.py:123:84: E226 missing whitespace around arithmetic operator
@pbrezina
Copy link
Contributor Author

Good catch. Fixed.

@pbrezina pbrezina merged commit 57cbba4 into next-actions:master Jun 12, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Find alternative to parallel-ssh
2 participants