Skip to content

Commit

Permalink
major refresh, minor fixes
Browse files Browse the repository at this point in the history
  • Loading branch information
Danylo Dubinin committed Dec 31, 2024
1 parent 47817e2 commit 461af6e
Show file tree
Hide file tree
Showing 26 changed files with 660 additions and 964 deletions.
9 changes: 5 additions & 4 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,15 +1,16 @@
/.venv/
logs/

/build/
/dist/

log/
.env
/.venv/

.ipynb_checkpoints/

.pytest_cache/
__pycache__
*.egg-info

.idea/
*.iml

/sandbox/
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# changelog

## [3.5.1] - 2024-12-31
### fixed
- issues with logo copying
- various minor fixes

## [3.5.0] - 2022-07-25
### added
- installation options to change display name and improved options help
Expand Down
4 changes: 1 addition & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,3 +1 @@
- update [version](/src/zsh_jupyter_kernel/version).
- [keep a changelog](https://keepachangelog.com/en/1.0.0/).
- [package and distribute](/dist/)
[keep a changelog](https://keepachangelog.com).
281 changes: 134 additions & 147 deletions Pipfile.lock

Large diffs are not rendered by default.

78 changes: 15 additions & 63 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,74 +1,26 @@
# zsh kernel for jupyter
# Zsh kernel for Jupyter

![screenshot](screenshots/example.png)

a simple z shell jupyter kernel powered by python 3, pexpect and enthusiasm.

i love experimentation and tinkering, but usual shell terminals do not allow developing multiple different code snippets at once conveniently.
with shell kernels you can turn your scripts into notebooks!

if you find this product useful, please consider supporting me with a one-time tip.

feedback and suggestions are welcome in [github issues](https://github.com/dan-oak/zsh-jupyter-kernel/issues).
![](https://raw.githubusercontent.com/dahn-zk/zsh-jupyter-kernel/master/screenshots/example.png)
a simple Z Shell Jupyter kernel powered by Python, IPyKernel, Pexpect,
and enthusiasm — turn your scripts into notebooks!

## installation

install the python package from [pypi](https://pypi.org/project/zsh-jupyter-kernel/) using available package manager like `pip` or `pipenv`

(optional) by default the installation script will install the package *and* the kernel. the kernel location will be the same as the python environment from which the installation is done. check `python -m zsh_jupyter_kernel.install -h` for possible installation options if you want to install the kernel in a different environment or to change display name.

### pipenv
[PyPi](https://pypi.org/project/zsh-jupyter-kernel/):

Pip
```sh
pipenv --python 3.10 install notebook zsh_jupyter_kernel
python -m pip install zsh_jupyter_kernel
```

### pip


Pipenv
```sh
python -m pip install notebook zsh_jupyter_kernel
pipenv install zsh_jupyter_kernel
```

## usage

expect the kernel to work as usual including your zsh configs *except without stdin (user input) support and prompt line* and in the boundaries of the current jupyter limitations like not all ansi codes will work in certain cases ('\r') and in general things that expect terminal will have unexpected behavior.
see below why user input is not supported.

## technical overview

the kernel launches zsh as if it was a regular process launched from your terminal with a few minor settings to make sure it works with jupyter. there is slight chance it wont work with super complicated zshrc setups, but it works with majority of configs including oh-my-zsh.

### how does code execution work

the kernel configures zsh prompt string to its own custom value.
when a user requests a cell execution, the code is sent to the kernel.
then the kernel puts the frontend on hold, sends the code to zsh process, and waits for the prompt string to release the frontend and let the user request more code execution.

### code completion
### install kernel file

code completion is powered by quite a non-trivial script that involves multiple steps, including spawning another temporary zsh process and capturing completion options into a data structure that jupyter frontend understands.

### code inspection

code inspection is done by `man --pager ul` which sends the whole man page to the frontend.

### code completeness

code completeness is checked with the temporary zsh process and help of `EXEC` zsh option, which allows switching off code execution and simply check if the code is complete using the exit code of the zsh process itself.

### stderr

stderr content is just sent to the front-end as regular stdout the same way it is in a terminal.

### stdin

stdin is not supported because of the execution system when a process spawned by a user waits for stdin, there is no way to detect it.

jupyter is a request-reply system, and zsh as a shell that constantly receives input and prints whatever current processes want to output. there is no clear start and end of a code execution in a shell unlike in jupyter system: a front-end sends a code from a cell to a kernel and waits until the kernel sends the full output back.

because of these two different ways of interacting with user zsh jupyter kernel cannot process stdin in a way python kernel does on `input()`, meaning you will not be able to enter a sudo password or answer y/n to prompts or use a pager like less. when a spawned program waits for a user input, you will need to interrupt the kernel and use any options which do not require input.

## troubleshooting

please let me know if you encounter an error or unexpected behavior using [github issues](https://github.com/dan-oak/zsh-jupyter-kernel/issues).
i will respond as soon as possible.
see the help command for details
```sh
python -m zsh_jupyter_kernel.install --help
```
142 changes: 142 additions & 0 deletions demos/example.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" ___ _ ___ _ ____ ___ \n",
" / / |__ (_)_ __ / / |__ __ _ ___| |__ | ___|/ _ \\ \n",
" / /| '_ \\| | '_ \\ / /| '_ \\ / _` / __| '_ \\ |___ \\ (_) |\n",
" / / | |_) | | | | |/ / | |_) | (_| \\__ \\ | | | ___) \\__, |\n",
"/_/ |_.__/|_|_| |_/_/ |_.__/ \\__,_|___/_| |_| |____(_)/_/ \n",
" \n"
]
}
],
"source": [
"figlet $SHELL $ZSH_VERSION"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"--------------------------------------------------------------------------------\n"
]
}
],
"source": [
"function hr { printf \"%-*s\\n\" $(tput cols) | tr ' ' '-' }\n",
"hr"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[33m47817e2\u001b[m\u001b[33m (\u001b[m\u001b[1;36mHEAD\u001b[m\u001b[33m -> \u001b[m\u001b[1;32mmaster\u001b[m\u001b[33m)\u001b[m major cleanup\n",
"\u001b[33m3a49d9c\u001b[m minor reformat refactor\n",
"\u001b[33m1bb6fe6\u001b[m\u001b[33m (\u001b[m\u001b[1;31morigin/master\u001b[m\u001b[33m, \u001b[m\u001b[1;31morigin/HEAD\u001b[m\u001b[33m)\u001b[m Fix bug on copying logo files\n",
"--------------------------------------------------------------------------------\n",
" \u001b[31mM\u001b[m ../.gitignore\n",
" \u001b[31mM\u001b[m ../CONTRIBUTING.md\n",
" \u001b[31mM\u001b[m ../Pipfile.lock\n",
" \u001b[31mM\u001b[m ../README.md\n",
"\u001b[32mR\u001b[m ../build-publish.md -> ../docs/build-install-publish.md\n",
"\u001b[32mA\u001b[m\u001b[31mM\u001b[m ../docs/readme.md\n",
"\u001b[32mR\u001b[m\u001b[31mM\u001b[m ../todo.md -> ../docs/todo.md\n",
" \u001b[31mD\u001b[m ../example.ipynb\n",
" \u001b[31mM\u001b[m ../pyproject.toml\n",
" \u001b[31mD\u001b[m ../tests/sandbox.zsh.ipynb\n",
" \u001b[31mM\u001b[m ../tests/zsh_kernel_test.py\n",
" \u001b[31mM\u001b[m ../tools/install-pipenv.sh\n",
"\u001b[32mD\u001b[m ../zsh_jupyter_kernel/LICENSE.txt\n",
"\u001b[32mD\u001b[m ../zsh_jupyter_kernel/README.md\n",
" \u001b[31mM\u001b[m ../zsh_jupyter_kernel/__init__.py\n",
" \u001b[31mM\u001b[m ../zsh_jupyter_kernel/config.py\n",
" \u001b[31mM\u001b[m ../zsh_jupyter_kernel/fun.py\n",
" \u001b[31mM\u001b[m ../zsh_jupyter_kernel/install.py\n",
" \u001b[31mM\u001b[m ../zsh_jupyter_kernel/kernel.py\n",
" \u001b[31mM\u001b[m ../zsh_jupyter_kernel/reference.md\n",
"\u001b[32mD\u001b[m ../zsh_jupyter_kernel/version\n",
"\u001b[32mA\u001b[m\u001b[31mM\u001b[m ../zsh_jupyter_kernel/version.txt\n",
"\u001b[31m??\u001b[m ./\n",
"--------------------------------------------------------------------------------\n",
"\u001b[1;33mexample.ipynb\u001b[0m \u001b[1;33msandbox.zsh.ipynb\u001b[0m\n"
]
}
],
"source": [
"git --no-pager log --oneline ...HEAD~3\n",
"hr\n",
"git status --short\n",
"hr\n",
"git ls-files src/zsh_jupyter_kernel | xargs exa"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Weather report: Chernivtsi\n",
"\n",
" \u001b[38;5;226m \\ / \u001b[0m Sunny\n",
" \u001b[38;5;226m .-. \u001b[0m \u001b[38;5;049m+3\u001b[0m(\u001b[38;5;049m2\u001b[0m) °C\u001b[0m \n",
" \u001b[38;5;226m ― ( ) ― \u001b[0m \u001b[1m↗\u001b[0m \u001b[38;5;118m4\u001b[0m km/h\u001b[0m \n",
" \u001b[38;5;226m `-’ \u001b[0m 10 km\u001b[0m \n",
" \u001b[38;5;226m / \\ \u001b[0m 0.0 mm\u001b[0m \n"
]
}
],
"source": [
"curl -s wttr.in/Chernivtsi | head -7"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Z shell (dev)",
"language": "zsh",
"name": "zsh-dev"
},
"language_info": {
"codemirror_mode": "shell",
"file_extension": ".zsh",
"mimetype": "text/x-zsh",
"name": "zsh",
"pygments_lexer": "shell",
"version": "5.7.1"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
15 changes: 3 additions & 12 deletions tests/sandbox.zsh.ipynb → demos/sandbox.zsh.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,7 @@
"name": "stdout",
"output_type": "stream",
"text": [
"fun_test.py msgspec_v5.py\t zsh_kernel_tests.py\n",
"__init__.py pexpect_zsh_test.py\n"
"example.ipynb sandbox.zsh.ipynb\n"
]
}
],
Expand All @@ -48,21 +47,13 @@
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"id": "2d379c32-6785-4c4e-854b-98b94bc37426",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Z shell",
"display_name": "Z shell (dev)",
"language": "zsh",
"name": "zsh"
"name": "zsh-dev"
},
"language_info": {
"codemirror_mode": "shell",
Expand Down
12 changes: 1 addition & 11 deletions build-publish.md → docs/build-install-publish.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,20 +9,10 @@ python -m build

twine upload --repository testpypi dist/*

and run in a separate window and a separate directory to test the
installation from testpypi:

```zsh
pipenv install --python=3.10 notebook
pipenv shell
pip install --index-url https://test.pypi.org/simple/ zsh-jupyter-kernel
python -m zsh_jupyter_kernel.install --sys-prefix
```

to upload to production pypi set the version and execute:

```zsh
version=`cat version`
#version=`cat version`
twine upload \
dist/zsh-jupyter-kernel-$version.tar.gz \
dist/zsh_jupyter_kernel-$version-py3-none-any.whl
Expand Down
61 changes: 61 additions & 0 deletions docs/readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Zsh kernel for Jupyter

![](../screenshots/example.png)
a simple Z Shell Jupyter kernel powered by Python, IPyKernel, Pexpect,
and enthusiasm — turn your scripts into notebooks!

## technical details

the kernel launches Zsh as if it was a regular process launched from your
terminal with a few minor settings to make sure it works with Jupyter. there is
slight chance it wont work with super complicated zshrc setups, but it works
with majority of configs including oh-my-zsh.

### code execution
the kernel configures Zsh prompt string to its own custom value.
when a user requests a cell execution, the code is sent to the kernel.
then the kernel puts the frontend on hold, sends the code to Zsh process, and
waits for the prompt string to release the frontend and let the user request
more code execution.

### code completion
![](../screenshots/completion.png)
code completion is powered by quite a non-trivial script that involves multiple
steps, including spawning another temporary Zsh process and capturing completion
options into a data structure that jupyter frontend understands.

### code inspection
![](../screenshots/inspection.png)
code inspection is done by `man --pager ul` which sends the whole man page to
the frontend.

### code completeness

code completeness is checked with the temporary Zsh process and help of `EXEC`
Zsh option, which allows switching off code execution and simply check if the
code is complete using the exit code of the Zsh process itself.

### stderr

stderr content is just sent to the front-end as regular stdout the same way it
is in a terminal.

### stdin

stdin is not supported because of the execution system when a process spawned by
a user waits for stdin, there is no way to detect it.

jupyter is a request-reply system, and Zsh as a shell that constantly receives
input and prints whatever current processes want to output. there is no clear
start and end of a code execution in a shell unlike in jupyter system: a
front-end sends a code from a cell to a kernel and waits until the kernel sends
the full output back.

because of these two different ways of interacting with user Zsh jupyter kernel
cannot process stdin in a way python kernel does on `input()`, meaning you will
not be able to enter a sudo password or answer y/n to prompts or use a pager
like less. when a spawned program waits for a user input, you will need to
interrupt the kernel and use any options which do not require input.
3 changes: 1 addition & 2 deletions zsh_jupyter_kernel/reference.md → docs/reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,9 @@
- zsh-options: https://linux.die.net/man/1/zshoptions
- zsh-functions: http://zsh.sourceforge.net/Doc/Release/Functions.html
- zsh-bracketed-paste: https://archive.zhimingwang.org/blog/2015-09-21-zsh-51-and-bracketed-paste.html
- zsh-prompts: https://jlk.fjfi.cvut.cz/arch/manpages/man/zshparam.1#PARAMETERS_USED_BY_THE_SHELL
- zsh-hooks: http://zsh.sourceforge.net/Doc/Release/User-Contributions.html
- Section "Manipulating Hook Functions"
Different plugins (e.g. oh-my-zsh with custom themes) use `precmd`/`preexec` hooks to set prompts. `add-zsh-hook -D <hook> <pattern>` allows to delete all assosiated functions from a hook.
- interrupt: https://jupyter-client.readthedocs.io/en/latest/messaging.html#kernel-interrupt
Interrupt by message does not work now but works by signal. This is
confusing as hell to me as previously it was not working. Whatever...
- kernel-specs: https://jupyter-client.readthedocs.io/en/latest/kernels.html#kernelspecs
Loading

0 comments on commit 461af6e

Please sign in to comment.