Migration to Lab 3.x (#54)

* mostly working

* more cleanup for server-side consistency, updated dependency in package.json for linux build, auto updates in progress, more fixes to UI

* adding docker pieces for a local slurm cluster

* updating dev instructions

* Cleaned out datatables dependency and surrounding pieces that were not needed, css modified, extension user settings added, jobsubmit modal moved to a form in a tab, added icons for some of the buttons for additional context, lots of cleanup

* mostly working

* more cleanup for server-side consistency, updated dependency in package.json for linux build, auto updates in progress, more fixes to UI

* adding docker pieces for a local slurm cluster

* updating dev instructions

* Cleaned out datatables dependency and surrounding pieces that were not needed, css modified, extension user settings added, jobsubmit modal moved to a form in a tab, added icons for some of the buttons for additional context, lots of cleanup

* fix for textarea of jobsubmit, was inserting serverroot path at the beginning incorrectly

* disable user input delay for input path

* working on auto-reload behavior, temp patch for table overflow issues

* added a search box, loading indicator (WIP), and config for slurm commands

* fix prettier/eslint formatting

* update build.yml

* added loader, fixed restorer

* fix prettier problems

* fix prettier problems

* style changes and button rearranging

* style changes & fixed filtering bug

* set max column width, change default sort

* work on submitting jobs from file

* change onChange for typed file path

* fix autoreload, fix update fileitems

* change when lastSqueueFetch is set, append alerts

* style updates, column sort, table auto loading

* only call updateFileitems() when JobSubmit tab is active

* swap filter and useronly inputs

* create base selenium test

* fix .eslintrc.js and comment out long logging messages

* update README.md

* update README.md

* missed some whitespace on merge

Co-authored-by: Josh Geden <[email protected]>
Co-authored-by: Josh Geden <[email protected]>

.eslintignore

@@ -0,0 +1,5 @@
+node_modules
+dist
+coverage
+**/*.d.ts
+tests

.eslintrc.js

@@ -0,0 +1,43 @@
+module.exports = {
+  env: {
+    browser: true,
+    node: true
+  },
+  extends: [
+    'eslint:recommended',
+    'plugin:@typescript-eslint/eslint-recommended',
+    'plugin:@typescript-eslint/recommended',
+    'plugin:prettier/recommended'
+  ],
+  parser: '@typescript-eslint/parser',
+  parserOptions: {
+    // project: 'tsconfig.json',
+    sourceType: 'module'
+  },
+  plugins: ['react', '@typescript-eslint', '@emotion'],
+  rules: {
+    '@typescript-eslint/naming-convention': [
+      'error',
+      {
+        selector: 'interface',
+        format: ['PascalCase'],
+        custom: {
+          regex: '^I[A-Z]',
+          match: true
+        }
+      }
+    ],
+    '@typescript-eslint/no-unused-vars': ['warn', { args: 'none' }],
+    '@typescript-eslint/no-explicit-any': 'off',
+    '@typescript-eslint/no-namespace': 'off',
+    '@typescript-eslint/no-use-before-define': 'off',
+    '@typescript-eslint/quotes': [
+      'error',
+      'single',
+      { avoidEscape: true, allowTemplateLiterals: false }
+    ],
+    curly: ['error', 'all'],
+    eqeqeq: 'error',
+    'prefer-arrow-callback': 'error'
+  }
+};

.github/workflows/build.yml

@@ -0,0 +1,58 @@
+name: Build
+
+on:
+  push:
+    branches: main
+  pull_request:
+    branches: '*'
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+    - name: Checkout
+      uses: actions/checkout@v2
+    - name: Install node
+      uses: actions/setup-node@v1
+      with:
+       node-version: '12.x'
+    - name: Install Python
+      uses: actions/setup-python@v2
+      with:
+        python-version: '3.7'
+        architecture: 'x64'
+
+
+    - name: Setup pip cache
+      uses: actions/cache@v2
+      with:
+        path: ~/.cache/pip
+        key: pip-3.7-${{ hashFiles('package.json') }}
+        restore-keys: |
+          pip-3.7-
+          pip-
+
+    - name: Get yarn cache directory path
+      id: yarn-cache-dir-path
+      run: echo "::set-output name=dir::$(yarn cache dir)"
+    - name: Setup yarn cache
+      uses: actions/cache@v2
+      id: yarn-cache # use this to check for `cache-hit` (`steps.yarn-cache.outputs.cache-hit != 'true'`)
+      with:
+        path: ${{ steps.yarn-cache-dir-path.outputs.dir }}
+        key: yarn-${{ hashFiles('**/yarn.lock') }}
+        restore-keys: |
+          yarn-
+    
+    - name: Install dependencies
+      run: python -m pip install -U jupyterlab~=3.0 jupyter_packaging~=0.7.9
+    - name: Build the extension
+      run: |
+        jlpm
+        jlpm run eslint:check
+        python -m pip install .
+
+        jupyter server extension list 2>&1 | grep -ie "jupyterlab_slurm.*OK"
+
+        jupyter labextension list 2>&1 | grep -ie "jupyterlab-slurm.*enabled"
+        python -m jupyterlab.browser_check

.gitignore

@@ -1,26 +1,25 @@
 *.bundle.*
+lib/
+node_modules/
+*.egg-info/
 .ipynb_checkpoints
-.DS_STORE
+*.tsbuildinfo
+jupyterlab_slurm/labextension
 
-# binaries
-*.pyc
-*.pyo
-*.pyd
-*.py[cod]
+# Created by https://www.gitignore.io/api/python
+# Edit at https://www.gitignore.io/?templates=python
+
+### Python ###
+# Byte-compiled / optimized / DLL files
 __pycache__/
+*.py[cod]
 *$py.class
-*.so
 
-# logs
-pip-log.txt
-pip-delete-this-directory.txt
-.cache
-*.log
+# C extensions
+*.so
 
-# distribution/packaging
-node_modules/
-package-lock.json
-bin/
+# Distribution / packaging
+.Python
 build/
 develop-eggs/
 dist/
@@ -33,10 +32,81 @@ parts/
 sdist/
 var/
 wheels/
-yarn.lock
-*.egg-info/
-*.egg
+pip-wheel-metadata/
+share/python-wheels/
 .installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+.hypothesis/
+.pytest_cache/
+
+# Translations
+*.mo
+*.pot
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# pyenv
+.python-version
+
+# celery beat schedule file
+celerybeat-schedule
+
+# SageMath parsed files
+*.sage.py
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# Mr Developer
+.mr.developer.cfg
+.project
+.pydevproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# End of https://www.gitignore.io/api/python
 
-# typescript
-tsconfig.tsbuildinfo
+# OSX files
+.DS_Store

.prettierignore

@@ -0,0 +1,5 @@
+node_modules
+**/node_modules
+**/lib
+**/package.json
+jupyterlab_slurm

.prettierrc

@@ -0,0 +1,5 @@
+{
+  "singleQuote": true,
+  "trailingComma": "none",
+  "arrowParens": "avoid"
+}

MANIFEST.in

@@ -1,2 +1,25 @@
+include LICENSE
+include README.md
+include pyproject.toml
+
 recursive-include jupyter-config *.json
-prune node_modules
+
+include package.json
+include install.json
+include ts*.json
+include yarn.lock
+
+graft jupyterlab_slurm/labextension
+
+# Javascript files
+graft src
+graft style
+prune **/node_modules
+prune lib
+
+# Patterns to exclude from any directory
+global-exclude *~
+global-exclude *.pyc
+global-exclude *.pyo
+global-exclude .git
+global-exclude .ipynb_checkpoints

README.md

@@ -7,8 +7,8 @@ providing simple and intuitive controls for viewing and managing jobs on the que
 
 ## Prerequisites
 
-* JupyterLab >= 1.0
-* Node.js 5+
+* JupyterLab >= 3.0
+* Node.js 14+
 * Slurm
 
 
@@ -19,7 +19,6 @@ Jupyter notebook server extension. Install these using the command line with
 
 ```bash
 pip install jupyterlab_slurm
-jupyter labextension install jupyterlab-slurm
 ```
 
 If you are running Notebook 5.2 or earlier, enable the server extension by running
@@ -37,24 +36,51 @@ of the palette and the launcher.
 
 As described in the [JupyterLab documentation](https://jupyterlab.readthedocs.io/en/stable/developer/extension_dev.html#extension-authoring) for a development install of the labextension you can run the following in this directory:
 
+### Setup a local slurm cluster
+
 ```bash
-jlpm install   # Install npm package dependencies
-jlpm run build  # Compile the TypeScript sources to Javascript
-jupyter labextension install  # Install the current directory as an extension
+git clone https://github.com/giovtorres/slurm-docker-cluster
+cd slurm-docker-cluster
+git clone --branch lab3 https://github.com/NERSC/jupyterlab-slurm.git
+cp jupyterlab-slurm/slurm_cluster/docker-compose.yml .
+# from slurm-docker-cluster README
+docker build -t slurm-docker-cluster:19.05.1 .
+# if you encounter an error with the PGP key step
+# update line 46 with gpg --keyserver pgp.mit.edu ...
+# this will build the jupyterlab image minimal-notebook with a slurm client
+docker-compose build
+# start the cluster
+docker-compose up -d
+# register the slurm cluster
+./register_cluster.sh
+# run munged on the jupyterlab instance to get the slurm commands to connect
+docker-compose exec jupyterlab bash
+runuser -u slurm -- munged
+# test that squeue comes back with a header, if it gets stuck you can't connect
+squeue
 ```
 
-To rebuild the extension:
+### Install jupyterlab-slurm into your environment
 
 ```bash
+docker-compose exec -u jovyan jupyterlab bash
+cd /usr/local/jupyterlab-slurm/
+# install jupyter_packaging which is a missing dependency
+pip install jupyter_packaging
+# this command takes a while the first it is run
+pip install -e .
+# point the labextension dev install at current dir
+jupyter labextension develop --overwrite .
+
+# rerun this if there are updates:
 jlpm run build
 ```
 
-If you run JupyterLab in watch mode (`jupyter lab --watch`) it will automatically pick
-up changes to the built extension and rebundle itself.
-
-To run an editable install of the server extension, run
-
+### Restart the jupyterlab docker container
 ```bash
-pip install -e .
-jupyter serverextension enable --sys-prefix jupyterlab_slurm
-```
+docker compose restart jupyterlab
+
+# rerun munged on the jupyterlab instance
+docker compose exec jupyterlab bash
+runuser -u slurm -- munged
+```

install.json

@@ -0,0 +1,5 @@
+{
+  "packageManager": "python",
+  "packageName": "jupyterlab_slurm",
+  "uninstallInstructions": "Use your Python package manager (pip, conda, etc.) to uninstall the package jupyterlab_slurm"
+}

jupyter-config/jupyter_server_config.d/jupyterlab_slurm.json

@@ -0,0 +1,7 @@
+{
+  "ServerApp": {
+    "jpserver_extensions": {
+      "jupyterlab_slurm": true
+    }
+  }
+}