flux commands

Improvement on sections and table for flux commands, and addition of flux accounting to container (likely will not easily work). Signed-off-by: vsoch <[email protected]>
flux-framework · Jul 20, 2024 · 74028b2 · 74028b2
1 parent c3012c6
commit 74028b2
Show file tree

Hide file tree

Showing 3 changed files with 935 additions and 270 deletions.
diff --git a/2024-RADIUSS-AWS/JupyterNotebook/README.md b/2024-RADIUSS-AWS/JupyterNotebook/README.md
@@ -32,7 +32,40 @@ to build them unless you are developing or changing them.
 
 If you do build (and use a different name) be sure to push your images to a public registry (or load them locally to your development cluster).
 
-
+### TODO
+
+After we add the flux-accounting:
+- after flux resource list, to see queues available (flux queue list)
+
+- more carbon copy examples
+- move flux batch aboe hierarchy
+  - transition into "what if I flux batch in my flux batch" (in my interactive allocation)
+  - yooo dawg
+  - check out riken tutorial for example
+  - drop the tree thing
+  - drop the throughput thing
+- better way to render script in the notebook
+- "construct a job submission object, called a jobspec"
+- Python, make handle, create job description (jobspec), submit and info (monitor)
+- collapse json dump
+- add watch / track events for some job
+ - reproduce cheese / pancakes example here
+ - how to list jobs
+ - hot to get output for a job
+- figure out way to collapse the last section
+- typo at top of chapter 2
+- do a section for flux exec? (show doing something across our "nodes"
+- move flux archive into main tutorial
+- Plumbing to Porcelain - "the toilet vs. the pipes" 💩️🚽️
+- squash Deep Dive into section above it
+- set up flux-accounting and see if it works
+  - how to specify a bank for a job 
+  - list banks (all) - flux account view-bank --tree
+  - specify banks - flux account view user $USER
+
+- Chapter 2: Flux Plumbing 💩️🚽️
+  - add flux job submit, show --dry-run
+
 ## Local Usage
 
 While the tutorial here is intended for deployment on AWS or Google Cloud, you can also give it a try on your local machine with a single container! You will need to [install Docker](https://docs.docker.com/engine/install/). When you have Docker available, you can build and run the tutorial with:

diff --git a/2024-RADIUSS-AWS/JupyterNotebook/docker/Dockerfile.spawn b/2024-RADIUSS-AWS/JupyterNotebook/docker/Dockerfile.spawn
@@ -79,6 +79,13 @@ RUN git clone https://github.com/flux-framework/dyad.git \
 COPY ./flux-tree/* /usr/libexec/flux/cmd/
 RUN chmod +x /usr/libexec/flux/cmd/flux-tree*
 
+# Flux accounting
+RUN git clone https://github.com/flux-framework/flux-accounting && \
+    cd flux-accounting && \
+    ./autogen.sh && \
+    ./configure --prefix=/usr && \
+    make && make install
+
 RUN apt-get update && apt-get install -y nodejs && apt-get clean && rm -rf /var/lib/apt/lists/*
 
 RUN wget https://nodejs.org/dist/v20.15.0/node-v20.15.0-linux-x64.tar.xz && \
@@ -119,6 +126,13 @@ COPY ./docker/start.sh /start.sh
 RUN mkdir -p $HOME/.local/share && \
     chmod 777 $HOME/.local/share
 
+# Quick setup of flux-accounting (not working)
+# RUN flux start /bin/bash -c "nohup flux account create-db && flux account-service & flux account add-bank root 1" && \
+#    flux start flux account add-bank --parent-bank=root default 1 && \
+#    flux start flux account add-user --username=jovyan --bank=default && \
+#    flux start flux jobtap load mf_priority.so && \
+#    flux start flux account-update-db
+
 USER ${NB_USER}
 
 CMD ["flux", "start", "--test-size=4", "jupyter", "lab"]