Skip to content

Latest commit

 

History

History
33 lines (21 loc) · 1.58 KB

README.md

File metadata and controls

33 lines (21 loc) · 1.58 KB

Distiller

Distiller is designed to streamline data acquisition and processing workflows for microscopy data. It integrates real-time monitoring, remote job launching, and post-processing capabilities, leveraging NERSC's high-performance computing (HPC) resources.

Features

  • Real-time Monitoring: Detects new microscopy data and displays metadata.
  • Automated HPC Processing: Launches jobs at NERSC for data transfer and electron counting using stempy.
  • Streaming Sessions: Enables real-time data processing on NERSC compute nodes.
  • Jupyter Notebooks: Facilitates post-processing via NERSC's Jupyter hub.

Technologies

  • Frontend: React
  • Backend: FastAPI, Kafka, Microservices
  • Integration: Superfacility API (NERSC)

Videos

  • Scans Page
    • Shows recent acquisitions. One can filter acquisitions to a date range on this page.
Distiller.Scans.mp4
  • Scan Page and Jupyter@NERSC
    • Shows individual scan, and launching an electron counting job on NERSC compute nodes. After the data is processed, we can create a templated Jupyter notebook running on NERSC's Jupyterhub for post-processing
Distiller.Scan.and.Jupyter.mp4
  • Streaming Session
    • A streaming session runs in Distiller (left) while collecting data on the TEAM 0.5 microscope with DigitalMicrograph (right).
Distiller.Streaming.mp4