Merge pull request #68 from streamsync-cloud/dev

Dev

CONTRIBUTING.md

@@ -0,0 +1,31 @@
+# Contributing to Streamsync
+
+Thank you for your interest in contributing to Streamsync.
+
+## Ways to contribute
+
+Beyond contributing to the repository, some ways to contribute to this project include: 
+
+- *Reporting bugs*. Bug reports are relatively easy to write, but have a big impact. Please include the steps required to reproduce the bug. Use "Issues" on GitHub. This is an example of a [wonderful bug report](https://github.com/streamsync-cloud/streamsync/issues/24).
+- *Creating content*. Think articles or tutorials. It doesn't have to be overwhelmingly positive; constructive criticism is appreciated. A great example is [this review](https://jreyesr.github.io/posts/streamsync-review/). A YouTube tutorial would be fantastic!
+- *Browse Issues and Discussions*. Browse these sections on GitHub and see if you can help.
+- *Suggesting valuable enhancements*. If you think of a feature that can have a positive impact, suggest it. Please use the "Discussions" on GitHub.
+- *Sponsoring the project*. Helps offset hosting and other expenses.
+- *Promoting the project*. Star it, share on LinkedIn or other social media.
+
+## Contributing to the repository
+
+If you wish to contribute to the repository, please first discuss the change you wish to make via issue, email, or any other method with the owners of this repository before making a change. Failure to discuss the changes beforehand will likely cause your pull request to be rejected, regrettably.
+
+Make sure to run the tests, which can be found in `/tests`, and pass mypy validation. Code formatting is important; Prettier is used in the frontend while autopep8 is used in the backend.
+
+Pull requests should be done on the `dev` branch. When the release is finalised, `dev` will be merged into `master`.
+
+## Setting up a development environment
+
+Whether you're interested in contributing to the repository, creating a fork, or just improving your understanding of Streamsync, these are the suggested steps for setting up a development environment.
+- Install streamsync[test] or streamsync[build].
+- You can install the package in editable mode using `pip install -e .`, which will make it more convenient if you intend to tweak the backend.
+- Run streamsync in port 5000. For example, `streamsync edit hello --port 5000`.
+- Install dependencies and run `npm run dev` in `/ui`. This runs the frontend for Streamsync in development mode while proxying requests to port 5000.
+- A breakdown of the steps required for packaging can be found in `./build.sh`. Notably, it includes compiling the frontend and taking it from `/ui` and into the Python package.

docs/docs/.vitepress/config.ts

@@ -5,11 +5,14 @@ export default {
   themeConfig: {
     logo: "/logo.svg",
     socialLinks: [
-      { icon: "github", link: "https://github.com/streamsync-cloud/streamsync" },
+      {
+        icon: "github",
+        link: "https://github.com/streamsync-cloud/streamsync",
+      },
     ],
     nav: [
       { text: "Documentation", link: "/getting-started" },
-      { text: "Components", link: "/component-list" }
+      { text: "Components", link: "/component-list" },
     ],
     sidebar: [
       {
@@ -30,6 +33,14 @@ export default {
             text: "Backend-initiated actions",
             link: "/backend-initiated-actions",
           },
+          {
+            text: "Stylesheets",
+            link: "/stylesheets",
+          },
+          {
+            text: "Frontend scripts",
+            link: "/frontend-scripts",
+          },
           { text: "Page routes", link: "/page-routes" },
           { text: "Sessions", link: "/sessions" },
           { text: "Custom server", link: "/custom-server" },

docs/docs/frontend-scripts.md

@@ -0,0 +1,72 @@
+# Frontend scripts
+
+Streamsync can import custom JavaScript/ES6 modules from the frontend. Module functions can be triggered from the backend.
+
+## Importing an ES6 module
+
+Similarly to [stylesheets](/stylesheets), frontend scripts are imported via Streamsync's `mail` capability. This allows you to trigger an import for all or specific sessions at any time during runtime. When the `import_frontend_module` method is called, this triggers a dynamic [import()](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/import) call in the frontend.
+
+The `import_frontend_module` method takes the `module_key` and `specifier` arguments. The `module_key` is an identifier used to store the reference to the module, which will be used later to call the module's functions. The `specifier` is the path to the module, such as `/static/mymodule.js`. It needs to be available to the frontend, so storing in the `/static/` folder is recommended.
+
+The following code imports a module during event handling.
+
+```py
+def handle_click(state):
+    state.import_script("my_script", "/static/script.js")
+```
+
+If you want the module to be imported during initialisation, use the initial state.
+
+```py
+initial_state = ss.init_state({
+    "counter": 1
+})
+
+initial_state.import_script("my_script", "/static/script.js")
+```
+
+::: tip Use versions to avoid caching
+Similarly to stylesheets, your browser may cache modules, preventing updates from being reflected. Append a querystring to invalidate the cache, e.g. use `/static/script.js?3`. 
+:::
+
+## Writing a module
+
+The module should be a standard ES6 module and export at least one function, enabling it to be triggered from the backend. As per JavaScript development best practices, modules should have no side effects.
+
+An example of a module is shown below.
+
+```js
+let i = 0;
+
+export function sendAlert(personName) {
+    i++;
+    alert(`${personName}, you've been alerted. This is alert ${i}.`);
+}
+```
+
+## Calling a function
+
+Once the module is imported, functions can be called from the backend using the `call_frontend_function` method of state. This function takes three arguments. The first, `module_key` is the identifier used to import the module. The second, `function_name` is the name of the exported frontend function. The third, `args` is a `List` containing the arguments for the call. 
+
+The following event handler triggers the frontend function defined in the section above.
+
+```py
+def handle_click(state):
+    state.call_frontend_function("myscript", "sendAlert", ["Bob"])
+```
+
+## Frontend core
+
+You can access Streamsync's frontend core via `globalThis.core`, unlocking all sorts of functionality. Notably, you can use `evaluate_expression` to get values from state.
+
+```js
+export function alertHueRotationValue() {
+    const core = globalThis.core;
+    const hueRotation = core.evaluateExpression("hue_rotation");
+    alert(`Value of hue_rotation is ${hueRotation}`);
+}
+```
+
+::: warning Here be dragons
+Effectively using Streamsync's core can be challenging and will likely entail reading its [source code](https://github.com/streamsync-cloud/streamsync/blob/master/ui/src/core/index.ts). Furthermore, it's considered an internal capability rather than a public API, so it may unexpectedly change between releases.
+:::

docs/docs/getting-started.md

@@ -29,7 +29,7 @@ A Streamsync app is a folder with the following items.
 
 - `main.py`. The entry point for the app. You can import anything you need from here.
 - `ui.json`. Contains the UI component declarations. Maintained by Builder, the framework's visual editor.
-- `static/`. This folder contains frontend-facing static files which you might want to distribute with your app. For example, images.
+- `static/`. This folder contains frontend-facing static files which you might want to distribute with your app. For example, images and stylesheets.
 
 ## Start the editor
 
@@ -44,11 +44,8 @@ streamsync edit testapp
 
 This command will provide you with a local URL which you can use to access Builder.
 
-::: warning Streamsync Builder shouldn't be directly exposed to the Internet.
-
-It's not safe and it won't work, as requests from non-local origins are rejected.
-
-If you need to access Builder remotely, we recommend setting up a SSH tunnel.
+::: warning It's not recommended to expose Streamsync Builder to the Internet.
+If you need to access Builder remotely, we recommend setting up a SSH tunnel. By default, requests from non-local origins are rejected, as a security measure to protect against drive-by attacks. If you need to disable this protection, use the flag `--enable-remote-edit`. 
 :::
 
 ## Run an app

docs/docs/stylesheets.md

@@ -0,0 +1,95 @@
+# Stylesheets
+
+The appearance of your application can be fully customised via CSS stylesheets. These are dynamically linked during runtime and can be switched from the backend, targeting all or specific sessions.
+
+## Importing a stylesheet
+
+Stylesheet imports are triggered via Streamsync's `mail`, similarly to other features discussed in [Backend-initiated actions](/backend-initiated-actions). When the import is triggered, the frontend downloads the specified stylesheet and creates a `style` element with its contents.
+
+The `import_stylesheet` method takes the `stylesheet_key` and `path` arguments. The first works as an identifier that will let you override the stylesheet later if needed. The second is the path to the CSS file.The path specified needs to be available to the frontend, so storing it in the `/static` folder of your app is recommended.
+
+The following code imports a stylesheet when handling an event.
+
+```py
+def handle_click(state):
+    state.import_stylesheet("theme", "/static/custom.css")
+```
+
+In many cases, you'll want to import a stylesheet during initialisation time, for all users. This is easily achievable via the initial state, as shown below.
+
+```py
+initial_state = ss.init_state({
+    "counter": 1
+})
+
+initial_state.import_stylesheet("theme", "/static/custom.css")
+```
+
+::: tip Use versions to avoid caching
+During development time, stylesheets may be cached by your browser, preventing updates from being reflected. Append a querystring to bust the cache, e.g. use `/static/custom.css?3`. 
+:::
+
+
+## Applying CSS classes
+
+You can use the property *Custom CSS classes* in Builder's *Component Settings* to apply classes to a component, separated by spaces. Internally, this will apply the classes to the root HTML element of the rendered component.
+
+![Stylesheets - Component Settings](./images/stylesheets.component-settings.png)
+
+## Tips for effective stylesheets
+
+The CSS code for the class used earlier, `bubblegum`, can be found below. Note how the `!important` flag is used when targetting style attributes that are configurable via Builder. If the flag isn't included, these declarations will not work, because built-in Streamsync styling is of higher specificity. 
+
+```css
+.bubblegum {
+    background: #ff63ca !important;
+    line-height: 1.5;
+    transform: rotate(-5deg);
+}
+
+/* Targeting an element inside the component root element */
+.bubblegum > h2 {
+    color: #f9ff94 !important;
+}
+```
+
+::: warning Component structure may change
+When targeting specific HTML elements inside components, take into account that the internal structure of components may change across Streamsync versions.
+:::
+
+Alternatively, you can override Streamsync's style variables. This behaves slightly differently though; style variables are inherited by children components. For example, if a *Section* has been assigned the `bubblegum` class, its children will also have a pink background by default.
+
+```css
+.bubblegum {
+    --containerBackgroundColor: #ff63ca;
+    --primaryTextColor: #f9ff94;
+    line-height: 1.5;	
+    transform: rotate(-5deg);
+}
+```
+
+The class can be used in *Component Settings*. If the stylesheet is imported, the effect will be immediate. In case the stylesheet has been modified since imported, it'll need to be imported again.
+
+![Stylesheets - Applied Classes](./images/stylesheets.applied-classes.png)
+
+## Targeting component types
+
+Streamsync components have root HTML elements with a class linked to their type. For example, *Dataframe* components use the class *CoreDataframe*. When writing a stylesheet, you can target all *Dataframe* components as shown below.
+
+```css
+.CoreDataframe {
+    line-height: 2.0;
+}
+```
+
+## Implementing themes
+
+It's possible to switch stylesheets during runtime, by specifying the same `stylesheet_key` in subsequent calls. This allows you to implement a "theme" logic if desired, targeting the whole or a specific part of your app. 
+
+```py
+def handle_cyberpunk(state):
+    state.import_stylesheet("theme", "/static/cyberpunk_theme.css")
+
+def handle_minimalist(state):
+    state.import_stylesheet("theme", "/static/minimalist_theme.css")
+```

pyproject.toml

@@ -12,7 +12,7 @@ keywords = ["data apps", "gui", "ui"]
 license = { text = "Apache 2.0" }
 classifiers = ["Development Status :: 4 - Beta"]
 dependencies = [
-    "pydantic == 1.10.11",
+    "pydantic >= 2.1.1, < 3",
     "fastapi >= 0.89.1, < 1",
     "websockets >= 10.4, < 11",
     "uvicorn >= 0.20.0, < 1",

src/streamsync/__init__.py

@@ -2,7 +2,7 @@
 from streamsync.core import Readable, FileWrapper, BytesWrapper, Config
 from streamsync.core import initial_state, component_manager, session_manager, session_verifier
 
-VERSION = "0.1.10"
+VERSION = "0.1.12"
 
 component_manager
 session_manager

src/streamsync/command_line.py

@@ -18,20 +18,23 @@ def main():
         "--port", help="The port on which to run the server.")
     parser.add_argument(
         "--host", help="The host on which to run the server. Use 0.0.0.0 to share in your local network.")
+    parser.add_argument(
+        "--enable-remote-edit", help="Set this flag to allow non-local requests in edit mode.", action='store_true')
 
     args = parser.parse_args()
     command = args.command
     default_port = 3006 if command in ("edit", "hello") else 3005
+    enable_remote_edit = args.enable_remote_edit
 
     port = int(args.port) if args.port else default_port
     absolute_app_path = _get_absolute_app_path(
         args.path) if args.path else None
     host = args.host if args.host else None
 
-    _perform_checks(command, absolute_app_path, host)
-    _route(command, absolute_app_path, port, host)
+    _perform_checks(command, absolute_app_path, host, enable_remote_edit)
+    _route(command, absolute_app_path, port, host, enable_remote_edit)
 
-def _perform_checks(command: str, absolute_app_path: str, host: Optional[str]):
+def _perform_checks(command: str, absolute_app_path: str, host: Optional[str], enable_remote_edit: Optional[bool]):
     is_path_folder = absolute_app_path is not None and os.path.isdir(absolute_app_path)
 
     if command in ("run", "edit") and is_path_folder is False:
@@ -44,7 +47,9 @@ def _perform_checks(command: str, absolute_app_path: str, host: Optional[str]):
 
     if command in ("edit", "hello") and host is not None:
         logging.warning("Streamsync has been enabled in edit mode with a host argument\nThis is enabled for local development purposes (such as a local VM).\nDon't expose Streamsync Builder to the Internet. We recommend using a SSH tunnel instead.")
-        logging.warning("Streamsync Builder will only accept local requests (via HTTP origin header).")
+
+    if command in ("edit", "hello") and enable_remote_edit is True:
+        logging.warning("The remote edit flag is active. Streamsync Builder will accept non-local requests. Please make sure the host is protected to avoid drive-by attacks.")
 
     if command in ("hello"):
         try:
@@ -56,19 +61,19 @@ def _perform_checks(command: str, absolute_app_path: str, host: Optional[str]):
             sys.exit(1)
 
 
-def _route(command: str, absolute_app_path: str, port: int, host: Optional[str]):
+def _route(command: str, absolute_app_path: str, port: int, host: Optional[str], enable_remote_edit: Optional[bool]):
     if host is None:
         host = "127.0.0.1"
     if command in ("edit"):
         streamsync.serve.serve(
-            absolute_app_path, mode="edit", port=port, host=host)
+            absolute_app_path, mode="edit", port=port, host=host, enable_remote_edit=enable_remote_edit)
     if command in ("run"):
         streamsync.serve.serve(
             absolute_app_path, mode="run", port=port, host=host)
     elif command in ("hello"):
         create_app("hello", template_name="hello", overwrite=True)
         streamsync.serve.serve("hello", mode="edit",
-                               port=port, host=host)
+                               port=port, host=host, enable_remote_edit=enable_remote_edit)
     elif command in ("create"):
         create_app(absolute_app_path)
 

src/streamsync/core.py

@@ -101,10 +101,6 @@ def serialise(self, v: Any) -> Union[Dict, List, str, bool, int, float, None]:
             return self._serialise_list_recursively(v)
         if isinstance(v, (str, bool)):
             return v
-        if isinstance(v, (int, float)):
-            if math.isnan(v):
-                return None
-            return v
         if v is None:
             return v
 
@@ -114,8 +110,17 @@ def serialise(self, v: Any) -> Union[Dict, List, str, bool, int, float, None]:
         v_mro = [
             f"{x.__module__}.{x.__name__}" for x in inspect.getmro(type(v))]
 
+        if isinstance(v, (int, float)):
+            if "numpy.float64" in v_mro:
+                return float(v)
+            if math.isnan(v):
+                return None
+            return v
+
         if "matplotlib.figure.Figure" in v_mro:
             return self._serialise_matplotlib_fig(v)
+        if "numpy.float64" in v_mro:
+            return float(v)
         if "numpy.ndarray" in v_mro:
             return self._serialise_list_recursively(v.tolist())
         if "pandas.core.frame.DataFrame" in v_mro:
@@ -357,8 +362,27 @@ def set_page(self, active_page_key: str) -> None:
     def set_route_vars(self, route_vars: Dict[str, str]) -> None:
         self.add_mail("routeVarsChange", route_vars)
 
-# TODO Consider switching Component to use Pydantic
+    def import_stylesheet(self, stylesheet_key: str, path: str) -> None:
+        self.add_mail("importStylesheet", {
+            "stylesheetKey": stylesheet_key,
+            "path": path
+        })
+
+    def import_frontend_module(self, module_key: str, specifier: str) -> None:
+        self.add_mail("importModule", {
+            "moduleKey": module_key,
+            "specifier": specifier
+        })
 
+    def call_frontend_function(self, module_key: str, function_name: str, args: List) -> None:
+        self.add_mail("functionCall", {
+            "moduleKey": module_key,
+            "functionName": function_name,
+            "args": args
+        })
+
+
+# TODO Consider switching Component to use Pydantic
 
 class Component: