Using Plotly-resampler for dash app with multiple users: problem with global variable

[malammar0]

Hi,
We are developing a dash app that plots a lot of data and we were looking into plotly-resampler to improve our app's performance. We were able to integrate plotly-resampler and it worked wonderfully. However, our app has multiple users, and we are using plotly-resampler with a global variable (fig: FigureResampler = FigureResampler() in the dash-app.py example). This means that different users can load different data and interfere with each other's sessions and graphs. Do you have any recommendations on how to solve this issue?
Thank you

[jonasvdd]

Related issues: #56

[malammar0]

Thank you Jonas,
Yes we did integrate plotly-resampler in our dash-app following the examples in the code you provided. The problem is that when we access the FigureResampler object in the callbacks, we access it as a global variable.

[jonasvdd]

Hi @malammar0,

As of now (stated in #56), plotly-resampler does not (yet) support this, but a workaround in mentioned there.

workaround: you use a global graph-dict in which you can create a new UUID per user/session and know which user/session session makes a request. Which would allow to use a graph_dict which looks like:

{ 
   "<user_uuid_str>": {
          "<graph_uuid_str>": FigureResampler(...), 
          ... 
    },
    ...
}

See also this example: https://github.com/predict-idlab/plotly-resampler/blob/2ed7423ac43cc6c500ef309d75c9e1f9d7b187f5/examples/dash_apps/construct_dynamic_figures.py

---

Next to this: we are looking into serialization properties of plotly-resampler (see #87 ) which might enable a more clean, non-global variable solution. (Will update this post accordingly with the findings we will get)

[malammar0]

Hi Jonas, thank you for recommending this dictionary solution. The problem we've had with implementing this solution is that when we've deployed the app with multiple workers, we have different global variables for each worker/process. It seems that the users get switched between different processes and thus they have access to a different global variable (and thus a different dictionary!) and so they don't consistently access their respective figures. Will also update this post when that finding is confirmed.

[malammar0]

Update: The plotly documentation confirms that multiple workers don't share memory: https://dash.plotly.com/sharing-data-between-callbacks ->
"Why Global Variables Will Break Your App" -> "When Dash apps run across multiple workers, their memory is not shared. This means that if you modify a global variable in one callback, that modification will not be applied to the other workers / processes."

[jonasvdd]

Hi @malammar0!

Very fair point! I have a p.o.c. of a solution (see this issue: #27) . Will share it later, when it totally works!

[jonasvdd]

Hi @malammar0,

From plotly-resampler>=0.8.0rc5 (pypi) on, we have FigureResampler serialization support. As a result, no global variables are needed and the figure can be cached per session using for example dash-extensions. This way, no global variables are used! 🔥

When #104 is merged, this will also be highlighted in our examples folder. For sake of completeness, I will copy a working example here:

"""Minimal dash app example.

Click on a button, and see a plotly-resampler graph of a noisy sinusoid.
No dynamic graph construction / pattern matching callbacks are needed.

This example uses the dash-extensions its ServersideOutput functionality to cache
the FigureResampler per user/session on the server side. This way, no global figure
variable is used and shows the best practice of using plotly-resampler Figures within
dash-apps.

"""

import dash
import numpy as np
import plotly.graph_objects as go
from dash import Input, Output, State, dcc, html
from dash_extensions.enrich import (
    DashProxy,
    ServersideOutput,
    ServersideOutputTransform,
)
from plotly_resampler import FigureResampler
from trace_updater import TraceUpdater

# Data that will be used for the plotly-resampler figures
x = np.arange(2_000_000)
noisy_sin = (3 + np.sin(x / 200) + np.random.randn(len(x)) / 10) * x / 1_000

# --------------------------------------Globals ---------------------------------------
app = DashProxy(__name__, transforms=[ServersideOutputTransform()])

app.layout = html.Div(
    [
        html.H1("plotly-resampler + dash-extensions", style={"textAlign": "center"}),
        html.Button("plot chart", id="plot-button", n_clicks=0),
        html.Hr(),
        # The graph and it's needed components to serialize and update efficiently
        # Note: we also add a dcc.Store component, which will be used to link the
        #       server side cached FigureResampler object
        dcc.Graph(id="graph-id"),
        dcc.Loading(dcc.Store(id="store")),
        TraceUpdater(id="trace-updater", gdID="graph-id"),
    ]
)


# ------------------------------------ DASH logic -------------------------------------
# The callback used to construct and store the FigureResampler on the serverside
@app.callback(
    [Output("graph-id", "figure"), ServersideOutput("store", "data")],
    Input("plot-button", "n_clicks"),
    prevent_initial_call=True,
    memoize=True,
)
def plot_graph(n_clicks):
    ctx = dash.callback_context
    if len(ctx.triggered) and "plot-button" in ctx.triggered[0]["prop_id"]:
        fig: FigureResampler = FigureResampler(go.Figure())

        # Figure construction logic
        fig.add_trace(go.Scattergl(name="log"), hf_x=x, hf_y=noisy_sin * 0.9999995**x)
        fig.add_trace(go.Scattergl(name="exp"), hf_x=x, hf_y=noisy_sin * 1.000002**x)

        return fig, fig
    else:
        raise dash.exceptions.PreventUpdate()


@app.callback(
    Output("trace-updater", "updateData"),
    Input("graph-id", "relayoutData"),
    State("store", "data"),  # The server side cached FigureResampler per session
    prevent_initial_call=True,
    memoize=True,
)
def update_fig(relayoutdata, fig):
    if fig is None:
        raise dash.exceptions.PreventUpdate()
    return fig.construct_update_data(relayoutdata)


# --------------------------------- Running the app ---------------------------------
if __name__ == "__main__":
    app.run_server(debug=True, port=9023)

Kind regards & Hope this answers your question,
Jonas

[malammar0]

Hi Jonas,
Thank you for the update! Sorry for the late response but I finally got the chance to try it and it works really well!

[GabrielReira]

Hi,
I believe this example no longer works because ServersideOutput has been deprecated.
When I try to use it I get the error: ImportError: cannot import name 'ServersideOutput' from 'dash_extensions.enrich’.
What would be the alternative way to modify this example to avoid the error on the line: [Output("graph-id", "figure"), ServersideOutput("store", "data”)]?

Thank you.

[galdir]

Hi,
I believe the solution no longer works because ServersideOutput has been deprecated.
When I try to use it I get the error: ImportError: cannot import name 'ServersideOutput' from 'dash_extensions.enrich’.
What would be the alternative way to modify this example to avoid the error on the line: [Output("graph-id", "figure"), ServersideOutput("store", "data”)]?

[jonasvdd]

Hi @galdir!

I will look into this somewhere later today!

[galdir]

Thank you @jonasvdd

After some research I found that we can use a Serverside component to wrap the return fig. The ServersideOutput has to be replaced by Output, and then the fig will keep resampling without being a global. https://www.dash-extensions.com/transforms/serverside_output_transform

Our app now works, but is slower than using a global variable, I don't know why yet. Any idea?

The code below is the adaptation of the Minimal dash app example.

from dash_extensions.enrich import Serverside

...

@app.callback(
    [Output("graph-id", "figure"), Output("store", "data")],
    Input("plot-button", "n_clicks"),
    prevent_initial_call=True,
    memoize=True,
)
def plot_graph(n_clicks):
    ctx = dash.callback_context
    if len(ctx.triggered) and "plot-button" in ctx.triggered[0]["prop_id"]:
        fig: FigureResampler = FigureResampler(go.Figure())
        # Figure construction logic
        fig.add_trace(go.Scattergl(name="log"), hf_x=x, hf_y=noisy_sin * 0.9999995**x)
        fig.add_trace(go.Scattergl(name="exp"), hf_x=x, hf_y=noisy_sin * 1.000002**x)
        return fig, Serverside(fig)
    else:
        raise dash.exceptions.PreventUpdate()