Hi all, I have recently built an app using Dash at work. While the app works for the most part, there are still crashes and I believe I need to find a better overall design, but I am not sure what the best one would be.
It is a “real-time” dashboard that updates every 15 seconds (“interval-component”). Update means it performs one “data retrieval” callback which runs 3 separate queries on a kdb database (using qpython), which return 3 pandas dataframes of reasonably small size (~10,000 x 10 each max.) containing numbers and short strings. That same callback performs a few operations on the frames (aggregation, computing a couple of new columns; i.e. nothing too expensive). This data is then JSON encoded and stored in a Store component (memory).
The store component is used as an input on several other callbacks which update various charts. Some of those charts are also using some dropdowns as inputs in addition to the store component, which basically act as a filter for the data.
How exactly does the store component manage the data in the background (in memory)? In my case, would it overwrite the data on every callback?
Occasionally, the queries don’t return anything or qpython can’t properly parse the data to a dataframe. They tend to be temporary problems and normally just need a rerun of the query for them to work properly. In this case, I would not want the store component to update on the callback, but rather just keep the data from the previous update and just try again in 15 seconds on the next update. I have added the following code to achieve this, although I am not entirely sure whether that does exactly what I want it to:
df = query_database(args)
If the app is deployed on a server, I think it would be best to have one data “repository” that is refreshed every 15 seconds (on the server) which is used in each users individual session as the input data will be the same. They might just apply different “filters” through the dropdowns but the source data doesnt change between users.
In terms of creating several instances of the app, does Dash do this automatically? What is the best way to have multiple users, say up to 15, use such an app that is deployed centrally on a server?
Having done some research, Example 3 on this page (https://dash.plot.ly/sharing-data-between-callbacks) seems like a good fit to manage the data, but would very much like to hear some thoughts on this and on the other points!