Sharing a dataframe between plots


#1

I have a dashboard that works as follows:

  1. User selects an item from a dropdown menu
  2. A database is queried to pull the data (as a dataframe) based on the users’s choice
  3. Three different plots are created using the same dataframe
  4. When the user selects a different item, steps 2 and 3 are rerun

Now my problem is, I am querying the database with exactly the same query to pull the data for all three app.callback decorators corresponding to the three plots I have. This is very inefficient and I want to figure out a way to query the database once and then share the dataframe between the decorators.

I am thinking of defining the dataframe as global but I feel there may be a cleaner solution.
Any pointer will be appreciated.


Working on large datasets -- comparison with shiny
ComputingAggregationsUpfront doesn't work well
Knowing what graph has been clicked
Problem when trying to share df between callbacks
Working on large datasets -- comparison with shiny
#2

See https://github.com/plotly/dash/issues/49#issuecomment-311511286 for now. In particular,

global_df = pd.read_csv('...')
app.layout = html.Div([
    dcc.Graph(id='graph'), 
    html.Table(id='table'),
    dcc.Dropdown(id='dropdown'),
    html.Div(id='intermediate-value', style={'display': 'none'})
])

@app.callback(Output('intermediate-value', 'children'), [Input('dropdown', 'value')])
def clean_data(value):
     # some expensive clean data step
     cleaned_df = your_expensive_clean_or_compute_step(value)
     return cleaned_df.to_json() # or, more generally, json.dumps(cleaned_df)

@app.callback(Output('graph', 'figure'), [Input('intermediate-value', 'children'])
def update_graph(jsonified_cleaned_data):
    dff = pd.read_json(jsonified_cleaned_data) # or, more generally json.loads(jsonified_cleaned_data)
    figure = create_figure(dff) 
    return figure

@app.callback(Output('table', 'children'), [Input('intermediate-value', 'children'])
def update_table(jsonified_cleaned_data):
    dff = pd.read_json(jsonified_cleaned_data) # or, more generally json.loads(jsonified_cleaned_data)
    table = create_table(dff) 
    return table

Append to a json list stored in a hidden div?
#3

Thanks you @chriddyp.
It worked like charm.


#4

I’ve pulled some of this discussion into a new chapter of the Dash user guide: https://plot.ly/dash/sharing-data-between-callbacks (source here: https://github.com/plotly/dash-docs/blob/master/tutorial/sharing_state.py).


#5

The part in example 2 where you return multiple dfs on their own keys in dictionaries doesn’t seem to work. When we are using a df as the key it complains about “DataFrame is mutable. Cannot be hashed”. If you set the key to be a string, it errors out with a “component.type is undefined” error.


ComputingAggregationsUpfront doesn't work well
#6

@cvax -

Thanks for reporting @cvax! Yes, this is a typo, it is supposed to be

    return {
        'df_1': df_1.to_json(orient='split'),
        'df_2': df_2.to_json(orient='split'),
        'df_3': df_3.to_json(orient='split'),
    }

(keys are supposed to be strings). Fixed in https://github.com/plotly/dash-docs/commit/b033b8dd67b06306a1d94a0caa78cb26fcb5e01d


ComputingAggregationsUpfront doesn't work well
#7

Hi Chris. Slightly confused about this part. It seems that you clean the data with a callback, then store it. Why not just clean the data when the page loads? Would there be any problems with that? Example:

global_df = pd.read_csv(’…’)
cleaned_df=some_expensive_step(global_df)

@app.callback(Output(‘graph’, ‘figure’), [Input(‘dropdown’, ‘value’])
def update_graph(val):
dff = cleaned_df[cleaned_df[‘col’]==val]
figure = create_figure(dff)
return figure


#8

No, there isn’t. This is only if you need to filter the data as response to a callback. See https://plot.ly/dash/sharing-data-between-callbacks for full context


#9

How does this work when building a multipage app? I must be missing something obvious here, because everytime I navigate to another page, all the data inside my hidden Div (which is part of the layout in my “index” app) is gone. Any help will be much appreciated.


#10

Also interested in how to share data across pages…

Simple use case would be pulling data from an API and being able to use it across multiple pages as opposed to having to hit the API again on every page load.