Time consuming initial background process


#1

I need to load a set of large set of cPickle or json files in the beginning. Then I want to keep those values in main memory in variables. I need a set of suggestion regarding this:

  1. How can I show verbose output to the user, for example file 1 loaded, now loading file 2, etc?
  2. How can I load them just once and use them for any instances without reloading again?

#2

Regarding showing verbose output to the user, you could try having an html.Div component and a dcc.Interval like this:

html.Div(
    id='status-box',
    children=[
        "Upload status",
        html.Div(
            id='status-display',
            children=[]
        ),
        dcc.Interval(
            id='update-status-interval',
            interval=1 * 1000,
            n_intervals=0
        ) 
    ]
)

Then attach a callback to the interval that updates the status:

@app.callback(
    Output('status-display', 'children'),
    [Input('update-status-interval', 'n_intervals')]
)
def update(_):
    ... # get the files that have been uploaded 

Hope this helps!