How to design an app that updates data at an interval, but not on page load?

Hi there

Looking for some advice on how to structure an app I want to try.

The end result is a number of charts, which are based on data retrieved from a number of APIs. The data is to be retrieved every 24 hours and some calculations will be applied (moving averages etc).

What I want to avoid, is the data being retrieved/calculations being done every time someone loads a page. I want the data/calcs done, and charts updated at a fixed time, and then when someone loads a page the chart is instantly there.

Would I be able to get away with global variables here?

So a dcc.Interval component set to every 24 hours, and that callback retrieves data, applies calcs, and creates/updates global figures for each chart. In my layout, my dcc.Graph components would use the global figures?

The data would be static once updated i.e. the app/user does not modify the data in any other way.

Any thoughts would be appreciated - thank you!

Anyone have any thoughts on this?

On my workplace, I have separated the “collect data and calculate some outcome” scripts and the Dash visualization scripts.
Roughly speaking, I have some python scripts which calls our database, do some analysis and save results in some .txt files.
In my Dash program, I read these text files and throw the figure.

I have then used Windows Scheduler to run my “collect and analyse” scripts once every 24 hours to make sure everything is up to date.

It’s not the most elegant solution, but it works. :slightly_smiling_face:

I do this as well and it’s a fine solution. I am mostly on Linux and use crontab instead. Some further things I do:

  • app.layout points to a function that gets the latest data
  • dcc.Interval runs every n seconds but only pulls an update if the csv file(s) are no older than ~n*1.5 seconds otherwise it raises PreventUpdate. E.g. if interval is 60 seconds and hasn’t been modified in over 90 seconds then don’t update. Different values may work better for you.
  • I also raise Prevent update then n_intervals is None or 0 as I include the data in the layout function, but you may want to think what works best for you
  • Include a timestamp in my app of when the data was last calculated - just in case there’s a problem with the calculations script

As long as reading the data is fast I think this is a pretty nice solution as updates to the Dash App and updates to the calculations can be done independently of each other.

1 Like

Also check out celery for running scheduled jobs & redis for caching data. A dcc.Interval could poll every 15 seconds but the scheduled task would update the underlying data on its own schedule & store the data in redis. Here’s an example: https://github.com/plotly/dash-redis-celery-periodic-updates/.

If you’re using Dash Deployment Server, then here are some docs on how to set up Redis on the server: https://dash.plot.ly/dash-deployment-server/redis-database

Thanks everyone. I am going to use a similar approach to what Blaceus has suggested.

I’m using Pythonanywhere and I’ve seen that it comes with a job scheduler. So I will put the “get data and do calcs” bit into a separate script that exports a csv of latest data every time it is run.

Thank you!

1 Like