My webApp is deployed on a VPS running on Ubuntu, with Gunicorn and NGinx. The app runs in a venv, and it’s monitored by supervisorctl. Besides what is needed to have the webApp running, there is nothing else.
Now, I would like to have a cron task, which would execute a python script every 24h. This script would download some zip file, extract the data, parse and cleansen them, and, eventually, save the relevant data in the /data directory of my dash webApp.
First question(maybe completely dumb): What’s the correct way to get scripts executed “in the background”, without they mess up with everything else (Especially my webApp) ?
Second question: what if i did need another script running permanently in the background, like a loop which would be executed every 2sec to grab new data? Do I have to set up something to have this running h24, 7d/7 ?
If I had several VPS I would try every potential solution I would find with Google, until I figure out what’s the best one, but given that I only have 1 VPS, I don’t want to go in this direction. Any hint, or link to decent explanations, would be warmly welcome:)
P.S: It seems that something similar was discussed here [Solved] Updating server side app data on a schedule but given that I would have several crawlers running in parallel, I would like to keep each of them totally independant from the dash app.