Stream graph blank after a few hours [Solved]

📊 Plotly Python
1

I am streaming tick data from my pandas dataframe and everything works fine until I leave me screen and come back to a blank graph. This has been happening for the past few days and this usually happens after 4 -5 hours of streaming. I don’t even get to a few hundred ticks before my graph loses all of its data. I plot new data every 5 minutes so I call s0.heartbeat() every second to tell Plotly not to close the streams. Should I be doing something different or is Plotly not working correctly? Thanks!

https://plot.ly/~tim3lord2/3/poloniex-smas-btc-lct-30480-5-min/

# plotly preparation
stream_0 = Stream(maxpoints=500, token=api_tokens[0])
stream_1 = Stream(maxpoints=500, token=api_tokens[1])
stream_2 = Stream(maxpoints=500, token=api_tokens[2])
stream_3 = Stream(maxpoints=500, token=api_tokens[3])
stream_4 = Stream(maxpoints=500, token=api_tokens[4])
stream_5 = Stream(maxpoints=500, token=api_tokens[5])

trace_0 = Scatter(x=[], y=[], name='tick data', mode='lines', stream=stream_0)
trace_1 = Scatter(x=[], y=[], name='SMA 1', mode='lines', stream=stream_1)
trace_2 = Scatter(x=[], y=[], name='SMA 2', mode='lines', stream=stream_2)
trace_3 = Scatter(x=[], y=[], name='Sell', mode='markers', marker=dict(size = 10, color = 'rgba(225, 0, 0, .8)'), stream=stream_3)
trace_4 = Scatter(x=[], y=[], name='Buy', mode='markers', marker=dict(size = 10, color = 'rgba(0, 225, 25, .8)'), stream=stream_4)
trace_5 = Scatter(x=[], y=[], xaxis='x2', yaxis='y2', name='Portfolio', mode='lines', stream=stream_5)

data = Data([trace_0, trace_1, trace_2, trace_3, trace_4, trace_5])

layout = {
  "title": "Poloniex SMAs BTC_LCT (30,480) - 5 Min", 
  "xaxis1": {
    "anchor": "y1", 
    "domain": [0, 1]
  }, 
  "xaxis2": {
  	"showgrid" : False,
  	"zeroline" : False,
  	"showline" : False,
  	"showticklabels" : False,
    "anchor": "y2", 
    "domain": [0, 1]
  },
  "yaxis1": {
    "anchor": "x1", 
    "domain": [0, 0.8]
  }, 
  "yaxis2": {
  	"showgrid" : False,
  	"zeroline" : False,
  	"showline" : False,
  	"showticklabels" : True,
    "anchor": "x2", 
    "domain": [0.85, 1]
  }
}

fig = Figure(data=data1, layout=layout)

ply.plot(fig, filename='Graph', auto_open=False)

s0 = ply.Stream(api_tokens[0])
s1 = ply.Stream(api_tokens[1])
s2 = ply.Stream(api_tokens[2])
s3 = ply.Stream(api_tokens[3])
s4 = ply.Stream(api_tokens[4])
s5 = ply.Stream(api_tokens[5])

s0.open()
s1.open()
s2.open()
s3.open()
s4.open()
s5.open()

while True:
	if(plot once every 5 mins is True):
		s0.write({'x': time, 'y': data['Price'].iloc[-1]})
		s1.write({'x': time, 'y': data['SMA1'].iloc[-1]})
		s2.write({'x': time, 'y': data['SMA2'].iloc[-1]})
		s3.write({'x': time, 'y': data['Sell'].iloc[-1]})
		s4.write({'x': time, 'y': data['Buy'].iloc[-1]})
		s5.write({'x': time, 'y': data['Portfolio'].iloc[-1]})
		time.sleep(1)
	else:
		s0.heartbeat()
		s1.heartbeat()
		s2.heartbeat()
		s3.heartbeat()
		s4.heartbeat()
		s5.heartbeat()
		time.sleep(1)
2

Could you please take a look if there’s anything suspicious going on that might relate it, such as constantly growing memory consumption by the browser, and what exactly happens besides the fact that it’s blank. E.g. is the entire browser page, or browser frozen; did all contents get lost from the page or only the plot (or its parts, and maybe the hover modebar still pops up etc.). If it looks like an error it’s best to file an issue at https://github.com/plotly/plotly.js (unless it seems to relate only to the Python or R binding, which have their own repos). If informative, a screenshot helps on the github issue, e.g. memory growth or what you see on the page, and what it was when it worked.

3

It appears as if all the the data goes missing when I start to not see anymore plot updates. I usually refresh the Plotly page and find that all of my point data is gone. I just restarted my bot and everything is starting to plot correctly again. I’m going to watch https://plot.ly/~tim3lord2/3#data and see if that gives any clues to what is going wrong.

4

Also, i’m new to Ploty. Does refreshing the page remove all of your data?

5

same problem here. clicking on autoscale I can see the updated plot but it’s not more “animated”.
I think the problem is related to some plotly update

6

I think I solved my problem. Instead of plotting using the stream method, I am now just updating my chart using, fileopt=‘extend’. I should of started out just updating my plots this way but I did not realize I could do this.

7

I initially made use of your solution but encountered two problems:

  1. The resulting graph does not update itself, you have to refresh the page.
  2. Each update is an API call, so for users on free tier this will run out at some point. (unclear if stream writes count as API hits, but I think they don’t. If someone knows for sure please reply!)

Have a look at my post here where I essentially ask and answer my own question, thus solving the problems above.

Hope this helps!

1 Like
8

Any update? I get blank graphs all the time now. Can’t get it to work at all.

9

Make sure your script is sending data under 1min constantly or else it will shut down the stream. Here is an example of how fast you can update data with Plotly https://plot.ly/~tim3lord2/267 (Live tick data from a crypto exchange) If you are not sending data this fast just update your plot periodically with fileopt=‘extend’