Problems Plotting Complete Pandas Dataset Scattermapbox

I’m having an issue plotting a relatively large pandas dataset (~79,000 points) in scattermapbox. It’s a dataset of points running from 2014-2018. I want to plot all of the points and then use a drop down menu to separate the data by each year and then use another drop down menu to separate each year by 2 different categories.

My issue is that one: not all of the data is being displayed. Only 14-part of 17 is being shown with the complete dataset and then the drop down menu isn’t working for separating the years other than showing no data for 2018.

The dataset work is done prior, but here is some info.

[79039 rows x 8 columns]

With column titles of datetime, year, lat, lon, ltgtyp, lat1, lat2, dist.

It’s based on a lightning detection system, lat1 and lat2 are set points and I have used the haversine formula to calculate the distance of lat/lon from lat1/lat2. I have further limited the dataset to any point within 150 miles.

#separate cloud to cloud and cloud to ground data
df_c2c = df[df['ltgtyp'] == 1]
df_c2g = df[df['ltgtyp'] == 0]

years = sorted(list(df.year.unique()))
print(years)


# Open mapbox account and pass your access token over here
# Mapbox account sign up and access token is free. 
# Mapbox information: https://www.mapbox.com/help/how-access-tokens-work/#rotating-tokens
mapbox_access_token = '*********'



# Use Scattermap box for satellite maps
data = [
go.Scattermapbox(
    lat= df_c2c['lat'],
    lon= df_c2c['lon'],
    text = [str(n) for n in df['datetime']],
    mode='markers',
    marker = dict(color='yellow', size=5),                      
    name = 'Cloud-Cloud',
    legendgroup = 'Cloud to cloud'
), 
go.Scattermapbox(
    lat= df_c2g['lat'],
    lon= df_c2g['lon'],
    text = [str(n) for n in df['datetime']],
    mode='markers',
    marker = dict(color='red', size=5),                      
    name = 'Cloud-Ground',
    legendgroup = 'Cloud to ground'
)   ]   

  
# Declare Layout    
layout = go.Layout(
title = 'LTG',
font=dict(family='Courier New, monospace', size=18, color='rgb(0,0,0)'),
autosize=False,
hovermode='closest',
showlegend=True,
width=1800,
height=1000,
mapbox=dict(
    accesstoken=mapbox_access_token,
    #width=1415,
    bearing=0,
    center=dict(
        lat=41.87,
        lon=-87.63
    ),
    pitch=0,
    zoom=8,
    style='satellite-streets'
),
)     

updatemenus=list([
# drop-down 1: map styles menu
# buttons containes as many dictionaries as many alternative map styles I want to offer
dict(
    buttons=list([
        dict(
            args=['mapbox.style', 'outdoors'],
            label='Outdoors',
            method='relayout'
        ),
        dict(
            args=['mapbox.style', 'satellite-streets'],
            label='Satellite with Streets',
            method='relayout'
        )                    
    ]),
    # direction where I want the menu to expand when I click on it
    direction = 'up',
  
    # here I specify where I want to place this drop-down on the map
    x = 0.75,
    xanchor = 'left',
    y = 0.05,
    yanchor = 'bottom',
  
    # specify font size and colors
    bgcolor = 'white',
    bordercolor = 'black',
    font = dict(size=11)
),    

# drop-down 2: select year
dict(
     buttons=list([
        dict(label = 'All',
             method = 'update',
             args = [{'visible': [True, True, True, True, True]}]),
        dict(label = '2014',
             method = 'update',
             args = [{'visible': [True, False, False, False, False]}]),
        dict(label = '2015',
             method = 'update',
             args = [{'visible': [False, True, False, False, False]}]),
         dict(label = '2016',
             method = 'update',
             args = [{'visible': [False, False, True, False, False]}]),
         dict(label = '2017',
             method = 'update',
             args = [{'visible': [False, False, False, True, False]}]),
         dict(label = '2018',
             method = 'update',
             args = [{'visible': [False, False, False, False, True]}])       
    ]),
    # direction where the drop-down expands when opened
    direction = 'down',
    # positional arguments
    x = 0.01,
    xanchor = 'left',
    y = 0.99,
    yanchor = 'bottom',
    # fonts and border
    bgcolor = 'white',
    bordercolor = 'black',
    font = dict(size=11)
),

# drop-down 3: select type lightning to visualize
dict(
     buttons=list([
        dict(label = 'All',
             method = 'update',
             args = [{'visible': [True, True]}]),
        dict(label = 'Cloud To Cloud',
             method = 'update',
             args = [{'visible': [True, False]}]),
        dict(label = 'Cloud To Ground',
             method = 'update',
             args = [{'visible': [False, True]}]),        
    ]),
    # direction where the drop-down expands when opened
    direction = 'down',
    # positional arguments
    x = 0.75,
    xanchor = 'left',
    y = 0.99,
    yanchor = 'bottom',
    # fonts and border
    bgcolor = 'white',
    bordercolor = 'black',
    font = dict(size=11)
)
])

# assign the list of dictionaries to the layout dictionary
layout['updatemenus'] = updatemenus



plotly.offline.plot({"data":data, "layout":layout},filename = 'ltg_plt.html', image='png')

So I have 3 drop down menu’s with years, ltg type, and map type. Probably a more efficient way to separate the data by ltg type, but it seems to work fine if I limit the dataset to one year. I just don’t want to make a plot for each year and end up with 5 plots.

Appreciate any help.

Hi mate
I’m new to Dash but been using Plotly cufflinks and map box / leaflet for ages

Showing too many points on the map is never a good idea

  1. Have you consider clustering them ? (Market cluster leaflet)
  2. Make logging entry just before the fig created - count actual records passed from df on back end side
  3. Not sure how to make it with dash yet but would be good to pass some checking info to front end and pass it to console.log() in js to see what’s actually reaching the front
    Let me know how it goes it’s quite interesting benchmark how much points is too many
    Cheers
    Alex
three reasons why you should buy plotly pro: support open source, get great support, host your plots and dashboards online