Take a single df and break it into two scattermapbox traces using list comprehension

jb23626436 · January 7, 2019, 9:47pm

I’ve got a df:

df = pd.DataFrame(
    {"BOROUGH": ['MANHATTAN','MANHATTAN','MANHATTAN','QUEENS', 'QUEENS', 'QUEENS'],
     "CALL_QTY":[100, 10, 5, 15, 30, 45],
     "lat":[40.75, 40.72, 40.73, 40.72, 70.71, 40.721],
     "lng":[-73.99, -73.98, -70.97, -73.74, -73.73, -73.72]})

And a tuple:

u_sel = [list(a) for a in zip(['MANHATTAN', 'QUEENS'], # names
                              ['blue', 'orange'],      # colors
                              [0.6, 0.7])]             # opacity

And here is the function I’ve created that uses list comprehension:

def scattermap_data(df, u_sel):
    return([go.Scattermapbox(
        lat = df.loc[df['BOROUGH']==b].lat.astype('object'),
        lon = df.loc[df['BOROUGH']==b].lng.astype('object'),
        mode = 'markers',
        marker = dict(
            size=df.loc[df['BOROUGH']==b].CALL_QTY,
            sizeref=0.9,
            sizemode='area',
            color=color,
            opacity=opacity
        )
    )] for b, color, opacity in u_sel
    )

The problem: When I try to run the following:

data = scattermap_data(df, u_sel)
layout = go.Layout(autosize=False,
                   mapbox= dict(
                       accesstoken=mapbox_access_token,
                       zoom=10,
                       style = 'dark',
                       center= dict(
                           lat=40.721319,
                           lon=-73.987130)
                   ),
                   title = "O God, Why?")
fig = dict(data=data, layout=layout)
py.iplot(fig, filename='tmapbox')

I get: ValueError: Invalid value of type 'builtins.generator' received for the 'data' property of Received value: <generator object scattermap_data.<locals>.<genexpr> at 0x000000000A72AF68>

Note that this example does work if I make two separate traces of go.Scattermapbox but I’ve got several plots I’m working on and I do not want to duplicate code for every plot (i.e. make 2 different trace instances for every plot. I feel that I am close to getting this to work but I just need some tweaks.

Ancillary info: Plotly v. 3.3.0, python 3.6, I am new to list comprehensions and plotly and any help would be very much appreciated.

jmmease · January 8, 2019, 10:38am

Hi @jb23626436,

The scattermap_data function currently returns something of the form

return([...] for ... in ...)

This will evaluate to a Python generator of length 1 lists. But I think what you want here is to return a single list with the length that matches the number of traces. So you want an expression more like

return([... for ... in ...])

In particular, try

def scattermap_data(df, u_sel):
    return([go.Scattermapbox(
        lat = df.loc[df['BOROUGH']==b].lat.astype('object'),
        lon = df.loc[df['BOROUGH']==b].lng.astype('object'),
        mode = 'markers',
        marker = dict(
            size=df.loc[df['BOROUGH']==b].CALL_QTY,
            sizeref=0.9,
            sizemode='area',
            color=color,
            opacity=opacity
        )
    ) for b, color, opacity in u_sel]
    )

Hope that helps!
-Jon

jb23626436 · January 8, 2019, 11:28pm

Hi @jmmease -

I actually just needed to encapsulate the return with listand that seemed to work but I am on to further issues:

I have several functions like the one mentioned above and I am trying to add them to the same figure.

One is the go.Scattermapbox, one is a go.Bar and another is ago.Box`- every single plot that I want to display is going to have 2 traces per plot just like my scattermapbox function does, above. Here’s the corrected version that works:

def scattermap_data(df, u_sel):
    return(list(go.Scattermapbox(
        lat = df.loc[df['BOROUGH']==b].lat.astype('object'),
        lon = df.loc[df['BOROUGH']==b].lng.astype('object'),
        mode = 'markers',
        marker = dict(
            size=df.loc[df['BOROUGH']==b].CALL_QTY,
            sizeref=0.9,
            sizemode='area',
            color=color,
            opacity=opacity
        )
    ) for b, color, opacity in u_sel)
    )

My updated question: Can you mix these 3 different types into subplots? To keep things (hopefully not too) simple, is it possible to add two side-by-side scattermapbox plots resulting from the function I created, above, in the same figure?

jmmease · January 9, 2019, 11:13am

Hi @jb23626436,

Yes, it’s possible to arrange different plot types in the same figure, but the way you position subplots depends on which trace types you’re working with.

Trace types that are positioned on 2D Cartesian axes (like scatter, bar, box, etc.) each have xaxis and yaxis properties that specify which Cartesian subplot axes they belong to (e.g. xaxis='x2'). The xaxis and xaxis locations are specified in the layout section of the figure (e.g. layout.xaxis2.domain.x=[0, 0.5]). See https://plot.ly/python/subplots/#multiple-subplots for more detailed example on how this works. But for your purposes ignore the examples the use the make_subplots function, this a higher-level convenience function that doesn’t support mapbox traces.

To position scattermapbox traces, you specify which scattermapbox subplot the trace belongs to using the subplot property (e.g. scattermapbox.subplot = 'mapbox2' for the second mapbox trace). Then you position the mapbox subplot itself in the layout section of the figure (e.g. `layout.mapbox2.domain.x=[0.5, 1.0].

Though it doesn’t include a scattermapbox trace, this example of mixed subplot types might be helpful https://plot.ly/python/mixed-subplots/.

Hope that helps!
-Jon

jb23626436 · January 9, 2019, 4:45pm

Hi @jmmease,

Thank you for the links. Unfortunately, my problem persists.

Question: Does there exist an example of “a list or tuple of dicts of string/value properties where” the data property is composed of at least 2 plots with at least 2 traces per plot? I think if I can just examine an example, I could get this to work but I haven’t found such an example.

To investigate this, I created a smaller data parameter containing only 2 plots (important: each plot has 2 traces) and combined them like:

data = [tracebar, tracebox]

Here is the printed version for tracebar (note that I created indents for easier reading):

[   {   'marker': {'color': 'blue', 'line': {'width': 1}, 'opacity': 0.6},
        'name': 'Manhattan',
        'type': 'bar',
        'x': array([0, 1, 2]),
        'y': array([ 6,  8, 18], dtype=int64)},
    {   'marker': {'color': 'orange', 'line': {'width': 1}, 'opacity': 0.5},
        'name': 'Queens',
        'type': 'bar',
        'x': array([0, 1, 2]),
        'y': array([ 3,  7, 16], dtype=int64)}]

and here is the printed version of tracebox:

[   {   'boxpoints': 'suspectedoutliers',
        'marker': {   'color': 'blue',
                      'line': {'outlierwidth': 1},
                      'opacity': 0.6,
                      'outliercolor': '#419ede',
                      'symbol': 'x'},
        'name': 'Manhattan',
        'type': 'box',
        'x': array([0, 0, 1, 2, 2]),
        'y': array([497.428 , 546.    , 798.3333, 539.5714, 481.8571])},
    {   'boxpoints': 'suspectedoutliers',
        'marker': {   'color': 'orange',
                      'line': {'outlierwidth': 1},
                      'opacity': 0.5,
                      'outliercolor': '#FFC999',
                      'symbol': 'x'},
        'name': 'Queens',
        'type': 'box',
        'x': array([0, 0, 1, 2, 2]),
        'y': array([712.8571, 842.    , 798.333 , 420.4286, 506.1429])}]

Each trace can be successfully plotted individually (i.e. tracebox successfully plots on its own and tracebar successfully plots on its own). For reference, here’s the code for each:

# bar layout and plotting
layoutbar = go.Layout( title="Monthly Total: Call Quantity by Hour", barmode="group")
figbar = go.Figure(data=tracebar, layout=layoutbar)
py.plot(figbar, filename='mybar')

# box layout and plotting
layoutbox = go.Layout(yaxis=dict(type = "log", title="Travel Seconds", zeroline=True),boxmode="group"))
figbox = go.Figure(data=tracebox, layout=layoutbox)
py.plot(figbox, filename='mybox')

However, when I attempt to marry these two traces via data = [tracebar, tracebox], the printed version looks like this:

[   [   {   'marker': {'color': 'blue', 'line': {'width': 1}, 'opacity': 0.6},
            'name': 'Manhattan',
            'type': 'bar',
            'x': array([0, 1, 2]),
            'y': array([ 6,  8, 18], dtype=int64)},
        {   'marker': {'color': 'orange', 'line': {'width': 1}, 'opacity': 0.5},
            'name': 'Queens',
            'type': 'bar',
            'x': array([0, 1, 2]),
            'y': array([ 3,  7, 16], dtype=int64)}],
    [   {   'boxpoints': 'suspectedoutliers',
            'marker': {   'color': 'blue',
                          'line': {'outlierwidth': 1},
                          'opacity': 0.6,
                          'outliercolor': '#419ede',
                          'symbol': 'x'},
            'name': 'Manhattan',
            'type': 'box',
            'x': array([0, 0, 1, 2, 2]),
            'y': array([497.428 , 546.    , 798.3333, 539.5714, 481.8571])},
        {   'boxpoints': 'suspectedoutliers',
            'marker': {   'color': 'orange',
                          'line': {'outlierwidth': 1},
                          'opacity': 0.5,
                          'outliercolor': '#FFC999',
                          'symbol': 'x'},
            'name': 'Queens',
            'type': 'box',
            'x': array([0, 0, 1, 2, 2]),
            'y': array([712.8571, 842.    , 798.333 , 420.4286, 506.1429])}]]

This looks ok to me as it’s in the form outlined by the error message:

(e.g. [{'type': 'scatter', ...}, {'type': 'bar, ...}])
All remaining properties are passed to the constructor of
          the specified trace type
        (e.g. [{'type': 'scatter', ...}, {'type': 'bar, ...}])

Nevertheless: When I attempt to make the fig via:

fig = go.Figure(data)

I receive the error message that indicates my data property is incorrectly formatted.

 Invalid element(s) received for the 'data' property of
            Invalid elements include: [[{'type': 'bar', 'x': array([0, 1, 2]), 'y': array([ 6,  8, 18], dtype=int64), 'name': 'Manhattan', 'marker': {'color': 'blue', 'opacity': 0.6, 'line': {'width': 1}}}, {'type': 'bar', 'x': array([0, 1, 2]), 'y': array([ 3,  7, 16], dtype=int64), 'name': 'Queens', 'marker': {'color': 'orange', 'opacity': 0.5, 'line': {'width': 1}}}], [{'type': 'box', 'x': array([0, 0, 1, 2, 2]), 'y': array([497.428 , 546.    , 798.3333, 539.5714, 481.8571]), 'name': 'Manhattan', 'boxpoints': 'suspectedoutliers', 'marker': {'color': 'blue', 'outliercolor': '#419ede', 'symbol': 'x', 'opacity': 0.6, 'line': {'outlierwidth': 1}}}, {'type': 'box', 'x': array([0, 0, 1, 2, 2]), 'y': array([712.8571, 842.    , 798.333 , 420.4286, 506.1429]), 'name': 'Queens', 'boxpoints': 'suspectedoutliers', 'marker': {'color': 'orange', 'outliercolor': '#FFC999', 'symbol': 'x', 'opacity': 0.5, 'line': {'outlierwidth': 1}}}]]

I feel that I’m so close to making this work but my attempt to combine the two plots (each plot has 2 traces) into the data parameter, I’m introducing an extra set of braces. I tried using go.Bar and go.Box versions of my plots as well but it’s still not allowing me to create the figure (problem with my data parameter).

Side: I haven’t seen a single example that uses list comprehension to create traces - are they uncommon for users because they introduce problems?

jmmease · January 10, 2019, 11:11am

Hi @jb23626436,

The list/tuple passed to figure.data must be a list of dicts or trace objects. It can’t be a nested list of lists of dicts as you have here. So you need something like

data = tracebar + tracebox

Then to control which axes the traces show up on, you need to set the xaxis/yaxis properties of each trace the way https://plot.ly/python/subplots/#multiple-subplots does.

Hope that helps clear things up a bit!
-Jon

jb23626436 · January 10, 2019, 5:23pm

Hi @jmmease,

Thank you. Using data = tracebar + tracebox did allow me to create the figure but the layout is a problem for me now.

Question1: Is there an example that shows stacked subplots without using the make_subplots function or without using cufflinks?

Question2: Does there exist an example that includes at least 2 plots with at least 2 traces per plot in a stacked orientation? I asked this before and I am only asking again because I really think this would help me a great deal.

Note: I was able to create the side-by-side plots using the below layout but I did not attempt stacked subplots because I’d eventually want to add in a scattermapbox (recall you suggested I avoid the make_subplots function). The box plots aren’t side-by-side and I wasn’t sure where to include the ‘boxmode’ and I want the plots stacked anyway so I didn’t pursue it much further.

layout = go.Layout(
    xaxis=dict(
        domain=[0, 0.7]
    ),
    xaxis2=dict(
        domain=[0.8, 1]
    ),
    yaxis2=dict(
        anchor='x2'
    )
)
fig = go.Figure(data=data, layout =layout)
py.plot(fig, filename='side-by-side-subplot')

Really: thank you for all your help thus far.

jmmease · January 11, 2019, 11:12am

Hi @jb23626436,

I’m not sure I understand exactly what you mean by “stacked subplots” , but here’s an example of one scattermabox subplot (with 2 traces) and two cartesian subplots (with two traces each) arranged in a column.

from plotly.offline import init_notebook_mode, iplot
import plotly.graph_objs as go
init_notebook_mode()

mapbox_access_token = '...'

data = [
    go.Scattermapbox(
        lat=['45.5017'],
        lon=['-73.5673'],
        mode='markers',
        marker=dict(
            size=14
        ),
        name='mapbox 1',
        text=['Montreal'],
        subplot='mapbox'
    ),
    go.Scattermapbox(
        lat=['45.7'],
        lon=['-73.7'],
        mode='markers',
        marker=dict(
            size=14
        ),
        name='mapbox 1',
        text=['Montreal 2'],
        subplot='mapbox'
    ),
    go.Scatter(
        y=[1, 3, 2],
        xaxis='x',
        yaxis='y',
        name='scatter 1'
    ),
    go.Scatter(
        y=[3, 2, 1],
        xaxis='x',
        yaxis='y',
        name='scatter 2'
    ),
    go.Bar(
        y=[1, 2, 3],
        xaxis='x2',
        yaxis='y2',
        name='scatter 3'
    ),
    go.Scatter(
        y=[3, 2, 1],
        xaxis='x2',
        yaxis='y2',
        name='scatter 4'
    ),
]

layout = go.Layout(
    autosize=True,
    hovermode='closest',
    mapbox=dict(
        accesstoken=mapbox_access_token,
        bearing=0,
        center=dict(
            lat=45,
            lon=-73
        ),
        pitch=0,
        zoom=5,
        domain={
            'x': [0.3, 0.6],
            'y': [0.7, 1.0]
        }
    ),
    xaxis={
        'domain': [0.3, 0.6]
    },
    yaxis={
        'domain': [0.36, 0.6]
    },
    xaxis2={
        'domain': [0.3, 0.6],
        'side': 'bottom',
        'anchor': 'y2'
    },
    yaxis2={
        'domain': [0, 0.3],
        'anchor': 'x2'
    },
    height=700,
    width=700
)

fig = go.Figure(data=data, layout=layout)
iplot(fig)

Does that help?
-Jon

jb23626436 · January 11, 2019, 7:26pm

@jmmease,

YES! That example you shared with me was the key. See the image below as proof I was able to translate it to work with my example. The example really clarified for me what the layout parameter was controlling in the fig (side: I feel it’s a good example to add to the user docs, too).

Thank you for being patient with me and taking to the time to provide that example. Have a great weekend!

matsujju · October 6, 2020, 9:50pm

How to use plotly express in fig.add_trace()?

Topic		Replies	Views
Two Plotly Express ScatterMapBoxs shown in one Figure (Help) 📊 Plotly Python	0	366	September 16, 2020
How to iterate through dataframe to add traces into go.scattermapbox? 📊 Plotly Python	1	1411	March 23, 2022
How do I combine multiple animated plotly.express.scatter_mapbox traces on a map? 📊 Plotly Python	2	1992	July 21, 2021
Scattermapbox with scaling 📊 Plotly Python	7	783	April 27, 2020
Go Scattermapbox with different traces like markers and bubbles/size possible? 📊 Plotly Python	4	2224	August 11, 2022

Take a single df and break it into two scattermapbox traces using list comprehension

Related Topics