Loading pandas dataframe into Data table through a callback

**Mike3** · February 10, 2019, 5:51pm

So, I have been excited by the continued improvement of Data Tables, but despite looking through many threads around here as well as guides, I can’t figure out the proper way to simply import a pandas dataframe into a data table.

Most examples illustrate how to manually pick certain columns/rows taken from a dataframe that is hardcoded within the example, and display that, but I have a dataframe that needs to be created via a callback, which will then be sent to dash_table.DataTable().

How can I make this work? Using the references, I’ve tried the following code to send a dict of my dataframe, but nothing displays.

## Imports
import dash
import dash_core_components as dcc
import dash_html_components as html
import dash_table

from dash.dependencies import Input, Output, State

import datetime as dt
import pandas as pd
import numpy as np
from twitter_functions import id_extractor, old_tweets, continuous_stream

app = dash.Dash(dev_tools_hot_reload=True)
app.scripts.config.serve_locally = True
app.config['suppress_callback_exceptions'] = True


app.layout = html.Div(children=[

    html.H3('Twitter App'),
    
    dcc.Input('ScreenName_Input', type='text'),
    
    html.Button(id='screenNames_submit_button', children='Submit'),
    
    dash_table.DataTable(id='tweet_table')
    
])
                      
@app.callback(
    Output(component_id='tweet_table', component_property='data'),
    [Input(component_id='screenNames_submit_button', component_property='n_clicks_timestamp')],
    [State(component_id='ScreenName_Input', component_property='value')]
)
def display_tweets(submit_button, screen_names):
    tweets = old_tweets(screen_names)
                      
    return tweets.to_dict(orient='records')

if __name__ == '__main__':
    app.run_server(debug=True)

How can I do this?

**sterlingbutters** · February 25, 2019, 3:19pm

If I’m not mistaken, you may have to define the DataTable with the columns you want and an empty list for data i.e.:

dash_table.DataTable(
    id='computed-table',
    columns=[
    {'name': 'Column 1', 'id': 'column1'},
    {'name': 'Column 2', 'id': 'column2'},
    {'name': 'Column 3', 'id': 'column3'},
    {'name': 'Column 4', 'id': 'column4'},
    {'name': 'Column 5', 'id': 'column5'},
    ...
                ],
    data=[],
    ...

where obviously the structure of you DataTable in declaration matches the df you are trying to display

**Mike3** · February 11, 2019, 3:46pm

This ended up working. Thank you so much, sterlingbutters; you saved a fellow coder in a great time of need.

Plotly really needs to do a better job of documenting these bedrock-basic features. I think I’ll make a forum post at some point documenting all the little tricks I’ve learned over the last couple months, in the hopes possibly they’ll help others like me in the future.

**sterlingbutters** · February 11, 2019, 3:48pm

yup, only way i figured it out was because I had two DataTables that were exactly the same except that was the only difference haha so I guess you could say I got lucky

**markus_zhang** · March 13, 2019, 6:20pm

Thanks, this however seems to put DataTable virtually useless for me…

**sterlingbutters** · March 13, 2019, 6:35pm

@markus_zhang While it may be a pain, you could likely create a function that parses your DataFrame to determine the shape of your data to create the list of dictionaries that occupy the columns property. Sure, it’d be nice if DataTable did everything for you but it could be a temporary solution for you. There could also be some potential issues depending on how “dynamic” your app is.

**markus_zhang** · March 13, 2019, 6:40pm

Thanks, this is what I’m doing at the moment. Basically I just save one row of data from SQL Server and parse the columns into the table. I’m new to Dash so I find it a bit awkard to pass objects around. I wish I could do everything with a controller program as a communication centre.

**sterlingbutters** · March 13, 2019, 6:52pm

Dash has a bit of a learning curve but it gets really fun after awhile

**markus_zhang** · March 13, 2019, 6:53pm

Yeah it’s extremely versatile. I know nothing about web dev and basically am leveraging the power of Dash to dev quick solutions for the other teams. When they ask I just say “Ask Dash, I know nothing…” lol

**sterlingbutters** · March 13, 2019, 7:06pm

Its a game-changer no doubt - Even if you knew web dev, it’d be a real pain to inject python in there. Probably one of the best tools for the scientific data and computing community

**Varlor** · March 28, 2019, 7:26am

@sterlingbutters So do I understand it right. I have to define the columns before I really know what columns I get from a callback?

**sterlingbutters** · March 28, 2019, 2:37pm

@Varlor Correct. In your callback you can likely determine the shape of your data you wish for DataTable to display and then return dcc.DataTable to the children property of a Div element. The point is that it doesn’t appear that DataTable is able to create it’s columns dynamically based on the data, this must be specified prior to loading the data into the table with a callback.

**Varlor** · March 28, 2019, 2:39pm

But what I tested now was to pass data and columns as output in the callback functions to the datatable and this worked for me! But now my data seems to large and the pagiation thing do not work for me. But if i shorten the dataframe to lets say 10 columns it works fine now!

**kna** · April 4, 2019, 10:57am

@sterlingbutters I did as you suggestet and defined the columns in my layout and and empty list for my data. I pass the actual data trough a callback. I can see the datatable with the columns now, but they remain empty. I used the the same data without a callback and it worked perfectly. I just can’t figure it out. What do I need to pass to my callback return statement?

**sterlingbutters** · April 5, 2019, 4:26pm

@kna Would you mind sharing your code?

**simonb** · April 8, 2019, 1:11pm

hey, here’s a fully working example that works for me, using Upload component.

import dash
import dash_table
import pandas as pd
import dash_core_components as dcc
import dash_html_components as html

app = dash.Dash()
app.css.config.serve_locally = True
app.scripts.config.serve_locally = True

app.layout = html.Div([
	dcc.Upload(
		id='upload-data',
		children=html.Div([
			'Drag and Drop or ',
			html.A('Select Files')
		]),
		style={
			'width': '100%',
			'height': '60px',
			'lineHeight': '60px',
			'borderWidth': '1px',
			'borderStyle': 'dashed',
			'borderRadius': '5px',
			'textAlign': 'center',
			'margin': '10px'
		},
		multiple=False
	),
	html.Div(id='output-data-upload'),
])

@app.callback(dash.dependencies.Output('output-data-upload', 'children'),
			 [dash.dependencies.Input('upload-data', 'contents'),
			  dash.dependencies.Input('upload-data', 'filename')])
def update_output(contents, filename):
	if contents is not None:
		df = pd.read_csv(filename, sep='\t')
		return html.Div([
			dash_table.DataTable(
				id='table',
				columns=[{"name": i, "id": i} for i in df.columns],
				data=df.to_dict("rows"),
				style_cell={'width': '300px',
				'height': '60px',
				'textAlign': 'left'})
			])

**kna** · April 9, 2019, 12:57pm

Thank you for your help and the additional code. Turns out it was a mistake in my dropdown menu options. I now testet your code and it works perfectly.