Show and Tell - Dash Pivottable


#1

I found out about pivottable.js last summer, and I really liked the idea of a library for interactive data exploration in JS, along with built-in aggregations (sums, average, count) and displays (table, scatter plots, bar charts), all done through an UI component.

I haven’t had the chance to try it out, so today I decided to get started by making it into a Dash component. Thus, I made this dash wrapping of the react-pivottable library (maintained by Plotly, and created by Nicholas who also made pivottable.js; awesome work!).

The github link: https://github.com/xhlulu/dash_pivottable

Here’s what it looks like when you run the usage.py file in the project:

Honestly all the heavy-lifting was already done in the react component, and the library is really easy to learn. Go check it out: https://github.com/plotly/react-pivottable/

Shoutout to Nicholas for making this awesome library (2 years late haha)!


#2

:heart_eyes::heart_eyes::heart_eyes:


#3

I really like this! I tried it out and it works great for data under 100,000 rows; any suggestions for working with much larger data?


#4

Thank you! This is based on react-pivottable, so I would recommend taking a look at how it optimizing rendering for very large datasets. Otherwise, using Pandas and Numpy to preprocess your dataset so that the table transmitted at the end is smaller could be helpful too!


#5

Took a look - turns out that all the pivoting happens client-side, and server-side integration is not great; this makes sense from a React-ive point of view but definitely not from a large-data-workability viewpoint. Sadly the design just doesn’t work for me.

My data is between 15M and 125M records, held in a pretty powerful server, accessed through a Dash app for business reporting. Right now it’s possible to interactively pivot server-side by generating a pivot table and rendering it with table.to_html() and making pivot charts with matplotlib before rendering them as images in the Dash. It’s not optimal though.

It could be interesting to hack out a component that mimics react-pivottable's pivoting and visualization logic on the server-side in order to make the larger data realistic.