Using external JS libraries in custom Dash component


#1

I need a Dash component that is a heatmap with clustering dendograms on its axes. I need the heatmap to be interactive so that a user can select a cluster in this heatmap (by clicking on a branch of the dendogram) and this selection needs to result in different data being plotted in another component.

It turns out Clustergrammer is a very nicely built tool to do this, so I figured I could create a custom Dash component out of it (similar to the visdcc example I saw referenced in this forum).

As part of this work, I needed my custom component to be able to read a JSON data file (given as input to clustergrammer). Some research online showed to use d3.json to do this, so I installed it with npm install d3. I created a very basic skeleton of the component following the guide for custom Dash plugins, and added a new React component file and also added a basic “renders” test for it, similar to the one for the ExampleComponent. However, I get this error:

10 07 2018 10:28:22.514:INFO [PhantomJS 2.1.1 (Windows 8.0.0)]: Connected on socket 98VtmWPwZEon4sC1AAAA with id 48627566
PhantomJS 2.1.1 (Windows 8.0.0) ERROR
  ReferenceError: Can't find variable: d3
  at webpack:///~/clustergrammer/clustergrammer.node.js:70:0 <- C:/Users/samir/scquery-components/test/main.js:49296

How do I properly import some external javascript library (such as d3)?

Also, any other pointers (or links to existing code) on how to go about creating a Dash component for Clustergrammer are most welcome!

Below is the very basic React code:

import React, {Component} from 'react';
import PropTypes from 'prop-types';

import Clustergrammer from 'clustergrammer';
import d3 from 'd3-fetch';

/**
 * ClusterGrammer is a port of the JS visualization library
 * of the same name.
 */
export default class ClusterGrammer extends Component {
	constructor(props) {
		super(props);
	}

	makeClust() {
		let networkData = d3.json(this.props.dataPath);
		let args = {
			'root':  this.props.id,
			'network_data': networkData
		};
		this.cgm = Clustergrammer(args);
	}

	componentDidMount() {
		this.makeClust();
	}
        
        // Not hooked up yet
	handleSelectCluster() {
		alert('You have selected a cluster!');
	}
	
	render() {
		const {id, style} = this.props;
		return (
			<div id={id} style={style}></div>
		);
	}
}

ClusterGrammer.propTypes = {
	/**
	 * The ID used to identify this compnent in Dash callbacks
	 */
	id: PropTypes.string,

	/**
	 * The path to JSON data file for ClusterGrammer
	 */
	dataPath: PropTypes.string,

	/**
	 * Dash-assigned callback that should be called whenever any of the
	 * properties change
	 */
	setProps: PropTypes.func
};


#2

In principle, this component looks good. I believe that the error that you are seeing is just related to the tests. You could remove that test command in package.json for now:

"scripts": {
     ...,
     "test": ""
}

For JSON itself, Dash handles serializing your python’s dictionaries as JSON and then converting it back into JS objects. So, you should be able to just pass in this.props.dataPath directly (without d3.json)


#3

Thanks for the response @chriddyp ! I removed the test script I made, and npm install works. However, when I then run python usage.py (see below for the app layout), the webpage just says “Error loading dependencies”, and when I inspect the console in Chrome, it has these errors (scquery_components is the name of my dash component package):

bundle.js?v=0.0.1:formatted:336 Uncaught ReferenceError: d3 is not defined
    at Object.<anonymous> (bundle.js?v=0.0.1:formatted:336)
    at e (bundle.js?v=0.0.1:formatted:247)
    at bundle.js?v=0.0.1:formatted:255
    at Object.<anonymous> (bundle.js?v=0.0.1:formatted:256)
    at e (bundle.js?v=0.0.1:formatted:10)
    at Object.<anonymous> (bundle.js?v=0.0.1:formatted:97)
    at e (bundle.js?v=0.0.1:formatted:10)
    at Object.<anonymous> (bundle.js?v=0.0.1:formatted:32)
    at e (bundle.js?v=0.0.1:formatted:10)
    at bundle.js?v=0.0.1:formatted:18
(anonymous) @ bundle.js?v=0.0.1:formatted:336
e @ bundle.js?v=0.0.1:formatted:247
(anonymous) @ bundle.js?v=0.0.1:formatted:255
(anonymous) @ bundle.js?v=0.0.1:formatted:256
e @ bundle.js?v=0.0.1:formatted:10
(anonymous) @ bundle.js?v=0.0.1:formatted:97
e @ bundle.js?v=0.0.1:formatted:10
(anonymous) @ bundle.js?v=0.0.1:formatted:32
e @ bundle.js?v=0.0.1:formatted:10
(anonymous) @ bundle.js?v=0.0.1:formatted:18
(anonymous) @ bundle.js?v=0.0.1:formatted:19

bundle.js?v=0.13.0:2 Error: scquery_components was not found.
    at Object.resolve (bundle.js?v=0.13.0:14)
    at s (bundle.js?v=0.13.0:14)
    at Array.map (<anonymous>)
    at s (bundle.js?v=0.13.0:14)
    at e.value (bundle.js?v=0.13.0:14)
    at p._renderValidatedComponentWithoutOwnerOrContext (react-dom@15.4.2.min.js?v=0.13.0:13)
    at p._renderValidatedComponent (react-dom@15.4.2.min.js?v=0.13.0:13)
    at performInitialMount (react-dom@15.4.2.min.js?v=0.13.0:13)
    at p.mountComponent (react-dom@15.4.2.min.js?v=0.13.0:13)
    at Object.mountComponent (react-dom@15.4.2.min.js?v=0.13.0:14)

So it seems that there is something wrong with the import, not just the test script.
I’m a little unclear on your suggestion for Dash serializing dictionaries. The situation I have is that Clustergrammer reads a JSON object to produce the plots (and this json file is produced by a backend program that is also part of Clustergrammer), and the constructor in the API for creating a plot needs this JSON object to be passed in. I have the JSON file on disk, and just want to read that into JS and pass it to Clustergrammer. Are you saying that Dash can help with that, without d3.json()?

Usage.py:

import scquery_components
import dash
import dash_html_components as html

app = dash.Dash('')

app.scripts.config.serve_locally = True

app.layout = html.Div([
    scquery_components.ClusterGrammer(
        id='cgm',
        dataPath='clustergrammer_example.json',
    )
])


if __name__ == '__main__':
    app.run_server(debug=True)


#4

Yeah, I’d try loading it into python memory rather than trying to make an API request to it. i.e.

import json

with open('clustergrammer_example.json') as f:
    data = json.loads(f.read())
app.layout = html.Div([
    scquery_components.ClusterGrammer(
        id='cgm',
        dataPath=data
    )
])

And then remove the d3.json from your component. I believe that d3.json is going to be making a network request to the file that you have which would mean that you would need to run a local file serving server. In any case, loading the data directly into memory and passing it through to the component is more the “dash” way anyway (as it makes it easier to update or listen to that property with Dash callbacks).

Regarding d3 not found, you probably just need to install it:

npm install d3 --save

#5

Ah, thanks for the explanation @chriddyp, that makes sense. However, even after getting rid of the d3.json call and using the Dash way above, I still can’t get a very basic app working, and get the bundle.js?v=0.0.1:1 Uncaught ReferenceError: d3 is not defined error.

I’ve tried to create a minimum working example, where all I do is copy the ExampleComponent, and just try to import the clustergrammer library (which I installed with npm i -S clustergrammer) (when this example didn’t work, I also ran npm i -S d3, since clustergrammer itself depends on d3, and it still didn’t resolve the errors). The source files are below, and I’d appreciate any advice! I feel like this is something very basic I must be missing…

ClusterGrammer.react.js (just a copy of ExampleComponent with only addition being an import statement)

import React, {Component} from 'react';
import PropTypes from 'prop-types';

import Clustergrammer from 'clustergrammer';

/**
 * ExampleComponent is an example component.
 * It takes a property, `label`, and
 * displays it.
 * It renders an input with the property `value`
 * which is editable by the user.
 */
export default class ClusterGrammer extends Component {
    render() {
        const {id, label, setProps, value} = this.props;
        typeof Clustergrammer // To get rid of unused var error
        return (
            <div id={id}>
                Clustergrammer: {label}
                <input
                    value={value}
                    onChange={e => {
                        /*
                         * Send the new value to the parent component.
                         * In a Dash app, this will send the data back to the
                         * Python Dash app server.
                         */
                         if (setProps) {
                             setProps({
                                value: e.target.value
                            });
                         }
                    }}
                />
            </div>
        );
    }
}

ClusterGrammer.propTypes = {
    /**
     * The ID used to identify this compnent in Dash callbacks
     */
    id: PropTypes.string,

    /**
     * A label that will be printed when this component is rendered.
     */
    label: PropTypes.string.isRequired,

    /**
     * The value displayed in the input
     */
    value: PropTypes.string,

    /**
     * Dash-assigned callback that should be called whenever any of the
     * properties change
     */
    setProps: PropTypes.func
};

index.js

/* eslint-disable import/prefer-default-export */
import ExampleComponent from './components/ExampleComponent.react';
import ClusterGrammer from './components/ClusterGrammer.react';

export {
    ExampleComponent,
    ClusterGrammer
};

usage.py (adds instance of the above component)

import clustergrammer_dash_component
import dash
import dash_html_components as html

app = dash.Dash('')

app.scripts.config.serve_locally = True

app.layout = html.Div([
    clustergrammer_dash_component.ExampleComponent(
        id='input',
        value='my-value',
        label='my-label'
    ),
    html.Div(id='output'),
    html.Hr(),
    clustergrammer_dash_component.ClusterGrammer(
        id='input-cgm',
        value='my-value-cgm',
        label='my-label-cgm'
    ),
    html.Div(id='output-cgm')
])

@app.callback(
	dash.dependencies.Output('output', 'children'),
	[dash.dependencies.Input('input', 'value')])
def display_output(value):
    return 'You have entered {}'.format(value)

@app.callback(
    dash.dependencies.Output('output-cgm', 'children'),
    [dash.dependencies.Input('input-cgm', 'value')])
def display_output(value):
    return 'You have entered {}'.format(value)

if __name__ == '__main__':
    app.run_server(debug=True)

package.json (clustergrammer and d3 were added by npm install)

{
  "name": "clustergrammer-dash-component",
  "version": "0.0.1",
  "description": "A dash UI component wrapper for Clustergrammer",
  "main": "lib/index.js",
  "repository": {
    "type": "git",
    "url": "https://github.com/AmirAlavi/clustergrammer-dash-component.git"
  },
  "license": "MIT",
  "bugs": {
    "url": "https://github.com/AmirAlavi/clustergrammer-dash-component/issues"
  },
  "homepage": "https://github.com/AmirAlavi/clustergrammer-dash-component",
  "scripts": {
    "copy-lib": "copyfiles -u 1 lib/* clustergrammer_dash_component",
    "demo": "builder run demo",
    "install-local": "npm run copy-lib && python setup.py install",
    "prepublish": "npm test && builder run build-dist && npm run copy-lib",
    "publish-all": "npm publish && python setup.py sdist upload",
    "publish-pypi": "npm run prepublish && python setup.py sdist upload",
    "start": "builder run build-dev",
    "test": "builder run check",
    "test-watch": "builder run test-frontend-watch",
    "test-debug": "builder run test-frontend-debug",
    "uninstall-local": "pip uninstall clustergrammer-dash-component -y"
  },
  "dependencies": {
    "builder": "3.2.2",
    "clustergrammer": "^1.19.5",
    "copyfiles": "^1.2.0",
    "d3": "^5.5.0",
    "dash-components-archetype": "^0.2.11",
    "prop-types": "^15.5.9",
    "react": "^15.5.4",
    "react-dom": "^15.5.4"
  },
  "devDependencies": {
    "dash-components-archetype-dev": "^0.2.11",
    "enzyme": "^2.8.2",
    "react-test-renderer": "^15.5.4"
  }
}

Finally, the 2 errors in Chrome’s console when I try to load the page on the server from python usage.py (displays “Error loading dependencies”):

bundle.js?v=0.0.1:1 Uncaught ReferenceError: d3 is not defined
    at Object.<anonymous> (bundle.js?v=0.0.1:1)
    at e (bundle.js?v=0.0.1:1)
    at bundle.js?v=0.0.1:1
    at Object.<anonymous> (bundle.js?v=0.0.1:1)
    at e (bundle.js?v=0.0.1:1)
    at Object.<anonymous> (bundle.js?v=0.0.1:1)
    at e (bundle.js?v=0.0.1:1)
    at Object.<anonymous> (bundle.js?v=0.0.1:1)
    at e (bundle.js?v=0.0.1:1)
    at bundle.js?v=0.0.1:1
(anonymous) @ bundle.js?v=0.0.1:1
e @ bundle.js?v=0.0.1:1
(anonymous) @ bundle.js?v=0.0.1:1
(anonymous) @ bundle.js?v=0.0.1:1
e @ bundle.js?v=0.0.1:1
(anonymous) @ bundle.js?v=0.0.1:1
e @ bundle.js?v=0.0.1:1
(anonymous) @ bundle.js?v=0.0.1:1
e @ bundle.js?v=0.0.1:1
(anonymous) @ bundle.js?v=0.0.1:1
(anonymous) @ bundle.js?v=0.0.1:1


bundle.js?v=0.13.0:2 Error: clustergrammer_dash_component was not found.
    at Object.resolve (bundle.js?v=0.13.0:14)
    at s (bundle.js?v=0.13.0:14)
    at Array.map (<anonymous>)
    at s (bundle.js?v=0.13.0:14)
    at e.value (bundle.js?v=0.13.0:14)
    at p._renderValidatedComponentWithoutOwnerOrContext (react-dom@15.4.2.min.js?v=0.13.0:formatted:2367)
    at p._renderValidatedComponent (react-dom@15.4.2.min.js?v=0.13.0:formatted:2374)
    at performInitialMount (react-dom@15.4.2.min.js?v=0.13.0:formatted:2222)
    at p.mountComponent (react-dom@15.4.2.min.js?v=0.13.0:formatted:2191)
    at Object.mountComponent (react-dom@15.4.2.min.js?v=0.13.0:formatted:4772)
(anonymous) @ bundle.js?v=0.13.0:2
Promise.catch (async)
(anonymous) @ bundle.js?v=0.13.0:2
(anonymous) @ bundle.js?v=0.13.0:28
value @ bundle.js?v=0.13.0:13
value @ bundle.js?v=0.13.0:13
e.notifyAll @ react-dom@15.4.2.min.js?v=0.13.0:formatted:494
close @ react-dom@15.4.2.min.js?v=0.13.0:formatted:4723
closeAll @ react-dom@15.4.2.min.js?v=0.13.0:formatted:6127
perform @ react-dom@15.4.2.min.js?v=0.13.0:formatted:6100
perform @ react-dom@15.4.2.min.js?v=0.13.0:formatted:6091
perform @ react-dom@15.4.2.min.js?v=0.13.0:formatted:5100
T @ react-dom@15.4.2.min.js?v=0.13.0:formatted:5108
closeAll @ react-dom@15.4.2.min.js?v=0.13.0:formatted:6127
perform @ react-dom@15.4.2.min.js?v=0.13.0:formatted:6100
batchedUpdates @ react-dom@15.4.2.min.js?v=0.13.0:formatted:3805
i @ react-dom@15.4.2.min.js?v=0.13.0:formatted:5019
_renderNewRootComponent @ react-dom@15.4.2.min.js?v=0.13.0:formatted:4352
_renderSubtreeIntoContainer @ react-dom@15.4.2.min.js?v=0.13.0:formatted:4391
render @ react-dom@15.4.2.min.js?v=0.13.0:formatted:4396
(anonymous) @ bundle.js?v=0.13.0:14
e @ bundle.js?v=0.13.0:1
(anonymous) @ bundle.js?v=0.13.0:1
e @ bundle.js?v=0.13.0:1
(anonymous) @ bundle.js?v=0.13.0:1
(anonymous) @ bundle.js?v=0.13.0:1

#6

Hm odd. I wonder if the old bundle is getting cached? Could you try opening up your Browser’s dev tools and disabling the network cache and reloading? See here: https://stackoverflow.com/questions/5690269/disabling-chrome-cache-for-website-development


#7

Ok, I’ve tried disabling the cache, and still no luck.

Just to make sure I understand how it’s supposed to work:

  1. You write your custom Dash component as a React component
  2. Any JS libraries that you need that are installed via npm are added to node_packages and also added to a list of dependencies in packages.json
  3. When you then create a build of your Dash component, some tool (is it builder or webpack?) will read the packages.json dependencies and basically concatenate the JS source code for each of the libraries into bundle.js
  4. When your Dash app runs, all of the JS it depends on should be in the bundle.js

So could it be that somehow d3.js isn’t being added to the bundle?
Or could it be that it is being added, but maybe order matters and it’s in the wrong order?


#8

Yeah, that’s the idea. And builder is running a webpack command.

So yeah, the probem is that d3.js isn’t being added to the bundle. The order shouldn’t matter as long as the import is happening in the file where you are referencing the variable.


#9

Ah ok, do you have any suggestions for how to fix this? I’ve been trying hard to look at other Dash components that users have written, but most are wrappers around React components. None seem to touch d3.