Django and Dash - ead's Method

Hello everyone. I’ve only been coding in Python for about 2 months now, but it has quickly become my favorite language because of incredible libraries like plotly, dash, pandas, etc. I discovered Dash about 1 month ago and have already managed to create some incredible interfaces for visualizing my company’s product quality data, using ead’s method to serve through Django: https://bitbucket.org/m_c_/sample-dash

The reason for this post is to show what I’ve done to separate my interfaces into different files in hopes that more experienced python developers will provide feedback: perhaps there is a more elegant solution. If so I’d love to update my rapidly growing project before it becomes extremely difficult to change due to size and complexity.

A quick overview of ead’s method for serving Dash apps through Django:

For a dash project named ‘chart’ we do the following in urls.py:

from chart import views as chartviews

urlpatterns=[
     path('dash-', chartviews.dash),
     path('_dash-', chartviews.dash_ajax),
]

And we do the following in views.py:

from chart.as_dash import dispatcher

def dash(request, **kwargs):
     ''' '''
     return HttpResponse(dispatcher(request))

@csrf_exempt
def dash_ajax(request,**kwargs):
     ''' '''
     return HttpResponse(dispatcher(request), content_type='application/json')

And the dispatcher function in as_dash.py:

def dispatcher(request):
’’‘
Main function
@param request: Request object
’’’

app = _create_app()
params = {
    'data': request.body,
    'method': request.method,
    'content_type': request.content_type
}
with app.server.test_request_context(request.path, **params):
    app.server.preprocess_request()
    try:
        response = app.server.full_dispatch_request()
    except Exception as e:
        response = app.server.make_response(app.server.handle_exception(e))
    return response.get_data()

Then we can define our app within create_app() to load whatever functions or callbacks we want. Basically, this allows us to load a dash function that is defined with a name that starts with 'dash’. So what I did is create a sub-directory ‘modules’ with various .py files defining such functions and their associated callbacks. And simply imported them into as_dash.py like so:

#for the callbacks
import chart.modules.proj

#for the main project function
from chart.modules.proj import dash_proj

Callbacks are applied to ‘app’ like so:

proj_update_graph = app.callback(
dash.dependencies.Output(‘pj-callback-loader’,‘children’),
[dash.dependencies.Input(‘pj-date-range-start’,‘value’),
dash.dependencies.Input(‘pj-date-range-end’,‘value’)]
)(chart.modules.proj.update_graph)

The only thing I have to be careful of is my naming conventions for my controls and html components, ensuring to not use the same ones between the different py files, and this method works pretty well. However, being new to Python and Dash, I feel like I am missing something and there must be a better way to do this. If anyone has any suggestions I would be most grateful for your input. Thank you for taking the time to read my post.

1 Like

I had been meaning to have a look at that Django trick, so I had a bit of play
around with that code. There’s a few issues I see with scaling the way some of
that code is laid out, but which can be smoothed over.

The main issues I see are:

  • those urlpatterns using ‘path’ won’t work
  • a new Dash instance is needlessly created on every request
  • does not support mounting the Dash app at a different prefix

Here’s how I fixed these issues. I also made a few other modifications not
necessarily related to the above problems, which you can use or ignore as you
see fit. Also, I’m using Python 3.

You can find the following code here.

Let’s work with a Django project called dash_test that contains a single
Django app viz that will house the Dash app. Here’s the project structure
(omitting the files that startproject and startapp create that I didn’t
touch):

dash_test/
  urls.py
viz/
  dashapp.py
  layouts.py
  router.py
  server.py
  urls.py
  views.py

dash_test/urls.py

from django.contrib import admin
from django.urls import path, include

urlpatterns = [
    path('', admin.site.urls),
    path('viz/', include('viz.urls')),
]

Your suggested approach assumes that the dash app will be mounted at the root of
the site. It will almost certainly be good to have some flexibility about where
it’s mounted. Let’s run it at viz/ to make sure we have this modularisation
working.

viz/urls.py

from django.urls import re_path

from . import views
from . import dashapp # this loads the Dash app

urlpatterns = [
    re_path('^_dash-', views.dash_ajax),
    re_path('^', views.dash_index),
]

Note that the path function you were using looks for an exact match. I think
this might be a recent change with Django, where url routes were always defined
as regexes. Since we want a catchall for the _dash-* route, we should use the
re_path function. The dash index function should also be a regex, because we
want users to be able to load a URL that was generated with Dash’s internal
router (eg viz/fig1) and have it take them to the correspond ending state of the
app. If it was an exact match then users would have to load the root of the app
and then navigate through the app to the relevant state.

Also note that the order matters, the more greedy index function would clobber
the ajax function if it came first.

viz/views.py

from django.http.response import HttpResponse
from django.views.decorators.csrf import csrf_exempt

from .server import server


def dispatcher(request):
    '''
    Main function
    @param request: Request object
    '''

    params = {
        'data': request.body,
        'method': request.method,
        'content_type': request.content_type
    }
    with server.test_request_context(request.path, **params):
        server.preprocess_request()
        try:
            response = server.full_dispatch_request()
        except Exception as e:
            response = server.make_response(server.handle_exception(e))
        return response.get_data()


def dash_index(request, **kwargs):
    ''' '''
    return HttpResponse(dispatcher(request))


@csrf_exempt
def dash_ajax(request):
    ''' '''
    return HttpResponse(dispatcher(request), content_type='application/json')

Similar to before but with the notable change that the Dash app is no longer
created within the dispatch function, which would have had the undesirable
effect of creating a new Dash app on every request. Instead we just import the
already defined Flask server from the server module into the global scope, where
the internals of the dispatch function can access it.

viz/dashapp.py

from dash.dependencies import Output, Input
import dash_core_components as dcc
import dash_html_components as html

from .server import app
from . import router


app.layout = html.Div(children=[
    dcc.Location(id='url', refresh=False),
    dcc.Link('Index', href=app.url_base_pathname),
    ', ',
    dcc.Link('Figure 1', href=f'{app.url_base_pathname}fig1'),
    ', ',
    dcc.Link('Figure 2', href=f'{app.url_base_pathname}fig2'),
    html.Br(),
    html.Br(),
    html.Div(id='content')
])


# callbacks could go here, or in another callback.py file with this at the top:
# from .server import app

The main driving module of the app. Importing it will result in the app being “run”.
Callbacks can go in here or if there’s many of them perhaps in a callback.py file.

viz/server.py

from flask import Flask
from dash import Dash

# should start and end with a '/'
URL_BASE_PATHNAME = '/viz/'

server = Flask(__name__)

app = Dash(
    __name__,
    server=server,
    url_base_pathname=URL_BASE_PATHNAME
)

app.config['suppress_callback_exceptions'] = True

This is where the actual Dash app is initialised. We do this in a module that
does as little as possible to reduce the chances of running into impossible to
meed import chains. For instance if the Dash app was created in dashapp.py
(which it might seem intuitive to do), then the router import would fail because
as it would need to import app from dashapp but this would not yet be initialised.

URL_BASE_PATHNAME is defined here once, and then accessed through the
corresponding app.url_base_pathname attribute elsewhere to ensure that all
generated links and routes have the correct prefix. To mount the app at the root
we would set this to / (and also make sure the project’s urls.py mounts the
Dash app at the root.)

One improvement might be to define a utility function get_url(path) or some such
that does the simple job of prefixing this to a given path, so that if the logic
need to change, you wouldn’t have to do this for every link/route.

viz/router.py

from dash.dependencies import Output, Input

from .server import app, server
from . import layouts


pages = (
    ('', layouts.index),
    ('fig1', layouts.fig1),
    ('fig2', layouts.fig2),
)

routes = {f"{app.url_base_pathname}{path}": layout for path, layout in pages}


@app.callback(Output('content', 'children'), [Input('url', 'pathname')])
def display_page(pathname):
    ''' '''
    if pathname is None:
        return ''

    page = routes.get(pathname, 'Unknown link') 
    
    if callable(page):
        # can add arguments to layout functions if needed etc
        layout = page()
    else:
        layout = page

    return layout

This could go in dashapp.py, but I like the idea of keeping the page routes in
a separate file, in the same way that Django does.

I’m not such a fan of pulling out functions with the dash prefix and
automatically adding them as routes. This is more of a subjective thing, but it
feels bit too magic and not-discoverable.

Instead, this router maps paths (which are expanded to full prefixed urls) with
corresponding layouts in our layout.py file. You could adapt this to post-process
the route object in any way you like

from random import randint
import dash_core_components as dcc
import dash_html_components as html


def index():
    return 'Welcome to index page'


def fig1():
    return dcc.Graph(
        id='main-graph',
        figure={
            'data': [{
                'name': 'Some name',
                'mode': 'line',
                'line': {
                    'color': 'rgb(0, 0, 0)',
                    'opacity': 1
                },
                'type': 'scatter',
                'x': [randint(1, 100) for x in range(20)],
                'y': [randint(1, 100) for x in range(20)]
            }],
            'layout': {
                'autosize': True,
                'scene': {
                    'bgcolor': 'rgb(255, 255, 255)',
                    'xaxis': {
                        'titlefont': {'color': 'rgb(0, 0, 0)'},
                        'title': 'X-AXIS',
                        'color': 'rgb(0, 0, 0)'
                    },
                    'yaxis': {
                        'titlefont': {'color': 'rgb(0, 0, 0)'},
                        'title': 'Y-AXIS',
                        'color': 'rgb(0, 0, 0)'
                    }
                }
            }
        }
    )

def fig2():
    return dcc.Graph(
        id='main-graph',
        figure={
            'data': [{
                'name': 'Some name',
                'mode': 'line',
                'line': {
                    'color': 'rgb(0, 0, 0)',
                    'opacity': 1
                },
                'type': 'scatter',
                'x': [randint(1, 100) for x in range(20)],
                'y': [randint(1, 100) for x in range(20)]
            }],
            'layout': {
                'autosize': True,
                'scene': {
                    'bgcolor': 'rgb(255, 255, 255)',
                    'xaxis': {
                        'titlefont': {'color': 'rgb(0, 0, 0)'},
                        'title': 'X-AXIS',
                        'color': 'rgb(0, 0, 0)'
                    },
                    'yaxis': {
                        'titlefont': {'color': 'rgb(0, 0, 0)'},
                        'title': 'Y-AXIS',
                        'color': 'rgb(0, 0, 0)'
                    }
                }
            }
        }
    )
10 Likes

Thank you so much for this extremely helpful feedback! I’ve only been able to quickly read your post and I’m going to have to spend some more time studying your code to fully understand everything that is happening, but this is exactly what I was looking for.

1 Like

No worries :slight_smile: More than happy to field any questions. And you should definitely question things! The projects that I’ve spun up with Dash have not yet had much time to be tested, so could well be more sensible ways of doing things.

Hey guys!

Cool stuff you’re doing here! In my current (first) Django project, I’m also trying to integrate a dash app. As a start I’m trying to get nedned’s example running on my machine, but I am having a few issues. When I try to navigate to one of the dash pages, I get the following error:

"POST /viz/_dash-update-component HTTP/1.1" 500 91822
Internal Server Error: /viz/_dash-update-component
Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\lib\site-packages\django\core\handlers\exception.py", line 41, in inner
    response = get_response(request)
  File "C:\ProgramData\Anaconda3\lib\site-packages\django\core\handlers\base.py", line 187, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "C:\ProgramData\Anaconda3\lib\site-packages\django\core\handlers\base.py", line 185, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\django\views\decorators\csrf.py", line 58, in wrapped_view
    return view_func(*args, **kwargs)
  File "C:\ProgramData\Anaconda3\dash-django-example-master\dash-django-example-master\dash_test\viz\views.py", line 35, in dash_ajax
    return HttpResponse(dispatcher(request), content_type='application/json')
  File "C:\ProgramData\Anaconda3\dash-django-example-master\dash-django-example-master\dash_test\viz\views.py", line 19, in dispatcher
    server.preprocess_request()
  File "C:\ProgramData\Anaconda3\lib\site-packages\flask\app.py", line 1831, in preprocess_request
    rv = func()
  File "C:\ProgramData\Anaconda3\lib\site-packages\flask_seasurf.py", line 283, in _before_request
    return abort(403)
  File "C:\ProgramData\Anaconda3\lib\site-packages\werkzeug\exceptions.py", line 707, in abort
    return _aborter(status, *args, **kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\werkzeug\exceptions.py", line 687, in __call__
    raise self.mapping[code](*args, **kwargs)
werkzeug.exceptions.Forbidden: 403 Forbidden: You don't have the permission to access the requested resource. It is either read-protected or not readable by the server.

I guess it has to do something with the csrf token. Then again shouldn’t the @csrf_exempt decorator prevent this? Anyone has an idea what’s the problem there?

edit: My issue was solved by including “csrf_protect=False” in server.py. :slight_smile:

Hey this is really neat example - thanks!
I was just wondering what is the best way to incorporate callbacks for each figure/page? I noticed in your viz.dashapp.py example you mention putting them there or in another file. I’m unsure how these would then link to the individual functions which are returning the figures.
Ideally I would like to make different callbacks on each page but the ways I’ve been trying end up with CantHaveMultipleOutputs error.
Thanks for any help!

Actually I’ve figured out how to do it! I didn’t realise it was as simple as adding mulitple callbacks into the dashapp.py just referencing the layout for the different figures/pages.
Thanks again for this example!

2 Likes

Is there any way to avoid this kind of routing being triggered for other pages as the ones in routes? Like if a user enters /viz/something, I would prefer django itself handling the routing instead of the user being redirected to the dash layout with the “unknown link” message.

For some reason also one of my prior django views is “hijacked” by this routing and displays the “unknown link” page instead of the usual content. The only distinct things I can think of is that this page is usually called from a form by a HttpResponseRedirect and depends on some context stored in the session. However, hijacking also occurs when i directly enter /viz/mypage.

My first instinct would be to change the regex in urls.py.
Currently its catching anything written as a URL and feeding it to views.dash because the regex is so broad. I unfortunately don’t know regex well enough to know what you could replace this with. If you made it so it only picked up URLS that matched something like ‘figure*’ then the URL wouldn’t get picked up by dash unless it had the word figure in it and therefore would be handled by django.
Maybe not the total solution but possibly somewhere to start?

1 Like

Wow, this worked surprisingly well, thanks!

I changed the regex from ‘^’ to ‘^dash/’. Also in router.py I changed the entries in pages to include ‘dash/’ at the beginning. Now all dash pages are available at /viz/dash/ and don’t interfere with my other stuff anymore.

1 Like

The other approach is to put all of your other views in a separate Django app that’s routed to something at the same level. eg /viz/ and /otherapp/ .

Giving Dash its own app and letting it take over routing of everything below the url prefix where the app is mounted was what I was going for. Seems like a good separation of concerns.

looks like a great solution,
i’ve been toying with the idea of moving my dash to django as a more complete solution and this seems to work

now all i need is figure out the best way to move my app selection menu out of the dash layout :slight_smile:

It’s probably worth pointing out that method used to embed Dash in Django here could well suffer from performance issues as there’s some double handling going on between Django and Flask, or maybe other surprises since its using flask machinery that’s intended to be used in a testing context rather than deployment. That said, for a lot of scenario’s it may well be fine.

Thank you for this super solution ead and nedned. It plugs in well with an existing solution I have.

I have another question about mixing context between Django and Dash:

If the “display_page” callback in router.py selects the layout, how can I get it to accept a context variable from Django and input in to ‘page()’?

I was thinking that the context could be a variable such as a country name stored as a json:

# From router.py:

@app.callback(Output('content', 'children'),
                     [Input('url', 'pathname')],
                      Input('django_context', 'children')])
def display_page(pathname, django_context):

   if pathname is None:
       return ''

   page = routes.get(pathname, 'Unknown link') 

   if callable(page):
       # can add arguments to layout functions if needed etc
       layout = page(**django_context**)
   else:
       layout = page

   return layout

I am also guessing that the ‘django_context’ variable would need to be passed as an arg to the ‘dispatcher’ method in views.py.

Are you able to give an indication as to how I go about passing this from the Flask server to Dash in this method? Any assistance, however vague, would be much appreciated at this stage!

I think I have found a (very hacky) solution but I need to test it before coming back to the forum. I will post it soon but may start a separate thread as it is running Dash within the context of Django, which is different from running a separated Dash app within Django.

1 Like

Great thank you a lot for your help @nedned @eddy_ojb

I have a question:

I understand with this solution We got one Server, one App that could handle multi sessions for serveral users.

Question: If I need to do 3 different graphs interfaces that are not related and I like those to be accessible through different urls. How many Dash apps should I do? How many Flask servers should I do? Should I have to do one Django app for each Dash App handling each graph interfaces? (of course I know that one Dash App will handle multi sessions for multi users)

I know the @nedned solution use the layouts.py functions to permit render different graphs, but I’m talking about completley differents graphs interfaces whit different sets of layouts, inputs and callbacks.

It depends on how you want to implement it really. You could have a single Django app that has three different Dash apps hooked up to different Django routes, or you could have one single Dash app attached to a single Django route (as in my example) and use the Dash router.py logic to add different endpoints within the Dash app. I suspect the second option might be simpler. The first option involves either having three separate Flask instances (which seems a bit wasteful) or having them all try to reuse the one Flask instance, which adds extra complexity.

Thank you a lot for the answer!

1 Like

Hi all,

I have produced an extension to ead’s and nedned’s code (thank you both very much!).

The code can be found here:

The package versions I used are:

  • Python 3.6.5
  • Django 2.0.2
  • dash 0.21.1
  • dash-core-components 0.22.1
  • dash-html-components 0.10.1
  • dash-renderer 0.12.1
  • numpy 1.14.3

The key differences are outlined below:

  • This is a Django app with Django controlling the page navigation, context and data models with a Dash incorporated in to a page

  • The database can be queried from a Dash layout using Django models and context. In the EU, this type of functionality is useful in staying compliant with GDPR regulations if you have a data security model tied to session context at the backend

  • Separation of Dash and Django files to try and make it more discoverable (since they aren’t cleanly interleaved)

  • There is also a filthy function called: “clean_dash_content”, which essentially removes characters such as ‘\n’ in the content returned by Flask (I am looking for a better solution here…)

  • Since most servers I have seen are offline, the Dash is served locally, which you can see in dash_server.py. This upset the CSS path that Dash tries to find so you will also notice that I put the Dash css in the ‘static’ folder and served it to fix this problem.

  • Since the base pathname might not be known in advance, I changed the “display_page” function which loops through each layout function name to check whether it is in the url - the applicable function is then returned. There may be an easier way using regex but I didn’t have the regex chops to do this at the time.

  • Lines 14 to 21 in dash_layouts.py show how context is extracted from the url and used in a Django model.

  • Lastly, if you look at the dash_django_page.html, you will see how the Dash content is injected:
    {{ dash_content | safe}}. It won’t work without the “safe” keyword.

The downside to this whole approach is some double handling by Flask and Django. However, I have hooked this Django app to MS IIS to serve pages and haven’t noticed a major problem yet. I imagine the Flask dev instance might fall over under heavy load. I don’t know what the security implications are either.

I think a more professional, integrated approach between Dash and Django is highly desirable moving forward. For me personally, I need time to prove to my stakeholders at work that there is value in our system (which heavily relies on Django) before I can approach the firm to get funding to work with Plotly to develop this as a project.

I am eager to receive any feedback from the community and any ways to improve the code!

4 Likes

We’ve taken a different approach and wrapped dash into a django app in a way that removes the need for a flask server. The main motivation is to reduce the number of moving parts.

You can find an initial version at https://github.com/GibbsConsulting/django-plotly-dash

It is very much at a preliminary stage and any and all comments and other feedback is most welcome.

5 Likes