How to plot 3-D boundary for any kernel SVM

📊 Plotly Python
1

Hi there!
I have trouble plotting a 3-D boundary for SVMs.

Here is the code that works with SVM:

from sklearn import svm
import numpy as np
from sklearn.datasets import make_classification
from sklearn.svm import SVC

X, y1 = make_classification(n_samples=100, n_features=3, n_redundant=1, n_informative=2,
                           random_state=332, n_clusters_per_class=1, hypercube=False)

clf = SVC(C=10, cache_size=200, class_weight=None, coef0=6,
   decision_function_shape='ovr', degree=6, gamma=0.30000000000000004,
   kernel='poly', max_iter=-1, probability=False, random_state=None,
   shrinking=True, tol=0.001, verbose=False)
clf.fit(X,y)

X_0 = np.array([x for idx, x in enumerate(X) if y[idx]==0 and not x in clf.support_vectors_])
X_1 = np.array([x for idx, x in enumerate(X) if y[idx]==1 and not x in clf.support_vectors_])

And here the part to build just separately test and trained data:

import plotly
import plotly.plotly as py
import plotly.graph_objs as go

import numpy as np

plotly.tools.set_credentials_file(username='KristinaKulivnyk', api_key='n1y0rfzdEbEjx63HoUx5')


trace1 = go.Scatter3d(
    x=X_0[:, 0],
    y=X_0[:, 1],
    z=X_0[:, 2],
    mode='markers',
    marker=dict(
        size=12,
        color=z,                # set color to an array/list of desired values
        colorscale='Greys',   # choose a colorscale
        opacity=0.8
    )
)

trace2 = go.Scatter3d(
    x=X_1[:, 0],
    y=X_1[:, 1],
    z=X_1[:, 2],
    mode='markers',
    marker=dict(
        size=12,
        color=z,                # set color to an array/list of desired values
        colorscale='Reds',   # choose a colorscale
        opacity=0.8
    )
)

data = [trace1, trace2]
layout = go.Layout(
    margin=dict(
        l=0,
        r=0,
        b=0,
        t=0
    )
)
fig = go.Figure(data=data, layout=layout)
py.iplot(fig, filename='3d-scatter-colorscale')

But it is not quite clear how to build a 3-D decision boundary. I’ll appreciate any help.

2

Hi @KrisKuliv,

Here is an example that I converted from matplotlib to plotly.py. Original example for this SO answer https://stackoverflow.com/questions/36232334/plotting-3d-decision-boundary-from-linear-svm.

import numpy as np
from sklearn.svm import SVC

rs = np.random.RandomState(1234)

# Generate some fake data.
n_samples = 200
# X is the input features by row.
X = np.zeros((200,3))
X[:n_samples//2] = rs.multivariate_normal( np.ones(3), np.eye(3), size=n_samples//2)
X[n_samples//2:] = rs.multivariate_normal(-np.ones(3), np.eye(3), size=n_samples//2)
# Y is the class labels for each row of X.
Y = np.zeros(n_samples); Y[n_samples//2:] = 1

# Fit the data with an svm
svc = SVC(kernel='linear')
svc.fit(X,Y)

# The equation of the separating plane is given by all x in R^3 such that:
# np.dot(svc.coef_[0], x) + b = 0. We should solve for the last coordinate
# to plot the plane in terms of x and y.

z = lambda x,y: (-svc.intercept_[0]-svc.coef_[0][0]*x-svc.coef_[0][1]*y) / svc.coef_[0][2]

tmp = np.linspace(-2,2,51)
x,y = np.meshgrid(tmp,tmp)

# Plot stuff.
fig = go.FigureWidget()
fig.add_surface(x=x, y=y, z=z(x,y), colorscale='Greys', showscale=False)
fig.add_scatter3d(x=X[Y==0,0], y=X[Y==0,1], z=X[Y==0,2], mode='markers', marker={'color': 'blue'})
fig.add_scatter3d(x=X[Y==1,0], y=X[Y==1,1], z=X[Y==1,2], mode='markers', marker={'color': 'red'})
fig

newplot%20(1)
newplot (1).png700×450 21.3 KB

Hope that helps get you started!
-Jon

4

z = lambda x,y: (-svc.intercept_[0]-svc.coef_[0][0]*x-svc.coef_[0][1]*y) / svc.coef_[0][2]<---- using this equation is not helping in my application…
how did you choose this [tmp = np.linspace(-2,2,51)] …?
can you pls explain how to calculate for other application[data]…?

5

@Aparna

To get a code working for you data, please, comment out these two lines:

tmp = np.linspace(-2,2,51)
x,y = np.meshgrid(tmp,tmp)

and replace them by:

xm, xM = X[:,0].min(), X[:, 0].max()
ym, yM = X[:,1].min(), X[:, 1].max()
x = np.linspace(xm, xM, 10)
y = np.linspace(ym, yM, 10)
x, y =np.meshgrid(x, y)

The last lines of code extracted the x- values, respectively y-values range. Then it is defined a meshgrid over the rectangle [xm, xM] x [ym, yM] and evaluated (via z(x,y) )the separating plane equation at the points of this grid, to get data for a go.Surface, that represents the plane.

I replaced the ‘Greys’ colorscale with a simple one (there is no reason to color the separating plane darker in its upper part):
my_colorscale = [[0, ‘rgb(230,230,230)’], [1, ‘rgb(230,230,230)’]]`,
and added opacity =0.9:

fig.add_surface(x=x, y=y, z=z(x,y), colorscale=my_colorscale, showscale=False, opacity=0.9)
and finally updated layout to width=800, height=800.

1 Like
6

Have updated… Still not getting the hyperplane to my application… as I am new user could not upload results… pls help.

type or paste code here
import plotly.graph_objects as go

xm, xM = X[:,0].min(), X[:, 0].max()
ym, yM = X[:,1].min(), X[:, 1].max()
x = np.linspace(xm, xM, 10)
y = np.linspace(ym, yM, 10)
x, y =np.meshgrid(x, y)
z = lambda x,y: (-clf.intercept_[0]-clf.coef_[0][0]*x -clf.coef_[0][1]*y) / clf.coef_[0][2]


fig = go.FigureWidget()
fig.add_surface(x=x, y=y, z=z(x,y), colorscale='Greys', showscale=False)
fig.add_scatter3d(x=X[Y==0,0], y=X[Y==0,1], z=X[Y==0,2], mode='markers', marker={'color': 'blue'})
fig.add_scatter3d(x=X[Y==1,0], y=X[Y==1,1], z=X[Y==1,2], mode='markers', marker={'color': 'red'})
fig
7

@Aparna

Here is the code:

import numpy as np
from sklearn.svm import SVC
import plotly.graph_objects as go
rs = np.random.RandomState(1234)

# Generate some fake data.
n_samples = 200
# X is the input features by row.
X = np.zeros((200,3))
X[:n_samples//2] = rs.multivariate_normal( np.ones(3), np.eye(3), size=n_samples//2)
X[n_samples//2:] = rs.multivariate_normal(-np.ones(3), np.eye(3), size=n_samples//2)
# Y is the class labels for each row of X.
Y = np.zeros(n_samples); Y[n_samples//2:] = 1

# Fit the data with an svm
svc = SVC(kernel='linear')
svc.fit(X,Y)

# The equation of the separating plane is given by all x in R^3 such that:
# np.dot(svc.coef_[0], x) + b = 0. We should solve for the last coordinate
# to plot the plane in terms of x and y.

z = lambda x,y: (-svc.intercept_[0]-svc.coef_[0][0]*x-svc.coef_[0][1]*y) / svc.coef_[0][2]

#tmp = np.linspace(-2,2,51)
#x,y = np.meshgrid(tmp,tmp)
xm, xM = X[:,0].min(), X[:, 0].max()
ym, yM = X[:,1].min(), X[:, 1].max()
x = np.linspace(xm, xM, 10)
y = np.linspace(ym, yM, 10)
x, y =np.meshgrid(x, y)

my_colorscale= [[0, 'rgb(230,230,230)'], [1, 'rgb(230,230,230)']]
fig = go.Figure()
fig.add_surface(x=x, y=y, z=z(x,y), colorscale=my_colorscale, showscale=False, opacity=0.9)
fig.add_scatter3d(x=X[Y==0,0], y=X[Y==0,1], z=X[Y==0,2], mode='markers', marker={'color': 'blue'})
fig.add_scatter3d(x=X[Y==1,0], y=X[Y==1,1], z=X[Y==1,2], mode='markers', marker={'color': 'red'})
fig.update_layout(width=800, height=800)
fig.show()
1 Like
9

@Aparna

I’m sorry, but with pasted, not formatted data, I do not work.
I gave you the pointers to change the code. From now on it’s not a plotly question.

10

Thank you Empet… :slight_smile: Your points and suggestions are working. Previously I had messed up in data reading…
Thank you so much :slight_smile:

1 Like
11

Hello guys

I tried this code with rbf kernel it was giving me an error, saying :AttributeError: coef_ is only available when using a linear kernel, so what would be the z for non linear kernels like rbf, I know that mathematically it is y(x)=\Sigma_{n=1}^N a_n t_n k(x,x_n) +b, is there a way to plot from the above code?