API Reference¶
Major classes are HDBSCAN
and RobustSingleLinkage
.
HDBSCAN¶
-
class
hdbscan.hdbscan_.
HDBSCAN
(min_cluster_size=5, min_samples=None, metric='euclidean', alpha=1.0, p=None, algorithm='best', leaf_size=40, memory=Memory(location=None), approx_min_span_tree=True, gen_min_span_tree=False, core_dist_n_jobs=4, cluster_selection_method='eom', allow_single_cluster=False, prediction_data=False, match_reference_implementation=False, **kwargs)¶ Perform HDBSCAN clustering from vector array or distance matrix.
HDBSCAN - Hierarchical Density-Based Spatial Clustering of Applications with Noise. Performs DBSCAN over varying epsilon values and integrates the result to find a clustering that gives the best stability over epsilon. This allows HDBSCAN to find clusters of varying densities (unlike DBSCAN), and be more robust to parameter selection.
- min_cluster_size : int, optional (default=5)
- The minimum size of clusters; single linkage splits that contain fewer points than this will be considered points “falling out” of a cluster rather than a cluster splitting into two new clusters.
- min_samples : int, optional (default=None)
- The number of samples in a neighbourhood for a point to be considered a core point.
- metric : string, or callable, optional (default=’euclidean’)
- The metric to use when calculating distance between instances in a feature array. If metric is a string or callable, it must be one of the options allowed by metrics.pairwise.pairwise_distances for its metric parameter. If metric is “precomputed”, X is assumed to be a distance matrix and must be square.
- p : int, optional (default=None)
- p value to use if using the minkowski metric.
- alpha : float, optional (default=1.0)
- A distance scaling parameter as used in robust single linkage. See [3] for more information.
- algorithm : string, optional (default=’best’)
Exactly which algorithm to use; hdbscan has variants specialised for different characteristics of the data. By default this is set to
best
which chooses the “best” algorithm given the nature of the data. You can force other options if you believe you know better. Options are:best
generic
prims_kdtree
prims_balltree
boruvka_kdtree
boruvka_balltree
- leaf_size: int, optional (default=40)
- If using a space tree algorithm (kdtree, or balltree) the number of points ina leaf node of the tree. This does not alter the resulting clustering, but may have an effect on the runtime of the algorithm.
- memory : Instance of joblib.Memory or string (optional)
- Used to cache the output of the computation of the tree. By default, no caching is done. If a string is given, it is the path to the caching directory.
- approx_min_span_tree : bool, optional (default=True)
- Whether to accept an only approximate minimum spanning tree. For some algorithms this can provide a significant speedup, but the resulting clustering may be of marginally lower quality. If you are willing to sacrifice speed for correctness you may want to explore this; in general this should be left at the default True.
- gen_min_span_tree: bool, optional (default=False)
- Whether to generate the minimum spanning tree with regard to mutual reachability distance for later analysis.
- core_dist_n_jobs : int, optional (default=4)
- Number of parallel jobs to run in core distance computations (if
supported by the specific algorithm). For
core_dist_n_jobs
below -1, (n_cpus + 1 + core_dist_n_jobs) are used. - cluster_selection_method : string, optional (default=’eom’)
The method used to select clusters from the condensed tree. The standard approach for HDBSCAN* is to use an Excess of Mass algorithm to find the most persistent clusters. Alternatively you can instead select the clusters at the leaves of the tree – this provides the most fine grained and homogeneous clusters. Options are:
eom
leaf
- allow_single_cluster : bool, optional (default=False)
- By default HDBSCAN* will not produce a single cluster, setting this to True will override this and allow single cluster results in the case that you feel this is a valid result for your dataset.
- prediction_data : boolean, optional
- Whether to generate extra cached data for predicting labels or membership vectors few new unseen points later. If you wish to persist the clustering object for later re-use you probably want to set this to True. (default False)
- match_reference_implementation : bool, optional (default=False)
- There exist some interpretational differences between this HDBSCAN* implementation and the original authors reference implementation in Java. This can result in very minor differences in clustering results. Setting this flag to True will, at a some performance cost, ensure that the clustering results match the reference implementation.
- **kwargs : optional
- Arguments passed to the distance metric
- labels_ : ndarray, shape (n_samples, )
- Cluster labels for each point in the dataset given to fit(). Noisy samples are given the label -1.
- probabilities_ : ndarray, shape (n_samples, )
- The strength with which each sample is a member of its assigned cluster. Noise points have probability zero; points in clusters have values assigned proportional to the degree that they persist as part of the cluster.
- cluster_persistence_ : ndarray, shape (n_clusters, )
- A score of how persistent each cluster is. A score of 1.0 represents a perfectly stable cluster that persists over all distance scales, while a score of 0.0 represents a perfectly ephemeral cluster. These scores can be guage the relative coherence of the clusters output by the algorithm.
- condensed_tree_ : CondensedTree object
- The condensed tree produced by HDBSCAN. The object has methods for converting to pandas, networkx, and plotting.
- single_linkage_tree_ : SingleLinkageTree object
- The single linkage tree produced by HDBSCAN. The object has methods for converting to pandas, networkx, and plotting.
- minimum_spanning_tree_ : MinimumSpanningTree object
- The minimum spanning tree of the mutual reachability graph generated by HDBSCAN. Note that this is not generated by default and will only be available if gen_min_span_tree was set to True on object creation. Even then in some optimized cases a tre may not be generated.
- outlier_scores_ : ndarray, shape (n_samples, )
- Outlier scores for clustered points; the larger the score the more outlier-like the point. Useful as an outlier detection technique. Based on the GLOSH algorithm by Campello, Moulavi, Zimek and Sander.
- prediction_data_ : PredictionData object
- Cached data used for predicting the cluster labels of new or
unseen points. Necessary only if you are using functions from
hdbscan.prediction
(seeapproximate_predict()
,membership_vector()
, andall_points_membership_vectors()
). - exemplars_ : list
- A list of exemplar points for clusters. Since HDBSCAN supports arbitrary shapes for clusters we cannot provide a single cluster exemplar per cluster. Instead a list is returned with each element of the list being a numpy array of exemplar points for a cluster – these points are the “most representative” points of the cluster.
[1] Campello, R. J., Moulavi, D., & Sander, J. (2013, April). Density-based clustering based on hierarchical density estimates. In Pacific-Asia Conference on Knowledge Discovery and Data Mining (pp. 160-172). Springer Berlin Heidelberg. [2] Campello, R. J., Moulavi, D., Zimek, A., & Sander, J. (2015). Hierarchical density estimates for data clustering, visualization, and outlier detection. ACM Transactions on Knowledge Discovery from Data (TKDD), 10(1), 5. [3] Chaudhuri, K., & Dasgupta, S. (2010). Rates of convergence for the cluster tree. In Advances in Neural Information Processing Systems (pp. 343-351). -
fit
(X, y=None)¶ Perform HDBSCAN clustering from features or distance matrix.
- X : array or sparse (CSR) matrix of shape (n_samples, n_features), or array of shape (n_samples, n_samples)
- A feature array, or array of distances between samples if
metric='precomputed'
.
- self : object
- Returns self
-
fit_predict
(X, y=None)¶ Performs clustering on X and returns cluster labels.
- X : array or sparse (CSR) matrix of shape (n_samples, n_features), or array of shape (n_samples, n_samples)
- A feature array, or array of distances between samples if
metric='precomputed'
.
- y : ndarray, shape (n_samples, )
- cluster labels
-
generate_prediction_data
()¶ Create data that caches intermediate results used for predicting the label of new/unseen points. This data is only useful if you are intending to use functions from
hdbscan.prediction
.
RobustSingleLinkage¶
-
class
hdbscan.robust_single_linkage_.
RobustSingleLinkage
(cut=0.4, k=5, alpha=1.4142135623730951, gamma=5, metric='euclidean', algorithm='best', core_dist_n_jobs=4, metric_params={})¶ Perform robust single linkage clustering from a vector array or distance matrix.
Robust single linkage is a modified version of single linkage that attempts to be more robust to noise. Specifically the goal is to more accurately approximate the level set tree of the unknown probability density function from which the sample data has been drawn.
- X : array or sparse (CSR) matrix of shape (n_samples, n_features), or \
- array of shape (n_samples, n_samples)
A feature array, or array of distances between samples if
metric='precomputed'
. - cut : float
- The reachability distance value to cut the cluster heirarchy at to derive a flat cluster labelling.
- k : int, optional (default=5)
- Reachability distances will be computed with regard to the k nearest neighbors.
- alpha : float, optional (default=np.sqrt(2))
- Distance scaling for reachability distance computation. Reachability distance is computed as $max { core_k(a), core_k(b), 1/alpha d(a,b) }$.
- gamma : int, optional (default=5)
- Ignore any clusters in the flat clustering with size less than gamma, and declare points in such clusters as noise points.
- metric : string, or callable, optional (default=’euclidean’)
- The metric to use when calculating distance between instances in a feature array. If metric is a string or callable, it must be one of the options allowed by metrics.pairwise.pairwise_distances for its metric parameter. If metric is “precomputed”, X is assumed to be a distance matrix and must be square.
- metric_params : dict, option (default={})
- Keyword parameter arguments for calling the metric (for example the p values if using the minkowski metric).
- algorithm : string, optional (default=’best’)
Exactly which algorithm to use; hdbscan has variants specialised for different characteristics of the data. By default this is set to
best
which chooses the “best” algorithm given the nature of the data. You can force other options if you believe you know better. Options are:small
small_kdtree
large_kdtree
large_kdtree_fastcluster
- core_dist_n_jobs : int, optional
- Number of parallel jobs to run in core distance computations (if
supported by the specific algorithm). For
core_dist_n_jobs
below -1, (n_cpus + 1 + core_dist_n_jobs) are used. (default 4)
- labels_ : ndarray, shape (n_samples, )
- Cluster labels for each point. Noisy samples are given the label -1.
- cluster_hierarchy_ : SingleLinkageTree object
The single linkage tree produced during clustering. This object provides several methods for:
- Plotting
- Generating a flat clustering
- Exporting to NetworkX
- Exporting to Pandas
[1] Chaudhuri, K., & Dasgupta, S. (2010). Rates of convergence for the cluster tree. In Advances in Neural Information Processing Systems (pp. 343-351). -
fit
(X, y=None)¶ Perform robust single linkage clustering from features or distance matrix.
- X : array or sparse (CSR) matrix of shape (n_samples, n_features), or array of shape (n_samples, n_samples)
- A feature array, or array of distances between samples if
metric='precomputed'
.
- self : object
- Returns self
-
fit_predict
(X, y=None)¶ Performs clustering on X and returns cluster labels.
- X : array or sparse (CSR) matrix of shape (n_samples, n_features), or array of shape (n_samples, n_samples)
- A feature array, or array of distances between samples if
metric='precomputed'
.
- y : ndarray, shape (n_samples, )
- cluster labels
Utilities¶
Other useful classes are contained in the plots module, the validity module, and the prediction module.
-
class
hdbscan.plots.
CondensedTree
(condensed_tree_array, cluster_selection_method='eom', allow_single_cluster=False)¶ The condensed tree structure, which provides a simplified or smoothed version of the
SingleLinkageTree
.- condensed_tree_array : numpy recarray from
HDBSCAN
- The raw numpy rec array version of the condensed tree as produced internally by hdbscan.
- cluster_selection_method : string, optional (default ‘eom’)
- The method of selecting clusters. One of ‘eom’ or ‘leaf’
- allow_single_cluster : Boolean, optional (default False)
- Whether to allow the root cluster as the only selected cluster
-
get_plot_data
(leaf_separation=1, log_size=False, max_rectangle_per_icicle=20)¶ Generates data for use in plotting the ‘icicle plot’ or dendrogram plot of the condensed tree generated by HDBSCAN.
- leaf_separation : float, optional
- How far apart to space the final leaves of the dendrogram. (default 1)
- log_size : boolean, optional
- Use log scale for the ‘size’ of clusters (i.e. number of points in the cluster at a given lambda value). (default False)
- max_rectangles_per_icicle : int, optional
- To simplify the plot this method will only emit
max_rectangles_per_icicle
bars per branch of the dendrogram. This ensures that we don’t suffer from massive overplotting in cases with a lot of data points.
- plot_data : dict
- Data associated to bars in a bar plot:
bar_centers x coordinate centers for bars bar_tops heights of bars in lambda scale bar_bottoms y coordinate of bottoms of bars bar_widths widths of the bars (in x coord scale) bar_bounds a 4-tuple of [left, right, bottom, top]
giving the bounds on a full set of cluster bars- Data associates with cluster splits:
- line_xs x coordinates for horizontal dendrogram lines line_ys y coordinates for horizontal dendrogram lines
-
plot
(leaf_separation=1, cmap='viridis', select_clusters=False, label_clusters=False, selection_palette=None, axis=None, colorbar=True, log_size=False, max_rectangles_per_icicle=20)¶ Use matplotlib to plot an ‘icicle plot’ dendrogram of the condensed tree.
Effectively this is a dendrogram where the width of each cluster bar is equal to the number of points (or log of the number of points) in the cluster at the given lambda value. Thus bars narrow as points progressively drop out of clusters. The make the effect more apparent the bars are also colored according the the number of points (or log of the number of points).
- leaf_separation : float, optional (default 1)
- How far apart to space the final leaves of the dendrogram.
- cmap : string or matplotlib colormap, optional (default viridis)
- The matplotlib colormap to use to color the cluster bars.
- select_clusters : boolean, optional (default False)
- Whether to draw ovals highlighting which cluster bar represent the clusters that were selected by HDBSCAN as the final clusters.
- label_clusters : boolean, optional (default False)
- If select_clusters is True then this determines whether to draw text labels on the clusters.
- selection_palette : list of colors, optional (default None)
- If not None, and at least as long as the number of clusters, draw ovals in colors iterating through this palette. This can aid in cluster identification when plotting.
- axis : matplotlib axis or None, optional (default None)
- The matplotlib axis to render to. If None then a new axis will be generated. The rendered axis will be returned.
- colorbar : boolean, optional (default True)
- Whether to draw a matplotlib colorbar displaying the range of cluster sizes as per the colormap.
- log_size : boolean, optional (default False)
- Use log scale for the ‘size’ of clusters (i.e. number of points in the cluster at a given lambda value).
- max_rectangles_per_icicle : int, optional (default 20)
- To simplify the plot this method will only emit
max_rectangles_per_icicle
bars per branch of the dendrogram. This ensures that we don’t suffer from massive overplotting in cases with a lot of data points.Returns
- axis : matplotlib axis
- The axis on which the ‘icicle plot’ has been rendered.
-
to_networkx
()¶ Return a NetworkX DiGraph object representing the condensed tree.
Edge weights in the graph are the lamba values at which child nodes ‘leave’ the parent cluster.
Nodes have a size attribute attached giving the number of points that are in the cluster (or 1 if it is a singleton point) at the point of cluster creation (fewer points may be in the cluster at larger lambda values).
-
to_numpy
()¶ Return a numpy structured array representation of the condensed tree.
-
to_pandas
()¶ Return a pandas dataframe representation of the condensed tree.
Each row of the dataframe corresponds to an edge in the tree. The columns of the dataframe are parent, child, lambda_val and child_size.
The parent and child are the ids of the parent and child nodes in the tree. Node ids less than the number of points in the original dataset represent individual points, while ids greater than the number of points are clusters.
The lambda_val value is the value (1/distance) at which the child node leaves the cluster.
The child_size is the number of points in the child node.
- condensed_tree_array : numpy recarray from
-
class
hdbscan.plots.
SingleLinkageTree
(linkage)¶ A single linkage format dendrogram tree, with plotting functionality and networkX support.
- linkage : ndarray (n_samples, 4)
- The numpy array that holds the tree structure. As output by scipy.cluster.hierarchy, hdbscan, of fastcluster.
-
get_clusters
(cut_distance, min_cluster_size=5)¶ Return a flat clustering from the single linkage hierarchy.
This represents the result of selecting a cut value for robust single linkage clustering. The min_cluster_size allows the flat clustering to declare noise points (and cluster smaller than min_cluster_size).
- cut_distance : float
- The mutual reachability distance cut value to use to generate a flat clustering.
- min_cluster_size : int, optional
- Clusters smaller than this value with be called ‘noise’ and remain unclustered in the resulting flat clustering.
- labels : array [n_samples]
- An array of cluster labels, one per datapoint. Unclustered points are assigned the label -1.
-
plot
(axis=None, truncate_mode=None, p=0, vary_line_width=True, cmap='viridis', colorbar=True)¶ Plot a dendrogram of the single linkage tree.
- truncate_mode : str, optional
- The dendrogram can be hard to read when the original observation matrix from which the linkage is derived is large. Truncation is used to condense the dendrogram. There are several modes:
None/'none'
- No truncation is performed (Default).
'lastp'
- The last p non-singleton formed in the linkage are the only non-leaf nodes in the linkage; they correspond to rows Z[n-p-2:end] in Z. All other non-singleton clusters are contracted into leaf nodes.
'level'/'mtica'
- No more than p levels of the dendrogram tree are displayed. This corresponds to Mathematica(TM) behavior.
- p : int, optional
- The
p
parameter fortruncate_mode
. - vary_line_width : boolean, optional
- Draw downward branches of the dendrogram with line thickness that varies depending on the size of the cluster.
- cmap : string or matplotlib colormap, optional
- The matplotlib colormap to use to color the cluster bars. A value of ‘none’ will result in black bars. (default ‘viridis’)
- colorbar : boolean, optional
- Whether to draw a matplotlib colorbar displaying the range of cluster sizes as per the colormap. (default True)
- axis : matplotlib axis
- The axis on which the dendrogram plot has been rendered.
-
to_networkx
()¶ Return a NetworkX DiGraph object representing the single linkage tree.
Edge weights in the graph are the distance values at which child nodes merge to form the parent cluster.
Nodes have a size attribute attached giving the number of points that are in the cluster.
-
to_numpy
()¶ Return a numpy array representation of the single linkage tree.
This representation conforms to the scipy.cluster.hierarchy notion of a single linkage tree, and can be used with all the associated scipy tools. Please see the scipy documentation for more details on the format.
-
to_pandas
()¶ Return a pandas dataframe representation of the single linkage tree.
Each row of the dataframe corresponds to an edge in the tree. The columns of the dataframe are parent, left_child, right_child, distance and size.
The parent, left_child and right_child are the ids of the parent and child nodes in the tree. Node ids less than the number of points in the original dataset represent individual points, while ids greater than the number of points are clusters.
The distance value is the at which the child nodes merge to form the parent node.
The size is the number of points in the parent node.
-
class
hdbscan.plots.
MinimumSpanningTree
(mst, data)¶ -
plot
(axis=None, node_size=40, node_color='k', node_alpha=0.8, edge_alpha=0.5, edge_cmap='viridis_r', edge_linewidth=2, vary_line_width=True, colorbar=True)¶ Plot the minimum spanning tree (as projected into 2D by t-SNE if required).
- axis : matplotlib axis, optional
- The axis to render the plot to
- node_size : int, optional
- The size of nodes in the plot (default 40).
- node_color : matplotlib color spec, optional
- The color to render nodes (default black).
- node_alpha : float, optional
- The alpha value (between 0 and 1) to render nodes with (default 0.8).
- edge_cmap : matplotlib colormap, optional
- The colormap to color edges by (varying color by edge
- weight/distance). Can be a cmap object or a string recognised by matplotlib. (default viridis_r)
- edge_alpha : float, optional
- The alpha value (between 0 and 1) to render edges with (default 0.5).
- edge_linewidth : float, optional
- The linewidth to use for rendering edges (default 2).
- vary_line_width : bool, optional
- Edge width is proportional to (log of) the inverse of the mutual reachability distance. (default True)
- colorbar : bool, optional
- Whether to draw a colorbar. (default True)
- axis : matplotlib axis
- The axis used the render the plot.
-
to_networkx
()¶ Return a NetworkX Graph object representing the minimum spanning tree.
Edge weights in the graph are the distance between the nodes they connect.
Nodes have a data attribute attached giving the data vector of the associated point.
-
to_numpy
()¶ Return a numpy array of weighted edges in the minimum spanning tree
-
to_pandas
()¶ Return a Pandas dataframe of the minimum spanning tree.
Each row is an edge in the tree; the columns are from, to, and distance giving the two vertices of the edge which are indices into the dataset, and the distance between those datapoints.
-
-
hdbscan.validity.
all_points_core_distance
(distance_matrix, d=2.0)¶ Compute the all-points-core-distance for all the points of a cluster.
- distance_matrix : array (cluster_size, cluster_size)
- The pairwise distance matrix between points in the cluster.
- d : integer
- The dimension of the data set, which is used in the computation of the all-point-core-distance as per the paper.
- core_distances : array (cluster_size,)
- The all-points-core-distance of each point in the cluster
Moulavi, D., Jaskowiak, P.A., Campello, R.J., Zimek, A. and Sander, J., 2014. Density-Based Clustering Validation. In SDM (pp. 839-847).
-
hdbscan.validity.
all_points_mutual_reachability
(X, labels, cluster_id, metric='euclidean', d=None, **kwd_args)¶ Compute the all-points-mutual-reachability distances for all the points of a cluster.
If metric is ‘precomputed’ then assume X is a distance matrix for the full dataset. Note that in this case you must pass in ‘d’ the dimension of the dataset.
- X : array (n_samples, n_features) or (n_samples, n_samples)
- The input data of the clustering. This can be the data, or, if metric is set to precomputed the pairwise distance matrix used for the clustering.
- labels : array (n_samples)
- The label array output by the clustering, providing an integral cluster label to each data point, with -1 for noise points.
- cluster_id : integer
- The cluster label for which to compute the all-points mutual-reachability (which should be done on a cluster by cluster basis).
- metric : string
- The metric used to compute distances for the clustering (and to be re-used in computing distances for mr distance). If set to precomputed then X is assumed to be the precomputed distance matrix between samples.
- d : integer (or None)
- The number of features (dimension) of the dataset. This need only be set in the case of metric being set to precomputed, where the ambient dimension of the data is unknown to the function.
- **kwd_args :
- Extra arguments to pass to the distance computation for other metrics, such as minkowski, Mahanalobis etc.
- mutual_reachaibility : array (n_samples, n_samples)
- The pairwise mutual reachability distances between all points in X with label equal to cluster_id.
- core_distances : array (n_samples,)
- The all-points-core_distance of all points in X with label equal to cluster_id.
Moulavi, D., Jaskowiak, P.A., Campello, R.J., Zimek, A. and Sander, J., 2014. Density-Based Clustering Validation. In SDM (pp. 839-847).
-
hdbscan.validity.
density_separation
(X, labels, cluster_id1, cluster_id2, internal_nodes1, internal_nodes2, core_distances1, core_distances2, metric='euclidean', **kwd_args)¶ Compute the density separation between two clusters. This is the minimum all-points mutual reachability distance between pairs of points, one from internal nodes of MSTs of each cluster.
- X : array (n_samples, n_features) or (n_samples, n_samples)
- The input data of the clustering. This can be the data, or, if metric is set to precomputed the pairwise distance matrix used for the clustering.
- labels : array (n_samples)
- The label array output by the clustering, providing an integral cluster label to each data point, with -1 for noise points.
- cluster_id1 : integer
- The first cluster label to compute separation between.
- cluster_id2 : integer
- The second cluster label to compute separation between.
- internal_nodes1 : array
- The vertices of the MST for cluster_id1 that were internal vertices.
- internal_nodes2 : array
- The vertices of the MST for cluster_id2 that were internal vertices.
- core_distances1 : array (size of cluster_id1,)
- The all-points-core_distances of all points in the cluster specified by cluster_id1.
- core_distances2 : array (size of cluster_id2,)
- The all-points-core_distances of all points in the cluster specified by cluster_id2.
- metric : string
- The metric used to compute distances for the clustering (and to be re-used in computing distances for mr distance). If set to precomputed then X is assumed to be the precomputed distance matrix between samples.
- **kwd_args :
- Extra arguments to pass to the distance computation for other metrics, such as minkowski, Mahanalobis etc.
The ‘density separation’ between the clusters specified by cluster_id1 and cluster_id2.
Moulavi, D., Jaskowiak, P.A., Campello, R.J., Zimek, A. and Sander, J., 2014. Density-Based Clustering Validation. In SDM (pp. 839-847).
-
hdbscan.validity.
internal_minimum_spanning_tree
(mr_distances)¶ Compute the ‘internal’ minimum spanning tree given a matrix of mutual reachability distances. Given a minimum spanning tree the ‘internal’ graph is the subgraph induced by vertices of degree greater than one.
- mr_distances : array (cluster_size, cluster_size)
- The pairwise mutual reachability distances, inferred to be the edge weights of a complete graph. Since MSTs are computed per cluster this is the all-points-mutual-reacability for points within a single cluster.
- internal_nodes : array
- An array listing the indices of the internal nodes of the MST
- internal_edges : array (?, 3)
- An array of internal edges in weighted edge list format; that is an edge is an array of length three listing the two vertices forming the edge and weight of the edge.
Moulavi, D., Jaskowiak, P.A., Campello, R.J., Zimek, A. and Sander, J., 2014. Density-Based Clustering Validation. In SDM (pp. 839-847).
-
hdbscan.validity.
validity_index
(X, labels, metric='euclidean', d=None, per_cluster_scores=False, **kwd_args)¶ Compute the density based cluster validity index for the clustering specified by labels and for each cluster in labels.
- X : array (n_samples, n_features) or (n_samples, n_samples)
- The input data of the clustering. This can be the data, or, if metric is set to precomputed the pairwise distance matrix used for the clustering.
- labels : array (n_samples)
- The label array output by the clustering, providing an integral cluster label to each data point, with -1 for noise points.
- metric : optional, string (default ‘euclidean’)
- The metric used to compute distances for the clustering (and to be re-used in computing distances for mr distance). If set to precomputed then X is assumed to be the precomputed distance matrix between samples.
- d : optional, integer (or None) (default None)
- The number of features (dimension) of the dataset. This need only be set in the case of metric being set to precomputed, where the ambient dimension of the data is unknown to the function.
- per_cluster_scores : optional, boolean (default False)
- Whether to return the validity index for individual clusters. Defaults to False with the function returning a single float value for the whole clustering.
- **kwd_args :
- Extra arguments to pass to the distance computation for other metrics, such as minkowski, Mahanalobis etc.
- validity_index : float
- The density based cluster validity index for the clustering. This is a numeric value between -1 and 1, with higher values indicating a ‘better’ clustering.
- per_cluster_validity_index : array (n_clusters,)
- The cluster validity index of each individual cluster as an array. The overall validity index is the weighted average of these values. Only returned if per_cluster_scores is set to True.
Moulavi, D., Jaskowiak, P.A., Campello, R.J., Zimek, A. and Sander, J., 2014. Density-Based Clustering Validation. In SDM (pp. 839-847).
-
class
hdbscan.prediction.
PredictionData
(data, condensed_tree, min_samples, tree_type='kdtree', metric='euclidean', **kwargs)¶ Extra data that allows for faster prediction if cached.
- data : array (n_samples, n_features)
- The original data set that was clustered
- condensed_tree : CondensedTree
- The condensed tree object created by a clustering
- min_samples : int
- The min_samples value used in clustering
- tree_type : string, optional
Which type of space tree to use for core distance computation. One of:
kdtree
balltree
- metric : string, optional
- The metric used to determine distance for the clustering. This is the metric that will be used for the space tree to determine core distances etc.
- **kwargs :
- Any further arguments to the metric.
- raw_data : array (n_samples, n_features)
- The original data set that was clustered
- tree : KDTree or BallTree
- A space partitioning tree that can be queried for nearest neighbors.
- core_distances : array (n_samples,)
- The core distances for every point in the original data set.
- cluster_map : dict
- A dictionary mapping cluster numbers in the condensed tree to labels in the final selected clustering.
- cluster_tree : structured array
- A version of the condensed tree that only contains clusters, not individual points.
- max_lambdas : dict
- A dictionary mapping cluster numbers in the condensed tree to the maximum lambda value seen in that cluster.
-
hdbscan.prediction.
all_points_membership_vectors
(clusterer)¶ Predict soft cluster membership vectors for all points in the original dataset the clusterer was trained on. This function is more efficient by making use of the fact that all points are already in the condensed tree, and processing in bulk.
- clusterer : HDBSCAN
- A clustering object that has been fit to the data and
either had
prediction_data=True
set, or called thegenerate_prediction_data
method after the fact. This method does not work if the clusterer was trained withmetric='precomputed'
.
- membership_vectors : array (n_samples, n_clusters)
- The probability that point
i
of the original dataset is a member of clusterj
is inmembership_vectors[i, j]
.
hdbscan.predict.predict()
hdbscan.predict.all_points_membership_vectors()
-
hdbscan.prediction.
approximate_predict
(clusterer, points_to_predict)¶ Predict the cluster label of new points. The returned labels will be those of the original clustering found by
clusterer
, and therefore are not (necessarily) the cluster labels that would be found by clustering the original data combined withpoints_to_predict
, hence the ‘approximate’ label.If you simply wish to assign new points to an existing clustering in the ‘best’ way possible, this is the function to use. If you want to predict how
points_to_predict
would cluster with the original data under HDBSCAN the most efficient existing approach is to simply recluster with the new point(s) added to the original dataset.- clusterer : HDBSCAN
- A clustering object that has been fit to the data and
either had
prediction_data=True
set, or called thegenerate_prediction_data
method after the fact. - points_to_predict : array, or array-like (n_samples, n_features)
- The new data points to predict cluster labels for. They should have the same dimensionality as the original dataset over which clusterer was fit.
- labels : array (n_samples,)
- The predicted labels of the
points_to_predict
- probabilities : array (n_samples,)
- The soft cluster scores for each of the
points_to_predict
hdbscan.predict.membership_vector()
hdbscan.predict.all_points_membership_vectors()
-
hdbscan.prediction.
membership_vector
(clusterer, points_to_predict)¶ Predict soft cluster membership. The result produces a vector for each point in
points_to_predict
that gives a probability that the given point is a member of a cluster for each of the selected clusters of theclusterer
.- clusterer : HDBSCAN
- A clustering object that has been fit to the data and
either had
prediction_data=True
set, or called thegenerate_prediction_data
method after the fact. - points_to_predict : array, or array-like (n_samples, n_features)
- The new data points to predict cluster labels for. They should have the same dimensionality as the original dataset over which clusterer was fit.
- membership_vectors : array (n_samples, n_clusters)
- The probability that point
i
is a member of clusterj
is inmembership_vectors[i, j]
.
hdbscan.predict.predict()
hdbscan.predict.all_points_membership_vectors()