Mathwords logoMathwords

Cluster Analysis — Definition, Formula & Example

Cluster analysis, also called clustering, is a method that sorts data points into groups so that points in the same group resemble each other more than they resemble points in other groups. No one tells the method what the groups are ahead of time. It finds them from the data alone.

Given observations x1,…,xn∈Rdx_1, \dots, x_n \in \mathbb{R}^d and a dissimilarity measure (often Euclidean distance), cluster analysis seeks a partition of the data into disjoint, nonempty subsets C1,…,CkC_1, \dots, C_k that keeps within-cluster dissimilarity small and between-cluster dissimilarity large. It is an unsupervised learning method because the observations carry no class labels. Partitioning methods such as k-means fix kk in advance. Hierarchical methods instead build a nested sequence of partitions, displayed as a dendrogram.

Key Formula

W=∑j=1k∑xi∈Cj∥xi−μj∥2W = \sum_{j=1}^{k} \sum_{x_i \in C_j} \lVert x_i - \mu_j \rVert^2
Where:
  • WW = within-cluster sum of squares, the quantity k-means tries to minimize
  • kk = number of clusters
  • CjC_j = the set of points assigned to cluster j
  • μj\mu_j = centroid (mean point) of cluster j
  • ∥xi−μj∥2\lVert x_i - \mu_j \rVert^2 = squared Euclidean distance from point xix_i to its cluster's centroid

Worked Example

Problem: Six students report (hours studying per week, hours gaming per week): A(1, 8), B(2, 9), C(3, 7), D(8, 2), E(9, 3), F(10, 1). Run k-means with k = 2, using A and F as the starting centroids.
Step 1: Assign each point to the nearer centroid using squared distance. For example, C is much closer to A(1, 8) than to F(10, 1). A, B, and C join cluster 1. D, E, and F join cluster 2.
d2(C,A)=(3−1)2+(7−8)2=5,d2(C,F)=(3−10)2+(7−1)2=85d^2(C, A) = (3-1)^2 + (7-8)^2 = 5, \qquad d^2(C, F) = (3-10)^2 + (7-1)^2 = 85
Step 2: Update each centroid to the mean of its cluster's points.
μ1=(1+2+33,8+9+73)=(2,8),μ2=(8+9+103,2+3+13)=(9,2)\mu_1 = \left(\tfrac{1+2+3}{3}, \tfrac{8+9+7}{3}\right) = (2, 8), \qquad \mu_2 = \left(\tfrac{8+9+10}{3}, \tfrac{2+3+1}{3}\right) = (9, 2)
Step 3: Reassign the points with the new centroids. Every point stays put. For example, D is 72 squared units from (2, 8) but only 1 from (9, 2). Since no point moved, the algorithm has converged. Compute W.
W=(1+1+2)+(1+1+2)=8W = (1+1+2) + (1+1+2) = 8
Answer: The two clusters are {A, B, C} (study less, game more) with centroid (2, 8) and {D, E, F} (study more, game less) with centroid (9, 2). The within-cluster sum of squares is W = 8.

Why It Matters

Clustering is a core topic in machine learning and data science courses. Businesses use it for customer segmentation, biologists use it to group genes with similar expression patterns, and image software uses it to compress colors. It is often the first step when you have a large data set and no labels.

Common Mistakes

Mistake: Clustering on variables measured in very different units, such as income in dollars and age in years, without rescaling them.
Correction: Distance-based methods let the variable with the largest numbers dominate. Standardize each variable first, for example by converting to z-scores, so that each one contributes fairly.

Related Terms

  • Cluster — The informal group this method finds systematically
  • Scatterplot — Displays two-variable data where clusters become visible
  • Bivariate Data — Paired measurements like those clustered in the example
  • Dot Plot — Shows one-variable clusters and gaps by eye
  • Histogram — Multiple peaks can hint at separate clusters
  • Euclidean Distance — Common way to measure how close data points are