Four Algorithms, One Honest Verdict
Data science case study — Unsupervised Machine Learning — automate playlist creation for a music startup using unsupervised clustering. What started as "run K-Means" became a genuine, iterative investigation across four methods, several real bugs, and one finding that overturned its own headline conclusion mid-analysis.
Moosic needs to automate playlist creation
Moosic, a fictional music startup built for the case study, hands over ~5,000 songs' worth of Spotify audio features and two open questions: can audio features identify "similar" songs the way a human listener does, and which clustering algorithm actually works best for turning that similarity into usable playlists?
Can features capture "similar"?
Danceability, energy, tempo and friends — do they track what a listener actually hears as alike?
Which algorithm works best?
K-Means, DBSCAN, or Hierarchical — each with real strengths and real failure modes to find out.
Feature selection → business-constrained K-Means
The starting 13 Spotify audio features were correlation-checked before anything else — loudness turned out almost redundant with energy and acousticness, while speechiness and instrumentalness carried genuine independent signal the original 5-feature course example missed entirely. Seven features made the final cut.
The elbow method alone suggested 3–5 clusters on the full dataset — mathematically defensible, practically useless (1,000+ songs per "playlist"). The actual fix: define the business constraint explicitly (a usable playlist is 20–60 songs) and build a recursive split → global reassignment → distance-based merge pipeline around that target, rather than trusting a metric to discover it unprompted.
Where the simple version hit a wall
K-Means with a business constraint works — but "works" isn't the same as "proven right." Three more investigations followed, each testing the result from a different angle rather than taking it on faith.
DBSCAN
Extensively tuned across two scalers and a wide eps/min_samples sweep — never found a workable middle ground between fragmentation and one dominant mega-cluster. But its noise bucket wasn't empty: it isolated a genuine classical/instrumental pocket and a hidden rap/hip-hop pocket the other methods hadn't named yet.
Hierarchical (Agglomerative)
Best-scoring cut (k=2–3) turned out to be the same misleading mega-cluster pattern as DBSCAN in disguise. k=8 gave 8 genuinely balanced groups — and independently re-found both of DBSCAN's hidden pockets, via a completely different mechanism.
The Hybrid Experiment
Agglomerative's 8 groups first, K-Means split within each. Looked like a clear win — 76 of 129 single-stage playlists "spanned" multiple broad categories. Checking one by ear: it was a completely coherent jazz & bossa nova playlist the single-stage pipeline got right. The hybrid would have split it apart.
Cross-Algorithm Agreement
K-Means and Agglomerative agree strongly (ARI 0.555) on genuinely different assumptions. DBSCAN's apparent disagreement dissolves entirely once resolution mismatch is accounted for — on the one distinction it committed to, agreement with K-Means is perfect.
Ship K-Means, business-constrained — with a human always in the loop
| Method | Coverage | Coherence | Verdict |
|---|---|---|---|
| K-Means | ~98% | Good, business-tuned | Recommended |
| DBSCAN | Up to 92%* | Diagnostic only | *hides a mega-cluster, not real playlists |
| Hierarchical | 100% | Good at k=8 | Confirms — not a replacement |
No algorithm has a concept of "good" on its own — every useful result here came from pairing the math with an explicit, human-defined goal. And audio features track sound, not genre: a rock song ("Teenage Dirtbag") landed in a dance-pop playlist on tempo/energy alone, while an unlabelled piano cover of "Shape of You" was correctly told apart from the original — two sides of the same limitation.
An individual project, built out loud
This was a solo unsupervised (machine) learning assignment, worked through in close collaboration with Claude (Anthropic) across the full arc — from "is fine clustering even a real technique" through nine documented bugs, a self-corrected finding, and a final cross-validated recommendation. Every course-correction is kept in the project's session report rather than edited out after the fact.
A parallel nod to Andreas, a colleague working the same course material — while this project stayed deliberately narrow (three core algorithms, tuned properly, one cross-validated conclusion), he built a full-stack playlist generation web app from the same Moosic brief — React/FastAPI, four algorithms including Gaussian Mixture Models, four scaler options, a real drag-and-drop UI. Different kind of ambitious, not a competing one: his own project notes independently land on almost the exact same conclusion this one did — that acoustically similar songs can span genres a listener would never group together, and that "no single clustering metric can fully measure musical usefulness." Reached from two completely different directions, same honest place.
The working metaphor for the whole project: unsupervised clustering is a blindfolded sort by touch alone. Sometimes the tools you're given (the features you chose) are like wearing boxing gloves — comically wrong for delicate work. The point isn't to pretend the blindfold isn't there. It's to be honest about what you can and can't feel, and check by listening whenever it's not sure.