[{"data":1,"prerenderedAt":698},["ShallowReactive",2],{"lang-switch-post-\u002Fen\u002Fplaylists\u002Fpattern-recognition\u002Fdbscan-semi-supervised":3,"post-en-pattern-recognition-dbscan-semi-supervised":4},"\u002Fplaylists\u002Fpattern-recognition\u002Fdbscan-semi-supervised",{"id":5,"title":6,"body":7,"cover":681,"date":682,"description":683,"extension":684,"meta":685,"navigation":686,"order":687,"path":688,"playlist":689,"seo":690,"status":691,"stem":692,"tags":693,"__hash__":697},"posts\u002Fen\u002Fplaylists\u002Fpattern-recognition\u002Fdbscan-semi-supervised.md","DBSCAN and Semi-Supervised Learning: When Clustering Helps You Label",{"type":8,"value":9,"toc":669},"minimark",[10,14,19,27,50,68,72,75,182,185,206,215,247,256,272,276,279,291,304,311,318,344,356,362,366,369,389,397,405,409,416,436,439,450,453,457,464,484,495,498,505,516,520,573,577,580,595,651,665],[11,12,13],"p",{},"Lectures 9a and 9b. Bishop's book covers neither DBSCAN nor semi-supervised learning (both became more prominent in the literature after 2006), so this post leans more on what the professor showed, with far less direct citing of the book than usual around here.",[15,16,18],"h2",{"id":17},"where-k-means-falls-flat","Where K-means falls flat",[11,20,21,22,26],{},"The professor generates a classic synthetic dataset, ",[23,24,25],"code",{},"make_moons",": two interleaving arcs, like two crescent moons locked together.",[28,29,34],"pre",{"className":30,"code":31,"language":32,"meta":33,"style":33},"language-python shiki shiki-themes github-light github-dark","from sklearn.datasets import make_moons\nX, y = make_moons(n_samples=1000, noise=0.05, random_state=42)\n","python","",[23,35,36,44],{"__ignoreMap":33},[37,38,41],"span",{"class":39,"line":40},"line",1,[37,42,43],{},"from sklearn.datasets import make_moons\n",[37,45,47],{"class":39,"line":46},2,[37,48,49],{},"X, y = make_moons(n_samples=1000, noise=0.05, random_state=42)\n",[11,51,52,53,58,59,63,64,67],{},"Running ",[54,55,57],"a",{"href":56},"\u002Fen\u002Fplaylists\u002Fpattern-recognition\u002Fkmeans","the K-means I already covered in the previous post"," with K=2 on this data gives a poor result: K-means always cuts space into convex regions (every point goes to its nearest centroid, so the boundary between groups is always a straight line), but the two moons aren't convex, they nest into each other's curve. K-means ends up cutting nearly down the middle, ignoring the arcs' actual shape. Measuring agreement between K-means's grouping and the real moon split (using the ",[60,61,62],"strong",{},"adjusted Rand index",", a metric that equals 1 when the groups perfectly match the real split and sits near 0 when it's about as good as guessing at random): ",[60,65,66],{},"0.24",". Barely any match.",[15,69,71],{"id":70},"dbscan-grouping-by-density-not-distance-to-a-center","DBSCAN: grouping by density, not distance to a center",[11,73,74],{},"The professor describes the algorithm in plain text, no formula:",[76,77,78,161,172,175],"ul",{},[79,80,81,82,128,129,160],"li",{},"For every point, count how many other points sit within a small distance ",[37,83,86,109],{"className":84},[85],"katex",[37,87,90],{"className":88},[89],"katex-mathml",[91,92,94],"math",{"xmlns":93},"http:\u002F\u002Fwww.w3.org\u002F1998\u002FMath\u002FMathML",[95,96,97,104],"semantics",{},[98,99,100],"mrow",{},[101,102,103],"mi",{},"ε",[105,106,108],"annotation",{"encoding":107},"application\u002Fx-tex","\\varepsilon",[37,110,114],{"className":111,"ariaHidden":113},[112],"katex-html","true",[37,115,118,123],{"className":116},[117],"base",[37,119],{"className":120,"style":122},[121],"strut","height:0.4306em;",[37,124,103],{"className":125},[126,127],"mord","mathnormal"," (epsilon) of it. That's the ",[60,130,131,159],{},[37,132,134,147],{"className":133},[85],[37,135,137],{"className":136},[89],[91,138,139],{"xmlns":93},[95,140,141,145],{},[98,142,143],{},[101,144,103],{},[105,146,108],{"encoding":107},[37,148,150],{"className":149,"ariaHidden":113},[112],[37,151,153,156],{"className":152},[117],[37,154],{"className":155,"style":122},[121],[37,157,103],{"className":158},[126,127],"-neighborhood",".",[79,162,163,164,167,168,171],{},"If a point has at least ",[23,165,166],{},"min_samples"," neighbors at that distance (counting itself), it's a ",[60,169,170],{},"core point"," (it lives in a dense region).",[79,173,174],{},"Every point in a core point's neighborhood belongs to the same group. Since that neighborhood can contain other core points, a chain of neighboring core points forms a single group, no matter how long that \"string\" gets.",[79,176,177,178,181],{},"Any point that isn't core and isn't in any core point's neighborhood is ",[60,179,180],{},"noise"," (label -1), it belongs to no group at all.",[11,183,184],{},"The core difference from K-means: DBSCAN never assumes a group has \"a center.\" It just follows dense regions, so it can trace crooked shapes, like two crescent moons.",[28,186,188],{"className":30,"code":187,"language":32,"meta":33,"style":33},"from sklearn.cluster import DBSCAN\ndbscan = DBSCAN(eps=0.05, min_samples=5)\ny_pred = dbscan.fit_predict(X)\n",[23,189,190,195,200],{"__ignoreMap":33},[37,191,192],{"class":39,"line":40},[37,193,194],{},"from sklearn.cluster import DBSCAN\n",[37,196,197],{"class":39,"line":46},[37,198,199],{},"dbscan = DBSCAN(eps=0.05, min_samples=5)\n",[37,201,203],{"class":39,"line":202},3,[37,204,205],{},"y_pred = dbscan.fit_predict(X)\n",[207,208,209],"blockquote",{},[11,210,211,214],{},[60,212,213],{},"Output (eps=0.05):"," 7 groups, 77 points marked as noise.",[11,216,217,218,246],{},"Too small an ",[37,219,221,234],{"className":220},[85],[37,222,224],{"className":223},[89],[91,225,226],{"xmlns":93},[95,227,228,232],{},[98,229,230],{},[101,231,103],{},[105,233,108],{"encoding":107},[37,235,237],{"className":236,"ariaHidden":113},[112],[37,238,240,243],{"className":239},[117],[37,241],{"className":242,"style":122},[121],[37,244,103],{"className":245},[126,127]," fragments the whole dataset into little pieces, because too few points fall inside such a tight neighborhood.",[28,248,250],{"className":30,"code":249,"language":32,"meta":33,"style":33},"dbscan = DBSCAN(eps=0.2, min_samples=5)\n",[23,251,252],{"__ignoreMap":33},[37,253,254],{"class":39,"line":40},[37,255,249],{},[207,257,258],{},[11,259,260,263,264,267,268,271],{},[60,261,262],{},"Output (eps=0.2):"," exactly 2 groups, ",[60,265,266],{},"zero"," noise points, adjusted Rand index of ",[60,269,270],{},"1.0",": a perfect match with the real moon split.",[15,273,275],{"id":274},"interactive-nudging-eps-and-min_samples-live","Interactive: nudging eps and min_samples live",[11,277,278],{},"My own reconstruction of the algorithm (on a sample of 300 points from the same two moons, to run fast in the browser). Move both controls and watch the number of groups and noise points shift:",[280,281],"dbscan-explorer",{":initial-eps":282,":initial-min-samples":283,":points":284,":x-max":285,":x-min":286,":y-max":287,":y-min":288,"x-label":289,"y-label":290},"0.2","5","[[0.6225,-0.3721],[1.9043,-0.1363],[-0.0694,0.4561],[0.9339,0.2375],[1.1804,-0.4908],[1.0254,-0.4625],[0.412,-0.286],[-0.6127,0.7298],[0.9481,0.0796],[0.1712,0.0016],[1.292,-0.3773],[0.3722,0.8922],[0.9163,-0.4881],[0.6263,0.866],[0.1712,-0.0556],[0.9715,-0.5275],[1.9931,0.3036],[0.7204,-0.4162],[-0.9279,0.0214],[0.1161,0.2384],[-0.0621,1.0061],[-0.7831,0.6132],[1.5022,-0.4123],[1.5101,-0.3856],[-0.6721,0.8306],[0.6121,0.7783],[1.8842,0.1128],[2.0745,0.4222],[0.0384,0.2894],[1.4003,-0.3676],[1.0046,0.3922],[0.044,0.2312],[0.8199,0.6096],[0.8525,0.4013],[0.7624,-0.3969],[-0.8232,0.5753],[1.005,0.0664],[-0.2738,1.0104],[0.5843,0.9074],[-0.0347,1.0917],[-0.4168,0.8609],[-0.0435,1.0931],[1.9617,0.1827],[0.2362,0.9484],[-0.1524,0.9336],[0.3173,1.0369],[1.1925,-0.5033],[1.0977,-0.3781],[-0.6756,0.7637],[1.1265,-0.4876],[0.2258,-0.0849],[-1.0893,0.1693],[1.2702,-0.3481],[0.0238,1.0035],[2.0211,0.4524],[1.8792,0.0819],[-0.4126,0.8663],[-0.3918,0.9149],[0.4147,-0.2016],[0.8824,0.5321],[1.126,-0.4437],[0.3486,0.9116],[0.9026,0.0651],[0.2267,-0.1147],[0.1929,0.0119],[0.7861,0.6043],[0.8949,0.4939],[1.5902,-0.24],[0.3683,0.9676],[0.261,-0.2189],[-0.5534,0.814],[-0.9743,0.4438],[1.176,-0.5531],[0.4221,-0.2541],[-0.9474,0.4089],[1.8319,0.2377],[-0.1729,1.0277],[-0.1683,1.0479],[0.1566,0.9942],[2.0012,0.3138],[0.095,0.2055],[-0.0297,0.4838],[0.819,0.4685],[-0.8794,0.5028],[-0.1289,0.9931],[-0.6763,0.7113],[-0.7693,0.6125],[-0.0196,0.9564],[0.9332,0.3339],[0.9813,0.2585],[0.424,-0.2975],[-0.4163,0.8682],[0.115,1.0836],[1.0495,-0.4091],[0.7053,0.7522],[0.871,0.3702],[0.1733,-0.1418],[-0.9475,-0.0484],[1.7686,0.0151],[0.186,1.036],[1.1108,-0.4234],[0.1507,0.9867],[0.7902,0.594],[0.2472,-0.1959],[1.7709,-0.1614],[1.0243,-0.486],[-1.1047,0.1436],[0.9234,0.3677],[0.2191,-0.0929],[-0.3621,0.9146],[-0.9521,0.2709],[1.6424,-0.1766],[-0.1871,1.0478],[1.4341,-0.3949],[1.431,-0.3057],[-0.7618,0.7186],[-0.9895,0.1209],[2.0405,0.3612],[0.4011,-0.35],[0.6942,-0.381],[0.378,0.8995],[0.0191,0.1248],[0.2904,-0.1639],[0.0736,0.0406],[-1.0022,0.276],[1.6145,-0.3083],[0.6951,0.758],[-0.8887,0.4015],[0.417,-0.2926],[1.9998,0.097],[-0.9743,0.2466],[-0.7659,0.7278],[0.1085,0.115],[-0.6781,0.6087],[1.872,0.0288],[1.79,-0.0846],[-0.0662,0.3852],[0.1494,-0.1353],[0.1642,-0.0546],[1.0248,0.1024],[-0.7113,0.626],[-1.0288,0.1551],[0.1303,-0.1004],[-0.7001,0.5585],[0.6677,-0.5632],[1.7231,-0.2821],[-0.8929,0.4958],[-0.0381,1.0039],[0.895,0.4512],[-0.2188,0.9388],[1.7972,-0.0936],[-0.6859,0.7415],[0.9297,-0.517],[0.0251,0.3777],[1.9736,0.1427],[1.3943,-0.3992],[1.066,-0.4479],[0.1769,1.0591],[0.4505,-0.3813],[-0.1498,0.9101],[0.6642,-0.4237],[0.0193,0.3177],[0.9501,0.2384],[0.7169,0.6415],[0.8116,0.536],[-1.0306,0.1198],[1.8253,0.2232],[1.6022,-0.317],[0.0995,0.1044],[0.4393,0.8531],[-0.3925,0.9184],[0.7612,-0.3591],[1.5035,-0.4277],[0.6208,0.7106],[-0.287,0.8166],[1.3767,-0.432],[-1.0389,0.3106],[0.7965,0.5345],[2.1312,0.2933],[-0.9932,0.1475],[0.476,-0.3315],[1.0553,0.0386],[1.7577,0.0136],[1.7538,-0.0907],[-0.5036,0.8634],[1.6893,-0.1679],[1.5937,-0.2946],[0.0667,0.101],[-0.4914,0.8638],[0.7527,0.74],[0.1704,0.9778],[-0.0135,0.5474],[1.2276,-0.4057],[0.5094,-0.5011],[1.4132,-0.4749],[0.8942,0.2307],[1.986,0.4779],[1.8255,-0.1199],[0.798,0.5534],[0.8926,-0.4648],[0.9527,0.0642],[-0.8168,0.6014],[0.8292,0.6288],[0.5567,-0.3878],[-0.0227,0.2088],[0.7545,-0.427],[1.9394,0.2505],[-0.9089,0.2076],[0.2391,0.9378],[-0.9086,0.3396],[-0.8343,0.4789],[1.9004,0.3254],[0.9329,0.1422],[0.1033,1.0373],[1.6953,-0.289],[0.2286,1.1078],[0.5895,0.8026],[2.0015,0.1202],[0.0188,0.3959],[0.6775,0.6899],[-0.9205,0.6155],[1.8839,0.0203],[0.8172,-0.4449],[0.1289,0.463],[0.8553,0.4595],[1.7494,-0.2473],[-0.6379,0.6167],[-0.1481,1.022],[0.7028,-0.4736],[0.6643,-0.4785],[0.6243,0.7253],[1.3207,-0.513],[2.026,0.3243],[1.7031,-0.353],[0.437,-0.3331],[0.9168,0.2827],[0.5479,0.8445],[-0.4335,0.9241],[1.6864,-0.1545],[1.0469,-0.4909],[1.982,-0.0765],[0.594,-0.5127],[-0.0771,0.2081],[0.5575,0.7976],[0.857,-0.5003],[-1.0039,0.1106],[0.3767,-0.3132],[0.8872,-0.499],[1.5544,-0.3866],[0.9935,0.0647],[-0.6349,0.7638],[0.4429,0.8925],[-0.9186,0.2783],[0.9822,0.0096],[-0.8881,0.3235],[-0.5537,0.8375],[0.5944,0.8086],[-0.1486,0.9464],[0.5168,-0.3284],[0.6918,0.6162],[-0.2751,0.8524],[0.4946,0.8189],[1.4033,-0.5486],[0.875,0.2434],[-0.2322,0.9582],[0.0396,0.2363],[2.1022,0.4284],[-0.9815,0.2901],[1.9037,0.2547],[1.2988,-0.3973],[0.5192,0.9439],[0.4704,0.8959],[0.2064,0.0938],[0.2693,-0.2072],[1.9919,0.5192],[0.3084,-0.3219],[0.0618,0.12],[0.2692,0.9691],[0.4539,0.808],[1.2866,-0.5198],[1.5766,-0.4242],[1.0076,0.4132],[0.0269,0.2997],[1.9151,-0.0421],[0.3507,-0.2937],[1.5484,-0.2249],[-1.0176,0.0121],[-0.9118,0.3732],[0.099,0.4766],[-0.5718,0.8495],[-0.2355,0.9921],[0.726,-0.4291],[-0.9189,0.6139],[1.934,0.4887],[0.9656,0.4153],[0.345,-0.1873],[0.138,1.0504],[-0.6391,0.8191],[1.7425,-0.077],[-0.507,0.8433]]","2.3","-1.3","1.2","-0.7","x1","x2",[11,292,293,294,297,298,300,301,303],{},"With a tiny ",[23,295,296],{},"eps",", almost everyone turns into noise (gray). Raising ",[23,299,296],{}," bit by bit, groups start forming, until at some point the two moons show up whole. Push past the right point and the two moons merge into one. ",[23,302,166],{}," works similarly: higher values demand a more crowded neighborhood before someone can become a core point, so groups get pickier (more noise, more \"solid\" groups).",[15,305,307,308],{"id":306},"the-problem-of-having-no-predict","The problem of having no ",[23,309,310],{},".predict()",[11,312,313,314,317],{},"DBSCAN keeps no centroid at all, so there's no way to ask \"which group does this new point belong to?\" the way K-means does. The professor's fix: train a separate KNN, using only DBSCAN's core points (",[23,315,316],{},"dbscan.components_",") and each one's group as the label.",[28,319,321],{"className":30,"code":320,"language":32,"meta":33,"style":33},"from sklearn.neighbors import KNeighborsClassifier\nknn = KNeighborsClassifier(n_neighbors=50)\nknn.fit(dbscan.components_, dbscan.labels_[dbscan.core_sample_indices_])\nprint(knn.predict(X_new))\n",[23,322,323,328,333,338],{"__ignoreMap":33},[37,324,325],{"class":39,"line":40},[37,326,327],{},"from sklearn.neighbors import KNeighborsClassifier\n",[37,329,330],{"class":39,"line":46},[37,331,332],{},"knn = KNeighborsClassifier(n_neighbors=50)\n",[37,334,335],{"class":39,"line":202},[37,336,337],{},"knn.fit(dbscan.components_, dbscan.labels_[dbscan.core_sample_indices_])\n",[37,339,341],{"class":39,"line":340},4,[37,342,343],{},"print(knn.predict(X_new))\n",[207,345,346],{},[11,347,348,351,352,355],{},[60,349,350],{},"Output:"," ",[23,353,354],{},"[1, 0, 1, 0]"," for the 4 new points tested.",[11,357,358,359,361],{},"But that has a catch: KNN always finds a nearest neighbor, no matter the distance, so it'll \"predict\" a group for any point, even one far from everything, that should really be noise. The fine-tuning: check the distance to the nearest neighbor and, if it's too large (the professor uses ",[23,360,296],{}," itself as the cutoff), mark it as noise (-1) instead of forcing it into a group.",[15,363,365],{"id":364},"switching-topics-what-if-labels-are-expensive","Switching topics: what if labels are expensive?",[11,367,368],{},"The second half of the lecture switches problems, but reuses the idea of grouping without labels. Scenario: 1400 handwritten digit images for training, but labeling each one is expensive manual work, so only 50 can be labeled.",[28,370,372],{"className":30,"code":371,"language":32,"meta":33,"style":33},"n_labeled = 50\nlog_reg = LogisticRegression(solver=\"liblinear\", max_iter=5000)\nlog_reg.fit(X_train[:n_labeled], y_train[:n_labeled])\n",[23,373,374,379,384],{"__ignoreMap":33},[37,375,376],{"class":39,"line":40},[37,377,378],{},"n_labeled = 50\n",[37,380,381],{"class":39,"line":46},[37,382,383],{},"log_reg = LogisticRegression(solver=\"liblinear\", max_iter=5000)\n",[37,385,386],{"class":39,"line":202},[37,387,388],{},"log_reg.fit(X_train[:n_labeled], y_train[:n_labeled])\n",[207,390,391],{},[11,392,393,396],{},[60,394,395],{},"Output (labeling the first 50 images, in whatever order they came):"," 0.766 test accuracy.",[207,398,399],{},[11,400,401,404],{},[60,402,403],{},"Output (if I had labels for all 1400, the theoretical ceiling):"," 0.902.",[15,406,408],{"id":407},"the-trick-choosing-which-50-to-label-not-which-come-first","The trick: choosing which 50 to label, not which come first",[11,410,411,412,415],{},"Instead of labeling the first 50 (an arbitrary order), the professor uses K-means with ",[60,413,414],{},"K=50"," (the same label budget!) to find 50 groups in the 1400 unlabeled images, and picks, from each group, the image closest to the center (the most \"typical\" one in that group):",[28,417,419],{"className":30,"code":418,"language":32,"meta":33,"style":33},"kmeans = KMeans(n_clusters=50)\nX_digits_dist = kmeans.fit_transform(X_train)\nrepresentative_digit_idx = np.argmin(X_digits_dist, axis=0)\n",[23,420,421,426,431],{"__ignoreMap":33},[37,422,423],{"class":39,"line":40},[37,424,425],{},"kmeans = KMeans(n_clusters=50)\n",[37,427,428],{"class":39,"line":46},[37,429,430],{},"X_digits_dist = kmeans.fit_transform(X_train)\n",[37,432,433],{"class":39,"line":202},[37,434,435],{},"representative_digit_idx = np.argmin(X_digits_dist, axis=0)\n",[11,437,438],{},"Only those 50 representative images get manually labeled (the professor pasted in the correct labels by hand, cell 8 of the notebook). Training on just those 50:",[207,440,441],{},[11,442,443,445,446,449],{},[60,444,350],{}," 0.834. Better than the 50 random labels (0.766), with the ",[60,447,448],{},"same"," number of labels.",[11,451,452],{},"That tracks: 50 images chosen to represent 50 different groups cover more variety than 50 images in whatever order they arrived (which might repeat the same \"7\" style several times and never show a crooked \"3\").",[15,454,456],{"id":455},"propagation-spreading-the-label-to-the-whole-group","Propagation: spreading the label to the whole group",[11,458,459,460,463],{},"If group 12's representative image is a \"7,\" it's reasonable to assume ",[60,461,462],{},"everyone"," in group 12 is also a \"7\" (that's why they landed in the same group, after all). Propagating the representative's label to every member of the cluster:",[28,465,467],{"className":30,"code":466,"language":32,"meta":33,"style":33},"y_train_propagated = np.empty(len(X_train), dtype=np.int32)\nfor i in range(k):\n    y_train_propagated[kmeans.labels_==i] = y_representative_digits[i]\n",[23,468,469,474,479],{"__ignoreMap":33},[37,470,471],{"class":39,"line":40},[37,472,473],{},"y_train_propagated = np.empty(len(X_train), dtype=np.int32)\n",[37,475,476],{"class":39,"line":46},[37,477,478],{},"for i in range(k):\n",[37,480,481],{"class":39,"line":202},[37,482,483],{},"    y_train_propagated[kmeans.labels_==i] = y_representative_digits[i]\n",[207,485,486],{},[11,487,488,490,491,494],{},[60,489,350],{}," training on all 1400 propagated labels (but only 50 truly hand-checked): 0.869. And checking against the real label (something only possible here because it's an exercise, in real life you wouldn't know it): the propagation is right ",[60,492,493],{},"95.4%"," of the time.",[11,496,497],{},"One more refinement: dropping, from each group, the 20% farthest from the center (the \"border\" points, more likely to have been assigned to the wrong group) before propagating:",[207,499,500],{},[11,501,502,504],{},[60,503,350],{}," 1111 examples left (out of 1400), with 97.7% propagation accuracy (up from 95.4%). Training on just those: 0.879, the lecture's best result using only 50 truly labeled examples.",[11,506,507,508,511,512,515],{},"And the pattern repeats swapping ",[23,509,510],{},"LogisticRegression"," for KNN, Random Forest, Gaussian Naive Bayes, and ",[23,513,514],{},"NearestCentroid",": in every classifier tested, \"representative\" beats \"50 random,\" and \"propagated\" beats \"representative alone.\"",[15,517,519],{"id":518},"wrapping-up","Wrapping up",[521,522,523,537],"table",{},[524,525,526],"thead",{},[527,528,529,534],"tr",{},[530,531,533],"th",{"align":532},"left","What I already knew",[530,535,536],{"align":532},"What this lecture settled",[538,539,540,549,565],"tbody",{},[527,541,542,546],{},[543,544,545],"td",{"align":532},"K-means groups by distance to a centroid",[543,547,548],{"align":532},"DBSCAN groups by density, with no assumption about a group's shape, so it handles non-convex shapes well",[527,550,551,554],{},[543,552,553],{"align":532},"Clustering is for exploring data with no label",[543,555,556,557,560,561,564],{"align":532},"Clustering also helps ",[60,558,559],{},"decide what to label"," when labeling is expensive, and helps ",[60,562,563],{},"extend"," a few labels to the rest of the data",[527,566,567,570],{},[543,568,569],{"align":532},"More labeled data is always better",[543,571,572],{"align":532},"Sometimes 50 well-chosen labels (via clustering) are worth more than 50 labels in whatever order they arrived",[15,574,576],{"id":575},"practical-application","Practical application",[11,578,579],{},"I reproduced the full chain (50 random → representative → propagated) on the same digits dataset, with a fixed seed, to confirm the gain wasn't a coincidence from one specific run.",[28,581,583],{"className":30,"code":582,"language":32,"meta":33,"style":33},"log_reg = LogisticRegression(solver=\"lbfgs\", max_iter=5000, random_state=42)\n# ... same procedure as the professor, with random_state fixed at every step\n",[23,584,585,590],{"__ignoreMap":33},[37,586,587],{"class":39,"line":40},[37,588,589],{},"log_reg = LogisticRegression(solver=\"lbfgs\", max_iter=5000, random_state=42)\n",[37,591,592],{"class":39,"line":46},[37,593,594],{},"# ... same procedure as the professor, with random_state fixed at every step\n",[521,596,597,609],{},[524,598,599],{},[527,600,601,605],{},[530,602,604],{"align":603},"center","Labeling strategy",[530,606,608],{"align":607},"right","Test accuracy",[538,610,611,619,627,635,643],{},[527,612,613,616],{},[543,614,615],{"align":603},"50 random labels (the first 50 images)",[543,617,618],{"align":607},"0.7582",[527,620,621,624],{},[543,622,623],{"align":603},"50 representative labels (1 per K-means group)",[543,625,626],{"align":607},"0.8388",[527,628,629,632],{},[543,630,631],{"align":603},"Labels propagated to the whole group",[543,633,634],{"align":607},"0.8589",[527,636,637,640],{},[543,638,639],{"align":603},"Propagation accuracy (against the real label)",[543,641,642],{"align":607},"0.9500",[527,644,645,648],{},[543,646,647],{"align":603},"All 1400 labels (theoretical ceiling)",[543,649,650],{"align":607},"0.9093",[11,652,653,654,657,658,661,662,664],{},"My numbers land close to the original notebook's (0.766 \u002F 0.834 \u002F 0.869 \u002F 0.954 \u002F 0.902), the difference sits only in the random seed and the logistic regression solver (I swapped ",[23,655,656],{},"liblinear"," for ",[23,659,660],{},"lbfgs",", because the newer scikit-learn version in my environment no longer accepts ",[23,663,656],{}," for problems with more than two classes). The order between strategies is identical in both runs: random loses to representative, which loses to propagated, confirming the gain is real, not luck from one specific run.",[666,667,668],"style",{},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":33,"searchDepth":46,"depth":46,"links":670},[671,672,673,674,676,677,678,679,680],{"id":17,"depth":46,"text":18},{"id":70,"depth":46,"text":71},{"id":274,"depth":46,"text":275},{"id":306,"depth":46,"text":675},"The problem of having no .predict()",{"id":364,"depth":46,"text":365},{"id":407,"depth":46,"text":408},{"id":455,"depth":46,"text":456},{"id":518,"depth":46,"text":519},{"id":575,"depth":46,"text":576},null,"2026-08-20","Lecture 9: the professor shows where K-means fails badly, introduces DBSCAN as a density-based alternative, then uses clustering to pick which examples to label better when labels are expensive.","md",{},true,10,"\u002Fen\u002Fplaylists\u002Fpattern-recognition\u002Fdbscan-semi-supervised","pattern-recognition",{"title":6,"description":683},"published","en\u002Fplaylists\u002Fpattern-recognition\u002Fdbscan-semi-supervised",[694,695,696],"dbscan","semi-supervised-learning","clustering","goon5DO08D62TkjwXY8CVWM2LgEXpQ1WRcCXq-TaghM",1787338984462]