[{"data":1,"prerenderedAt":1675},["ShallowReactive",2],{"lang-switch-post-\u002Fplaylists\u002Fpattern-recognition\u002Fkmeans":3,"post-pt-pattern-recognition-kmeans":4},"\u002Fen\u002Fplaylists\u002Fpattern-recognition\u002Fkmeans",{"id":5,"title":6,"body":7,"cover":1660,"date":1661,"description":1662,"extension":1663,"meta":1664,"navigation":56,"order":95,"path":1665,"playlist":1666,"seo":1667,"status":1668,"stem":1669,"tags":1670,"__hash__":1674},"posts\u002Fpt\u002Fplaylists\u002Fpattern-recognition\u002Fkmeans.md","K-means: Quando Agrupar Vira um Truque de Engenharia de Features",{"type":8,"value":9,"toc":1650},"minimark",[10,19,24,30,34,176,195,202,218,225,726,1133,1137,1175,1190,1193,1197,1233,1248,1257,1269,1273,1280,1314,1326,1355,1363,1391,1394,1409,1412,1416,1422,1479,1482,1486,1526,1530,1540,1570,1640,1646],[11,12,13,14,18],"p",{},"Aula 8a, 8b e 8c, e a matéria muda de regime pela primeira vez: até aqui, todo modelo aprendia a prever um rótulo que já vinha no dado (",[15,16,17],"code",{},"y","). K-means não recebe rótulo nenhum, só encontra grupos.",[20,21,23],"h2",{"id":22},"o-problema-separar-sem-saber-a-resposta","O problema: separar sem saber a resposta",[11,25,26,27,29],{},"O professor volta pro Iris, mas dessa vez esconde a espécie (",[15,28,17],{},") e passa só as duas variáveis (comprimento e largura da pétala) pro algoritmo. A pergunta muda de \"que espécie é essa flor?\" pra \"quantos grupos naturais existem aqui, e onde ficam?\".",[20,31,33],{"id":32},"k-means-na-unha","K-means na unha",[35,36,41],"pre",{"className":37,"code":38,"language":39,"meta":40,"style":40},"language-python shiki shiki-themes github-light github-dark","from sklearn.base import BaseEstimator, ClusterMixin, TransformerMixin\n\nclass KMeans(BaseEstimator, ClusterMixin, TransformerMixin):\n    def __init__(self, n_clusters=3, max_iter=100):\n        self.n_clusters = n_clusters\n        self.max_iter = max_iter\n\n    def fit(self, X, y=None):\n        self.centroids = X[random.sample(range(len(X)), self.n_clusters)]\n        max_iter = self.max_iter\n        while max_iter > 0:\n            max_iter -= 1\n            y_pred = self.predict(X)\n            for i in range(self.n_clusters):\n                self.centroids[i] = np.mean(X[y_pred==i], axis=0)\n            if np.allclose(self.centroids, self.previous_centroids[-1], atol=1e-9):\n                break\n        return self\n\n    def predict(self, X):\n        # atribui cada ponto ao centroide mais próximo\n        ...\n","python","",[15,42,43,51,58,64,70,76,82,87,93,99,105,111,117,123,129,135,141,147,153,158,164,170],{"__ignoreMap":40},[44,45,48],"span",{"class":46,"line":47},"line",1,[44,49,50],{},"from sklearn.base import BaseEstimator, ClusterMixin, TransformerMixin\n",[44,52,54],{"class":46,"line":53},2,[44,55,57],{"emptyLinePlaceholder":56},true,"\n",[44,59,61],{"class":46,"line":60},3,[44,62,63],{},"class KMeans(BaseEstimator, ClusterMixin, TransformerMixin):\n",[44,65,67],{"class":46,"line":66},4,[44,68,69],{},"    def __init__(self, n_clusters=3, max_iter=100):\n",[44,71,73],{"class":46,"line":72},5,[44,74,75],{},"        self.n_clusters = n_clusters\n",[44,77,79],{"class":46,"line":78},6,[44,80,81],{},"        self.max_iter = max_iter\n",[44,83,85],{"class":46,"line":84},7,[44,86,57],{"emptyLinePlaceholder":56},[44,88,90],{"class":46,"line":89},8,[44,91,92],{},"    def fit(self, X, y=None):\n",[44,94,96],{"class":46,"line":95},9,[44,97,98],{},"        self.centroids = X[random.sample(range(len(X)), self.n_clusters)]\n",[44,100,102],{"class":46,"line":101},10,[44,103,104],{},"        max_iter = self.max_iter\n",[44,106,108],{"class":46,"line":107},11,[44,109,110],{},"        while max_iter > 0:\n",[44,112,114],{"class":46,"line":113},12,[44,115,116],{},"            max_iter -= 1\n",[44,118,120],{"class":46,"line":119},13,[44,121,122],{},"            y_pred = self.predict(X)\n",[44,124,126],{"class":46,"line":125},14,[44,127,128],{},"            for i in range(self.n_clusters):\n",[44,130,132],{"class":46,"line":131},15,[44,133,134],{},"                self.centroids[i] = np.mean(X[y_pred==i], axis=0)\n",[44,136,138],{"class":46,"line":137},16,[44,139,140],{},"            if np.allclose(self.centroids, self.previous_centroids[-1], atol=1e-9):\n",[44,142,144],{"class":46,"line":143},17,[44,145,146],{},"                break\n",[44,148,150],{"class":46,"line":149},18,[44,151,152],{},"        return self\n",[44,154,156],{"class":46,"line":155},19,[44,157,57],{"emptyLinePlaceholder":56},[44,159,161],{"class":46,"line":160},20,[44,162,163],{},"    def predict(self, X):\n",[44,165,167],{"class":46,"line":166},21,[44,168,169],{},"        # atribui cada ponto ao centroide mais próximo\n",[44,171,173],{"class":46,"line":172},22,[44,174,175],{},"        ...\n",[11,177,178,179,182,183,182,187,190,191,194],{},"Repara na assinatura da classe: ",[15,180,181],{},"ClusterMixin"," ",[184,185,186],"strong",{},"e",[15,188,189],{},"TransformerMixin"," juntos, a primeira vez que uma classe dessa playlist herda de duas coisas ao mesmo tempo além de ",[15,192,193],{},"BaseEstimator",". Isso não é acaso, e o motivo vai ficar claro mais adiante no post.",[11,196,197,198,201],{},"O algoritmo (chamado ",[184,199,200],{},"K-means de Lloyd",", o mais comum) é só dois passos repetidos até parar de mudar:",[203,204,205,212],"ol",{},[206,207,208,211],"li",{},[184,209,210],{},"Atribuir",": cada ponto vai pro centroide mais próximo (distância euclidiana).",[206,213,214,217],{},[184,215,216],{},"Recalcular",": cada centroide vira a média dos pontos que foram atribuídos a ele.",[11,219,220,221,224],{},"O Bishop formaliza os dois passos como a minimização de uma ",[184,222,223],{},"medida de distorção",",",[11,226,227],{},[44,228,231,337],{"className":229},[230],"katex",[44,232,235],{"className":233},[234],"katex-mathml",[236,237,239],"math",{"xmlns":238},"http:\u002F\u002Fwww.w3.org\u002F1998\u002FMath\u002FMathML",[240,241,242,332],"semantics",{},[243,244,245,249,253,273,289,301,305,313,316,324],"mrow",{},[246,247,248],"mi",{},"J",[250,251,252],"mo",{},"=",[254,255,256,259,270],"msubsup",{},[250,257,258],{},"∑",[243,260,261,264,266],{},[246,262,263],{},"n",[250,265,252],{},[267,268,269],"mn",{},"1",[246,271,272],{},"N",[254,274,275,277,286],{},[250,276,258],{},[243,278,279,282,284],{},[246,280,281],{},"k",[250,283,252],{},[267,285,269],{},[246,287,288],{},"K",[290,291,292,295],"msub",{},[246,293,294],{},"r",[243,296,297,299],{},[246,298,263],{},[246,300,281],{},[246,302,304],{"mathvariant":303},"normal","∥",[290,306,307,311],{},[246,308,310],{"mathvariant":309},"bold","x",[246,312,263],{},[250,314,315],{},"−",[290,317,318,322],{},[246,319,321],{"mathvariant":320},"bold-italic","μ",[246,323,281],{},[325,326,327,329],"msup",{},[246,328,304],{"mathvariant":303},[267,330,331],{},"2",[333,334,336],"annotation",{"encoding":335},"application\u002Fx-tex","J = \\sum_{n=1}^{N}\\sum_{k=1}^{K} r_{nk}\\|\\mathbf{x}_n - \\boldsymbol{\\mu}_k\\|^2",[44,338,342,369,638],{"className":339,"ariaHidden":341},[340],"katex-html","true",[44,343,346,351,357,362,366],{"className":344},[345],"base",[44,347],{"className":348,"style":350},[349],"strut","height:0.6833em;",[44,352,248],{"className":353,"style":356},[354,355],"mord","mathnormal","margin-right:0.0962em;",[44,358],{"className":359,"style":361},[360],"mspace","margin-right:0.2778em;",[44,363,252],{"className":364},[365],"mrel",[44,367],{"className":368,"style":361},[360],[44,370,372,376,461,465,530,533,581,584,627,631,635],{"className":371},[345],[44,373],{"className":374,"style":375},[349],"height:1.2809em;vertical-align:-0.2997em;",[44,377,380,386],{"className":378},[379],"mop",[44,381,258],{"className":382,"style":385},[379,383,384],"op-symbol","small-op","position:relative;top:0em;",[44,387,390],{"className":388},[389],"msupsub",[44,391,395,452],{"className":392},[393,394],"vlist-t","vlist-t2",[44,396,399,447],{"className":397},[398],"vlist-r",[44,400,404,431],{"className":401,"style":403},[402],"vlist","height:0.9812em;",[44,405,407,412],{"style":406},"top:-2.4003em;margin-left:0em;margin-right:0.05em;",[44,408],{"className":409,"style":411},[410],"pstrut","height:2.7em;",[44,413,419],{"className":414},[415,416,417,418],"sizing","reset-size6","size3","mtight",[44,420,422,425,428],{"className":421},[354,418],[44,423,263],{"className":424},[354,355,418],[44,426,252],{"className":427},[365,418],[44,429,269],{"className":430},[354,418],[44,432,434,437],{"style":433},"top:-3.2029em;margin-right:0.05em;",[44,435],{"className":436,"style":411},[410],[44,438,440],{"className":439},[415,416,417,418],[44,441,443],{"className":442},[354,418],[44,444,272],{"className":445,"style":446},[354,355,418],"margin-right:0.109em;",[44,448,451],{"className":449},[450],"vlist-s","​",[44,453,455],{"className":454},[398],[44,456,459],{"className":457,"style":458},[402],"height:0.2997em;",[44,460],{},[44,462],{"className":463,"style":464},[360],"margin-right:0.1667em;",[44,466,468,471],{"className":467},[379],[44,469,258],{"className":470,"style":385},[379,383,384],[44,472,474],{"className":473},[389],[44,475,477,522],{"className":476},[393,394],[44,478,480,519],{"className":479},[398],[44,481,483,504],{"className":482,"style":403},[402],[44,484,485,488],{"style":406},[44,486],{"className":487,"style":411},[410],[44,489,491],{"className":490},[415,416,417,418],[44,492,494,498,501],{"className":493},[354,418],[44,495,281],{"className":496,"style":497},[354,355,418],"margin-right:0.0315em;",[44,499,252],{"className":500},[365,418],[44,502,269],{"className":503},[354,418],[44,505,506,509],{"style":433},[44,507],{"className":508,"style":411},[410],[44,510,512],{"className":511},[415,416,417,418],[44,513,515],{"className":514},[354,418],[44,516,288],{"className":517,"style":518},[354,355,418],"margin-right:0.0715em;",[44,520,451],{"className":521},[450],[44,523,525],{"className":524},[398],[44,526,528],{"className":527,"style":458},[402],[44,529],{},[44,531],{"className":532,"style":464},[360],[44,534,536,540],{"className":535},[354],[44,537,294],{"className":538,"style":539},[354,355],"margin-right:0.0278em;",[44,541,543],{"className":542},[389],[44,544,546,572],{"className":545},[393,394],[44,547,549,569],{"className":548},[398],[44,550,553],{"className":551,"style":552},[402],"height:0.3361em;",[44,554,556,559],{"style":555},"top:-2.55em;margin-left:-0.0278em;margin-right:0.05em;",[44,557],{"className":558,"style":411},[410],[44,560,562],{"className":561},[415,416,417,418],[44,563,565],{"className":564},[354,418],[44,566,568],{"className":567,"style":497},[354,355,418],"nk",[44,570,451],{"className":571},[450],[44,573,575],{"className":574},[398],[44,576,579],{"className":577,"style":578},[402],"height:0.15em;",[44,580],{},[44,582,304],{"className":583},[354],[44,585,587,591],{"className":586},[354],[44,588,310],{"className":589},[354,590],"mathbf",[44,592,594],{"className":593},[389],[44,595,597,619],{"className":596},[393,394],[44,598,600,616],{"className":599},[398],[44,601,604],{"className":602,"style":603},[402],"height:0.1514em;",[44,605,607,610],{"style":606},"top:-2.55em;margin-left:0em;margin-right:0.05em;",[44,608],{"className":609,"style":411},[410],[44,611,613],{"className":612},[415,416,417,418],[44,614,263],{"className":615},[354,355,418],[44,617,451],{"className":618},[450],[44,620,622],{"className":621},[398],[44,623,625],{"className":624,"style":578},[402],[44,626],{},[44,628],{"className":629,"style":630},[360],"margin-right:0.2222em;",[44,632,315],{"className":633},[634],"mbin",[44,636],{"className":637,"style":630},[360],[44,639,641,645,695],{"className":640},[345],[44,642],{"className":643,"style":644},[349],"height:1.0641em;vertical-align:-0.25em;",[44,646,648,658],{"className":647},[354],[44,649,651],{"className":650},[354],[44,652,654],{"className":653},[354],[44,655,321],{"className":656},[354,657],"boldsymbol",[44,659,661],{"className":660},[389],[44,662,664,686],{"className":663},[393,394],[44,665,667,683],{"className":666},[398],[44,668,671],{"className":669,"style":670},[402],"height:0.242em;",[44,672,674,677],{"style":673},"top:-2.4559em;margin-right:0.05em;",[44,675],{"className":676,"style":411},[410],[44,678,680],{"className":679},[415,416,417,418],[44,681,281],{"className":682,"style":497},[354,355,418],[44,684,451],{"className":685},[450],[44,687,689],{"className":688},[398],[44,690,693],{"className":691,"style":692},[402],"height:0.2441em;",[44,694],{},[44,696,698,701],{"className":697},[354],[44,699,304],{"className":700},[354],[44,702,704],{"className":703},[389],[44,705,707],{"className":706},[393],[44,708,710],{"className":709},[398],[44,711,714],{"className":712,"style":713},[402],"height:0.8141em;",[44,715,717,720],{"style":716},"top:-3.063em;margin-right:0.05em;",[44,718],{"className":719,"style":411},[410],[44,721,723],{"className":722},[415,416,417,418],[44,724,331],{"className":725},[354,418],[11,727,728,729,830,831,860,861,890,891,968,969,1045,1046,1074,1075,1103,1104,1132],{},"onde ",[44,730,732,758],{"className":731},[230],[44,733,735],{"className":734},[234],[236,736,737],{"xmlns":238},[240,738,739,755],{},[243,740,741,751,753],{},[290,742,743,745],{},[246,744,294],{},[243,746,747,749],{},[246,748,263],{},[246,750,281],{},[250,752,252],{},[267,754,269],{},[333,756,757],{"encoding":335},"r_{nk}=1",[44,759,761,820],{"className":760,"ariaHidden":341},[340],[44,762,764,768,811,814,817],{"className":763},[345],[44,765],{"className":766,"style":767},[349],"height:0.5806em;vertical-align:-0.15em;",[44,769,771,774],{"className":770},[354],[44,772,294],{"className":773,"style":539},[354,355],[44,775,777],{"className":776},[389],[44,778,780,803],{"className":779},[393,394],[44,781,783,800],{"className":782},[398],[44,784,786],{"className":785,"style":552},[402],[44,787,788,791],{"style":555},[44,789],{"className":790,"style":411},[410],[44,792,794],{"className":793},[415,416,417,418],[44,795,797],{"className":796},[354,418],[44,798,568],{"className":799,"style":497},[354,355,418],[44,801,451],{"className":802},[450],[44,804,806],{"className":805},[398],[44,807,809],{"className":808,"style":578},[402],[44,810],{},[44,812],{"className":813,"style":361},[360],[44,815,252],{"className":816},[365],[44,818],{"className":819,"style":361},[360],[44,821,823,827],{"className":822},[345],[44,824],{"className":825,"style":826},[349],"height:0.6444em;",[44,828,269],{"className":829},[354]," se o ponto ",[44,832,834,847],{"className":833},[230],[44,835,837],{"className":836},[234],[236,838,839],{"xmlns":238},[240,840,841,845],{},[243,842,843],{},[246,844,263],{},[333,846,263],{"encoding":335},[44,848,850],{"className":849,"ariaHidden":341},[340],[44,851,853,857],{"className":852},[345],[44,854],{"className":855,"style":856},[349],"height:0.4306em;",[44,858,263],{"className":859},[354,355]," pertence ao grupo ",[44,862,864,877],{"className":863},[230],[44,865,867],{"className":866},[234],[236,868,869],{"xmlns":238},[240,870,871,875],{},[243,872,873],{},[246,874,281],{},[333,876,281],{"encoding":335},[44,878,880],{"className":879,"ariaHidden":341},[340],[44,881,883,887],{"className":882},[345],[44,884],{"className":885,"style":886},[349],"height:0.6944em;",[44,888,281],{"className":889,"style":497},[354,355]," (e 0 caso contrário). Fixando os centroides, o ",[44,892,894,916],{"className":893},[230],[44,895,897],{"className":896},[234],[236,898,899],{"xmlns":238},[240,900,901,913],{},[243,902,903],{},[290,904,905,907],{},[246,906,294],{},[243,908,909,911],{},[246,910,263],{},[246,912,281],{},[333,914,915],{"encoding":335},"r_{nk}",[44,917,919],{"className":918,"ariaHidden":341},[340],[44,920,922,925],{"className":921},[345],[44,923],{"className":924,"style":767},[349],[44,926,928,931],{"className":927},[354],[44,929,294],{"className":930,"style":539},[354,355],[44,932,934],{"className":933},[389],[44,935,937,960],{"className":936},[393,394],[44,938,940,957],{"className":939},[398],[44,941,943],{"className":942,"style":552},[402],[44,944,945,948],{"style":555},[44,946],{"className":947,"style":411},[410],[44,949,951],{"className":950},[415,416,417,418],[44,952,954],{"className":953},[354,418],[44,955,568],{"className":956,"style":497},[354,355,418],[44,958,451],{"className":959},[450],[44,961,963],{"className":962},[398],[44,964,966],{"className":965,"style":578},[402],[44,967],{}," ótimo é óbvio (atribui pro mais próximo, é exatamente o passo 1). Fixando os ",[44,970,972,993],{"className":971},[230],[44,973,975],{"className":974},[234],[236,976,977],{"xmlns":238},[240,978,979,991],{},[243,980,981],{},[290,982,983,985],{},[246,984,294],{},[243,986,987,989],{},[246,988,263],{},[246,990,281],{},[333,992,915],{"encoding":335},[44,994,996],{"className":995,"ariaHidden":341},[340],[44,997,999,1002],{"className":998},[345],[44,1000],{"className":1001,"style":767},[349],[44,1003,1005,1008],{"className":1004},[354],[44,1006,294],{"className":1007,"style":539},[354,355],[44,1009,1011],{"className":1010},[389],[44,1012,1014,1037],{"className":1013},[393,394],[44,1015,1017,1034],{"className":1016},[398],[44,1018,1020],{"className":1019,"style":552},[402],[44,1021,1022,1025],{"style":555},[44,1023],{"className":1024,"style":411},[410],[44,1026,1028],{"className":1027},[415,416,417,418],[44,1029,1031],{"className":1030},[354,418],[44,1032,568],{"className":1033,"style":497},[354,355,418],[44,1035,451],{"className":1036},[450],[44,1038,1040],{"className":1039},[398],[44,1041,1043],{"className":1042,"style":578},[402],[44,1044],{},", o centroide ótimo é a média dos pontos atribuídos a ele, porque é ali que a derivada de ",[44,1047,1049,1062],{"className":1048},[230],[44,1050,1052],{"className":1051},[234],[236,1053,1054],{"xmlns":238},[240,1055,1056,1060],{},[243,1057,1058],{},[246,1059,248],{},[333,1061,248],{"encoding":335},[44,1063,1065],{"className":1064,"ariaHidden":341},[340],[44,1066,1068,1071],{"className":1067},[345],[44,1069],{"className":1070,"style":350},[349],[44,1072,248],{"className":1073,"style":356},[354,355]," zera (é exatamente o passo 2). Cada passo só pode diminuir ",[44,1076,1078,1091],{"className":1077},[230],[44,1079,1081],{"className":1080},[234],[236,1082,1083],{"xmlns":238},[240,1084,1085,1089],{},[243,1086,1087],{},[246,1088,248],{},[333,1090,248],{"encoding":335},[44,1092,1094],{"className":1093,"ariaHidden":341},[340],[44,1095,1097,1100],{"className":1096},[345],[44,1098],{"className":1099,"style":350},[349],[44,1101,248],{"className":1102,"style":356},[354,355]," ou empatar, nunca aumentar, então o algoritmo sempre converge, só que pode parar num mínimo local, não necessariamente o melhor agrupamento possível (por isso, na prática, o scikit-learn roda o K-means várias vezes com centroides iniciais diferentes e fica com o resultado de menor ",[44,1105,1107,1120],{"className":1106},[230],[44,1108,1110],{"className":1109},[234],[236,1111,1112],{"xmlns":238},[240,1113,1114,1118],{},[243,1115,1116],{},[246,1117,248],{},[333,1119,248],{"encoding":335},[44,1121,1123],{"className":1122,"ariaHidden":341},[340],[44,1124,1126,1129],{"className":1125},[345],[44,1127],{"className":1128,"style":350},[349],[44,1130,248],{"className":1131,"style":356},[354,355],").",[20,1134,1136],{"id":1135},"interativo-vendo-os-centroides-se-moverem","Interativo: vendo os centroides se moverem",[11,1138,1139,1140,1145,1146,1174],{},"Reconstrução minha do mesmo algoritmo (atribuir, recalcular, repetir), nos 150 pontos reais do Iris, ",[1141,1142,1144],"a",{"href":1143},"\u002Fplaylists\u002Fpattern-recognition\u002Fdecision-trees","as mesmas duas variáveis do post de árvores de decisão",". Clica em \"Passo\" e acompanha a inércia (",[44,1147,1149,1162],{"className":1148},[230],[44,1150,1152],{"className":1151},[234],[236,1153,1154],{"xmlns":238},[240,1155,1156,1160],{},[243,1157,1158],{},[246,1159,248],{},[333,1161,248],{"encoding":335},[44,1163,1165],{"className":1164,"ariaHidden":341},[340],[44,1166,1168,1171],{"className":1167},[345],[44,1169],{"className":1170,"style":350},[349],[44,1172,248],{"className":1173,"style":356},[354,355],") cair a cada rodada, e as regiões (quem pertence a qual centroide) se ajustarem:",[1176,1177],"k-means-explorer",{":n-clusters":1178,":points":1179,":true-classes":1180,":x-max":1181,":x-min":1182,":y-max":1183,":y-min":1184,"class0-label":1185,"class1-label":1186,"class2-label":1187,"x-label":1188,"y-label":1189},"3","[[1.4,0.2],[1.4,0.2],[1.3,0.2],[1.5,0.2],[1.4,0.2],[1.7,0.4],[1.4,0.3],[1.5,0.2],[1.4,0.2],[1.5,0.1],[1.5,0.2],[1.6,0.2],[1.4,0.1],[1.1,0.1],[1.2,0.2],[1.5,0.4],[1.3,0.4],[1.4,0.3],[1.7,0.3],[1.5,0.3],[1.7,0.2],[1.5,0.4],[1.0,0.2],[1.7,0.5],[1.9,0.2],[1.6,0.2],[1.6,0.4],[1.5,0.2],[1.4,0.2],[1.6,0.2],[1.6,0.2],[1.5,0.4],[1.5,0.1],[1.4,0.2],[1.5,0.2],[1.2,0.2],[1.3,0.2],[1.4,0.1],[1.3,0.2],[1.5,0.2],[1.3,0.3],[1.3,0.3],[1.3,0.2],[1.6,0.6],[1.9,0.4],[1.4,0.3],[1.6,0.2],[1.4,0.2],[1.5,0.2],[1.4,0.2],[4.7,1.4],[4.5,1.5],[4.9,1.5],[4.0,1.3],[4.6,1.5],[4.5,1.3],[4.7,1.6],[3.3,1.0],[4.6,1.3],[3.9,1.4],[3.5,1.0],[4.2,1.5],[4.0,1.0],[4.7,1.4],[3.6,1.3],[4.4,1.4],[4.5,1.5],[4.1,1.0],[4.5,1.5],[3.9,1.1],[4.8,1.8],[4.0,1.3],[4.9,1.5],[4.7,1.2],[4.3,1.3],[4.4,1.4],[4.8,1.4],[5.0,1.7],[4.5,1.5],[3.5,1.0],[3.8,1.1],[3.7,1.0],[3.9,1.2],[5.1,1.6],[4.5,1.5],[4.5,1.6],[4.7,1.5],[4.4,1.3],[4.1,1.3],[4.0,1.3],[4.4,1.2],[4.6,1.4],[4.0,1.2],[3.3,1.0],[4.2,1.3],[4.2,1.2],[4.2,1.3],[4.3,1.3],[3.0,1.1],[4.1,1.3],[6.0,2.5],[5.1,1.9],[5.9,2.1],[5.6,1.8],[5.8,2.2],[6.6,2.1],[4.5,1.7],[6.3,1.8],[5.8,1.8],[6.1,2.5],[5.1,2.0],[5.3,1.9],[5.5,2.1],[5.0,2.0],[5.1,2.4],[5.3,2.3],[5.5,1.8],[6.7,2.2],[6.9,2.3],[5.0,1.5],[5.7,2.3],[4.9,2.0],[6.7,2.0],[4.9,2.0],[5.7,1.8],[6.0,2.1],[4.8,1.8],[4.9,1.8],[5.6,2.1],[5.8,1.6],[6.1,1.9],[6.4,2.0],[5.6,2.2],[5.1,1.5],[5.6,1.4],[6.1,2.3],[5.6,2.4],[5.5,1.8],[4.8,1.8],[5.4,2.1],[5.6,2.4],[5.1,2.3],[5.1,1.9],[5.9,2.3],[5.7,2.5],[5.2,2.3],[5.0,1.9],[5.2,2.0],[5.4,2.3],[5.1,1.8]]","[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2]","7.1","0.8","2.6","0","Setosa","Versicolor","Virginica","comprimento da pétala (cm)","largura da pétala (cm)",[11,1191,1192],{},"Os pontos continuam coloridos pela espécie real, só pra você comparar visualmente: as regiões que o K-means encontra sozinho, sem nunca ver o rótulo, batem bastante com as espécies de verdade, principalmente pra Setosa (que já era claramente separada desde o post de árvores). Clica em \"Reiniciar\" algumas vezes: dependendo de onde os centroides caem por sorteio, o resultado final pode variar um pouco, exatamente a limitação de mínimo local que o Bishop descreve.",[20,1194,1196],{"id":1195},"escolhendo-k-o-método-do-cotovelo","Escolhendo K: o método do cotovelo",[11,1198,1199,1200,1203,1204,1232],{},"Faltou uma pergunta: como escolher quantos grupos (K) procurar? O K-means do scikit-learn expõe ",[15,1201,1202],{},"inertia_",", que é exatamente o ",[44,1205,1207,1220],{"className":1206},[230],[44,1208,1210],{"className":1209},[234],[236,1211,1212],{"xmlns":238},[240,1213,1214,1218],{},[243,1215,1216],{},[246,1217,248],{},[333,1219,248],{"encoding":335},[44,1221,1223],{"className":1222,"ariaHidden":341},[340],[44,1224,1226,1229],{"className":1225},[345],[44,1227],{"className":1228,"style":350},[349],[44,1230,248],{"className":1231,"style":356},[354,355]," do Bishop no final do treino:",[35,1234,1236],{"className":37,"code":1235,"language":39,"meta":40,"style":40},"kmeans_k3 = KMeans(n_clusters=3).fit(X)\nkmeans_k8 = KMeans(n_clusters=8).fit(X)\n",[15,1237,1238,1243],{"__ignoreMap":40},[44,1239,1240],{"class":46,"line":47},[44,1241,1242],{},"kmeans_k3 = KMeans(n_clusters=3).fit(X)\n",[44,1244,1245],{"class":46,"line":53},[44,1246,1247],{},"kmeans_k8 = KMeans(n_clusters=8).fit(X)\n",[1249,1250,1251],"blockquote",{},[11,1252,1253,1256],{},[184,1254,1255],{},"Saída:"," inércia com K=3: 31.37. Com K=8: 8.31.",[11,1258,1259,1260,1263,1264,1268],{},"Mais grupos sempre reduz (ou empata) a inércia, no limite (K = número de pontos) ela chega a zero, cada ponto vira seu próprio grupo, o que não serve pra nada. O truque é plotar a inércia contra vários valores de K e procurar o ",[184,1261,1262],{},"cotovelo",": o ponto onde a curva para de cair rápido e passa a cair devagar. Antes do cotovelo, cada grupo a mais ainda captura estrutura real. Depois dele, cada grupo a mais só está fatiando ruído. O professor repete esse gráfico pro Iris, pro dataset de vinhos ",[1141,1265,1267],{"href":1266},"\u002Fplaylists\u002Fpattern-recognition\u002Fknn-classifier","que já apareceu nessa playlist",", e pro Digits (1797 imagens de dígitos escritos à mão, 8×8 pixels cada), sem tirar um número fechado de nenhum, o objetivo é só mostrar a forma da curva.",[20,1270,1272],{"id":1271},"o-truque-usar-os-grupos-como-feature-nova","O truque: usar os grupos como feature nova",[11,1274,1275,1276,1279],{},"Aqui que a aula 8b vira o post inteiro de cabeça pra baixo. Em vez de usar K-means só pra agrupar, o professor usa ele ",[184,1277,1278],{},"dentro de um pipeline supervisionado",", como uma etapa de pré-processamento:",[35,1281,1283],{"className":37,"code":1282,"language":39,"meta":40,"style":40},"from sklearn.linear_model import RidgeClassifier\nfrom sklearn.model_selection import cross_val_score\nfrom sklearn.pipeline import make_pipeline\n\nmodel = RidgeClassifier()\nscores = cross_val_score(model, X, y, cv=5)\n",[15,1284,1285,1290,1295,1300,1304,1309],{"__ignoreMap":40},[44,1286,1287],{"class":46,"line":47},[44,1288,1289],{},"from sklearn.linear_model import RidgeClassifier\n",[44,1291,1292],{"class":46,"line":53},[44,1293,1294],{},"from sklearn.model_selection import cross_val_score\n",[44,1296,1297],{"class":46,"line":60},[44,1298,1299],{},"from sklearn.pipeline import make_pipeline\n",[44,1301,1302],{"class":46,"line":66},[44,1303,57],{"emptyLinePlaceholder":56},[44,1305,1306],{"class":46,"line":72},[44,1307,1308],{},"model = RidgeClassifier()\n",[44,1310,1311],{"class":46,"line":78},[44,1312,1313],{},"scores = cross_val_score(model, X, y, cv=5)\n",[1249,1315,1316],{},[11,1317,1318,1325],{},[184,1319,1320,1321,1324],{},"Saída (Digits, ",[15,1322,1323],{},"RidgeClassifier"," sozinho, direto nos 64 pixels):"," 0.888 de acurácia média.",[35,1327,1329],{"className":37,"code":1328,"language":39,"meta":40,"style":40},"model = make_pipeline(\n    KMeans(n_clusters=50),\n    RidgeClassifier()\n)\nscores = cross_val_score(model, X, y, cv=5)\n",[15,1330,1331,1336,1341,1346,1351],{"__ignoreMap":40},[44,1332,1333],{"class":46,"line":47},[44,1334,1335],{},"model = make_pipeline(\n",[44,1337,1338],{"class":46,"line":53},[44,1339,1340],{},"    KMeans(n_clusters=50),\n",[44,1342,1343],{"class":46,"line":60},[44,1344,1345],{},"    RidgeClassifier()\n",[44,1347,1348],{"class":46,"line":66},[44,1349,1350],{},")\n",[44,1352,1353],{"class":46,"line":72},[44,1354,1313],{},[1249,1356,1357],{},[11,1358,1359,1362],{},[184,1360,1361],{},"Saída (mesmo dataset, com K-means de 50 grupos antes do classificador):"," 0.939.",[11,1364,1365,1366,1368,1369,1372,1373,1376,1377,1380,1381,1383,1384,1386,1387,1390],{},"Um salto e tanto, só de encaixar um K-means no meio do caminho. Como isso funciona? É aqui que o ",[15,1367,189],{}," que eu mencionei lá em cima entra em cena. Dentro de um ",[15,1370,1371],{},"Pipeline",", toda etapa que não é a última precisa ter ",[15,1374,1375],{},".transform()",", não ",[15,1378,1379],{},".predict()",". O K-means do scikit-learn implementa os dois: ",[15,1382,1379],{}," devolve o índice do grupo mais próximo (um número só), mas ",[15,1385,1375],{}," devolve a ",[184,1388,1389],{},"distância até cada um dos K centroides"," (K números). Uma imagem de 64 pixels vira um vetor de 50 números, cada um dizendo \"quão parecida essa imagem é com o protótipo do grupo 1, do grupo 2, ..., do grupo 50\". Cada centroide funciona como um \"dígito prototípico\" aprendido sem rótulo nenhum, e a distância até cada um vira uma feature nova, mais informativa pro classificador linear do que o pixel cru.",[11,1392,1393],{},"O professor ainda usa o Optuna pra buscar o número ideal de grupos (entre 10 e 200):",[1249,1395,1396],{},[11,1397,1398,1400,1401,1404,1405,1408],{},[184,1399,1255],{}," o melhor encontrado foi ",[15,1402,1403],{},"n_clusters=136",", com 0.963 de acurácia média em validação cruzada. Testando ",[15,1406,1407],{},"n_clusters=250"," na mão (mais grupos que o Optuna sequer tentou): 0.966, ainda melhor.",[11,1410,1411],{},"Mais grupos, mais \"protótipos\" diferentes pra comparar, mais informação pro classificador final, até onde o professor testou.",[20,1413,1415],{"id":1414},"o-mesmo-truque-num-dataset-maior-mnist","O mesmo truque, num dataset maior: MNIST",[11,1417,1418,1421],{},[15,1419,1420],{},"aula08c"," repete a receita exata no MNIST (60 mil imagens de treino, 28×28 pixels cada, então 784 variáveis por imagem, bem mais que o Digits):",[1423,1424,1425,1440],"table",{},[1426,1427,1428],"thead",{},[1429,1430,1431,1436],"tr",{},[1432,1433,1435],"th",{"align":1434},"left","Abordagem",[1432,1437,1439],{"align":1438},"right","Acurácia de teste",[1441,1442,1443,1454,1467],"tbody",{},[1429,1444,1445,1451],{},[1446,1447,1448,1450],"td",{"align":1434},[15,1449,1323],{}," direto nos 784 pixels",[1446,1452,1453],{"align":1438},"0.8603",[1429,1455,1456,1464],{},[1446,1457,1458,1461,1462],{"align":1434},[15,1459,1460],{},"StandardScaler"," + ",[15,1463,1323],{},[1446,1465,1466],{"align":1438},"(praticamente igual, normalizar pixel não ajudou aqui)",[1429,1468,1469,1474],{},[1446,1470,1471,1472],{"align":1434},"K-means (250 grupos) + ",[15,1473,1323],{},[1446,1475,1476],{"align":1438},[184,1477,1478],{},"0.9403",[11,1480,1481],{},"Oito pontos percentuais de ganho, no mesmo dataset, no mesmo classificador linear, só trocando \"64\u002F784 valores de pixel cru\" por \"250 distâncias até protótipos aprendidos sem rótulo\". A ideia de reaproveitar um algoritmo não supervisionado como fonte de features pra um problema supervisionado é mais geral do que parece: qualquer vetor de \"distância até um protótipo\" carrega informação que o pixel cru não carrega sozinho.",[20,1483,1485],{"id":1484},"fechando","Fechando",[1423,1487,1488,1498],{},[1426,1489,1490],{},[1429,1491,1492,1495],{},[1432,1493,1494],{"align":1434},"O que eu já sabia",[1432,1496,1497],{"align":1434},"O que essa aula assentou",[1441,1499,1500,1508,1518],{},[1429,1501,1502,1505],{},[1446,1503,1504],{"align":1434},"Todo modelo até aqui aprendia a prever um rótulo dado",[1446,1506,1507],{"align":1434},"K-means agrupa sem nenhum rótulo, só a estrutura geométrica dos dados",[1429,1509,1510,1515],{},[1446,1511,1512,1514],{"align":1434},[15,1513,189],{}," serve pra etapas que só transformam, sem prever",[1446,1516,1517],{"align":1434},"Um algoritmo de clustering também pode ser um transformador: a distância até cada centroide vira uma feature nova",[1429,1519,1520,1523],{},[1446,1521,1522],{"align":1434},"Mais complexidade nem sempre ajuda",[1446,1524,1525],{"align":1434},"Aqui ajudou bastante: trocar pixel cru por distância-até-protótipo rendeu ganhos de 5 a 8 pontos percentuais em dois datasets diferentes",[20,1527,1529],{"id":1528},"aplicação-prática","Aplicação Prática",[11,1531,1532,1533,1535,1536,1539],{},"Reproduzi a mesma receita (K-means como transformador antes do ",[15,1534,1323],{},") no Digits, com semente fixa e ",[15,1537,1538],{},"n_init=10"," (o K-means roda 10 vezes com centroides iniciais diferentes e fica com o de menor inércia, o padrão do scikit-learn), pra confirmar o efeito de forma reprodutível.",[35,1541,1543],{"className":37,"code":1542,"language":39,"meta":40,"style":40},"cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)\nridge_scores = cross_val_score(RidgeClassifier(random_state=42), X, y, cv=cv)\nfor k in [50, 136, 250]:\n    model = make_pipeline(KMeans(n_clusters=k, random_state=42, n_init=10), RidgeClassifier(random_state=42))\n    scores = cross_val_score(model, X, y, cv=cv)\n",[15,1544,1545,1550,1555,1560,1565],{"__ignoreMap":40},[44,1546,1547],{"class":46,"line":47},[44,1548,1549],{},"cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)\n",[44,1551,1552],{"class":46,"line":53},[44,1553,1554],{},"ridge_scores = cross_val_score(RidgeClassifier(random_state=42), X, y, cv=cv)\n",[44,1556,1557],{"class":46,"line":60},[44,1558,1559],{},"for k in [50, 136, 250]:\n",[44,1561,1562],{"class":46,"line":66},[44,1563,1564],{},"    model = make_pipeline(KMeans(n_clusters=k, random_state=42, n_init=10), RidgeClassifier(random_state=42))\n",[44,1566,1567],{"class":46,"line":72},[44,1568,1569],{},"    scores = cross_val_score(model, X, y, cv=cv)\n",[1423,1571,1572,1584],{},[1426,1573,1574],{},[1429,1575,1576,1578,1581],{},[1432,1577,1435],{"align":1434},[1432,1579,1580],{"align":1438},"Acurácia média (5 dobras)",[1432,1582,1583],{"align":1438},"Desvio padrão",[1441,1585,1586,1599,1612,1625],{},[1429,1587,1588,1593,1596],{},[1446,1589,1590,1592],{"align":1434},[15,1591,1323],{}," direto nos pixels",[1446,1594,1595],{"align":1438},"0.9343",[1446,1597,1598],{"align":1438},"0.0072",[1429,1600,1601,1606,1609],{},[1446,1602,1603,1604],{"align":1434},"K-means (50) + ",[15,1605,1323],{},[1446,1607,1608],{"align":1438},"0.9666",[1446,1610,1611],{"align":1438},"0.0128",[1429,1613,1614,1619,1622],{},[1446,1615,1616,1617],{"align":1434},"K-means (136) + ",[15,1618,1323],{},[1446,1620,1621],{"align":1438},"0.9844",[1446,1623,1624],{"align":1438},"0.0045",[1429,1626,1627,1632,1637],{},[1446,1628,1629,1630],{"align":1434},"K-means (250) + ",[15,1631,1323],{},[1446,1633,1634],{"align":1438},[184,1635,1636],{},"0.9916",[1446,1638,1639],{"align":1438},"0.0056",[11,1641,1642,1643,1645],{},"Meus números saem mais altos que os do notebook original (0.888\u002F0.939\u002F0.963), provavelmente porque fixei ",[15,1644,1538],{}," (o K-means tenta várias inicializações e fica com a melhor, evitando os mínimos locais ruins que o Bishop avisou lá em cima) e uma semente que meu ambiente conseguiu reproduzir de forma estável. Mas a tendência é idêntica à do notebook: quanto mais grupos, melhor a acurácia, e a melhora de \"pixel cru\" pra \"distância até protótipo\" é grande e consistente em qualquer contagem de grupos testada.",[1647,1648,1649],"style",{},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":40,"searchDepth":53,"depth":53,"links":1651},[1652,1653,1654,1655,1656,1657,1658,1659],{"id":22,"depth":53,"text":23},{"id":32,"depth":53,"text":33},{"id":1135,"depth":53,"text":1136},{"id":1195,"depth":53,"text":1196},{"id":1271,"depth":53,"text":1272},{"id":1414,"depth":53,"text":1415},{"id":1484,"depth":53,"text":1485},{"id":1528,"depth":53,"text":1529},null,"2026-08-20","Aula 8: o professor entra em aprendizado não supervisionado com K-means feito à mão, o método do cotovelo pra escolher K, e um truque que eu não esperava: usar os grupos encontrados como features novas pra um classificador supervisionado.","md",{},"\u002Fpt\u002Fplaylists\u002Fpattern-recognition\u002Fkmeans","pattern-recognition",{"title":6,"description":1662},"published","pt\u002Fplaylists\u002Fpattern-recognition\u002Fkmeans",[1671,1672,1673],"k-means","nao-supervisionado","engenharia-de-features","H_gKXhe92KNF_F8iU2dBZdA6JqmGE0bYvaf7v2-JS4w",1787338983385]