[{"data":1,"prerenderedAt":1952},["ShallowReactive",2],{"lang-switch-post-\u002Fen\u002Fplaylists\u002Fpattern-recognition\u002Flinear-regression-estimator":3,"post-en-pattern-recognition-linear-regression-estimator":4},"\u002Fplaylists\u002Fpattern-recognition\u002Flinear-regression-estimator",{"id":5,"title":6,"body":7,"cover":1937,"date":1938,"description":1939,"extension":1940,"meta":1941,"navigation":240,"order":45,"path":1942,"playlist":1943,"seo":1944,"status":1945,"stem":1946,"tags":1947,"__hash__":1951},"posts\u002Fen\u002Fplaylists\u002Fpattern-recognition\u002Flinear-regression-estimator.md","Linear Regression by Hand: Building My Own Estimator",{"type":8,"value":9,"toc":1926},"minimark",[10,14,19,32,67,78,82,212,221,253,268,285,289,297,327,373,382,391,851,854,1092,1098,1102,1117,1137,1145,1153,1178,1365,1372,1376,1387,1431,1476,1483,1526,1532,1564,1572,1579,1583,1586,1679,1704,1707,1716,1739,1742,1757,1769,1773,1779,1794,1805,1808,1812,1861,1877,1881,1884,1904,1911,1919,1922],[11,12,13],"p",{},"Lecture 1 of Dr. Boldt's course. His notebook has almost no markdown cells, it's him live-coding straight through, so most of the work in this post is mine: retelling the \"why\" behind each block of code, with Bishop's book as reference. Let's go.",[15,16,18],"h2",{"id":17},"the-dataset-442-real-patients","The dataset: 442 real patients",[11,20,21,22,26,27,31],{},"Unlike the Andrew Ng specialization playlist, here the dataset ships built into scikit-learn itself: ",[23,24,25],"code",{},"load_diabetes",", with ",[28,29,30],"strong",{},"442 real patients",", 10 input variables (age, sex, body mass index, average blood pressure, and six blood serum measurements) and one target, a numeric measure of how much the diabetes progressed after one year.",[33,34,39],"pre",{"className":35,"code":36,"language":37,"meta":38,"style":38},"language-python shiki shiki-themes github-light github-dark","from sklearn.datasets import load_diabetes\ndata = load_diabetes()\nX = data.data   # (442, 10)\ny = data.target # (442,)\n","python","",[23,40,41,49,55,61],{"__ignoreMap":38},[42,43,46],"span",{"class":44,"line":45},"line",1,[42,47,48],{},"from sklearn.datasets import load_diabetes\n",[42,50,52],{"class":44,"line":51},2,[42,53,54],{},"data = load_diabetes()\n",[42,56,58],{"class":44,"line":57},3,[42,59,60],{},"X = data.data   # (442, 10)\n",[42,62,64],{"class":44,"line":63},4,[42,65,66],{},"y = data.target # (442,)\n",[11,68,69,70,73,74,77],{},"One detail that catches a lot of people off guard: the 10 columns of ",[23,71,72],{},"X"," are ",[28,75,76],{},"not"," in their original units. Scikit-learn ships this dataset already mean-centered and rescaled, column by column. That's why the body mass index column (the one I'll use throughout this post) ranges from -0.09 to 0.17 instead of sitting in the normal 15 to 40 range. Keep that in mind, it's why the coefficients that show up later look like giant numbers: they're compensating for a tiny input variable.",[15,79,81],{"id":80},"the-simplest-possible-model","The simplest possible model",[11,83,84],{},[42,85,88,130],{"className":86},[87],"katex",[42,89,92],{"className":90},[91],"katex-mathml",[93,94,96],"math",{"xmlns":95},"http:\u002F\u002Fwww.w3.org\u002F1998\u002FMath\u002FMathML",[97,98,99,125],"semantics",{},[100,101,102,106,110,113,116,119,122],"mrow",{},[103,104,105],"mi",{},"y",[107,108,109],"mo",{},"=",[103,111,112],{},"a",[107,114,115],{},"⋅",[103,117,118],{},"x",[107,120,121],{},"+",[103,123,124],{},"b",[126,127,129],"annotation",{"encoding":128},"application\u002Fx-tex","y = a \\cdot x + b",[42,131,135,162,183,202],{"className":132,"ariaHidden":134},[133],"katex-html","true",[42,136,139,144,150,155,159],{"className":137},[138],"base",[42,140],{"className":141,"style":143},[142],"strut","height:0.625em;vertical-align:-0.1944em;",[42,145,105],{"className":146,"style":149},[147,148],"mord","mathnormal","margin-right:0.0359em;",[42,151],{"className":152,"style":154},[153],"mspace","margin-right:0.2778em;",[42,156,109],{"className":157},[158],"mrel",[42,160],{"className":161,"style":154},[153],[42,163,165,169,172,176,180],{"className":164},[138],[42,166],{"className":167,"style":168},[142],"height:0.4445em;",[42,170,112],{"className":171},[147,148],[42,173],{"className":174,"style":175},[153],"margin-right:0.2222em;",[42,177,115],{"className":178},[179],"mbin",[42,181],{"className":182,"style":175},[153],[42,184,186,190,193,196,199],{"className":185},[138],[42,187],{"className":188,"style":189},[142],"height:0.6667em;vertical-align:-0.0833em;",[42,191,118],{"className":192},[147,148],[42,194],{"className":195,"style":175},[153],[42,197,121],{"className":198},[179],[42,200],{"className":201,"style":175},[153],[42,203,205,209],{"className":204},[138],[42,206],{"className":207,"style":208},[142],"height:0.6944em;",[42,210,124],{"className":211},[147,148],[11,213,214,215,217,218,220],{},"The same familiar shape: a line, where ",[23,216,112],{}," is the slope (how much the target changes per unit of body mass index) and ",[23,219,124],{}," is the intercept. The professor tries two hand-picked guesses, no optimization yet, just to see what the line looks like:",[33,222,224],{"className":35,"code":223,"language":37,"meta":38,"style":38},"def modelo_linear(X, a, b):\n    return a*X + b\n\nypred = modelo_linear(X2, 1000, 100)  # first guess\nypred = modelo_linear(X2, 1500, 150)  # second guess\n",[23,225,226,231,236,242,247],{"__ignoreMap":38},[42,227,228],{"class":44,"line":45},[42,229,230],{},"def modelo_linear(X, a, b):\n",[42,232,233],{"class":44,"line":51},[42,234,235],{},"    return a*X + b\n",[42,237,238],{"class":44,"line":57},[42,239,241],{"emptyLinePlaceholder":240},true,"\n",[42,243,244],{"class":44,"line":63},[42,245,246],{},"ypred = modelo_linear(X2, 1000, 100)  # first guess\n",[42,248,250],{"class":44,"line":249},5,[42,251,252],{},"ypred = modelo_linear(X2, 1500, 150)  # second guess\n",[11,254,255,256,259,260,262,263,259,265,267],{},"Drag the sliders below (here on the blog I use the standard ",[23,257,258],{},"w","\u002F",[23,261,124],{}," notation, it's exactly the same ",[23,264,112],{},[23,266,124],{}," the professor uses) over a real sample of 45 patients from the training set, and see for yourself which line fits the points better:",[269,270],"model-playground",{":b-max":271,":b-min":272,":b-step":273,":initial-b":274,":initial-w":275,":w-max":276,":w-min":272,":w-step":277,":x-train":278,":y-train":279,"dataLabel":280,"error-label":281,"prediction-label":282,"x-label":283,"y-label":284},"300","0","5","100","1000","1600","20","[-0.0062, -0.0903, 0.0714, 0.0445, 0.0143, -0.0698, -0.0849, -0.0461, -0.0655, -0.0213, -0.0041, 0.0358, 0.0692, 0.0207, -0.0504, 0.024, -0.0094, 0.1102, 0.0477, 0.0143, -0.0256, -0.0385, 0.08, -0.0041, 0.0412, -0.0418, 0.093, 0.01, 0.0606, 0.0391, 0.0595, 0.0164, -0.0105, 0.0067, 0.0013, -0.0299, -0.0235, -0.072, -0.0084, 0.1609, -0.0224, 0.0595, -0.0073, -0.0105, 0.0067]","[219, 94, 295, 129, 90, 48, 90, 72, 214, 281, 68, 184, 277, 197, 189, 121, 257, 258, 317, 191, 252, 127, 257, 198, 198, 103, 128, 150, 245, 246, 178, 268, 168, 109, 229, 118, 71, 77, 81, 346, 84, 85, 52, 25, 67]","Patient","Total error (sum of absolute errors)","Model","BMI (normalized)","diabetes progression",[15,286,288],{"id":287},"how-to-tell-if-a-guess-is-any-good","How to tell if a guess is any good",[11,290,291,292,296],{},"\"Fits better\" needs to become a number. The professor implements three metrics by hand, the same trio that already showed up ",[112,293,295],{"href":294},"\u002Fen\u002Fplaylists\u002Fmachine-learning-specialization\u002Fw2-lab05-scikit-learn","in the specialization playlist"," (if you've read that one, feel free to skim):",[298,299,300,311,321],"ul",{},[301,302,303,306,307,310],"li",{},[28,304,305],{},"MAE"," (mean absolute error): the average of ",[23,308,309],{},"|predicted - actual|",". Easy to read (it's in patient units), but treats every error the same.",[301,312,313,316,317,320],{},[28,314,315],{},"MSE"," (mean squared error): the average of ",[23,318,319],{},"(predicted - actual)²",". Squaring means one big miss weighs disproportionately more than several small ones.",[301,322,323,326],{},[28,324,325],{},"RMSE",": the square root of MSE, to get back to the original unit after having squared everything.",[33,328,330],{"className":35,"code":329,"language":37,"meta":38,"style":38},"def mae(y, ypred):\n    return np.sum(np.abs(y - ypred)) \u002F len(y)\n\ndef mse(y, ypred):\n    return np.sum((y - ypred)**2) \u002F len(y)\n\ndef rmse(y, ypred):\n    return np.sqrt(mse(y, ypred))\n",[23,331,332,337,342,346,351,356,361,367],{"__ignoreMap":38},[42,333,334],{"class":44,"line":45},[42,335,336],{},"def mae(y, ypred):\n",[42,338,339],{"class":44,"line":51},[42,340,341],{},"    return np.sum(np.abs(y - ypred)) \u002F len(y)\n",[42,343,344],{"class":44,"line":57},[42,345,241],{"emptyLinePlaceholder":240},[42,347,348],{"class":44,"line":63},[42,349,350],{},"def mse(y, ypred):\n",[42,352,353],{"class":44,"line":249},[42,354,355],{},"    return np.sum((y - ypred)**2) \u002F len(y)\n",[42,357,359],{"class":44,"line":358},6,[42,360,241],{"emptyLinePlaceholder":240},[42,362,364],{"class":44,"line":363},7,[42,365,366],{},"def rmse(y, ypred):\n",[42,368,370],{"class":44,"line":369},8,[42,371,372],{},"    return np.sqrt(mse(y, ypred))\n",[374,375,376],"blockquote",{},[11,377,378,381],{},[28,379,380],{},"Output on the full dataset (442 patients):"," guess (1000, 100) → MAE 64.93, MSE 6614.14, RMSE 81.33. Guess (1500, 150) → MAE 53.93, MSE 4580.80, RMSE 67.68. The second guess wins on all three metrics.",[11,383,384,385,390],{},"Bishop calls this an ",[386,387,389],"glossary-term",{"definition":388},"the single number that summarizes how far the model is from the training data","error function",", and defines it almost exactly like MSE, except without dividing by the number of points and with a bonus factor of 1\u002F2:",[11,392,393],{},[42,394,396,490],{"className":395},[87],[42,397,399],{"className":398},[91],[93,400,401],{"xmlns":95},[97,402,403,487],{},[100,404,405,408,412,415,418,420,430,448,451,453,455,462,465,467,469,472,479],{},[103,406,407],{},"E",[107,409,411],{"stretchy":410},"false","(",[103,413,258],{"mathvariant":414},"bold",[107,416,417],{"stretchy":410},")",[107,419,109],{},[421,422,423,427],"mfrac",{},[424,425,426],"mn",{},"1",[424,428,429],{},"2",[431,432,433,436,445],"msubsup",{},[107,434,435],{},"∑",[100,437,438,441,443],{},[103,439,440],{},"n",[107,442,109],{},[424,444,426],{},[103,446,447],{},"N",[107,449,450],{"stretchy":410},"{",[103,452,105],{},[107,454,411],{"stretchy":410},[456,457,458,460],"msub",{},[103,459,118],{},[103,461,440],{},[107,463,464],{"separator":134},",",[103,466,258],{"mathvariant":414},[107,468,417],{"stretchy":410},[107,470,471],{},"−",[456,473,474,477],{},[103,475,476],{},"t",[103,478,440],{},[480,481,482,485],"msup",{},[107,483,484],{"stretchy":410},"}",[424,486,429],{},[126,488,489],{"encoding":128},"E(\\mathbf{w}) = \\frac{1}{2}\\sum_{n=1}^{N}\\{y(x_n,\\mathbf{w}) - t_n\\}^2",[42,491,493,526,773],{"className":492,"ariaHidden":134},[133],[42,494,496,500,504,508,513,517,520,523],{"className":495},[138],[42,497],{"className":498,"style":499},[142],"height:1em;vertical-align:-0.25em;",[42,501,407],{"className":502,"style":503},[147,148],"margin-right:0.0576em;",[42,505,411],{"className":506},[507],"mopen",[42,509,258],{"className":510,"style":512},[147,511],"mathbf","margin-right:0.016em;",[42,514,417],{"className":515},[516],"mclose",[42,518],{"className":519,"style":154},[153],[42,521,109],{"className":522},[158],[42,524],{"className":525,"style":154},[153],[42,527,529,533,621,625,699,702,705,708,751,755,758,761,764,767,770],{"className":528},[138],[42,530],{"className":531,"style":532},[142],"height:1.3262em;vertical-align:-0.345em;",[42,534,536,540,618],{"className":535},[147],[42,537],{"className":538},[507,539],"nulldelimiter",[42,541,543],{"className":542},[421],[42,544,548,609],{"className":545},[546,547],"vlist-t","vlist-t2",[42,549,552,604],{"className":550},[551],"vlist-r",[42,553,557,578,589],{"className":554,"style":556},[555],"vlist","height:0.8451em;",[42,558,560,565],{"style":559},"top:-2.655em;",[42,561],{"className":562,"style":564},[563],"pstrut","height:3em;",[42,566,572],{"className":567},[568,569,570,571],"sizing","reset-size6","size3","mtight",[42,573,575],{"className":574},[147,571],[42,576,429],{"className":577},[147,571],[42,579,581,584],{"style":580},"top:-3.23em;",[42,582],{"className":583,"style":564},[563],[42,585],{"className":586,"style":588},[587],"frac-line","border-bottom-width:0.04em;",[42,590,592,595],{"style":591},"top:-3.394em;",[42,593],{"className":594,"style":564},[563],[42,596,598],{"className":597},[568,569,570,571],[42,599,601],{"className":600},[147,571],[42,602,426],{"className":603},[147,571],[42,605,608],{"className":606},[607],"vlist-s","​",[42,610,612],{"className":611},[551],[42,613,616],{"className":614,"style":615},[555],"height:0.345em;",[42,617],{},[42,619],{"className":620},[516,539],[42,622],{"className":623,"style":624},[153],"margin-right:0.1667em;",[42,626,629,635],{"className":627},[628],"mop",[42,630,435],{"className":631,"style":634},[628,632,633],"op-symbol","small-op","position:relative;top:0em;",[42,636,639],{"className":637},[638],"msupsub",[42,640,642,690],{"className":641},[546,547],[42,643,645,687],{"className":644},[551],[42,646,649,671],{"className":647,"style":648},[555],"height:0.9812em;",[42,650,652,656],{"style":651},"top:-2.4003em;margin-left:0em;margin-right:0.05em;",[42,653],{"className":654,"style":655},[563],"height:2.7em;",[42,657,659],{"className":658},[568,569,570,571],[42,660,662,665,668],{"className":661},[147,571],[42,663,440],{"className":664},[147,148,571],[42,666,109],{"className":667},[158,571],[42,669,426],{"className":670},[147,571],[42,672,674,677],{"style":673},"top:-3.2029em;margin-right:0.05em;",[42,675],{"className":676,"style":655},[563],[42,678,680],{"className":679},[568,569,570,571],[42,681,683],{"className":682},[147,571],[42,684,447],{"className":685,"style":686},[147,148,571],"margin-right:0.109em;",[42,688,608],{"className":689},[607],[42,691,693],{"className":692},[551],[42,694,697],{"className":695,"style":696},[555],"height:0.2997em;",[42,698],{},[42,700,450],{"className":701},[507],[42,703,105],{"className":704,"style":149},[147,148],[42,706,411],{"className":707},[507],[42,709,711,714],{"className":710},[147],[42,712,118],{"className":713},[147,148],[42,715,717],{"className":716},[638],[42,718,720,742],{"className":719},[546,547],[42,721,723,739],{"className":722},[551],[42,724,727],{"className":725,"style":726},[555],"height:0.1514em;",[42,728,730,733],{"style":729},"top:-2.55em;margin-left:0em;margin-right:0.05em;",[42,731],{"className":732,"style":655},[563],[42,734,736],{"className":735},[568,569,570,571],[42,737,440],{"className":738},[147,148,571],[42,740,608],{"className":741},[607],[42,743,745],{"className":744},[551],[42,746,749],{"className":747,"style":748},[555],"height:0.15em;",[42,750],{},[42,752,464],{"className":753},[754],"mpunct",[42,756],{"className":757,"style":624},[153],[42,759,258],{"className":760,"style":512},[147,511],[42,762,417],{"className":763},[516],[42,765],{"className":766,"style":175},[153],[42,768,471],{"className":769},[179],[42,771],{"className":772,"style":175},[153],[42,774,776,780,820],{"className":775},[138],[42,777],{"className":778,"style":779},[142],"height:1.0641em;vertical-align:-0.25em;",[42,781,783,786],{"className":782},[147],[42,784,476],{"className":785},[147,148],[42,787,789],{"className":788},[638],[42,790,792,812],{"className":791},[546,547],[42,793,795,809],{"className":794},[551],[42,796,798],{"className":797,"style":726},[555],[42,799,800,803],{"style":729},[42,801],{"className":802,"style":655},[563],[42,804,806],{"className":805},[568,569,570,571],[42,807,440],{"className":808},[147,148,571],[42,810,608],{"className":811},[607],[42,813,815],{"className":814},[551],[42,816,818],{"className":817,"style":748},[555],[42,819],{},[42,821,823,826],{"className":822},[516],[42,824,484],{"className":825},[516],[42,827,829],{"className":828},[638],[42,830,832],{"className":831},[546],[42,833,835],{"className":834},[551],[42,836,839],{"className":837,"style":838},[555],"height:0.8141em;",[42,840,842,845],{"style":841},"top:-3.063em;margin-right:0.05em;",[42,843],{"className":844,"style":655},[563],[42,846,848],{"className":847},[568,569,570,571],[42,849,429],{"className":850},[147,571],[11,852,853],{},"The 1\u002F2 exists purely for convenience: when you take the derivative to find the minimum, the square comes down multiplying and cancels the 1\u002F2 exactly. A cosmetic detail, it doesn't change where the minimum points to. And the \"root\" version he uses to compare datasets of different sizes,",[11,855,856],{},[42,857,859,906],{"className":858},[87],[42,860,862],{"className":861},[91],[93,863,864],{"xmlns":95},[97,865,866,903],{},[100,867,868,876,878],{},[456,869,870,872],{},[103,871,407],{},[873,874,875],"mtext",{},"RMS",[107,877,109],{},[879,880,881],"msqrt",{},[100,882,883,885,887,889,896,898,901],{},[424,884,429],{},[103,886,407],{},[107,888,411],{"stretchy":410},[480,890,891,893],{},[103,892,258],{"mathvariant":414},[107,894,895],{},"∗",[107,897,417],{"stretchy":410},[103,899,259],{"mathvariant":900},"normal",[103,902,447],{},[126,904,905],{"encoding":128},"E_{\\text{RMS}} = \\sqrt{2E(\\mathbf{w}^*)\u002FN}",[42,907,909,974],{"className":908,"ariaHidden":134},[133],[42,910,912,916,965,968,971],{"className":911},[138],[42,913],{"className":914,"style":915},[142],"height:0.8333em;vertical-align:-0.15em;",[42,917,919,922],{"className":918},[147],[42,920,407],{"className":921,"style":503},[147,148],[42,923,925],{"className":924},[638],[42,926,928,957],{"className":927},[546,547],[42,929,931,954],{"className":930},[551],[42,932,935],{"className":933,"style":934},[555],"height:0.3283em;",[42,936,938,941],{"style":937},"top:-2.55em;margin-left:-0.0576em;margin-right:0.05em;",[42,939],{"className":940,"style":655},[563],[42,942,944],{"className":943},[568,569,570,571],[42,945,947],{"className":946},[147,571],[42,948,951],{"className":949},[147,950,571],"text",[42,952,875],{"className":953},[147,571],[42,955,608],{"className":956},[607],[42,958,960],{"className":959},[551],[42,961,963],{"className":962,"style":748},[555],[42,964],{},[42,966],{"className":967,"style":154},[153],[42,969,109],{"className":970},[158],[42,972],{"className":973,"style":154},[153],[42,975,977,981],{"className":976},[138],[42,978],{"className":979,"style":980},[142],"height:1.24em;vertical-align:-0.305em;",[42,982,985],{"className":983},[147,984],"sqrt",[42,986,988,1083],{"className":987},[546,547],[42,989,991,1080],{"className":990},[551],[42,992,995,1057],{"className":993,"style":994},[555],"height:0.935em;",[42,996,1000,1004],{"className":997,"style":999},[998],"svg-align","top:-3.2em;",[42,1001],{"className":1002,"style":1003},[563],"height:3.2em;",[42,1005,1008,1011,1014,1017,1048,1051,1054],{"className":1006,"style":1007},[147],"padding-left:1em;",[42,1009,429],{"className":1010},[147],[42,1012,407],{"className":1013,"style":503},[147,148],[42,1015,411],{"className":1016},[507],[42,1018,1020,1023],{"className":1019},[147],[42,1021,258],{"className":1022,"style":512},[147,511],[42,1024,1026],{"className":1025},[638],[42,1027,1029],{"className":1028},[546],[42,1030,1032],{"className":1031},[551],[42,1033,1036],{"className":1034,"style":1035},[555],"height:0.6147em;",[42,1037,1039,1042],{"style":1038},"top:-2.989em;margin-right:0.05em;",[42,1040],{"className":1041,"style":655},[563],[42,1043,1045],{"className":1044},[568,569,570,571],[42,1046,895],{"className":1047},[179,571],[42,1049,417],{"className":1050},[516],[42,1052,259],{"className":1053},[147],[42,1055,447],{"className":1056,"style":686},[147,148],[42,1058,1060,1063],{"style":1059},"top:-2.895em;",[42,1061],{"className":1062,"style":1003},[563],[42,1064,1068],{"className":1065,"style":1067},[1066],"hide-tail","min-width:1.02em;height:1.28em;",[1069,1070,1076],"svg",{"xmlns":1071,"width":1072,"height":1073,"viewBox":1074,"preserveAspectRatio":1075},"http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg","400em","1.28em","0 0 400000 1296","xMinYMin slice",[1077,1078],"path",{"d":1079},"M263,681c0.7,0,18,39.7,52,119\nc34,79.3,68.167,158.7,102.5,238c34.3,79.3,51.8,119.3,52.5,120\nc340,-704.7,510.7,-1060.3,512,-1067\nl0 -0\nc4.7,-7.3,11,-11,19,-11\nH40000v40H1012.3\ns-271.3,567,-271.3,567c-38.7,80.7,-84,175,-136,283c-52,108,-89.167,185.3,-111.5,232\nc-22.3,46.7,-33.8,70.3,-34.5,71c-4.7,4.7,-12.3,7,-23,7s-12,-1,-12,-1\ns-109,-253,-109,-253c-72.7,-168,-109.3,-252,-110,-252c-10.7,8,-22,16.7,-34,26\nc-22,17.3,-33.3,26,-34,26s-26,-26,-26,-26s76,-59,76,-59s76,-60,76,-60z\nM1001 80h400000v40h-400000z",[42,1081,608],{"className":1082},[607],[42,1084,1086],{"className":1085},[551],[42,1087,1090],{"className":1088,"style":1089},[555],"height:0.305em;",[42,1091],{},[11,1093,1094,1095,1097],{},"is the same RMSE I just computed, with the same division by ",[23,1096,447],{}," hidden behind the factor of 2 (because his function skips the division that our MSE already does).",[15,1099,1101],{"id":1100},"the-dumb-model-that-becomes-a-ruler","The dumb model that becomes a ruler",[11,1103,1104,1105,1108,1109,1112,1113,1116],{},"Before any real regression, the second notebook (",[23,1106,1107],{},"aula02a",") does something smart: it splits off 20% of the data for testing (",[23,1110,1111],{},"train_test_split"," implemented by hand, ",[23,1114,1115],{},"random_state=42"," so it always gives the same result) and defines the laziest model that exists, always guessing the mean:",[33,1118,1120],{"className":35,"code":1119,"language":37,"meta":38,"style":38},"def modelo_media(y):\n    media = np.mean(y)\n    return np.ones_like(y) * media\n",[23,1121,1122,1127,1132],{"__ignoreMap":38},[42,1123,1124],{"class":44,"line":45},[42,1125,1126],{},"def modelo_media(y):\n",[42,1128,1129],{"class":44,"line":51},[42,1130,1131],{},"    media = np.mean(y)\n",[42,1133,1134],{"class":44,"line":57},[42,1135,1136],{},"    return np.ones_like(y) * media\n",[374,1138,1139],{},[11,1140,1141,1144],{},[28,1142,1143],{},"Output (353 training patients):"," MSE of guess (1500, 150) = 4646.20. MSE of \"always the mean\" = 5978.59.",[11,1146,1147,1148,1152],{},"Why do I need this? Because an MSE of 4646 on its own says nothing, it's just a number. Compared against the mean baseline, it's smaller, so the model is learning something. That's literally the definition of ",[386,1149,1151],{"definition":1150},"the fraction of the target's variance the model explains, compared to just always guessing the mean, where 1 is perfect, 0 is the same as guessing the mean, negative is worse than guessing the mean","R²",":",[33,1154,1156],{"className":35,"code":1155,"language":37,"meta":38,"style":38},"def r2_score(y, ypred):\n    ss_res = mse(y, ypred)\n    ss_tot = mse(y, modelo_media(y))\n    return 1 - (ss_res \u002F ss_tot)\n",[23,1157,1158,1163,1168,1173],{"__ignoreMap":38},[42,1159,1160],{"class":44,"line":45},[42,1161,1162],{},"def r2_score(y, ypred):\n",[42,1164,1165],{"class":44,"line":51},[42,1166,1167],{},"    ss_res = mse(y, ypred)\n",[42,1169,1170],{"class":44,"line":57},[42,1171,1172],{},"    ss_tot = mse(y, modelo_media(y))\n",[42,1174,1175],{"class":44,"line":63},[42,1176,1177],{},"    return 1 - (ss_res \u002F ss_tot)\n",[11,1179,1180],{},[42,1181,1183,1216],{"className":1182},[87],[42,1184,1186],{"className":1185},[91],[93,1187,1188],{"xmlns":95},[97,1189,1190,1213],{},[100,1191,1192,1199,1201,1203,1205],{},[480,1193,1194,1197],{},[103,1195,1196],{},"R",[424,1198,429],{},[107,1200,109],{},[424,1202,426],{},[107,1204,471],{},[421,1206,1207,1210],{},[873,1208,1209],{},"model MSE",[873,1211,1212],{},"baseline MSE",[126,1214,1215],{"encoding":128},"R^2 = 1 - \\frac{\\text{model MSE}}{\\text{baseline MSE}}",[42,1217,1219,1264,1283],{"className":1218,"ariaHidden":134},[133],[42,1220,1222,1225,1255,1258,1261],{"className":1221},[138],[42,1223],{"className":1224,"style":838},[142],[42,1226,1228,1232],{"className":1227},[147],[42,1229,1196],{"className":1230,"style":1231},[147,148],"margin-right:0.0077em;",[42,1233,1235],{"className":1234},[638],[42,1236,1238],{"className":1237},[546],[42,1239,1241],{"className":1240},[551],[42,1242,1244],{"className":1243,"style":838},[555],[42,1245,1246,1249],{"style":841},[42,1247],{"className":1248,"style":655},[563],[42,1250,1252],{"className":1251},[568,569,570,571],[42,1253,429],{"className":1254},[147,571],[42,1256],{"className":1257,"style":154},[153],[42,1259,109],{"className":1260},[158],[42,1262],{"className":1263,"style":154},[153],[42,1265,1267,1271,1274,1277,1280],{"className":1266},[138],[42,1268],{"className":1269,"style":1270},[142],"height:0.7278em;vertical-align:-0.0833em;",[42,1272,426],{"className":1273},[147],[42,1275],{"className":1276,"style":175},[153],[42,1278,471],{"className":1279},[179],[42,1281],{"className":1282,"style":175},[153],[42,1284,1286,1290],{"className":1285},[138],[42,1287],{"className":1288,"style":1289},[142],"height:1.2251em;vertical-align:-0.345em;",[42,1291,1293,1296,1362],{"className":1292},[147],[42,1294],{"className":1295},[507,539],[42,1297,1299],{"className":1298},[421],[42,1300,1302,1354],{"className":1301},[546,547],[42,1303,1305,1351],{"className":1304},[551],[42,1306,1309,1326,1334],{"className":1307,"style":1308},[555],"height:0.8801em;",[42,1310,1311,1314],{"style":559},[42,1312],{"className":1313,"style":564},[563],[42,1315,1317],{"className":1316},[568,569,570,571],[42,1318,1320],{"className":1319},[147,571],[42,1321,1323],{"className":1322},[147,950,571],[42,1324,1212],{"className":1325},[147,571],[42,1327,1328,1331],{"style":580},[42,1329],{"className":1330,"style":564},[563],[42,1332],{"className":1333,"style":588},[587],[42,1335,1336,1339],{"style":591},[42,1337],{"className":1338,"style":564},[563],[42,1340,1342],{"className":1341},[568,569,570,571],[42,1343,1345],{"className":1344},[147,571],[42,1346,1348],{"className":1347},[147,950,571],[42,1349,1209],{"className":1350},[147,571],[42,1352,608],{"className":1353},[607],[42,1355,1357],{"className":1356},[551],[42,1358,1360],{"className":1359,"style":615},[555],[42,1361],{},[42,1363],{"className":1364},[516,539],[11,1366,1367,1368,1371],{},"With guess (1500, 150): R² = 1 - 4646.20\u002F5978.59 = ",[28,1369,1370],{},"0.2229",". Just one variable explaining 22% of the variation is already a start, considering diabetes depends on a lot that isn't even in these 10 columns. And the baseline's own R², by definition, is always zero.",[15,1373,1375],{"id":1374},"scikit-learns-api-built-from-the-inside","Scikit-learn's API, built from the inside",[11,1377,1378,1379,1382,1383,1386],{},"This is where the lecture gets genuinely interesting. Instead of just calling a ready-made ",[23,1380,1381],{},"LinearRegression()",", the professor teaches how to ",[28,1384,1385],{},"build an estimator that follows the same convention",", inheriting from two base classes:",[33,1388,1390],{"className":35,"code":1389,"language":37,"meta":38,"style":38},"from sklearn.base import BaseEstimator, RegressorMixin\n\nclass AverageRegressor(BaseEstimator, RegressorMixin):\n    def fit(self, X, y):\n        self.media_ = np.mean(y)\n        return self\n    def predict(self, X):\n        return np.ones(shape=(X.shape[0],)) * self.media_\n",[23,1391,1392,1397,1401,1406,1411,1416,1421,1426],{"__ignoreMap":38},[42,1393,1394],{"class":44,"line":45},[42,1395,1396],{},"from sklearn.base import BaseEstimator, RegressorMixin\n",[42,1398,1399],{"class":44,"line":51},[42,1400,241],{"emptyLinePlaceholder":240},[42,1402,1403],{"class":44,"line":57},[42,1404,1405],{},"class AverageRegressor(BaseEstimator, RegressorMixin):\n",[42,1407,1408],{"class":44,"line":63},[42,1409,1410],{},"    def fit(self, X, y):\n",[42,1412,1413],{"class":44,"line":249},[42,1414,1415],{},"        self.media_ = np.mean(y)\n",[42,1417,1418],{"class":44,"line":358},[42,1419,1420],{},"        return self\n",[42,1422,1423],{"class":44,"line":363},[42,1424,1425],{},"    def predict(self, X):\n",[42,1427,1428],{"class":44,"line":369},[42,1429,1430],{},"        return np.ones(shape=(X.shape[0],)) * self.media_\n",[11,1432,1433,1436,1437,1440,1441,1444,1445,1448,1449,1452,1453,1456,1457,1460,1461,1464,1465,1472,1473,1475],{},[23,1434,1435],{},"BaseEstimator"," gives you a whole bunch of infrastructure for free (parameter comparison, cloning, integration with ",[23,1438,1439],{},"Pipeline"," and ",[23,1442,1443],{},"GridSearchCV",", things that'll show up later in the course). ",[23,1446,1447],{},"RegressorMixin"," adds the standard ",[23,1450,1451],{},".score()"," method. And notice the underscore convention: ",[23,1454,1455],{},"self.media_",", ending in ",[23,1458,1459],{},"_",", marks \"this was ",[28,1462,1463],{},"learned"," from the data,\" as opposed to a parameter configured by hand. It's the same detail I'd already seen from the outside, ",[112,1466,1467,1468,1471],{"href":294},"using ",[23,1469,1470],{},"SGDRegressor"," in the other playlist",", except now I'm writing the ",[23,1474,1459],{}," with my own hands.",[11,1477,1478,1479,1482],{},"The next step shows something subtle: a regressor with ",[28,1480,1481],{},"fixed"," coefficients, passed in the constructor, learning nothing at all:",[33,1484,1486],{"className":35,"code":1485,"language":37,"meta":38,"style":38},"class LinearRegressor(BaseEstimator, RegressorMixin):\n    def __init__(self, a, b):\n        self.a_ = a\n        self.b_ = b\n    def fit(self, X, y):\n        return self  # learns nothing, the coefficients already arrived ready\n    def predict(self, X):\n        return (self.a_*X + self.b_).reshape(X.shape[0],)\n",[23,1487,1488,1493,1498,1503,1508,1512,1517,1521],{"__ignoreMap":38},[42,1489,1490],{"class":44,"line":45},[42,1491,1492],{},"class LinearRegressor(BaseEstimator, RegressorMixin):\n",[42,1494,1495],{"class":44,"line":51},[42,1496,1497],{},"    def __init__(self, a, b):\n",[42,1499,1500],{"class":44,"line":57},[42,1501,1502],{},"        self.a_ = a\n",[42,1504,1505],{"class":44,"line":63},[42,1506,1507],{},"        self.b_ = b\n",[42,1509,1510],{"class":44,"line":249},[42,1511,1410],{},[42,1513,1514],{"class":44,"line":358},[42,1515,1516],{},"        return self  # learns nothing, the coefficients already arrived ready\n",[42,1518,1519],{"class":44,"line":363},[42,1520,1425],{},[42,1522,1523],{"class":44,"line":369},[42,1524,1525],{},"        return (self.a_*X + self.b_).reshape(X.shape[0],)\n",[11,1527,1528,1529,1152],{},"And then, the version that ",[28,1530,1531],{},"initializes randomly",[33,1533,1535],{"className":35,"code":1534,"language":37,"meta":38,"style":38},"class LinearRegressor(BaseEstimator, RegressorMixin):\n    def fit(self, X, y):\n        self.a_ = np.random.rand()\n        self.b_ = np.random.rand()\n        return self\n    # ...\n",[23,1536,1537,1541,1545,1550,1555,1559],{"__ignoreMap":38},[42,1538,1539],{"class":44,"line":45},[42,1540,1492],{},[42,1542,1543],{"class":44,"line":51},[42,1544,1410],{},[42,1546,1547],{"class":44,"line":57},[42,1548,1549],{},"        self.a_ = np.random.rand()\n",[42,1551,1552],{"class":44,"line":63},[42,1553,1554],{},"        self.b_ = np.random.rand()\n",[42,1556,1557],{"class":44,"line":249},[42,1558,1420],{},[42,1560,1561],{"class":44,"line":358},[42,1562,1563],{},"    # ...\n",[374,1565,1566],{},[11,1567,1568,1571],{},[28,1569,1570],{},"Output:"," MSE of the professor's guess (1500, 150) = 4646.20. MSE of the random coefficient = 29993.30.",[11,1573,1574,1575,1578],{},"That works out to an R² of ",[28,1576,1577],{},"-4.02"," for the random guess. Negative: worse than simply guessing the mean every time. It's proof that \"learning\" isn't magic, it's moving from a bad spot (random) toward a better one, and a random number on its own has no reason to be good.",[15,1580,1582],{"id":1581},"from-random-guess-to-gradient","From random guess to gradient",[11,1584,1585],{},"The last version of the class swaps \"random and stuck\" for \"random and walking\":",[33,1587,1589],{"className":35,"code":1588,"language":37,"meta":38,"style":38},"class LinearRegressor(BaseEstimator, RegressorMixin):\n    def __init__(self, max_iter=1000, learning_rate=0.001):\n        self.max_iter = max_iter\n        self.learning_rate = learning_rate\n\n    def fit(self, X, y):\n        self.coefs_ = np.random.rand(X.shape[1])\n        self.intercept_ = np.random.rand()\n        for i in range(self.max_iter):\n            y_pred = self.predict(X)\n            error = y - y_pred\n            self.coefs_ += X.T @ error * self.learning_rate\n            self.intercept_ += error.sum() * self.learning_rate\n        return self\n\n    def predict(self, X):\n        return (X @ self.coefs_ + self.intercept_).reshape(X.shape[0],)\n",[23,1590,1591,1595,1600,1605,1610,1614,1618,1623,1628,1634,1640,1646,1652,1658,1663,1668,1673],{"__ignoreMap":38},[42,1592,1593],{"class":44,"line":45},[42,1594,1492],{},[42,1596,1597],{"class":44,"line":51},[42,1598,1599],{},"    def __init__(self, max_iter=1000, learning_rate=0.001):\n",[42,1601,1602],{"class":44,"line":57},[42,1603,1604],{},"        self.max_iter = max_iter\n",[42,1606,1607],{"class":44,"line":63},[42,1608,1609],{},"        self.learning_rate = learning_rate\n",[42,1611,1612],{"class":44,"line":249},[42,1613,241],{"emptyLinePlaceholder":240},[42,1615,1616],{"class":44,"line":358},[42,1617,1410],{},[42,1619,1620],{"class":44,"line":363},[42,1621,1622],{},"        self.coefs_ = np.random.rand(X.shape[1])\n",[42,1624,1625],{"class":44,"line":369},[42,1626,1627],{},"        self.intercept_ = np.random.rand()\n",[42,1629,1631],{"class":44,"line":1630},9,[42,1632,1633],{},"        for i in range(self.max_iter):\n",[42,1635,1637],{"class":44,"line":1636},10,[42,1638,1639],{},"            y_pred = self.predict(X)\n",[42,1641,1643],{"class":44,"line":1642},11,[42,1644,1645],{},"            error = y - y_pred\n",[42,1647,1649],{"class":44,"line":1648},12,[42,1650,1651],{},"            self.coefs_ += X.T @ error * self.learning_rate\n",[42,1653,1655],{"class":44,"line":1654},13,[42,1656,1657],{},"            self.intercept_ += error.sum() * self.learning_rate\n",[42,1659,1661],{"class":44,"line":1660},14,[42,1662,1420],{},[42,1664,1666],{"class":44,"line":1665},15,[42,1667,241],{"emptyLinePlaceholder":240},[42,1669,1671],{"class":44,"line":1670},16,[42,1672,1425],{},[42,1674,1676],{"class":44,"line":1675},17,[42,1677,1678],{},"        return (X @ self.coefs_ + self.intercept_).reshape(X.shape[0],)\n",[11,1680,1681,1682,1686,1687,1691,1692,1695,1696,1699,1700,1703],{},"If you've already gone through ",[112,1683,1685],{"href":1684},"\u002Fen\u002Fplaylists\u002Fmachine-learning-specialization\u002Flab04-gradient-descent","the specialization playlist",", this math looks familiar: it's the same batch ",[386,1688,1690],{"definition":1689},"the algorithm that repeatedly adjusts parameters in the direction that most reduces the error, until it stops improving","gradient descent"," as always, ",[23,1693,1694],{},"coefficient += X.T @ error * rate",", just without dividing by the number of examples (which only pushes the effect into the ",[23,1697,1698],{},"learning_rate",", a slightly smaller one compensates for the missing division). This specific shape, updating a weight proportionally to the error times the input, has its own name in the literature: the ",[28,1701,1702],{},"delta rule"," (or Widrow-Hoff rule), one of the oldest learning algorithms that exists, decades before the term \"gradient descent\" caught on.",[11,1705,1706],{},"Run it live below, on the same sample of 45 patients, and watch it start far off and converge toward the ideal fit:",[1708,1709],"gradient-descent-simulator",{":alpha-presets":1710,":alpha-slider-max":1711,":alpha-slider-min":1712,":alpha-slider-step":1712,":b-range":1713,":initial-alpha":1714,":initial-b":272,":initial-w":272,":w-range":1715,":x-train":278,":y-train":279,"b-label":124,"w-label":112},"[0.01, 0.1, 0.5, 1, 1.5]","1.8","0.01","[-100, 300]","0.5","[0, 1000]",[11,1717,1718,1719,1722,1723,1726,1727,1729,1730,1732,1733,1735,1736,1738],{},"On this 45-point sample (smaller than the full 353-patient training set, so the ideal fit here isn't exactly the number the lecture reports), with ",[23,1720,1721],{},"alpha=0.5"," clicking \"Rodar 2000\" already lands close to the bottom of the valley. Try a bigger ",[23,1724,1725],{},"alpha"," too, like 1.5: ",[23,1728,124],{}," starts oscillating before converging, because the scale of ",[23,1731,112],{}," (multiplied by a tiny ",[23,1734,118],{},", between -0.09 and 0.17) and the scale of ",[23,1737,124],{}," (added directly) are quite different, the same scale problem normalization solves, just showing up here hidden inside a dataset that's already normalized.",[11,1740,1741],{},"On the full training set (353 patients), running this same class with a single feature (body mass index):",[374,1743,1744],{},[11,1745,1746,1748,1749,1752,1753,1756],{},[28,1747,1570],{}," ",[23,1750,1751],{},"coefs_ = [529.65]",", ",[23,1754,1755],{},"intercept_ = 154.82",", MSE = 4302.16.",[11,1758,1759,1760,1763,1764,1440,1766,1768],{},"That already beats the best manual guess (4646.20 → 4302.16) and works out to R² = 1 - 4302.16\u002F5978.59 = ",[28,1761,1762],{},"0.2804",". Without me picking ",[23,1765,112],{},[23,1767,124],{}," by hand, the gradient found a better fit on its own.",[15,1770,1772],{"id":1771},"from-one-variable-to-ten","From one variable to ten",[11,1774,1775,1776,1152],{},"The dataset has 10 columns, and so far I've only used one (body mass index). The exact same class, no code changes at all, accepts the full ",[23,1777,1778],{},"X_train",[33,1780,1782],{"className":35,"code":1781,"language":37,"meta":38,"style":38},"regressor = LinearRegressor()\nregressor.fit(X_train, y_train)  # now with all 10 columns\n",[23,1783,1784,1789],{"__ignoreMap":38},[42,1785,1786],{"class":44,"line":45},[42,1787,1788],{},"regressor = LinearRegressor()\n",[42,1790,1791],{"class":44,"line":51},[42,1792,1793],{},"regressor.fit(X_train, y_train)  # now with all 10 columns\n",[374,1795,1796],{},[11,1797,1798,1800,1801,1804],{},[28,1799,1570],{}," MSE = 3027.12 (versus 4302.16 using just one variable). RMSE drops from 65.59 to 55.02. R² climbs from 0.2804 to ",[28,1802,1803],{},"0.4937",".",[11,1806,1807],{},"Almost double the explained variance, just from letting the model see the other 9 variables (age, sex, blood pressure, the six blood measurements) it simply didn't have access to before. Nobody changed the algorithm, only the amount of information it received.",[15,1809,1811],{"id":1810},"wrapping-up","Wrapping up",[1813,1814,1815,1829],"table",{},[1816,1817,1818],"thead",{},[1819,1820,1821,1826],"tr",{},[1822,1823,1825],"th",{"align":1824},"left","What I already knew",[1822,1827,1828],{"align":1824},"What this lecture settled",[1830,1831,1832,1845,1853],"tbody",{},[1819,1833,1834,1838],{},[1835,1836,1837],"td",{"align":1824},"Linear regression fits a line that minimizes error",[1835,1839,1840,259,1842,1844],{"align":1824},[23,1841,1435],{},[23,1843,1447],{}," is the contract that makes any class of mine behave like a real scikit-learn estimator",[1819,1846,1847,1850],{},[1835,1848,1849],{"align":1824},"MAE, MSE, RMSE measure how wrong the model is",[1835,1851,1852],{"align":1824},"Without comparing against a baseline (like guessing the mean), those numbers say nothing on their own: R² fixes that",[1819,1854,1855,1858],{},[1835,1856,1857],{"align":1824},"Gradient descent adjusts weights in the direction that reduces error",[1835,1859,1860],{"align":1824},"That update has its own name in the classic literature (delta rule), and \"learning\" is nothing more than walking from a bad guess in the right direction",[11,1862,1863,1864,1872,1873,1876],{},"Two things stayed open on purpose. First: solving this by iterating, step by step, works, but linear regression has an exact, closed-form solution in a single computation, and that's exactly what ",[112,1865,1867,1868,1871],{"href":1866},"\u002Fen\u002Fplaylists\u002Fpattern-recognition\u002Fnormal-equation","the next lecture (",[23,1869,1870],{},"aula02b",", normal equation) tackles",". Second: I didn't touch the other models that lecture compares (",[23,1874,1875],{},"aula02c","), that's next post material too.",[15,1878,1880],{"id":1879},"practical-application","Practical application",[11,1882,1883],{},"I use the same diabetes dataset that's been running through this whole post, without introducing anything new: the goal here is just to visualize how good the 10-variable fit I just showed actually is.",[33,1885,1887],{"className":35,"code":1886,"language":37,"meta":38,"style":38},"regressor = LinearRegressor(max_iter=1000, learning_rate=0.001)\nregressor.fit(X_train, y_train)\ny_pred = regressor.predict(X_train)\n",[23,1888,1889,1894,1899],{"__ignoreMap":38},[42,1890,1891],{"class":44,"line":45},[42,1892,1893],{},"regressor = LinearRegressor(max_iter=1000, learning_rate=0.001)\n",[42,1895,1896],{"class":44,"line":51},[42,1897,1898],{},"regressor.fit(X_train, y_train)\n",[42,1900,1901],{"class":44,"line":57},[42,1902,1903],{},"y_pred = regressor.predict(X_train)\n",[11,1905,1906,1907,1910],{},"I reproduced this same class (with a different random seed, so the coefficients only match the professor's to the second decimal place, final MSE landed at 3027.43, practically identical to the 3027.12 the lecture reports) to pull out the actual-versus-predicted pairs for 40 patients from the training set. Each point is one patient: the closer to the dashed line (the perfect prediction, ",[23,1908,1909],{},"predicted = actual","), the closer the model got that specific case.",[1912,1913],"predicted-vs-actual-scatter",{":actual":1914,":predicted":1915,"point-label":280,"reference-label":1916,"x-label":1917,"y-label":1918},"[42, 51, 52, 59, 60, 67, 71, 72, 72, 77, 77, 78, 83, 89, 96, 102, 104, 144, 146, 150, 150, 150, 170, 171, 172, 174, 179, 190, 198, 212, 219, 262, 265, 275, 276, 281, 292, 308, 317, 341]","[143.2, 80.1, 78.4, 120.6, 140.3, 183.2, 95.9, 94.7, 92.1, 91.1, 174.5, 86.5, 138.4, 157.2, 132.4, 125.7, 94.1, 165.1, 157.4, 163.9, 111.3, 204.9, 172.6, 183.3, 140.6, 172.7, 175.6, 122.7, 192.1, 198.0, 160.9, 177.3, 185.5, 198.8, 143.2, 236.4, 209.2, 234.2, 230.8, 242.5]","Perfect prediction","actual diabetes progression","predicted progression",[11,1920,1921],{},"You can see the cloud of points tracking the diagonal, but with plenty of spread, exactly what an R² of 0.49 means in practice: the model captured a good chunk of the pattern, but still misses quite a bit case by case. Not bad at all for a plain linear model, no new features, no regularization, just 10 raw numbers and 1000 steps of gradient descent.",[1923,1924,1925],"style",{},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":38,"searchDepth":51,"depth":51,"links":1927},[1928,1929,1930,1931,1932,1933,1934,1935,1936],{"id":17,"depth":51,"text":18},{"id":80,"depth":51,"text":81},{"id":287,"depth":51,"text":288},{"id":1100,"depth":51,"text":1101},{"id":1374,"depth":51,"text":1375},{"id":1581,"depth":51,"text":1582},{"id":1771,"depth":51,"text":1772},{"id":1810,"depth":51,"text":1811},{"id":1879,"depth":51,"text":1880},null,"2026-08-19","Lecture one of my Pattern Recognition course: the professor grabs a real diabetes dataset, hand-picks coefficients, and builds an estimator following scikit-learn's own API. I retell the why behind each step, with Bishop's book alongside.","md",{},"\u002Fen\u002Fplaylists\u002Fpattern-recognition\u002Flinear-regression-estimator","pattern-recognition",{"title":6,"description":1939},"published","en\u002Fplaylists\u002Fpattern-recognition\u002Flinear-regression-estimator",[1948,1949,1950],"linear-regression","scikit-learn","gradient-descent","L_iUimVMvKneVDUVJ45usLRO5OFHAmN8SYGmLbylCqY",1787338984126]