[{"data":1,"prerenderedAt":3773},["ShallowReactive",2],{"lang-switch-post-\u002Fen\u002Fplaylists\u002Fmachine-learning-specialization\u002Fw2-lab05-scikit-learn":3,"post-en-machine-learning-specialization-w2-lab05-scikit-learn":4},"\u002Fplaylists\u002Fmachine-learning-specialization\u002Fw2-lab05-scikit-learn",{"id":5,"title":6,"body":7,"cover":3758,"date":3759,"description":3760,"extension":3761,"meta":3762,"navigation":3702,"order":3763,"path":3764,"playlist":3765,"seo":3766,"status":3767,"stem":3768,"tags":3769,"__hash__":3772},"posts\u002Fen\u002Fplaylists\u002Fmachine-learning-specialization\u002Fw2-lab05-scikit-learn.md","Linear Regression with Scikit-Learn",{"type":8,"value":9,"toc":3742},"minimark",[10,18,21,26,41,137,157,184,191,473,480,491,529,1020,1030,1309,1348,1353,1365,1370,1373,1376,1387,1632,1639,1646,1669,1679,1683,1696,2092,2095,2553,2665,2793,2837,2841,2855,2859,2892,2941,2948,2958,3291,3438,3454,3458,3496,3499,3537,3541,3656,3663,3710,3722,3729,3732,3735,3738],[11,12,13],"p",{},[14,15],"img",{"alt":16,"src":17},"A three-panel \"Rick and Morty\" meme: in the first panel, a robot labeled \"scikit-learn\" asks \"WHAT IS MY PURPOSE?\" In the second, Rick answers \"YOU SPLIT THE DATA.\" In the third, the robot says \"OH MY GOD.\"","\u002Fimages\u002Fposts\u002Fmachine-learning-specialization\u002Fw2-lab05-scikit-learn\u002Fmeme-scikitlearn.jpeg",[11,19,20],{},"After four posts implementing gradient descent, normalization, and feature engineering entirely by hand, it's finally time to use the real tool. And the first shock was finding out the \"default\" model the course uses here isn't quite what it looks like.",[22,23,25],"h2",{"id":24},"the-api-convention-more-important-than-the-model-itself","The API convention, more important than the model itself",[11,27,28,35,36,40],{},[29,30,34],"a",{"href":31,"rel":32},"https:\u002F\u002Fscikit-learn.org\u002F",[33],"nofollow","scikit-learn"," ships ready-made, tested implementations of most of what I've already built by hand. But what matters most to learn here isn't the specific model, it's the ",[37,38,39],"strong",{},"API convention",", because it repeats identically across hundreds of models and transformers:",[42,43,44,61],"table",{},[45,46,47],"thead",{},[48,49,50,55,58],"tr",{},[51,52,54],"th",{"align":53},"left","Method",[51,56,57],{"align":53},"What it does",[51,59,60],{"align":53},"Who has it",[62,63,64,79,92,105,124],"tbody",{},[48,65,66,73,76],{},[67,68,69],"td",{"align":53},[70,71,72],"code",{},".fit(X, y)",[67,74,75],{"align":53},"learns parameters from the data",[67,77,78],{"align":53},"every estimator",[48,80,81,86,89],{},[67,82,83],{"align":53},[70,84,85],{},".predict(X)",[67,87,88],{"align":53},"uses the learned parameters to predict",[67,90,91],{"align":53},"predictive models",[48,93,94,99,102],{},[67,95,96],{"align":53},[70,97,98],{},".transform(X)",[67,100,101],{"align":53},"applies an already-learned transformation",[67,103,104],{"align":53},"transformers",[48,106,107,112,122],{},[67,108,109],{"align":53},[70,110,111],{},".fit_transform(X)",[67,113,114,115,118,119],{"align":53},"shortcut for ",[70,116,117],{},"fit"," followed by ",[70,120,121],{},"transform",[67,123,104],{"align":53},[48,125,126,131,134],{},[67,127,128],{"align":53},[70,129,130],{},".score(X, y)",[67,132,133],{"align":53},"the estimator's default metric (R² for regression)",[67,135,136],{"align":53},"almost all of them",[11,138,139,140,143,144,143,147,143,150,143,153,156],{},"Learned attributes end with a trailing underscore: ",[70,141,142],{},"coef_",", ",[70,145,146],{},"intercept_",[70,148,149],{},"mean_",[70,151,152],{},"scale_",[70,154,155],{},"n_iter_",". That trailing underscore is the convention distinguishing \"learned from the data\" from \"configured by me.\"",[11,158,159,160,162,163,166,167,169,170,173,174,178,179,183],{},"The golden rule that comes with it: ",[70,161,117],{}," only ever gets to see ",[37,164,165],{},"training"," data. On test data, only ",[70,168,121],{}," and ",[70,171,172],{},"predict",". Breaking that is data leakage, and it's the most common mistake for anyone starting out (I already hit this note in the ",[29,175,177],{"href":176},"\u002Fen\u002Fplaylists\u002Fmachine-learning-specialization\u002Ffeature-scaling-pitfalls","previous two"," ",[29,180,182],{"href":181},"\u002Fen\u002Fplaylists\u002Fmachine-learning-specialization\u002Ffeature-engineering-pitfalls","bonus posts",").",[22,185,187,190],{"id":186},"standardscaler-the-z-score-i-already-built-by-hand",[70,188,189],{},"StandardScaler",": the z-score I already built by hand",[11,192,193,195,196,200,201,385,386,415,416,472],{},[70,194,189],{}," does exactly the math I implemented in ",[29,197,199],{"href":198},"\u002Fen\u002Fplaylists\u002Fmachine-learning-specialization\u002Fw2-lab03-feature-scaling","the normalization post",": ",[202,203,206,249],"span",{"className":204},[205],"katex",[202,207,210],{"className":208},[209],"katex-mathml",[211,212,214],"math",{"xmlns":213},"http:\u002F\u002Fwww.w3.org\u002F1998\u002FMath\u002FMathML",[215,216,217,244],"semantics",{},[218,219,220,224,228],"mrow",{},[221,222,223],"mi",{},"x",[225,226,227],"mo",{},"←",[229,230,231,241],"mfrac",{},[218,232,233,235,238],{},[221,234,223],{},[225,236,237],{},"−",[221,239,240],{},"μ",[221,242,243],{},"σ",[245,246,248],"annotation",{"encoding":247},"application\u002Fx-tex","x \\leftarrow \\frac{x-\\mu}{\\sigma}",[202,250,254,280],{"className":251,"ariaHidden":253},[252],"katex-html","true",[202,255,258,263,268,273,277],{"className":256},[257],"base",[202,259],{"className":260,"style":262},[261],"strut","height:0.4306em;",[202,264,223],{"className":265},[266,267],"mord","mathnormal",[202,269],{"className":270,"style":272},[271],"mspace","margin-right:0.2778em;",[202,274,227],{"className":275},[276],"mrel",[202,278],{"className":279,"style":272},[271],[202,281,283,287],{"className":282},[257],[202,284],{"className":285,"style":286},[261],"height:1.1994em;vertical-align:-0.345em;",[202,288,290,295,381],{"className":289},[266],[202,291],{"className":292},[293,294],"mopen","nulldelimiter",[202,296,298],{"className":297},[229],[202,299,303,372],{"className":300},[301,302],"vlist-t","vlist-t2",[202,304,307,367],{"className":305},[306],"vlist-r",[202,308,312,334,345],{"className":309,"style":311},[310],"vlist","height:0.8544em;",[202,313,315,320],{"style":314},"top:-2.655em;",[202,316],{"className":317,"style":319},[318],"pstrut","height:3em;",[202,321,327],{"className":322},[323,324,325,326],"sizing","reset-size6","size3","mtight",[202,328,330],{"className":329},[266,326],[202,331,243],{"className":332,"style":333},[266,267,326],"margin-right:0.0359em;",[202,335,337,340],{"style":336},"top:-3.23em;",[202,338],{"className":339,"style":319},[318],[202,341],{"className":342,"style":344},[343],"frac-line","border-bottom-width:0.04em;",[202,346,348,351],{"style":347},"top:-3.4461em;",[202,349],{"className":350,"style":319},[318],[202,352,354],{"className":353},[323,324,325,326],[202,355,357,360,364],{"className":356},[266,326],[202,358,223],{"className":359},[266,267,326],[202,361,237],{"className":362},[363,326],"mbin",[202,365,240],{"className":366},[266,267,326],[202,368,371],{"className":369},[370],"vlist-s","​",[202,373,375],{"className":374},[306],[202,376,379],{"className":377,"style":378},[310],"height:0.345em;",[202,380],{},[202,382],{"className":383},[384,294],"mclose",", column by column. I checked and it matches my manual z-score digit for digit, including the same population-standard-deviation convention (divide by ",[202,387,389,403],{"className":388},[205],[202,390,392],{"className":391},[209],[211,393,394],{"xmlns":213},[215,395,396,401],{},[218,397,398],{},[221,399,400],{},"m",[245,402,400],{"encoding":247},[202,404,406],{"className":405,"ariaHidden":253},[252],[202,407,409,412],{"className":408},[257],[202,410],{"className":411,"style":262},[261],[202,413,400],{"className":414},[266,267],", not ",[202,417,419,439],{"className":418},[205],[202,420,422],{"className":421},[209],[211,423,424],{"xmlns":213},[215,425,426,436],{},[218,427,428,430,432],{},[221,429,400],{},[225,431,237],{},[433,434,435],"mn",{},"1",[245,437,438],{"encoding":247},"m-1",[202,440,442,462],{"className":441,"ariaHidden":253},[252],[202,443,445,449,452,456,459],{"className":444},[257],[202,446],{"className":447,"style":448},[261],"height:0.6667em;vertical-align:-0.0833em;",[202,450,400],{"className":451},[266,267],[202,453],{"className":454,"style":455},[271],"margin-right:0.2222em;",[202,457,237],{"className":458},[363],[202,460],{"className":461,"style":455},[271],[202,463,465,469],{"className":464},[257],[202,466],{"className":467,"style":468},[261],"height:0.6444em;",[202,470,435],{"className":471},[266],") I was already using.",[22,474,476,479],{"id":475},"sgdregressor-and-what-the-s-means",[70,477,478],{},"SGDRegressor"," and what the \"S\" means",[11,481,482,483,485,486,490],{},"The course uses ",[70,484,478],{}," without ever explaining the acronym. Worth pausing on, because it's the real difference between this model and the gradient descent I implemented ",[29,487,489],{"href":488},"\u002Fen\u002Fplaylists\u002Fmachine-learning-specialization\u002Flab04-gradient-descent","in the previous posts",".",[11,492,493,496,497,178,500,528],{},[37,494,495],{},"Batch gradient descent"," (what I've done so far): every step uses ",[37,498,499],{},"all",[202,501,503,516],{"className":502},[205],[202,504,506],{"className":505},[209],[211,507,508],{"xmlns":213},[215,509,510,514],{},[218,511,512],{},[221,513,400],{},[245,515,400],{"encoding":247},[202,517,519],{"className":518,"ariaHidden":253},[252],[202,520,522,525],{"className":521},[257],[202,523],{"className":524,"style":262},[261],[202,526,400],{"className":527},[266,267]," examples to compute the gradient.",[11,530,531],{},[202,532,534,650],{"className":533},[205],[202,535,537],{"className":536},[209],[211,538,539],{"xmlns":213},[215,540,541,647],{},[218,542,543,547,549,551,553,556,560,566,591,635],{},[221,544,546],{"mathvariant":545},"bold","w",[225,548,227],{},[221,550,546],{"mathvariant":545},[225,552,237],{},[221,554,555],{},"α",[557,558,559],"mtext",{}," ",[229,561,562,564],{},[433,563,435],{},[221,565,400],{},[567,568,569,572,583],"msubsup",{},[225,570,571],{},"∑",[218,573,574,577,580],{},[221,575,576],{},"i",[225,578,579],{},"=",[433,581,582],{},"0",[218,584,585,587,589],{},[221,586,400],{},[225,588,237],{},[433,590,435],{},[218,592,593,596,599,602,616,618,620,633],{},[225,594,595],{"fence":253},"(",[221,597,598],{},"f",[225,600,595],{"stretchy":601},"false",[603,604,605,607],"msup",{},[221,606,223],{"mathvariant":545},[218,608,609,611,613],{},[225,610,595],{"stretchy":601},[221,612,576],{},[225,614,615],{"stretchy":601},")",[225,617,615],{"stretchy":601},[225,619,237],{},[603,621,622,625],{},[221,623,624],{},"y",[218,626,627,629,631],{},[225,628,595],{"stretchy":601},[221,630,576],{},[225,632,615],{"stretchy":601},[225,634,615],{"fence":253},[603,636,637,639],{},[221,638,223],{"mathvariant":545},[218,640,641,643,645],{},[225,642,595],{"stretchy":601},[221,644,576],{},[225,646,615],{"stretchy":601},[245,648,649],{"encoding":247},"\\mathbf{w} \\leftarrow \\mathbf{w} - \\alpha\\,\\frac{1}{m}\\sum_{i=0}^{m-1}\\left(f(\\mathbf{x}^{(i)}) - y^{(i)}\\right)\\mathbf{x}^{(i)}",[202,651,653,674,692],{"className":652,"ariaHidden":253},[252],[202,654,656,660,665,668,671],{"className":655},[257],[202,657],{"className":658,"style":659},[261],"height:0.4444em;",[202,661,546],{"className":662,"style":664},[266,663],"mathbf","margin-right:0.016em;",[202,666],{"className":667,"style":272},[271],[202,669,227],{"className":670},[276],[202,672],{"className":673,"style":272},[271],[202,675,677,680,683,686,689],{"className":676},[257],[202,678],{"className":679,"style":448},[261],[202,681,546],{"className":682,"style":664},[266,663],[202,684],{"className":685,"style":455},[271],[202,687,237],{"className":688},[363],[202,690],{"className":691,"style":455},[271],[202,693,695,699,703,707,777,780,859,862,979,982],{"className":694},[257],[202,696],{"className":697,"style":698},[261],"height:1.304em;vertical-align:-0.35em;",[202,700,555],{"className":701,"style":702},[266,267],"margin-right:0.0037em;",[202,704],{"className":705,"style":706},[271],"margin-right:0.1667em;",[202,708,710,713,774],{"className":709},[266],[202,711],{"className":712},[293,294],[202,714,716],{"className":715},[229],[202,717,719,766],{"className":718},[301,302],[202,720,722,763],{"className":721},[306],[202,723,726,740,748],{"className":724,"style":725},[310],"height:0.8451em;",[202,727,728,731],{"style":314},[202,729],{"className":730,"style":319},[318],[202,732,734],{"className":733},[323,324,325,326],[202,735,737],{"className":736},[266,326],[202,738,400],{"className":739},[266,267,326],[202,741,742,745],{"style":336},[202,743],{"className":744,"style":319},[318],[202,746],{"className":747,"style":344},[343],[202,749,751,754],{"style":750},"top:-3.394em;",[202,752],{"className":753,"style":319},[318],[202,755,757],{"className":756},[323,324,325,326],[202,758,760],{"className":759},[266,326],[202,761,435],{"className":762},[266,326],[202,764,371],{"className":765},[370],[202,767,769],{"className":768},[306],[202,770,772],{"className":771,"style":378},[310],[202,773],{},[202,775],{"className":776},[384,294],[202,778],{"className":779,"style":706},[271],[202,781,784,790],{"className":782},[783],"mop",[202,785,571],{"className":786,"style":789},[783,787,788],"op-symbol","small-op","position:relative;top:0em;",[202,791,794],{"className":792},[793],"msupsub",[202,795,797,850],{"className":796},[301,302],[202,798,800,847],{"className":799},[306],[202,801,804,826],{"className":802,"style":803},[310],"height:0.954em;",[202,805,807,811],{"style":806},"top:-2.4003em;margin-left:0em;margin-right:0.05em;",[202,808],{"className":809,"style":810},[318],"height:2.7em;",[202,812,814],{"className":813},[323,324,325,326],[202,815,817,820,823],{"className":816},[266,326],[202,818,576],{"className":819},[266,267,326],[202,821,579],{"className":822},[276,326],[202,824,582],{"className":825},[266,326],[202,827,829,832],{"style":828},"top:-3.2029em;margin-right:0.05em;",[202,830],{"className":831,"style":810},[318],[202,833,835],{"className":834},[323,324,325,326],[202,836,838,841,844],{"className":837},[266,326],[202,839,400],{"className":840},[266,267,326],[202,842,237],{"className":843},[363,326],[202,845,435],{"className":846},[266,326],[202,848,371],{"className":849},[370],[202,851,853],{"className":852},[306],[202,854,857],{"className":855,"style":856},[310],"height:0.2997em;",[202,858],{},[202,860],{"className":861,"style":706},[271],[202,863,866,876,880,883,923,926,929,932,935,973],{"className":864},[865],"minner",[202,867,871],{"className":868,"style":870},[293,869],"delimcenter","top:0em;",[202,872,595],{"className":873},[874,875],"delimsizing","size1",[202,877,598],{"className":878,"style":879},[266,267],"margin-right:0.1076em;",[202,881,595],{"className":882},[293],[202,884,886,889],{"className":885},[266],[202,887,223],{"className":888},[266,663],[202,890,892],{"className":891},[793],[202,893,895],{"className":894},[301],[202,896,898],{"className":897},[306],[202,899,902],{"className":900,"style":901},[310],"height:0.888em;",[202,903,905,908],{"style":904},"top:-3.063em;margin-right:0.05em;",[202,906],{"className":907,"style":810},[318],[202,909,911],{"className":910},[323,324,325,326],[202,912,914,917,920],{"className":913},[266,326],[202,915,595],{"className":916},[293,326],[202,918,576],{"className":919},[266,267,326],[202,921,615],{"className":922},[384,326],[202,924,615],{"className":925},[384],[202,927],{"className":928,"style":455},[271],[202,930,237],{"className":931},[363],[202,933],{"className":934,"style":455},[271],[202,936,938,941],{"className":937},[266],[202,939,624],{"className":940,"style":333},[266,267],[202,942,944],{"className":943},[793],[202,945,947],{"className":946},[301],[202,948,950],{"className":949},[306],[202,951,953],{"className":952,"style":901},[310],[202,954,955,958],{"style":904},[202,956],{"className":957,"style":810},[318],[202,959,961],{"className":960},[323,324,325,326],[202,962,964,967,970],{"className":963},[266,326],[202,965,595],{"className":966},[293,326],[202,968,576],{"className":969},[266,267,326],[202,971,615],{"className":972},[384,326],[202,974,976],{"className":975,"style":870},[384,869],[202,977,615],{"className":978},[874,875],[202,980],{"className":981,"style":706},[271],[202,983,985,988],{"className":984},[266],[202,986,223],{"className":987},[266,663],[202,989,991],{"className":990},[793],[202,992,994],{"className":993},[301],[202,995,997],{"className":996},[306],[202,998,1000],{"className":999,"style":901},[310],[202,1001,1002,1005],{"style":904},[202,1003],{"className":1004,"style":810},[318],[202,1006,1008],{"className":1007},[323,324,325,326],[202,1009,1011,1014,1017],{"className":1010},[266,326],[202,1012,595],{"className":1013},[293,326],[202,1015,576],{"className":1016},[266,267,326],[202,1018,615],{"className":1019},[384,326],[11,1021,1022,1025,1026,1029],{},[37,1023,1024],{},"Stochastic gradient descent (SGD)",": every step uses a ",[37,1027,1028],{},"single"," example, picked at random.",[11,1031,1032],{},[202,1033,1035,1107],{"className":1034},[205],[202,1036,1038],{"className":1037},[209],[211,1039,1040],{"xmlns":213},[215,1041,1042,1104],{},[218,1043,1044,1046,1048,1050,1052,1054,1092],{},[221,1045,546],{"mathvariant":545},[225,1047,227],{},[221,1049,546],{"mathvariant":545},[225,1051,237],{},[221,1053,555],{},[218,1055,1056,1058,1060,1062,1074,1076,1078,1090],{},[225,1057,595],{"fence":253},[221,1059,598],{},[225,1061,595],{"stretchy":601},[603,1063,1064,1066],{},[221,1065,223],{"mathvariant":545},[218,1067,1068,1070,1072],{},[225,1069,595],{"stretchy":601},[221,1071,576],{},[225,1073,615],{"stretchy":601},[225,1075,615],{"stretchy":601},[225,1077,237],{},[603,1079,1080,1082],{},[221,1081,624],{},[218,1083,1084,1086,1088],{},[225,1085,595],{"stretchy":601},[221,1087,576],{},[225,1089,615],{"stretchy":601},[225,1091,615],{"fence":253},[603,1093,1094,1096],{},[221,1095,223],{"mathvariant":545},[218,1097,1098,1100,1102],{},[225,1099,595],{"stretchy":601},[221,1101,576],{},[225,1103,615],{"stretchy":601},[245,1105,1106],{"encoding":247},"\\mathbf{w} \\leftarrow \\mathbf{w} - \\alpha\\left(f(\\mathbf{x}^{(i)}) - y^{(i)}\\right)\\mathbf{x}^{(i)}",[202,1108,1110,1128,1146],{"className":1109,"ariaHidden":253},[252],[202,1111,1113,1116,1119,1122,1125],{"className":1112},[257],[202,1114],{"className":1115,"style":659},[261],[202,1117,546],{"className":1118,"style":664},[266,663],[202,1120],{"className":1121,"style":272},[271],[202,1123,227],{"className":1124},[276],[202,1126],{"className":1127,"style":272},[271],[202,1129,1131,1134,1137,1140,1143],{"className":1130},[257],[202,1132],{"className":1133,"style":448},[261],[202,1135,546],{"className":1136,"style":664},[266,663],[202,1138],{"className":1139,"style":455},[271],[202,1141,237],{"className":1142},[363],[202,1144],{"className":1145,"style":455},[271],[202,1147,1149,1153,1156,1159,1268,1271],{"className":1148},[257],[202,1150],{"className":1151,"style":1152},[261],"height:1.238em;vertical-align:-0.35em;",[202,1154,555],{"className":1155,"style":702},[266,267],[202,1157],{"className":1158,"style":706},[271],[202,1160,1162,1168,1171,1174,1212,1215,1218,1221,1224,1262],{"className":1161},[865],[202,1163,1165],{"className":1164,"style":870},[293,869],[202,1166,595],{"className":1167},[874,875],[202,1169,598],{"className":1170,"style":879},[266,267],[202,1172,595],{"className":1173},[293],[202,1175,1177,1180],{"className":1176},[266],[202,1178,223],{"className":1179},[266,663],[202,1181,1183],{"className":1182},[793],[202,1184,1186],{"className":1185},[301],[202,1187,1189],{"className":1188},[306],[202,1190,1192],{"className":1191,"style":901},[310],[202,1193,1194,1197],{"style":904},[202,1195],{"className":1196,"style":810},[318],[202,1198,1200],{"className":1199},[323,324,325,326],[202,1201,1203,1206,1209],{"className":1202},[266,326],[202,1204,595],{"className":1205},[293,326],[202,1207,576],{"className":1208},[266,267,326],[202,1210,615],{"className":1211},[384,326],[202,1213,615],{"className":1214},[384],[202,1216],{"className":1217,"style":455},[271],[202,1219,237],{"className":1220},[363],[202,1222],{"className":1223,"style":455},[271],[202,1225,1227,1230],{"className":1226},[266],[202,1228,624],{"className":1229,"style":333},[266,267],[202,1231,1233],{"className":1232},[793],[202,1234,1236],{"className":1235},[301],[202,1237,1239],{"className":1238},[306],[202,1240,1242],{"className":1241,"style":901},[310],[202,1243,1244,1247],{"style":904},[202,1245],{"className":1246,"style":810},[318],[202,1248,1250],{"className":1249},[323,324,325,326],[202,1251,1253,1256,1259],{"className":1252},[266,326],[202,1254,595],{"className":1255},[293,326],[202,1257,576],{"className":1258},[266,267,326],[202,1260,615],{"className":1261},[384,326],[202,1263,1265],{"className":1264,"style":870},[384,869],[202,1266,615],{"className":1267},[874,875],[202,1269],{"className":1270,"style":706},[271],[202,1272,1274,1277],{"className":1273},[266],[202,1275,223],{"className":1276},[266,663],[202,1278,1280],{"className":1279},[793],[202,1281,1283],{"className":1282},[301],[202,1284,1286],{"className":1285},[306],[202,1287,1289],{"className":1288,"style":901},[310],[202,1290,1291,1294],{"style":904},[202,1292],{"className":1293,"style":810},[318],[202,1295,1297],{"className":1296},[323,324,325,326],[202,1298,1300,1303,1306],{"className":1299},[266,326],[202,1301,595],{"className":1302},[293,326],[202,1304,576],{"className":1305},[266,267,326],[202,1307,615],{"className":1308},[384,326],[11,1310,1311,1312,1340,1341,1344,1345,1347],{},"Dirt-cheap step, noisy direction, a zigzag path, but ",[202,1313,1315,1328],{"className":1314},[205],[202,1316,1318],{"className":1317},[209],[211,1319,1320],{"xmlns":213},[215,1321,1322,1326],{},[218,1323,1324],{},[221,1325,400],{},[245,1327,400],{"encoding":247},[202,1329,1331],{"className":1330,"ariaHidden":253},[252],[202,1332,1334,1337],{"className":1333},[257],[202,1335],{"className":1336,"style":262},[261],[202,1338,400],{"className":1339},[266,267]," updates for the price of one batch step. I built an SGD engine from scratch (the same ",[70,1342,1343],{},"GradientDescentSimulator"," as always, just swapping the gear underneath) so you can see the difference live, on the same 8-house dataset from ",[29,1346,199],{"href":198},":",[11,1349,1350],{},[37,1351,1352],{},"Batch:",[1354,1355],"gradient-descent-simulator",{":alpha-slider-max":1356,":alpha-slider-min":1357,":alpha-slider-step":1357,":b-range":1358,":initial-alpha":1359,":initial-b":582,":initial-w":582,":w-range":1360,":x-train":1361,":y-train":1362,"b-label":1363,"mode":1364,"w-label":546},"2e-6","1e-8","[-250, 250]","8e-7","[0, 0.6]","[952, 1244, 1947, 1725, 1959, 1314, 864, 1836]","[271.5, 300, 509.8, 394, 540, 415, 230, 560]","b","batch",[11,1366,1367],{},[37,1368,1369],{},"Stochastic:",[1354,1371],{":alpha-slider-max":1356,":alpha-slider-min":1357,":alpha-slider-step":1357,":b-range":1358,":initial-alpha":1359,":initial-b":582,":initial-w":582,":w-range":1360,":x-train":1361,":y-train":1362,"b-label":1363,"mode":1372,"w-label":546},"stochastic",[11,1374,1375],{},"Click \"Rodar 100\" on both with the same default alpha. After 50 steps, batch already has cost near 922 (very close to the real minimum, 919), while stochastic, with the same number of steps but each one only seeing one house at a time, is still bouncing around 7102. The stochastic path on the chart is visibly \"messier\" too, back and forth instead of a smooth descent.",[11,1377,1378,1379,1382,1383,1386],{},"One full pass through the examples is called an ",[37,1380,1381],{},"epoch",". In scikit-learn, ",[70,1384,1385],{},"max_iter"," counts epochs, not individual updates.",[42,1388,1389,1401],{},[45,1390,1391],{},[48,1392,1393,1395,1398],{},[51,1394],{"align":53},[51,1396,1397],{"align":53},"Batch GD",[51,1399,1400],{"align":53},"SGD",[62,1402,1403,1506,1543,1554,1565],{},[48,1404,1405,1408,1460],{},[67,1406,1407],{"align":53},"cost per update",[67,1409,1410],{"align":53},[202,1411,1413,1437],{"className":1412},[205],[202,1414,1416],{"className":1415},[209],[211,1417,1418],{"xmlns":213},[215,1419,1420,1434],{},[218,1421,1422,1425,1427,1429,1432],{},[221,1423,1424],{},"O",[225,1426,595],{"stretchy":601},[221,1428,400],{},[221,1430,1431],{},"n",[225,1433,615],{"stretchy":601},[245,1435,1436],{"encoding":247},"O(mn)",[202,1438,1440],{"className":1439,"ariaHidden":253},[252],[202,1441,1443,1447,1451,1454,1457],{"className":1442},[257],[202,1444],{"className":1445,"style":1446},[261],"height:1em;vertical-align:-0.25em;",[202,1448,1424],{"className":1449,"style":1450},[266,267],"margin-right:0.0278em;",[202,1452,595],{"className":1453},[293],[202,1455,433],{"className":1456},[266,267],[202,1458,615],{"className":1459},[384],[67,1461,1462],{"align":53},[202,1463,1465,1485],{"className":1464},[205],[202,1466,1468],{"className":1467},[209],[211,1469,1470],{"xmlns":213},[215,1471,1472,1482],{},[218,1473,1474,1476,1478,1480],{},[221,1475,1424],{},[225,1477,595],{"stretchy":601},[221,1479,1431],{},[225,1481,615],{"stretchy":601},[245,1483,1484],{"encoding":247},"O(n)",[202,1486,1488],{"className":1487,"ariaHidden":253},[252],[202,1489,1491,1494,1497,1500,1503],{"className":1490},[257],[202,1492],{"className":1493,"style":1446},[261],[202,1495,1424],{"className":1496,"style":1450},[266,267],[202,1498,595],{"className":1499},[293],[202,1501,1431],{"className":1502},[266,267],[202,1504,615],{"className":1505},[384],[48,1507,1508,1511,1513],{},[67,1509,1510],{"align":53},"updates per epoch",[67,1512,435],{"align":53},[67,1514,1515],{"align":53},[202,1516,1518,1531],{"className":1517},[205],[202,1519,1521],{"className":1520},[209],[211,1522,1523],{"xmlns":213},[215,1524,1525,1529],{},[218,1526,1527],{},[221,1528,400],{},[245,1530,400],{"encoding":247},[202,1532,1534],{"className":1533,"ariaHidden":253},[252],[202,1535,1537,1540],{"className":1536},[257],[202,1538],{"className":1539,"style":262},[261],[202,1541,400],{"className":1542},[266,267],[48,1544,1545,1548,1551],{},[67,1546,1547],{"align":53},"trajectory",[67,1549,1550],{"align":53},"smooth",[67,1552,1553],{"align":53},"noisy",[48,1555,1556,1559,1562],{},[67,1557,1558],{"align":53},"deterministic?",[67,1560,1561],{"align":53},"yes",[67,1563,1564],{"align":53},"no (depends on the sampled order)",[48,1566,1567,1570,1601],{},[67,1568,1569],{"align":53},"good when",[67,1571,1572,1573],{"align":53},"small\u002Fmedium ",[202,1574,1576,1589],{"className":1575},[205],[202,1577,1579],{"className":1578},[209],[211,1580,1581],{"xmlns":213},[215,1582,1583,1587],{},[218,1584,1585],{},[221,1586,400],{},[245,1588,400],{"encoding":247},[202,1590,1592],{"className":1591,"ariaHidden":253},[252],[202,1593,1595,1598],{"className":1594},[257],[202,1596],{"className":1597,"style":262},[261],[202,1599,400],{"className":1600},[266,267],[67,1602,1603,1604],{"align":53},"very large ",[202,1605,1607,1620],{"className":1606},[205],[202,1608,1610],{"className":1609},[209],[211,1611,1612],{"xmlns":213},[215,1613,1614,1618],{},[218,1615,1616],{},[221,1617,400],{},[245,1619,400],{"encoding":247},[202,1621,1623],{"className":1622,"ariaHidden":253},[252],[202,1624,1626,1629],{"className":1625},[257],[202,1627],{"className":1628,"style":262},[261],[202,1630,400],{"className":1631},[266,267],[11,1633,1634,1635,1638],{},"With a constant learning rate, SGD never fully stops trembling around the minimum. That's why scikit-learn defaults to a rate that ",[37,1636,1637],{},"decreases"," over training.",[22,1640,1642,1643,1645],{"id":1641},"two-things-max_iter-hides","Two things ",[70,1644,1385],{}," hides",[11,1647,1648,1649,169,1652,1654,1655,169,1658,1661,1662,1664,1665,1668],{},"I ask for ",[70,1650,1651],{},"max_iter=1000",[70,1653,478],{}," usually stops well before that. Not a bug: there's early stopping (",[70,1656,1657],{},"tol",[70,1659,1660],{},"n_iter_no_change",") that detects when the loss has genuinely stopped improving and halts on its own. ",[70,1663,1385],{}," is a ",[37,1666,1667],{},"ceiling",", not a target.",[11,1670,1671,1672,1675,1676,1678],{},"And without fixing ",[70,1673,1674],{},"random_state",", every ",[70,1677,117],{}," samples a different order for the examples, and the result changes run to run. The variation is usually small, but real, especially on smaller or harder datasets. Fixing the seed is what makes the result reproducible.",[22,1680,1682],{"id":1681},"the-surprise-by-default-this-is-ridge-not-plain-least-squares","The surprise: by default, this is Ridge, not plain least squares",[11,1684,1685,1686,1688,1689,169,1692,1695],{},"This is the one that caught me most off guard. ",[70,1687,478],{},"'s defaults are ",[70,1690,1691],{},"penalty='l2'",[70,1693,1694],{},"alpha=0.0001",". In other words, by default it doesn't minimize",[11,1697,1698],{},[202,1699,1701,1789],{"className":1700},[205],[202,1702,1704],{"className":1703},[209],[211,1705,1706],{"xmlns":213},[215,1707,1708,1786],{},[218,1709,1710,1713,1715,1717,1720,1722,1724,1726,1737,1744],{},[221,1711,1712],{},"J",[225,1714,595],{"stretchy":601},[221,1716,546],{"mathvariant":545},[225,1718,1719],{"separator":253},",",[221,1721,1363],{},[225,1723,615],{"stretchy":601},[225,1725,579],{},[229,1727,1728,1730],{},[433,1729,435],{},[218,1731,1732,1735],{},[433,1733,1734],{},"2",[221,1736,400],{},[1738,1739,1740,1742],"msub",{},[225,1741,571],{},[221,1743,576],{},[603,1745,1746,1784],{},[218,1747,1748,1750,1752,1754,1766,1768,1770,1782],{},[225,1749,595],{"fence":253},[221,1751,598],{},[225,1753,595],{"stretchy":601},[603,1755,1756,1758],{},[221,1757,223],{"mathvariant":545},[218,1759,1760,1762,1764],{},[225,1761,595],{"stretchy":601},[221,1763,576],{},[225,1765,615],{"stretchy":601},[225,1767,615],{"stretchy":601},[225,1769,237],{},[603,1771,1772,1774],{},[221,1773,624],{},[218,1775,1776,1778,1780],{},[225,1777,595],{"stretchy":601},[221,1779,576],{},[225,1781,615],{"stretchy":601},[225,1783,615],{"fence":253},[433,1785,1734],{},[245,1787,1788],{"encoding":247},"J(\\mathbf{w},b) = \\frac{1}{2m}\\sum_i \\left(f(\\mathbf{x}^{(i)}) - y^{(i)}\\right)^2",[202,1790,1792,1830],{"className":1791,"ariaHidden":253},[252],[202,1793,1795,1798,1802,1805,1808,1812,1815,1818,1821,1824,1827],{"className":1794},[257],[202,1796],{"className":1797,"style":1446},[261],[202,1799,1712],{"className":1800,"style":1801},[266,267],"margin-right:0.0962em;",[202,1803,595],{"className":1804},[293],[202,1806,546],{"className":1807,"style":664},[266,663],[202,1809,1719],{"className":1810},[1811],"mpunct",[202,1813],{"className":1814,"style":706},[271],[202,1816,1363],{"className":1817},[266,267],[202,1819,615],{"className":1820},[384],[202,1822],{"className":1823,"style":272},[271],[202,1825,579],{"className":1826},[276],[202,1828],{"className":1829,"style":272},[271],[202,1831,1833,1837,1908,1911,1952,1955],{"className":1832},[257],[202,1834],{"className":1835,"style":1836},[261],"height:1.442em;vertical-align:-0.35em;",[202,1838,1840,1843,1905],{"className":1839},[266],[202,1841],{"className":1842},[293,294],[202,1844,1846],{"className":1845},[229],[202,1847,1849,1897],{"className":1848},[301,302],[202,1850,1852,1894],{"className":1851},[306],[202,1853,1855,1872,1880],{"className":1854,"style":725},[310],[202,1856,1857,1860],{"style":314},[202,1858],{"className":1859,"style":319},[318],[202,1861,1863],{"className":1862},[323,324,325,326],[202,1864,1866,1869],{"className":1865},[266,326],[202,1867,1734],{"className":1868},[266,326],[202,1870,400],{"className":1871},[266,267,326],[202,1873,1874,1877],{"style":336},[202,1875],{"className":1876,"style":319},[318],[202,1878],{"className":1879,"style":344},[343],[202,1881,1882,1885],{"style":750},[202,1883],{"className":1884,"style":319},[318],[202,1886,1888],{"className":1887},[323,324,325,326],[202,1889,1891],{"className":1890},[266,326],[202,1892,435],{"className":1893},[266,326],[202,1895,371],{"className":1896},[370],[202,1898,1900],{"className":1899},[306],[202,1901,1903],{"className":1902,"style":378},[310],[202,1904],{},[202,1906],{"className":1907},[384,294],[202,1909],{"className":1910,"style":706},[271],[202,1912,1914,1917],{"className":1913},[783],[202,1915,571],{"className":1916,"style":789},[783,787,788],[202,1918,1920],{"className":1919},[793],[202,1921,1923,1944],{"className":1922},[301,302],[202,1924,1926,1941],{"className":1925},[306],[202,1927,1930],{"className":1928,"style":1929},[310],"height:0.162em;",[202,1931,1932,1935],{"style":806},[202,1933],{"className":1934,"style":810},[318],[202,1936,1938],{"className":1937},[323,324,325,326],[202,1939,576],{"className":1940},[266,267,326],[202,1942,371],{"className":1943},[370],[202,1945,1947],{"className":1946},[306],[202,1948,1950],{"className":1949,"style":856},[310],[202,1951],{},[202,1953],{"className":1954,"style":706},[271],[202,1956,1958,2067],{"className":1957},[865],[202,1959,1961,1967,1970,1973,2011,2014,2017,2020,2023,2061],{"className":1960},[865],[202,1962,1964],{"className":1963,"style":870},[293,869],[202,1965,595],{"className":1966},[874,875],[202,1968,598],{"className":1969,"style":879},[266,267],[202,1971,595],{"className":1972},[293],[202,1974,1976,1979],{"className":1975},[266],[202,1977,223],{"className":1978},[266,663],[202,1980,1982],{"className":1981},[793],[202,1983,1985],{"className":1984},[301],[202,1986,1988],{"className":1987},[306],[202,1989,1991],{"className":1990,"style":901},[310],[202,1992,1993,1996],{"style":904},[202,1994],{"className":1995,"style":810},[318],[202,1997,1999],{"className":1998},[323,324,325,326],[202,2000,2002,2005,2008],{"className":2001},[266,326],[202,2003,595],{"className":2004},[293,326],[202,2006,576],{"className":2007},[266,267,326],[202,2009,615],{"className":2010},[384,326],[202,2012,615],{"className":2013},[384],[202,2015],{"className":2016,"style":455},[271],[202,2018,237],{"className":2019},[363],[202,2021],{"className":2022,"style":455},[271],[202,2024,2026,2029],{"className":2025},[266],[202,2027,624],{"className":2028,"style":333},[266,267],[202,2030,2032],{"className":2031},[793],[202,2033,2035],{"className":2034},[301],[202,2036,2038],{"className":2037},[306],[202,2039,2041],{"className":2040,"style":901},[310],[202,2042,2043,2046],{"style":904},[202,2044],{"className":2045,"style":810},[318],[202,2047,2049],{"className":2048},[323,324,325,326],[202,2050,2052,2055,2058],{"className":2051},[266,326],[202,2053,595],{"className":2054},[293,326],[202,2056,576],{"className":2057},[266,267,326],[202,2059,615],{"className":2060},[384,326],[202,2062,2064],{"className":2063,"style":870},[384,869],[202,2065,615],{"className":2066},[874,875],[202,2068,2070],{"className":2069},[793],[202,2071,2073],{"className":2072},[301],[202,2074,2076],{"className":2075},[306],[202,2077,2080],{"className":2078,"style":2079},[310],"height:1.092em;",[202,2081,2083,2086],{"style":2082},"top:-3.3409em;margin-right:0.05em;",[202,2084],{"className":2085,"style":810},[318],[202,2087,2089],{"className":2088},[323,324,325,326],[202,2090,1734],{"className":2091},[266,326],[11,2093,2094],{},"it minimizes",[11,2096,2097],{},[202,2098,2100,2201],{"className":2099},[205],[202,2101,2103],{"className":2102},[209],[211,2104,2105],{"xmlns":213},[215,2106,2107,2198],{},[218,2108,2109,2111,2113,2115,2117,2119,2121,2123,2133,2139,2181,2184,2186,2190,2192],{},[221,2110,1712],{},[225,2112,595],{"stretchy":601},[221,2114,546],{"mathvariant":545},[225,2116,1719],{"separator":253},[221,2118,1363],{},[225,2120,615],{"stretchy":601},[225,2122,579],{},[229,2124,2125,2127],{},[433,2126,435],{},[218,2128,2129,2131],{},[433,2130,1734],{},[221,2132,400],{},[1738,2134,2135,2137],{},[225,2136,571],{},[221,2138,576],{},[603,2140,2141,2179],{},[218,2142,2143,2145,2147,2149,2161,2163,2165,2177],{},[225,2144,595],{"fence":253},[221,2146,598],{},[225,2148,595],{"stretchy":601},[603,2150,2151,2153],{},[221,2152,223],{"mathvariant":545},[218,2154,2155,2157,2159],{},[225,2156,595],{"stretchy":601},[221,2158,576],{},[225,2160,615],{"stretchy":601},[225,2162,615],{"stretchy":601},[225,2164,237],{},[603,2166,2167,2169],{},[221,2168,624],{},[218,2170,2171,2173,2175],{},[225,2172,595],{"stretchy":601},[221,2174,576],{},[225,2176,615],{"stretchy":601},[225,2178,615],{"fence":253},[433,2180,1734],{},[225,2182,2183],{},"+",[221,2185,555],{},[221,2187,2189],{"mathvariant":2188},"normal","∥",[221,2191,546],{"mathvariant":545},[603,2193,2194,2196],{},[221,2195,2189],{"mathvariant":2188},[433,2197,1734],{},[245,2199,2200],{"encoding":247},"J(\\mathbf{w},b) = \\frac{1}{2m}\\sum_i \\left(f(\\mathbf{x}^{(i)}) - y^{(i)}\\right)^2 + \\alpha\\|\\mathbf{w}\\|^2",[202,2202,2204,2240,2507],{"className":2203,"ariaHidden":253},[252],[202,2205,2207,2210,2213,2216,2219,2222,2225,2228,2231,2234,2237],{"className":2206},[257],[202,2208],{"className":2209,"style":1446},[261],[202,2211,1712],{"className":2212,"style":1801},[266,267],[202,2214,595],{"className":2215},[293],[202,2217,546],{"className":2218,"style":664},[266,663],[202,2220,1719],{"className":2221},[1811],[202,2223],{"className":2224,"style":706},[271],[202,2226,1363],{"className":2227},[266,267],[202,2229,615],{"className":2230},[384],[202,2232],{"className":2233,"style":272},[271],[202,2235,579],{"className":2236},[276],[202,2238],{"className":2239,"style":272},[271],[202,2241,2243,2246,2317,2320,2360,2363,2498,2501,2504],{"className":2242},[257],[202,2244],{"className":2245,"style":1836},[261],[202,2247,2249,2252,2314],{"className":2248},[266],[202,2250],{"className":2251},[293,294],[202,2253,2255],{"className":2254},[229],[202,2256,2258,2306],{"className":2257},[301,302],[202,2259,2261,2303],{"className":2260},[306],[202,2262,2264,2281,2289],{"className":2263,"style":725},[310],[202,2265,2266,2269],{"style":314},[202,2267],{"className":2268,"style":319},[318],[202,2270,2272],{"className":2271},[323,324,325,326],[202,2273,2275,2278],{"className":2274},[266,326],[202,2276,1734],{"className":2277},[266,326],[202,2279,400],{"className":2280},[266,267,326],[202,2282,2283,2286],{"style":336},[202,2284],{"className":2285,"style":319},[318],[202,2287],{"className":2288,"style":344},[343],[202,2290,2291,2294],{"style":750},[202,2292],{"className":2293,"style":319},[318],[202,2295,2297],{"className":2296},[323,324,325,326],[202,2298,2300],{"className":2299},[266,326],[202,2301,435],{"className":2302},[266,326],[202,2304,371],{"className":2305},[370],[202,2307,2309],{"className":2308},[306],[202,2310,2312],{"className":2311,"style":378},[310],[202,2313],{},[202,2315],{"className":2316},[384,294],[202,2318],{"className":2319,"style":706},[271],[202,2321,2323,2326],{"className":2322},[783],[202,2324,571],{"className":2325,"style":789},[783,787,788],[202,2327,2329],{"className":2328},[793],[202,2330,2332,2352],{"className":2331},[301,302],[202,2333,2335,2349],{"className":2334},[306],[202,2336,2338],{"className":2337,"style":1929},[310],[202,2339,2340,2343],{"style":806},[202,2341],{"className":2342,"style":810},[318],[202,2344,2346],{"className":2345},[323,324,325,326],[202,2347,576],{"className":2348},[266,267,326],[202,2350,371],{"className":2351},[370],[202,2353,2355],{"className":2354},[306],[202,2356,2358],{"className":2357,"style":856},[310],[202,2359],{},[202,2361],{"className":2362,"style":706},[271],[202,2364,2366,2475],{"className":2365},[865],[202,2367,2369,2375,2378,2381,2419,2422,2425,2428,2431,2469],{"className":2368},[865],[202,2370,2372],{"className":2371,"style":870},[293,869],[202,2373,595],{"className":2374},[874,875],[202,2376,598],{"className":2377,"style":879},[266,267],[202,2379,595],{"className":2380},[293],[202,2382,2384,2387],{"className":2383},[266],[202,2385,223],{"className":2386},[266,663],[202,2388,2390],{"className":2389},[793],[202,2391,2393],{"className":2392},[301],[202,2394,2396],{"className":2395},[306],[202,2397,2399],{"className":2398,"style":901},[310],[202,2400,2401,2404],{"style":904},[202,2402],{"className":2403,"style":810},[318],[202,2405,2407],{"className":2406},[323,324,325,326],[202,2408,2410,2413,2416],{"className":2409},[266,326],[202,2411,595],{"className":2412},[293,326],[202,2414,576],{"className":2415},[266,267,326],[202,2417,615],{"className":2418},[384,326],[202,2420,615],{"className":2421},[384],[202,2423],{"className":2424,"style":455},[271],[202,2426,237],{"className":2427},[363],[202,2429],{"className":2430,"style":455},[271],[202,2432,2434,2437],{"className":2433},[266],[202,2435,624],{"className":2436,"style":333},[266,267],[202,2438,2440],{"className":2439},[793],[202,2441,2443],{"className":2442},[301],[202,2444,2446],{"className":2445},[306],[202,2447,2449],{"className":2448,"style":901},[310],[202,2450,2451,2454],{"style":904},[202,2452],{"className":2453,"style":810},[318],[202,2455,2457],{"className":2456},[323,324,325,326],[202,2458,2460,2463,2466],{"className":2459},[266,326],[202,2461,595],{"className":2462},[293,326],[202,2464,576],{"className":2465},[266,267,326],[202,2467,615],{"className":2468},[384,326],[202,2470,2472],{"className":2471,"style":870},[384,869],[202,2473,615],{"className":2474},[874,875],[202,2476,2478],{"className":2477},[793],[202,2479,2481],{"className":2480},[301],[202,2482,2484],{"className":2483},[306],[202,2485,2487],{"className":2486,"style":2079},[310],[202,2488,2489,2492],{"style":2082},[202,2490],{"className":2491,"style":810},[318],[202,2493,2495],{"className":2494},[323,324,325,326],[202,2496,1734],{"className":2497},[266,326],[202,2499],{"className":2500,"style":455},[271],[202,2502,2183],{"className":2503},[363],[202,2505],{"className":2506,"style":455},[271],[202,2508,2510,2514,2517,2520,2523],{"className":2509},[257],[202,2511],{"className":2512,"style":2513},[261],"height:1.0641em;vertical-align:-0.25em;",[202,2515,555],{"className":2516,"style":702},[266,267],[202,2518,2189],{"className":2519},[266],[202,2521,546],{"className":2522,"style":664},[266,663],[202,2524,2526,2529],{"className":2525},[266],[202,2527,2189],{"className":2528},[266],[202,2530,2532],{"className":2531},[793],[202,2533,2535],{"className":2534},[301],[202,2536,2538],{"className":2537},[306],[202,2539,2542],{"className":2540,"style":2541},[310],"height:0.8141em;",[202,2543,2544,2547],{"style":904},[202,2545],{"className":2546,"style":810},[318],[202,2548,2550],{"className":2549},[323,324,325,326],[202,2551,1734],{"className":2552},[266,326],[11,2554,2555,2556,2559,2560,2612,2613,2664],{},"That extra term is exactly the L2 regularization I built from scratch in ",[29,2557,2558],{"href":181},"the previous bonus post",". With ",[202,2561,2563,2582],{"className":2562},[205],[202,2564,2566],{"className":2565},[209],[211,2567,2568],{"xmlns":213},[215,2569,2570,2579],{},[218,2571,2572,2574,2576],{},[221,2573,555],{},[225,2575,579],{},[433,2577,2578],{},"0.0001",[245,2580,2581],{"encoding":247},"\\alpha=0.0001",[202,2583,2585,2603],{"className":2584,"ariaHidden":253},[252],[202,2586,2588,2591,2594,2597,2600],{"className":2587},[257],[202,2589],{"className":2590,"style":262},[261],[202,2592,555],{"className":2593,"style":702},[266,267],[202,2595],{"className":2596,"style":272},[271],[202,2598,579],{"className":2599},[276],[202,2601],{"className":2602,"style":272},[271],[202,2604,2606,2609],{"className":2605},[257],[202,2607],{"className":2608,"style":468},[261],[202,2610,2578],{"className":2611},[266]," (the default) the effect is too small to notice on this dataset, but the mechanism is real. I fit with ",[202,2614,2616,2634],{"className":2615},[205],[202,2617,2619],{"className":2618},[209],[211,2620,2621],{"xmlns":213},[215,2622,2623,2631],{},[218,2624,2625,2627,2629],{},[221,2626,555],{},[225,2628,579],{},[433,2630,435],{},[245,2632,2633],{"encoding":247},"\\alpha=1",[202,2635,2637,2655],{"className":2636,"ariaHidden":253},[252],[202,2638,2640,2643,2646,2649,2652],{"className":2639},[257],[202,2641],{"className":2642,"style":262},[261],[202,2644,555],{"className":2645,"style":702},[266,267],[202,2647],{"className":2648,"style":272},[271],[202,2650,579],{"className":2651},[276],[202,2653],{"className":2654,"style":272},[271],[202,2656,2658,2661],{"className":2657},[257],[202,2659],{"className":2660,"style":468},[261],[202,2662,435],{"className":2663},[266]," (much stronger, just to make it visible) on our 100-house dataset:",[42,2666,2667,2719],{},[45,2668,2669],{},[48,2670,2671,2674,2678],{},[51,2672,2673],{"align":53},"Model",[51,2675,2677],{"align":2676},"right","RMSE",[51,2679,2680],{"align":2676},[202,2681,2683,2701],{"className":2682},[205],[202,2684,2686],{"className":2685},[209],[211,2687,2688],{"xmlns":213},[215,2689,2690,2698],{},[218,2691,2692,2694,2696],{},[221,2693,2189],{"mathvariant":2188},[221,2695,546],{"mathvariant":545},[221,2697,2189],{"mathvariant":2188},[245,2699,2700],{"encoding":247},"\\|\\mathbf{w}\\|",[202,2702,2704],{"className":2703,"ariaHidden":253},[252],[202,2705,2707,2710,2713,2716],{"className":2706},[257],[202,2708],{"className":2709,"style":1446},[261],[202,2711,2189],{"className":2712},[266],[202,2714,546],{"className":2715,"style":664},[266,663],[202,2717,2189],{"className":2718},[266],[62,2720,2721,2732],{},[48,2722,2723,2726,2729],{},[67,2724,2725],{"align":53},"No regularization",[67,2727,2728],{"align":2676},"20.96",[67,2730,2731],{"align":2676},"123.25",[48,2733,2734,2787,2790],{},[67,2735,2736,2737,615],{"align":53},"Ridge (",[202,2738,2740,2757],{"className":2739},[205],[202,2741,2743],{"className":2742},[209],[211,2744,2745],{"xmlns":213},[215,2746,2747,2755],{},[218,2748,2749,2751,2753],{},[221,2750,555],{},[225,2752,579],{},[433,2754,435],{},[245,2756,2633],{"encoding":247},[202,2758,2760,2778],{"className":2759,"ariaHidden":253},[252],[202,2761,2763,2766,2769,2772,2775],{"className":2762},[257],[202,2764],{"className":2765,"style":262},[261],[202,2767,555],{"className":2768,"style":702},[266,267],[202,2770],{"className":2771,"style":272},[271],[202,2773,579],{"className":2774},[276],[202,2776],{"className":2777,"style":272},[271],[202,2779,2781,2784],{"className":2780},[257],[202,2782],{"className":2783,"style":468},[261],[202,2785,435],{"className":2786},[266],[67,2788,2789],{"align":2676},"21.04",[67,2791,2792],{"align":2676},"120.39",[11,2794,2795,2796,2799,2800,2803,2804,2807,2808,490],{},"The weight shrinks, the training error gets a little worse, exactly the trade-off I already saw in ",[29,2797,2798],{"href":181},"the Ridge post",". ",[37,2801,2802],{},"Know the defaults of the library you're using",": an ",[70,2805,2806],{},"SGDRegressor()"," called with no arguments isn't \"raw linear regression,\" it's Ridge with a small, discrete ",[202,2809,2811,2825],{"className":2810},[205],[202,2812,2814],{"className":2813},[209],[211,2815,2816],{"xmlns":213},[215,2817,2818,2822],{},[218,2819,2820],{},[221,2821,555],{},[245,2823,2824],{"encoding":247},"\\alpha",[202,2826,2828],{"className":2827,"ariaHidden":253},[252],[202,2829,2831,2834],{"className":2830},[257],[202,2832],{"className":2833,"style":262},[261],[202,2835,555],{"className":2836,"style":702},[266,267],[22,2838,2840],{"id":2839},"predicting-and-a-fragile-way-to-compare","Predicting, and a fragile way to compare",[11,2842,2843,2844,2847,2848,2851,2852,2854],{},"Two mathematically equivalent calculations can differ in the last bit of precision because of floating-point operation order. Comparing predictions with ",[70,2845,2846],{},"=="," is fragile (",[70,2849,2850],{},"0.1 + 0.2 == 0.3"," is ",[70,2853,601],{}," in any language using 64-bit floats, for the exact same reason). The right way is to check the difference falls within a small tolerance, not demand exact equality.",[22,2856,2858],{"id":2857},"metrics-how-good-is-this-model-really","Metrics: how good is this model, really",[11,2860,2861,2862,2891],{},"Up to now I'd used the cost ",[202,2863,2865,2878],{"className":2864},[205],[202,2866,2868],{"className":2867},[209],[211,2869,2870],{"xmlns":213},[215,2871,2872,2876],{},[218,2873,2874],{},[221,2875,1712],{},[245,2877,1712],{"encoding":247},[202,2879,2881],{"className":2880,"ariaHidden":253},[252],[202,2882,2884,2888],{"className":2883},[257],[202,2885],{"className":2886,"style":2887},[261],"height:0.6833em;",[202,2889,1712],{"className":2890,"style":1801},[266,267]," to train, but never a metric meant to be read by a person. I fit the model with all 4 features on the full 100-house dataset and computed three standard metrics:",[42,2893,2894,2907],{},[45,2895,2896],{},[48,2897,2898,2901,2904],{},[51,2899,2900],{"align":53},"Metric",[51,2902,2903],{"align":2676},"Value",[51,2905,2906],{"align":53},"What it measures",[62,2908,2909,2919,2930],{},[48,2910,2911,2913,2916],{},[67,2912,2677],{"align":53},[67,2914,2915],{"align":2676},"20.96 thousand US$",[67,2917,2918],{"align":53},"typical error, punishes big misses disproportionately",[48,2920,2921,2924,2927],{},[67,2922,2923],{"align":53},"MAE",[67,2925,2926],{"align":2676},"16.91 thousand US$",[67,2928,2929],{"align":53},"mean absolute error, more robust to outliers",[48,2931,2932,2935,2938],{},[67,2933,2934],{"align":53},"R²",[67,2936,2937],{"align":2676},"0.9594",[67,2939,2940],{"align":53},"fraction of price's variance explained by the model",[11,2942,2943,2944,2947],{},"For reference: a \"dumb\" model that always guesses the mean has an RMSE of 104.07. Ours misses ",[37,2945,2946],{},"4.96 times less",". That's the real payoff of having a model, not just the R² number alone.",[22,2949,2951,2953,2954,2957],{"id":2950},"sgdregressor-or-linearregression-the-real-choice",[70,2952,478],{}," or ",[70,2955,2956],{},"LinearRegression","? The real choice",[11,2959,2960,2961,2963,2964,2967,2968,2970,2971,3162,3163,3166,3167,2799,3242,3244,3245,3290],{},"The course presents ",[70,2962,478],{}," as \"scikit-learn's linear regression,\" but in practice it's the ",[37,2965,2966],{},"least"," common choice. ",[70,2969,2956],{}," solves the normal equation in closed form, ",[202,2972,2974,3022],{"className":2973},[205],[202,2975,2977],{"className":2976},[209],[211,2978,2979],{"xmlns":213},[215,2980,2981,3019],{},[218,2982,2983,2987,2989,2991,2999,3001,3011,3017],{},[221,2984,2986],{"mathvariant":2985},"bold-italic","θ",[225,2988,579],{},[225,2990,595],{"stretchy":601},[603,2992,2993,2996],{},[221,2994,2995],{"mathvariant":545},"X",[221,2997,2998],{"mathvariant":2188},"⊤",[221,3000,2995],{"mathvariant":545},[603,3002,3003,3005],{},[225,3004,615],{"stretchy":601},[218,3006,3007,3009],{},[225,3008,237],{},[433,3010,435],{},[603,3012,3013,3015],{},[221,3014,2995],{"mathvariant":545},[221,3016,2998],{"mathvariant":2188},[221,3018,624],{"mathvariant":545},[245,3020,3021],{"encoding":247},"\\boldsymbol{\\theta} = (\\mathbf{X}^\\top\\mathbf{X})^{-1}\\mathbf{X}^\\top\\mathbf{y}",[202,3023,3025,3052],{"className":3024,"ariaHidden":253},[252],[202,3026,3028,3032,3043,3046,3049],{"className":3027},[257],[202,3029],{"className":3030,"style":3031},[261],"height:0.6944em;",[202,3033,3035],{"className":3034},[266],[202,3036,3038],{"className":3037},[266],[202,3039,2986],{"className":3040,"style":3042},[266,3041],"boldsymbol","margin-right:0.0319em;",[202,3044],{"className":3045,"style":272},[271],[202,3047,579],{"className":3048},[276],[202,3050],{"className":3051,"style":272},[271],[202,3053,3055,3059,3062,3092,3095,3130,3159],{"className":3054},[257],[202,3056],{"className":3057,"style":3058},[261],"height:1.0991em;vertical-align:-0.25em;",[202,3060,595],{"className":3061},[293],[202,3063,3065,3068],{"className":3064},[266],[202,3066,2995],{"className":3067},[266,663],[202,3069,3071],{"className":3070},[793],[202,3072,3074],{"className":3073},[301],[202,3075,3077],{"className":3076},[306],[202,3078,3081],{"className":3079,"style":3080},[310],"height:0.8491em;",[202,3082,3083,3086],{"style":904},[202,3084],{"className":3085,"style":810},[318],[202,3087,3089],{"className":3088},[323,324,325,326],[202,3090,2998],{"className":3091},[266,326],[202,3093,2995],{"className":3094},[266,663],[202,3096,3098,3101],{"className":3097},[384],[202,3099,615],{"className":3100},[384],[202,3102,3104],{"className":3103},[793],[202,3105,3107],{"className":3106},[301],[202,3108,3110],{"className":3109},[306],[202,3111,3113],{"className":3112,"style":2541},[310],[202,3114,3115,3118],{"style":904},[202,3116],{"className":3117,"style":810},[318],[202,3119,3121],{"className":3120},[323,324,325,326],[202,3122,3124,3127],{"className":3123},[266,326],[202,3125,237],{"className":3126},[266,326],[202,3128,435],{"className":3129},[266,326],[202,3131,3133,3136],{"className":3132},[266],[202,3134,2995],{"className":3135},[266,663],[202,3137,3139],{"className":3138},[793],[202,3140,3142],{"className":3141},[301],[202,3143,3145],{"className":3144},[306],[202,3146,3148],{"className":3147,"style":3080},[310],[202,3149,3150,3153],{"style":904},[202,3151],{"className":3152,"style":810},[318],[202,3154,3156],{"className":3155},[323,324,325,326],[202,3157,2998],{"className":3158},[266,326],[202,3160,624],{"className":3161,"style":664},[266,663],", no iteration, no learning rate, no random seed. The cost grows with the ",[37,3164,3165],{},"cube"," of the number of features, ",[202,3168,3170,3195],{"className":3169},[205],[202,3171,3173],{"className":3172},[209],[211,3174,3175],{"xmlns":213},[215,3176,3177,3192],{},[218,3178,3179,3181,3183,3190],{},[221,3180,1424],{},[225,3182,595],{"stretchy":601},[603,3184,3185,3187],{},[221,3186,1431],{},[433,3188,3189],{},"3",[225,3191,615],{"stretchy":601},[245,3193,3194],{"encoding":247},"O(n^3)",[202,3196,3198],{"className":3197,"ariaHidden":253},[252],[202,3199,3201,3204,3207,3210,3239],{"className":3200},[257],[202,3202],{"className":3203,"style":2513},[261],[202,3205,1424],{"className":3206,"style":1450},[266,267],[202,3208,595],{"className":3209},[293],[202,3211,3213,3216],{"className":3212},[266],[202,3214,1431],{"className":3215},[266,267],[202,3217,3219],{"className":3218},[793],[202,3220,3222],{"className":3221},[301],[202,3223,3225],{"className":3224},[306],[202,3226,3228],{"className":3227,"style":2541},[310],[202,3229,3230,3233],{"style":904},[202,3231],{"className":3232,"style":810},[318],[202,3234,3236],{"className":3235},[323,324,325,326],[202,3237,3189],{"className":3238},[266,326],[202,3240,615],{"className":3241},[384],[70,3243,478],{}," iterates, costs ",[202,3246,3248,3269],{"className":3247},[205],[202,3249,3251],{"className":3250},[209],[211,3252,3253],{"xmlns":213},[215,3254,3255,3267],{},[218,3256,3257,3259,3261,3263,3265],{},[221,3258,1424],{},[225,3260,595],{"stretchy":601},[221,3262,400],{},[221,3264,1431],{},[225,3266,615],{"stretchy":601},[245,3268,1436],{"encoding":247},[202,3270,3272],{"className":3271,"ariaHidden":253},[252],[202,3273,3275,3278,3281,3284,3287],{"className":3274},[257],[202,3276],{"className":3277,"style":1446},[261],[202,3279,1424],{"className":3280,"style":1450},[266,267],[202,3282,595],{"className":3283},[293],[202,3285,433],{"className":3286},[266,267],[202,3288,615],{"className":3289},[384]," per epoch, and never needs everything loaded into memory at once.",[42,3292,3293,3308],{},[45,3294,3295],{},[48,3296,3297,3300,3304],{},[51,3298,3299],{"align":53},"Criterion",[51,3301,3302],{"align":53},[70,3303,2956],{},[51,3305,3306],{"align":53},[70,3307,478],{},[62,3309,3310,3321,3332,3345,3415,3429],{},[48,3311,3312,3315,3318],{},[67,3313,3314],{"align":53},"accuracy",[67,3316,3317],{"align":53},"exact solution",[67,3319,3320],{"align":53},"approximate",[48,3322,3323,3326,3329],{},[67,3324,3325],{"align":53},"determinism",[67,3327,3328],{"align":53},"full",[67,3330,3331],{"align":53},"depends on the seed",[48,3333,3334,3337,3340],{},[67,3335,3336],{"align":53},"needs normalizing?",[67,3338,3339],{"align":53},"no",[67,3341,3342],{"align":53},[37,3343,3344],{},"yes, mandatorily",[48,3346,3347,3350,3412],{},[67,3348,3349],{"align":53},"huge feature count",[67,3351,3352,3353,615],{"align":53},"gets expensive (",[202,3354,3356,3374],{"className":3355},[205],[202,3357,3359],{"className":3358},[209],[211,3360,3361],{"xmlns":213},[215,3362,3363,3371],{},[218,3364,3365],{},[603,3366,3367,3369],{},[221,3368,1431],{},[433,3370,3189],{},[245,3372,3373],{"encoding":247},"n^3",[202,3375,3377],{"className":3376,"ariaHidden":253},[252],[202,3378,3380,3383],{"className":3379},[257],[202,3381],{"className":3382,"style":2541},[261],[202,3384,3386,3389],{"className":3385},[266],[202,3387,1431],{"className":3388},[266,267],[202,3390,3392],{"className":3391},[793],[202,3393,3395],{"className":3394},[301],[202,3396,3398],{"className":3397},[306],[202,3399,3401],{"className":3400,"style":2541},[310],[202,3402,3403,3406],{"style":904},[202,3404],{"className":3405,"style":810},[318],[202,3407,3409],{"className":3408},[323,324,325,326],[202,3410,3189],{"className":3411},[266,326],[67,3413,3414],{"align":53},"fine",[48,3416,3417,3420,3423],{},[67,3418,3419],{"align":53},"huge example count",[67,3421,3422],{"align":53},"needs to fit in memory",[67,3424,3425,3426],{"align":53},"scales well, supports ",[70,3427,3428],{},"partial_fit",[48,3430,3431,3434,3436],{},[67,3432,3433],{"align":53},"incremental learning",[67,3435,3339],{"align":53},[67,3437,1561],{"align":53},[11,3439,3440,3443,3444,3446,3447,3450,3451,3453],{},[37,3441,3442],{},"Rule of thumb:"," I default to ",[70,3445,2956],{}," (or ",[70,3448,3449],{},"Ridge","). I only reach for ",[70,3452,478],{}," when the data doesn't fit in memory, arrives as a stream, or the feature count is huge.",[22,3455,3457],{"id":3456},"wrapping-up","Wrapping up",[42,3459,3460,3470],{},[45,3461,3462],{},[48,3463,3464,3467],{},[51,3465,3466],{"align":53},"What I already knew",[51,3468,3469],{"align":53},"What this post settled",[62,3471,3472,3480,3488],{},[48,3473,3474,3477],{},[67,3475,3476],{"align":53},"Gradient descent uses the whole dataset every step",[67,3478,3479],{"align":53},"There's a version that uses one example at a time, cheaper per step, noisier",[48,3481,3482,3485],{},[67,3483,3484],{"align":53},"I implemented all of this by hand up to now",[67,3486,3487],{"align":53},"The field's standard library does the same thing, with the same API convention across hundreds of models",[48,3489,3490,3493],{},[67,3491,3492],{"align":53},"Ridge is a choice I made explicitly",[67,3494,3495],{"align":53},"Scikit-learn's \"default\" stochastic gradient model already ships with Ridge turned on, without saying so",[11,3497,3498],{},"Three takeaways:",[3500,3501,3502,3520,3526],"ol",{},[3503,3504,3505,3519],"li",{},[37,3506,3507,3508,3510,3511,3510,3513,3510,3515,3518],{},"The ",[70,3509,117],{},"\u002F",[70,3512,172],{},[70,3514,121],{},[70,3516,3517],{},"score"," convention matters more than memorizing one specific model",", it repeats across the whole library.",[3503,3521,3522,3525],{},[37,3523,3524],{},"SGD trades precision for speed per step",": same general direction, a cheaper, noisier path.",[3503,3527,3528,200,3531,3533,3534,3536],{},[37,3529,3530],{},"Know the defaults of what you're using",[70,3532,1691],{}," turned on by default in ",[70,3535,478],{}," is exactly the kind of detail that changes what your code is actually doing.",[22,3538,3540],{"id":3539},"practical-application","Practical application",[11,3542,3543,3544,169,3547,3550,3551,3651,3652,3655],{},"Same real housing dataset from the previous posts. I already know batch gradient descent, with ",[70,3545,3546],{},"square_feet\u002F100",[70,3548,3549],{},"price\u002F1000",", alpha=0.01, needs 4000 full iterations to reach ",[202,3552,3554,3590],{"className":3553},[205],[202,3555,3557],{"className":3556},[209],[211,3558,3559],{"xmlns":213},[215,3560,3561,3587],{},[218,3562,3563,3565,3567,3569,3571,3573,3575,3577,3580,3582,3585],{},[225,3564,595],{"stretchy":601},[221,3566,546],{},[225,3568,1719],{"separator":253},[221,3570,1363],{},[225,3572,615],{"stretchy":601},[225,3574,579],{},[225,3576,595],{"stretchy":601},[433,3578,3579],{},"116.5",[225,3581,1719],{"separator":253},[433,3583,3584],{},"398.3",[225,3586,615],{"stretchy":601},[245,3588,3589],{"encoding":247},"(w,b) = (116.5, 398.3)",[202,3591,3593,3627],{"className":3592,"ariaHidden":253},[252],[202,3594,3596,3599,3602,3606,3609,3612,3615,3618,3621,3624],{"className":3595},[257],[202,3597],{"className":3598,"style":1446},[261],[202,3600,595],{"className":3601},[293],[202,3603,546],{"className":3604,"style":3605},[266,267],"margin-right:0.0269em;",[202,3607,1719],{"className":3608},[1811],[202,3610],{"className":3611,"style":706},[271],[202,3613,1363],{"className":3614},[266,267],[202,3616,615],{"className":3617},[384],[202,3619],{"className":3620,"style":272},[271],[202,3622,579],{"className":3623},[276],[202,3625],{"className":3626,"style":272},[271],[202,3628,3630,3633,3636,3639,3642,3645,3648],{"className":3629},[257],[202,3631],{"className":3632,"style":1446},[261],[202,3634,595],{"className":3635},[293],[202,3637,3579],{"className":3638},[266],[202,3640,1719],{"className":3641},[1811],[202,3643],{"className":3644,"style":706},[271],[202,3646,3584],{"className":3647},[266],[202,3649,615],{"className":3650},[384],", final cost 5189.72. Each one of those 4000 iterations looks at all 50 houses, so that's ",[37,3653,3654],{},"200 thousand"," example evaluations total.",[11,3657,3658,3659,3662],{},"I ran the stochastic version with the same alpha, but counting ",[37,3660,3661],{},"individual steps"," instead of full iterations:",[3664,3665,3670],"pre",{"className":3666,"code":3667,"language":3668,"meta":3669,"style":3669},"language-python shiki shiki-themes github-light github-dark","w, b, hist = sgd_gradient_descent(\n    square_feet_norm, price,\n    w_in=0, b_in=0,\n    alpha=0.01, num_steps=4000)  # 4000 examples seen, not 4000 full passes\n\nprint(f\"(w, b) found: ({w:.1f}, {b:.1f})\")\n","python","",[70,3671,3672,3679,3685,3691,3697,3704],{"__ignoreMap":3669},[202,3673,3676],{"class":3674,"line":3675},"line",1,[202,3677,3678],{},"w, b, hist = sgd_gradient_descent(\n",[202,3680,3682],{"class":3674,"line":3681},2,[202,3683,3684],{},"    square_feet_norm, price,\n",[202,3686,3688],{"class":3674,"line":3687},3,[202,3689,3690],{},"    w_in=0, b_in=0,\n",[202,3692,3694],{"class":3674,"line":3693},4,[202,3695,3696],{},"    alpha=0.01, num_steps=4000)  # 4000 examples seen, not 4000 full passes\n",[202,3698,3700],{"class":3674,"line":3699},5,[202,3701,3703],{"emptyLinePlaceholder":3702},true,"\n",[202,3705,3707],{"class":3674,"line":3706},6,[202,3708,3709],{},"print(f\"(w, b) found: ({w:.1f}, {b:.1f})\")\n",[3711,3712,3713],"blockquote",{},[11,3714,3715,178,3718,3721],{},[37,3716,3717],{},"Output:",[70,3719,3720],{},"(w, b) found: (112.1, 396.9)",", final cost 5233.79",[11,3723,3724,3725,3728],{},"Practically the same result as batch (cost 5233.79 versus 5189.72), using ",[37,3726,3727],{},"50 times fewer"," example evaluations (4000 versus 200 thousand). Compare both live:",[3730,3731],"housing-gradient-descent-simulator",{},[3733,3734],"housing-stochastic-gradient-descent-simulator",{},[11,3736,3737],{},"Click \"Rodar 2000\" on each and notice: the stochastic one reaches a good neighborhood much faster in terms of total work, even with the messier path on the chart.",[3739,3740,3741],"style",{},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":3669,"searchDepth":3681,"depth":3681,"links":3743},[3744,3745,3747,3749,3751,3752,3753,3754,3756,3757],{"id":24,"depth":3681,"text":25},{"id":186,"depth":3681,"text":3746},"StandardScaler: the z-score I already built by hand",{"id":475,"depth":3681,"text":3748},"SGDRegressor and what the \"S\" means",{"id":1641,"depth":3681,"text":3750},"Two things max_iter hides",{"id":1681,"depth":3681,"text":1682},{"id":2839,"depth":3681,"text":2840},{"id":2857,"depth":3681,"text":2858},{"id":2950,"depth":3681,"text":3755},"SGDRegressor or LinearRegression? The real choice",{"id":3456,"depth":3681,"text":3457},{"id":3539,"depth":3681,"text":3540},null,"2026-08-19","After building gradient descent, normalization, and feature engineering by hand, I finally use scikit-learn, and find out the course's 'default' model hides two surprises nobody warns you about.","md",{},10,"\u002Fen\u002Fplaylists\u002Fmachine-learning-specialization\u002Fw2-lab05-scikit-learn","machine-learning-specialization",{"title":6,"description":3760},"published","en\u002Fplaylists\u002Fmachine-learning-specialization\u002Fw2-lab05-scikit-learn",[34,3770,3771],"stochastic-gradient-descent","sgd","yR3cXoVOUASqEpiV0kbz68EbkdzyppXZRqzgG0_FqX8",1787338984905]