[{"data":1,"prerenderedAt":2246},["ShallowReactive",2],{"lang-switch-post-\u002Fen\u002Fplaylists\u002Fmachine-learning-specialization\u002Floss-functions":3,"post-en-machine-learning-specialization-loss-functions":4},"\u002Fplaylists\u002Fmachine-learning-specialization\u002Floss-functions",{"id":5,"title":6,"body":7,"cover":2229,"date":2230,"description":2231,"extension":2232,"meta":2233,"navigation":1984,"order":1988,"path":2234,"playlist":2235,"seo":2236,"status":2237,"stem":2238,"tags":2239,"__hash__":2245},"posts\u002Fen\u002Fplaylists\u002Fmachine-learning-specialization\u002Floss-functions.md","Just a Little Extra: Loss Functions (MSE, MAE, Huber)",{"type":8,"value":9,"toc":2220},"minimark",[10,18,32,37,50,600,606,610,618,1118,1124,1128,1131,1138,1141,1145,1186,1772,1805,1809,1825,1837,1840,1844,1911,1914,1936,1940,1950,2032,2050,2209,2213,2216],[11,12,13],"p",{},[14,15],"img",{"alt":16,"src":17},"A \"look what they need to mimic a fraction of our power\" meme: a GPU and a 3D plot of a bumpy cost surface, next to a giant brain, with the caption underneath","\u002Fimages\u002Fposts\u002Fmachine-learning-specialization\u002Floss-functions\u002Fmeme-loss-function.jpeg",[11,19,20,21,26,27,31],{},"This post doesn't come from any specific course lab, it's a bonus we earned from hammering on the cost function so much in ",[22,23,25],"a",{"href":24},"\u002Fen\u002Fplaylists\u002Fmachine-learning-specialization\u002Flab03-cost-function","posts 2"," ",[22,28,30],{"href":29},"\u002Fen\u002Fplaylists\u002Fmachine-learning-specialization\u002Flab04-gradient-descent","and 3",". Back there we squared the error without questioning the choice much, but \"squaring it\" is a choice, not the only option. Let's open that up.",[33,34,36],"h2",{"id":35},"naming-what-you-already-built-mse","Naming what you already built: MSE",[11,38,39,40,43,44,49],{},"The cost function you've been using since ",[22,41,42],{"href":24},"post 2"," has an official name: ",[45,46,48],"glossary-term",{"definition":47},"Mean Squared Error, the same cost function you've already been using since post 2","MSE"," (Mean Squared Error).",[11,51,52],{},[53,54,57,184],"span",{"className":55},[56],"katex",[53,58,61],{"className":59},[60],"katex-mathml",[62,63,65],"math",{"xmlns":64},"http:\u002F\u002Fwww.w3.org\u002F1998\u002FMath\u002FMathML",[66,67,68,179],"semantics",{},[69,70,71,74,78,89,114],"mrow",{},[72,73,48],"mtext",{},[75,76,77],"mo",{},"=",[79,80,81,85],"mfrac",{},[82,83,84],"mn",{},"1",[86,87,88],"mi",{},"m",[90,91,92,95,105],"msubsup",{},[75,93,94],{},"∑",[69,96,97,100,102],{},[86,98,99],{},"i",[75,101,77],{},[82,103,104],{},"0",[69,106,107,109,112],{},[86,108,88],{},[75,110,111],{},"−",[82,113,84],{},[115,116,117,176],"msup",{},[69,118,119,123,140,143,157,159,161,174],{},[75,120,122],{"fence":121},"true","(",[124,125,126,129],"msub",{},[86,127,128],{},"f",[69,130,131,134,137],{},[86,132,133],{},"w",[75,135,136],{"separator":121},",",[86,138,139],{},"b",[75,141,122],{"stretchy":142},"false",[115,144,145,148],{},[86,146,147],{},"x",[69,149,150,152,154],{},[75,151,122],{"stretchy":142},[86,153,99],{},[75,155,156],{"stretchy":142},")",[75,158,156],{"stretchy":142},[75,160,111],{},[115,162,163,166],{},[86,164,165],{},"y",[69,167,168,170,172],{},[75,169,122],{"stretchy":142},[86,171,99],{},[75,173,156],{"stretchy":142},[75,175,156],{"fence":121},[82,177,178],{},"2",[180,181,183],"annotation",{"encoding":182},"application\u002Fx-tex","\\text{MSE} = \\frac{1}{m}\\sum_{i=0}^{m-1}\\left(f_{w,b}(x^{(i)}) - y^{(i)}\\right)^2",[53,185,188,217],{"className":186,"ariaHidden":121},[187],"katex-html",[53,189,192,197,205,210,214],{"className":190},[191],"base",[53,193],{"className":194,"style":196},[195],"strut","height:0.6833em;",[53,198,202],{"className":199},[200,201],"mord","text",[53,203,48],{"className":204},[200],[53,206],{"className":207,"style":209},[208],"mspace","margin-right:0.2778em;",[53,211,77],{"className":212},[213],"mrel",[53,215],{"className":216,"style":209},[208],[53,218,220,224,315,319,399,402],{"className":219},[191],[53,221],{"className":222,"style":223},[195],"height:1.442em;vertical-align:-0.35em;",[53,225,227,232,311],{"className":226},[200],[53,228],{"className":229},[230,231],"mopen","nulldelimiter",[53,233,235],{"className":234},[79],[53,236,240,302],{"className":237},[238,239],"vlist-t","vlist-t2",[53,241,244,297],{"className":242},[243],"vlist-r",[53,245,249,271,282],{"className":246,"style":248},[247],"vlist","height:0.8451em;",[53,250,252,257],{"style":251},"top:-2.655em;",[53,253],{"className":254,"style":256},[255],"pstrut","height:3em;",[53,258,264],{"className":259},[260,261,262,263],"sizing","reset-size6","size3","mtight",[53,265,267],{"className":266},[200,263],[53,268,88],{"className":269},[200,270,263],"mathnormal",[53,272,274,277],{"style":273},"top:-3.23em;",[53,275],{"className":276,"style":256},[255],[53,278],{"className":279,"style":281},[280],"frac-line","border-bottom-width:0.04em;",[53,283,285,288],{"style":284},"top:-3.394em;",[53,286],{"className":287,"style":256},[255],[53,289,291],{"className":290},[260,261,262,263],[53,292,294],{"className":293},[200,263],[53,295,84],{"className":296},[200,263],[53,298,301],{"className":299},[300],"vlist-s","​",[53,303,305],{"className":304},[243],[53,306,309],{"className":307,"style":308},[247],"height:0.345em;",[53,310],{},[53,312],{"className":313},[314,231],"mclose",[53,316],{"className":317,"style":318},[208],"margin-right:0.1667em;",[53,320,323,329],{"className":321},[322],"mop",[53,324,94],{"className":325,"style":328},[322,326,327],"op-symbol","small-op","position:relative;top:0em;",[53,330,333],{"className":331},[332],"msupsub",[53,334,336,390],{"className":335},[238,239],[53,337,339,387],{"className":338},[243],[53,340,343,365],{"className":341,"style":342},[247],"height:0.954em;",[53,344,346,350],{"style":345},"top:-2.4003em;margin-left:0em;margin-right:0.05em;",[53,347],{"className":348,"style":349},[255],"height:2.7em;",[53,351,353],{"className":352},[260,261,262,263],[53,354,356,359,362],{"className":355},[200,263],[53,357,99],{"className":358},[200,270,263],[53,360,77],{"className":361},[213,263],[53,363,104],{"className":364},[200,263],[53,366,368,371],{"style":367},"top:-3.2029em;margin-right:0.05em;",[53,369],{"className":370,"style":349},[255],[53,372,374],{"className":373},[260,261,262,263],[53,375,377,380,384],{"className":376},[200,263],[53,378,88],{"className":379},[200,270,263],[53,381,111],{"className":382},[383,263],"mbin",[53,385,84],{"className":386},[200,263],[53,388,301],{"className":389},[300],[53,391,393],{"className":392},[243],[53,394,397],{"className":395,"style":396},[247],"height:0.2997em;",[53,398],{},[53,400],{"className":401,"style":318},[208],[53,403,406,575],{"className":404},[405],"minner",[53,407,409,419,474,477,517,520,524,527,530,569],{"className":408},[405],[53,410,414],{"className":411,"style":413},[230,412],"delimcenter","top:0em;",[53,415,122],{"className":416},[417,418],"delimsizing","size1",[53,420,422,426],{"className":421},[200],[53,423,128],{"className":424,"style":425},[200,270],"margin-right:0.1076em;",[53,427,429],{"className":428},[332],[53,430,432,465],{"className":431},[238,239],[53,433,435,462],{"className":434},[243],[53,436,439],{"className":437,"style":438},[247],"height:0.3361em;",[53,440,442,445],{"style":441},"top:-2.55em;margin-left:-0.1076em;margin-right:0.05em;",[53,443],{"className":444,"style":349},[255],[53,446,448],{"className":447},[260,261,262,263],[53,449,451,455,459],{"className":450},[200,263],[53,452,133],{"className":453,"style":454},[200,270,263],"margin-right:0.0269em;",[53,456,136],{"className":457},[458,263],"mpunct",[53,460,139],{"className":461},[200,270,263],[53,463,301],{"className":464},[300],[53,466,468],{"className":467},[243],[53,469,472],{"className":470,"style":471},[247],"height:0.2861em;",[53,473],{},[53,475,122],{"className":476},[230],[53,478,480,483],{"className":479},[200],[53,481,147],{"className":482},[200,270],[53,484,486],{"className":485},[332],[53,487,489],{"className":488},[238],[53,490,492],{"className":491},[243],[53,493,496],{"className":494,"style":495},[247],"height:0.888em;",[53,497,499,502],{"style":498},"top:-3.063em;margin-right:0.05em;",[53,500],{"className":501,"style":349},[255],[53,503,505],{"className":504},[260,261,262,263],[53,506,508,511,514],{"className":507},[200,263],[53,509,122],{"className":510},[230,263],[53,512,99],{"className":513},[200,270,263],[53,515,156],{"className":516},[314,263],[53,518,156],{"className":519},[314],[53,521],{"className":522,"style":523},[208],"margin-right:0.2222em;",[53,525,111],{"className":526},[383],[53,528],{"className":529,"style":523},[208],[53,531,533,537],{"className":532},[200],[53,534,165],{"className":535,"style":536},[200,270],"margin-right:0.0359em;",[53,538,540],{"className":539},[332],[53,541,543],{"className":542},[238],[53,544,546],{"className":545},[243],[53,547,549],{"className":548,"style":495},[247],[53,550,551,554],{"style":498},[53,552],{"className":553,"style":349},[255],[53,555,557],{"className":556},[260,261,262,263],[53,558,560,563,566],{"className":559},[200,263],[53,561,122],{"className":562},[230,263],[53,564,99],{"className":565},[200,270,263],[53,567,156],{"className":568},[314,263],[53,570,572],{"className":571,"style":413},[314,412],[53,573,156],{"className":574},[417,418],[53,576,578],{"className":577},[332],[53,579,581],{"className":580},[238],[53,582,584],{"className":583},[243],[53,585,588],{"className":586,"style":587},[247],"height:1.092em;",[53,589,591,594],{"style":590},"top:-3.3409em;margin-right:0.05em;",[53,592],{"className":593,"style":349},[255],[53,595,597],{"className":596},[260,261,262,263],[53,598,178],{"className":599},[200,263],[11,601,602,603,605],{},"Notice it's basically the same formula from ",[22,604,42],{"href":24},", just without the extra \"2\" in the denominator (that 2 only existed to make gradient descent's derivative cleaner, remember?). The name changes, the substance doesn't.",[33,607,609],{"id":608},"the-most-direct-alternative-mae","The most direct alternative: MAE",[11,611,612,613,617],{},"What if, instead of squaring the error, we just took its absolute value? That's ",[45,614,616],{"definition":615},"Mean Absolute Error, uses the absolute value of the error instead of squaring it","MAE"," (Mean Absolute Error):",[11,619,620],{},[53,621,623,714],{"className":622},[56],[53,624,626],{"className":625},[60],[62,627,628],{"xmlns":64},[66,629,630,711],{},[69,631,632,634,636,642,662],{},[72,633,616],{},[75,635,77],{},[79,637,638,640],{},[82,639,84],{},[86,641,88],{},[90,643,644,646,654],{},[75,645,94],{},[69,647,648,650,652],{},[86,649,99],{},[75,651,77],{},[82,653,104],{},[69,655,656,658,660],{},[86,657,88],{},[75,659,111],{},[82,661,84],{},[69,663,664,667,679,681,693,695,697,709],{},[75,665,666],{"fence":121},"∣",[124,668,669,671],{},[86,670,128],{},[69,672,673,675,677],{},[86,674,133],{},[75,676,136],{"separator":121},[86,678,139],{},[75,680,122],{"stretchy":142},[115,682,683,685],{},[86,684,147],{},[69,686,687,689,691],{},[75,688,122],{"stretchy":142},[86,690,99],{},[75,692,156],{"stretchy":142},[75,694,156],{"stretchy":142},[75,696,111],{},[115,698,699,701],{},[86,700,165],{},[69,702,703,705,707],{},[75,704,122],{"stretchy":142},[86,706,99],{},[75,708,156],{"stretchy":142},[75,710,666],{"fence":121},[180,712,713],{"encoding":182},"\\text{MAE} = \\frac{1}{m}\\sum_{i=0}^{m-1}\\left|f_{w,b}(x^{(i)}) - y^{(i)}\\right|",[53,715,717,738],{"className":716,"ariaHidden":121},[187],[53,718,720,723,729,732,735],{"className":719},[191],[53,721],{"className":722,"style":196},[195],[53,724,726],{"className":725},[200,201],[53,727,616],{"className":728},[200],[53,730],{"className":731,"style":209},[208],[53,733,77],{"className":734},[213],[53,736],{"className":737,"style":209},[208],[53,739,741,745,813,816,885,888],{"className":740},[191],[53,742],{"className":743,"style":744},[195],"height:1.304em;vertical-align:-0.35em;",[53,746,748,751,810],{"className":747},[200],[53,749],{"className":750},[230,231],[53,752,754],{"className":753},[79],[53,755,757,802],{"className":756},[238,239],[53,758,760,799],{"className":759},[243],[53,761,763,777,785],{"className":762,"style":248},[247],[53,764,765,768],{"style":251},[53,766],{"className":767,"style":256},[255],[53,769,771],{"className":770},[260,261,262,263],[53,772,774],{"className":773},[200,263],[53,775,88],{"className":776},[200,270,263],[53,778,779,782],{"style":273},[53,780],{"className":781,"style":256},[255],[53,783],{"className":784,"style":281},[280],[53,786,787,790],{"style":284},[53,788],{"className":789,"style":256},[255],[53,791,793],{"className":792},[260,261,262,263],[53,794,796],{"className":795},[200,263],[53,797,84],{"className":798},[200,263],[53,800,301],{"className":801},[300],[53,803,805],{"className":804},[243],[53,806,808],{"className":807,"style":308},[247],[53,809],{},[53,811],{"className":812},[314,231],[53,814],{"className":815,"style":318},[208],[53,817,819,822],{"className":818},[322],[53,820,94],{"className":821,"style":328},[322,326,327],[53,823,825],{"className":824},[332],[53,826,828,877],{"className":827},[238,239],[53,829,831,874],{"className":830},[243],[53,832,834,854],{"className":833,"style":342},[247],[53,835,836,839],{"style":345},[53,837],{"className":838,"style":349},[255],[53,840,842],{"className":841},[260,261,262,263],[53,843,845,848,851],{"className":844},[200,263],[53,846,99],{"className":847},[200,270,263],[53,849,77],{"className":850},[213,263],[53,852,104],{"className":853},[200,263],[53,855,856,859],{"style":367},[53,857],{"className":858,"style":349},[255],[53,860,862],{"className":861},[260,261,262,263],[53,863,865,868,871],{"className":864},[200,263],[53,866,88],{"className":867},[200,270,263],[53,869,111],{"className":870},[383,263],[53,872,84],{"className":873},[200,263],[53,875,301],{"className":876},[300],[53,878,880],{"className":879},[243],[53,881,883],{"className":882,"style":396},[247],[53,884],{},[53,886],{"className":887,"style":318},[208],[53,889,891,941,990,993,1031,1034,1037,1040,1043,1081],{"className":890},[405],[53,892,894],{"className":893},[230],[53,895,898],{"className":896},[417,897],"mult",[53,899,901,932],{"className":900},[238,239],[53,902,904,929],{"className":903},[243],[53,905,908],{"className":906,"style":907},[247],"height:0.85em;",[53,909,911,915],{"style":910},"top:-2.85em;",[53,912],{"className":913,"style":914},[255],"height:3.2em;",[53,916,918],{"style":917},"width:0.333em;height:1.2em;",[919,920,925],"svg",{"xmlns":921,"width":922,"height":923,"viewBox":924},"http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg","0.333em","1.2em","0 0 333 1200",[926,927],"path",{"d":928},"M145 15 v585 v0 v585 c2.667,10,9.667,15,21,15\nc10,0,16.667,-5,20,-15 v-585 v0 v-585 c-2.667,-10,-9.667,-15,-21,-15\nc-10,0,-16.667,5,-20,15z M188 15 H145 v585 v0 v585 h43z",[53,930,301],{"className":931},[300],[53,933,935],{"className":934},[243],[53,936,939],{"className":937,"style":938},[247],"height:0.35em;",[53,940],{},[53,942,944,947],{"className":943},[200],[53,945,128],{"className":946,"style":425},[200,270],[53,948,950],{"className":949},[332],[53,951,953,982],{"className":952},[238,239],[53,954,956,979],{"className":955},[243],[53,957,959],{"className":958,"style":438},[247],[53,960,961,964],{"style":441},[53,962],{"className":963,"style":349},[255],[53,965,967],{"className":966},[260,261,262,263],[53,968,970,973,976],{"className":969},[200,263],[53,971,133],{"className":972,"style":454},[200,270,263],[53,974,136],{"className":975},[458,263],[53,977,139],{"className":978},[200,270,263],[53,980,301],{"className":981},[300],[53,983,985],{"className":984},[243],[53,986,988],{"className":987,"style":471},[247],[53,989],{},[53,991,122],{"className":992},[230],[53,994,996,999],{"className":995},[200],[53,997,147],{"className":998},[200,270],[53,1000,1002],{"className":1001},[332],[53,1003,1005],{"className":1004},[238],[53,1006,1008],{"className":1007},[243],[53,1009,1011],{"className":1010,"style":495},[247],[53,1012,1013,1016],{"style":498},[53,1014],{"className":1015,"style":349},[255],[53,1017,1019],{"className":1018},[260,261,262,263],[53,1020,1022,1025,1028],{"className":1021},[200,263],[53,1023,122],{"className":1024},[230,263],[53,1026,99],{"className":1027},[200,270,263],[53,1029,156],{"className":1030},[314,263],[53,1032,156],{"className":1033},[314],[53,1035],{"className":1036,"style":523},[208],[53,1038,111],{"className":1039},[383],[53,1041],{"className":1042,"style":523},[208],[53,1044,1046,1049],{"className":1045},[200],[53,1047,165],{"className":1048,"style":536},[200,270],[53,1050,1052],{"className":1051},[332],[53,1053,1055],{"className":1054},[238],[53,1056,1058],{"className":1057},[243],[53,1059,1061],{"className":1060,"style":495},[247],[53,1062,1063,1066],{"style":498},[53,1064],{"className":1065,"style":349},[255],[53,1067,1069],{"className":1068},[260,261,262,263],[53,1070,1072,1075,1078],{"className":1071},[200,263],[53,1073,122],{"className":1074},[230,263],[53,1076,99],{"className":1077},[200,270,263],[53,1079,156],{"className":1080},[314,263],[53,1082,1084],{"className":1083},[314],[53,1085,1087],{"className":1086},[417,897],[53,1088,1090,1110],{"className":1089},[238,239],[53,1091,1093,1107],{"className":1092},[243],[53,1094,1096],{"className":1095,"style":907},[247],[53,1097,1098,1101],{"style":910},[53,1099],{"className":1100,"style":914},[255],[53,1102,1103],{"style":917},[919,1104,1105],{"xmlns":921,"width":922,"height":923,"viewBox":924},[926,1106],{"d":928},[53,1108,301],{"className":1109},[300],[53,1111,1113],{"className":1112},[243],[53,1114,1116],{"className":1115,"style":938},[247],[53,1117],{},[11,1119,1120,1121,1123],{},"The difference looks small on paper, but it changes everything about the behavior. With MSE, missing by twice as much costs four times as much (remember ",[22,1122,42],{"href":24},"?). With MAE, missing by twice as much costs exactly twice as much, no more, no less. It's the difference between a judge who loses their mind when you miss badly and a judge who just counts points proportionally, no extra drama.",[33,1125,1127],{"id":1126},"both-shapes-side-by-side","Both shapes side by side",[11,1129,1130],{},"No need to imagine the shape, here it is:",[1132,1133],"loss-shape-chart",{":error-max":1134,":initial-delta":1135,"x-label":1136,"y-label":1137},"10","4","error (prediction - actual)","loss",[11,1139,1140],{},"Notice: MSE is a parabola (grows faster and faster), MAE is a V (grows at a constant rate). Hold that image in your head, it explains everything that follows.",[33,1142,1144],{"id":1143},"the-best-of-both-worlds-huber","The best of both worlds: Huber",[11,1146,1147,1148,1152,1153,1185],{},"MSE is too sensitive to big errors. MAE is too harsh, even on small ones (look at the V: even tiny errors cost proportionally to their size, without the \"discount\" squaring gives near zero). The ",[45,1149,1151],{"definition":1150},"a hybrid loss function, a parabola for small errors and a straight line for big ones, with a threshold delta deciding where the switch happens","Huber loss"," tries to get the best of both: a smooth parabola right near zero, a straight line past a threshold ",[53,1154,1156,1171],{"className":1155},[56],[53,1157,1159],{"className":1158},[60],[62,1160,1161],{"xmlns":64},[66,1162,1163,1168],{},[69,1164,1165],{},[86,1166,1167],{},"δ",[180,1169,1170],{"encoding":182},"\\delta",[53,1172,1174],{"className":1173,"ariaHidden":121},[187],[53,1175,1177,1181],{"className":1176},[191],[53,1178],{"className":1179,"style":1180},[195],"height:0.6944em;",[53,1182,1167],{"className":1183,"style":1184},[200,270],"margin-right:0.0379em;"," (delta).",[11,1187,1188],{},[53,1189,1191,1325],{"className":1190},[56],[53,1192,1194],{"className":1193},[60],[62,1195,1196],{"xmlns":64},[66,1197,1198,1322],{},[69,1199,1200,1207,1209,1212,1214,1216],{},[124,1201,1202,1205],{},[86,1203,1204],{},"L",[86,1206,1167],{},[75,1208,122],{"stretchy":142},[86,1210,1211],{},"e",[75,1213,156],{"stretchy":142},[75,1215,77],{},[69,1217,1218,1221],{},[75,1219,1220],{"fence":121},"{",[1222,1223,1227,1271],"mtable",{"rowspacing":1224,"columnalign":1225,"columnspacing":1226},"0.36em","left left","1em",[1228,1229,1230,1250],"mtr",{},[1231,1232,1233],"mtd",{},[1234,1235,1236],"mstyle",{"scriptlevel":104,"displaystyle":142},[69,1237,1238,1244],{},[79,1239,1240,1242],{},[82,1241,84],{},[82,1243,178],{},[115,1245,1246,1248],{},[86,1247,1211],{},[82,1249,178],{},[1231,1251,1252],{},[1234,1253,1254],{"scriptlevel":104,"displaystyle":142},[69,1255,1256,1259,1262,1264,1266,1269],{},[72,1257,1258],{},"if ",[86,1260,666],{"mathvariant":1261},"normal",[86,1263,1211],{},[86,1265,666],{"mathvariant":1261},[75,1267,1268],{},"≤",[86,1270,1167],{},[1228,1272,1273,1303],{},[1231,1274,1275],{},[1234,1276,1277],{"scriptlevel":104,"displaystyle":142},[69,1278,1279,1281],{},[86,1280,1167],{},[69,1282,1283,1285,1287,1289,1291,1293,1299,1301],{},[75,1284,122],{"fence":121},[86,1286,666],{"mathvariant":1261},[86,1288,1211],{},[86,1290,666],{"mathvariant":1261},[75,1292,111],{},[79,1294,1295,1297],{},[82,1296,84],{},[82,1298,178],{},[86,1300,1167],{},[75,1302,156],{"fence":121},[1231,1304,1305],{},[1234,1306,1307],{"scriptlevel":104,"displaystyle":142},[69,1308,1309,1311,1313,1315,1317,1320],{},[72,1310,1258],{},[86,1312,666],{"mathvariant":1261},[86,1314,1211],{},[86,1316,666],{"mathvariant":1261},[75,1318,1319],{},">",[86,1321,1167],{},[180,1323,1324],{"encoding":182},"L_\\delta(e) = \\begin{cases} \\frac{1}{2}e^2 & \\text{if } |e| \\le \\delta \\\\ \\delta\\left(|e| - \\frac{1}{2}\\delta\\right) & \\text{if } |e| > \\delta \\end{cases}",[53,1326,1328,1395],{"className":1327,"ariaHidden":121},[187],[53,1329,1331,1335,1377,1380,1383,1386,1389,1392],{"className":1330},[191],[53,1332],{"className":1333,"style":1334},[195],"height:1em;vertical-align:-0.25em;",[53,1336,1338,1341],{"className":1337},[200],[53,1339,1204],{"className":1340},[200,270],[53,1342,1344],{"className":1343},[332],[53,1345,1347,1368],{"className":1346},[238,239],[53,1348,1350,1365],{"className":1349},[243],[53,1351,1353],{"className":1352,"style":438},[247],[53,1354,1356,1359],{"style":1355},"top:-2.55em;margin-left:0em;margin-right:0.05em;",[53,1357],{"className":1358,"style":349},[255],[53,1360,1362],{"className":1361},[260,261,262,263],[53,1363,1167],{"className":1364,"style":1184},[200,270,263],[53,1366,301],{"className":1367},[300],[53,1369,1371],{"className":1370},[243],[53,1372,1375],{"className":1373,"style":1374},[247],"height:0.15em;",[53,1376],{},[53,1378,122],{"className":1379},[230],[53,1381,1211],{"className":1382},[200,270],[53,1384,156],{"className":1385},[314],[53,1387],{"className":1388,"style":209},[208],[53,1390,77],{"className":1391},[213],[53,1393],{"className":1394,"style":209},[208],[53,1396,1398,1402],{"className":1397},[191],[53,1399],{"className":1400,"style":1401},[195],"height:3em;vertical-align:-1.25em;",[53,1403,1405,1412,1769],{"className":1404},[405],[53,1406,1408],{"className":1407,"style":413},[230,412],[53,1409,1220],{"className":1410},[417,1411],"size4",[53,1413,1415],{"className":1414},[200],[53,1416,1418,1671,1676],{"className":1417},[1222],[53,1419,1422],{"className":1420},[1421],"col-align-l",[53,1423,1425,1662],{"className":1424},[238,239],[53,1426,1428,1659],{"className":1427},[243],[53,1429,1432,1540],{"className":1430,"style":1431},[247],"height:1.69em;",[53,1433,1435,1439],{"style":1434},"top:-3.69em;",[53,1436],{"className":1437,"style":1438},[255],"height:3.008em;",[53,1440,1442,1510],{"className":1441},[200],[53,1443,1445,1448,1507],{"className":1444},[200],[53,1446],{"className":1447},[230,231],[53,1449,1451],{"className":1450},[79],[53,1452,1454,1499],{"className":1453},[238,239],[53,1455,1457,1496],{"className":1456},[243],[53,1458,1460,1474,1482],{"className":1459,"style":248},[247],[53,1461,1462,1465],{"style":251},[53,1463],{"className":1464,"style":256},[255],[53,1466,1468],{"className":1467},[260,261,262,263],[53,1469,1471],{"className":1470},[200,263],[53,1472,178],{"className":1473},[200,263],[53,1475,1476,1479],{"style":273},[53,1477],{"className":1478,"style":256},[255],[53,1480],{"className":1481,"style":281},[280],[53,1483,1484,1487],{"style":284},[53,1485],{"className":1486,"style":256},[255],[53,1488,1490],{"className":1489},[260,261,262,263],[53,1491,1493],{"className":1492},[200,263],[53,1494,84],{"className":1495},[200,263],[53,1497,301],{"className":1498},[300],[53,1500,1502],{"className":1501},[243],[53,1503,1505],{"className":1504,"style":308},[247],[53,1506],{},[53,1508],{"className":1509},[314,231],[53,1511,1513,1516],{"className":1512},[200],[53,1514,1211],{"className":1515},[200,270],[53,1517,1519],{"className":1518},[332],[53,1520,1522],{"className":1521},[238],[53,1523,1525],{"className":1524},[243],[53,1526,1529],{"className":1527,"style":1528},[247],"height:0.8141em;",[53,1530,1531,1534],{"style":498},[53,1532],{"className":1533,"style":349},[255],[53,1535,1537],{"className":1536},[260,261,262,263],[53,1538,178],{"className":1539},[200,263],[53,1541,1543,1546],{"style":1542},"top:-2.25em;",[53,1544],{"className":1545,"style":1438},[255],[53,1547,1549,1552,1555],{"className":1548},[200],[53,1550,1167],{"className":1551,"style":1184},[200,270],[53,1553],{"className":1554,"style":318},[208],[53,1556,1558,1564,1567,1570,1573,1576,1579,1582,1650,1653],{"className":1557},[405],[53,1559,1561],{"className":1560,"style":413},[230,412],[53,1562,122],{"className":1563},[417,418],[53,1565,666],{"className":1566},[200],[53,1568,1211],{"className":1569},[200,270],[53,1571,666],{"className":1572},[200],[53,1574],{"className":1575,"style":523},[208],[53,1577,111],{"className":1578},[383],[53,1580],{"className":1581,"style":523},[208],[53,1583,1585,1588,1647],{"className":1584},[200],[53,1586],{"className":1587},[230,231],[53,1589,1591],{"className":1590},[79],[53,1592,1594,1639],{"className":1593},[238,239],[53,1595,1597,1636],{"className":1596},[243],[53,1598,1600,1614,1622],{"className":1599,"style":248},[247],[53,1601,1602,1605],{"style":251},[53,1603],{"className":1604,"style":256},[255],[53,1606,1608],{"className":1607},[260,261,262,263],[53,1609,1611],{"className":1610},[200,263],[53,1612,178],{"className":1613},[200,263],[53,1615,1616,1619],{"style":273},[53,1617],{"className":1618,"style":256},[255],[53,1620],{"className":1621,"style":281},[280],[53,1623,1624,1627],{"style":284},[53,1625],{"className":1626,"style":256},[255],[53,1628,1630],{"className":1629},[260,261,262,263],[53,1631,1633],{"className":1632},[200,263],[53,1634,84],{"className":1635},[200,263],[53,1637,301],{"className":1638},[300],[53,1640,1642],{"className":1641},[243],[53,1643,1645],{"className":1644,"style":308},[247],[53,1646],{},[53,1648],{"className":1649},[314,231],[53,1651,1167],{"className":1652,"style":1184},[200,270],[53,1654,1656],{"className":1655,"style":413},[314,412],[53,1657,156],{"className":1658},[417,418],[53,1660,301],{"className":1661},[300],[53,1663,1665],{"className":1664},[243],[53,1666,1669],{"className":1667,"style":1668},[247],"height:1.19em;",[53,1670],{},[53,1672],{"className":1673,"style":1675},[1674],"arraycolsep","width:1em;",[53,1677,1679],{"className":1678},[1421],[53,1680,1682,1761],{"className":1681},[238,239],[53,1683,1685,1758],{"className":1684},[243],[53,1686,1688,1723],{"className":1687,"style":1431},[247],[53,1689,1690,1693],{"style":1434},[53,1691],{"className":1692,"style":1438},[255],[53,1694,1696,1702,1705,1708,1711,1714,1717,1720],{"className":1695},[200],[53,1697,1699],{"className":1698},[200,201],[53,1700,1258],{"className":1701},[200],[53,1703,666],{"className":1704},[200],[53,1706,1211],{"className":1707},[200,270],[53,1709,666],{"className":1710},[200],[53,1712],{"className":1713,"style":209},[208],[53,1715,1268],{"className":1716},[213],[53,1718],{"className":1719,"style":209},[208],[53,1721,1167],{"className":1722,"style":1184},[200,270],[53,1724,1725,1728],{"style":1542},[53,1726],{"className":1727,"style":1438},[255],[53,1729,1731,1737,1740,1743,1746,1749,1752,1755],{"className":1730},[200],[53,1732,1734],{"className":1733},[200,201],[53,1735,1258],{"className":1736},[200],[53,1738,666],{"className":1739},[200],[53,1741,1211],{"className":1742},[200,270],[53,1744,666],{"className":1745},[200],[53,1747],{"className":1748,"style":209},[208],[53,1750,1319],{"className":1751},[213],[53,1753],{"className":1754,"style":209},[208],[53,1756,1167],{"className":1757,"style":1184},[200,270],[53,1759,301],{"className":1760},[300],[53,1762,1764],{"className":1763},[243],[53,1765,1767],{"className":1766,"style":1668},[247],[53,1768],{},[53,1770],{"className":1771},[314,231],[11,1773,1774,1775,1804],{},"Where ",[53,1776,1778,1791],{"className":1777},[56],[53,1779,1781],{"className":1780},[60],[62,1782,1783],{"xmlns":64},[66,1784,1785,1789],{},[69,1786,1787],{},[86,1788,1211],{},[180,1790,1211],{"encoding":182},[53,1792,1794],{"className":1793,"ariaHidden":121},[187],[53,1795,1797,1801],{"className":1796},[191],[53,1798],{"className":1799,"style":1800},[195],"height:0.4306em;",[53,1802,1211],{"className":1803},[200,270]," is one example's error. Drag the delta slider above again and watch the orange curve: with a small delta, Huber turns nearly into MAE, and with a big delta, it turns nearly into MSE. Delta is literally the control for \"past what error size do I stop giving a discount\".",[33,1806,1808],{"id":1807},"the-impact-of-an-outlier","The impact of an outlier",[11,1810,1811,1812,1814,1815,1819,1820,1824],{},"This is where picking a loss function stops being theory and becomes a real decision. Take the 6-house dataset from ",[22,1813,42],{"href":24}," and add ",[1816,1817,1818],"strong",{},"one"," more house, at a fixed position. You control its price with the slider, dragging from a reasonable value to one way off the trend (an ",[45,1821,1823],{"definition":1822},"a data point way outside the pattern of the others, whether from a typo, a weird measurement, or a genuinely rare case","outlier","), and watch three fitted lines live, one per loss function:",[1826,1827],"outlier-impact-explorer",{":base-x":1828,":base-y":1829,":huber-delta":1830,":initial-outlier-y":1831,":outlier-x":1832,":outlier-y-max":1833,":outlier-y-min":1834,"x-label":1835,"y-label":1836},"[1.0, 1.7, 2.0, 2.5, 3.0, 3.2]","[250, 300, 480, 430, 630, 730]","100","730","2.2","1800","200","Size (1000 sqft)","Price (1000 dollars)",[11,1838,1839],{},"Drag it all the way to the max and watch: the blue line (MSE) runs after the red dot, twisting itself to try to \"please\" the outlier. The green line (MAE) barely budges. The orange one (Huber) sits in between. That's not a coincidence of this one example, it's the direct consequence of the two shapes you saw above: squaring punishes big errors without limit, absolute value doesn't.",[33,1841,1843],{"id":1842},"wrapping-up","Wrapping up",[1845,1846,1847,1867],"table",{},[1848,1849,1850],"thead",{},[1851,1852,1853,1858,1861,1864],"tr",{},[1854,1855,1857],"th",{"align":1856},"left","Loss function",[1854,1859,1860],{"align":1856},"Formula",[1854,1862,1863],{"align":1856},"Reaction to an outlier",[1854,1865,1866],{"align":1856},"When to use",[1868,1869,1870,1884,1897],"tbody",{},[1851,1871,1872,1875,1878,1881],{},[1873,1874,48],"td",{"align":1856},[1873,1876,1877],{"align":1856},"squared error",[1873,1879,1880],{"align":1856},"sensitive, gets pulled",[1873,1882,1883],{"align":1856},"clean data, no meaningful outliers",[1851,1885,1886,1888,1891,1894],{},[1873,1887,616],{"align":1856},[1873,1889,1890],{"align":1856},"absolute error",[1873,1892,1893],{"align":1856},"robust, nearly immune",[1873,1895,1896],{"align":1856},"data with outliers you want to ignore",[1851,1898,1899,1902,1905,1908],{},[1873,1900,1901],{"align":1856},"Huber",[1873,1903,1904],{"align":1856},"hybrid (squared near zero, linear far)",[1873,1906,1907],{"align":1856},"adjustable middle ground",[1873,1909,1910],{"align":1856},"when you want a bit of both, tuned via delta",[11,1912,1913],{},"Three takeaways:",[1915,1916,1917,1924,1930],"ol",{},[1918,1919,1920,1923],"li",{},[1816,1921,1922],{},"The loss function isn't a hidden technical detail, it's a design choice"," that changes the entire model's behavior, especially in the presence of real, messy data.",[1918,1925,1926,1929],{},[1816,1927,1928],{},"Squaring punishes big errors disproportionately",", great when you trust your dataset, dangerous when you don't.",[1918,1931,1932,1935],{},[1816,1933,1934],{},"Huber exists exactly so you don't have to choose blind",": delta is a dial that slides between the two extremes.",[33,1937,1939],{"id":1938},"practical-application","Practical application",[11,1941,1942,1943,1949],{},"Same real housing dataset from the previous posts (",[22,1944,1948],{"href":1945,"rel":1946},"https:\u002F\u002Fwww.kaggle.com\u002Fdatasets\u002Fdenkuznetz\u002Fhousing-prices-regression",[1947],"nofollow","Housing Prices Regression, Kaggle","). This time, a very realistic scenario: a typo. A 60 sqft house (tiny) got mistakenly logged as costing almost $2 million.",[1951,1952,1957],"pre",{"className":1953,"code":1954,"language":1955,"meta":1956,"style":1956},"language-python shiki shiki-themes github-light github-dark","# 50 real houses + 1 typo: 60 sqft, $1.9 million\nx_with_typo = [*x_sqft, 0.6]\ny_with_typo = [*y_price, 1900]\n\nols   = fit_mse(x_with_typo, y_with_typo)                 # ordinary least squares fit\nmae   = fit_irls(x_with_typo, y_with_typo, \"mae\")         # robust to the outlier\nhuber = fit_irls(x_with_typo, y_with_typo, \"huber\", delta=50)\n\nnew_house = 1.5   # 150 sqft, same scale as the previous posts\n\nfor name, (w, b) in [(\"MSE\", ols), (\"MAE\", mae), (\"Huber\", huber)]:\n    print(f\"{name}: ${(w * new_house + b) * 1000:,.0f}\")\n","python","",[1958,1959,1960,1967,1973,1979,1986,1992,1998,2004,2009,2015,2020,2026],"code",{"__ignoreMap":1956},[53,1961,1964],{"class":1962,"line":1963},"line",1,[53,1965,1966],{},"# 50 real houses + 1 typo: 60 sqft, $1.9 million\n",[53,1968,1970],{"class":1962,"line":1969},2,[53,1971,1972],{},"x_with_typo = [*x_sqft, 0.6]\n",[53,1974,1976],{"class":1962,"line":1975},3,[53,1977,1978],{},"y_with_typo = [*y_price, 1900]\n",[53,1980,1982],{"class":1962,"line":1981},4,[53,1983,1985],{"emptyLinePlaceholder":1984},true,"\n",[53,1987,1989],{"class":1962,"line":1988},5,[53,1990,1991],{},"ols   = fit_mse(x_with_typo, y_with_typo)                 # ordinary least squares fit\n",[53,1993,1995],{"class":1962,"line":1994},6,[53,1996,1997],{},"mae   = fit_irls(x_with_typo, y_with_typo, \"mae\")         # robust to the outlier\n",[53,1999,2001],{"class":1962,"line":2000},7,[53,2002,2003],{},"huber = fit_irls(x_with_typo, y_with_typo, \"huber\", delta=50)\n",[53,2005,2007],{"class":1962,"line":2006},8,[53,2008,1985],{"emptyLinePlaceholder":1984},[53,2010,2012],{"class":1962,"line":2011},9,[53,2013,2014],{},"new_house = 1.5   # 150 sqft, same scale as the previous posts\n",[53,2016,2018],{"class":1962,"line":2017},10,[53,2019,1985],{"emptyLinePlaceholder":1984},[53,2021,2023],{"class":1962,"line":2022},11,[53,2024,2025],{},"for name, (w, b) in [(\"MSE\", ols), (\"MAE\", mae), (\"Huber\", huber)]:\n",[53,2027,2029],{"class":1962,"line":2028},12,[53,2030,2031],{},"    print(f\"{name}: ${(w * new_house + b) * 1000:,.0f}\")\n",[2033,2034,2035],"blockquote",{},[11,2036,2037,26,2040,2043,2044,2043,2047],{},[1816,2038,2039],{},"Output:",[1958,2041,2042],{},"MSE: $608,900"," \u002F ",[1958,2045,2046],{},"MAE: $570,700",[1958,2048,2049],{},"Huber: $569,800",[11,2051,2052,2053,2102,2103,2205,2206,2208],{},"Without the typo, all three give roughly the same guess for this house (around ",[53,2054,2056,2080],{"className":2055},[56],[53,2057,2059],{"className":2058},[60],[62,2060,2061],{"xmlns":64},[66,2062,2063,2077],{},[69,2064,2065,2068,2071,2074],{},[82,2066,2067],{},"567",[86,2069,2070],{},"k",[86,2072,2073],{},"t",[86,2075,2076],{},"o",[180,2078,2079],{"encoding":182},"567k to ",[53,2081,2083],{"className":2082,"ariaHidden":121},[187],[53,2084,2086,2089,2092,2096,2099],{"className":2085},[191],[53,2087],{"className":2088,"style":1180},[195],[53,2090,2067],{"className":2091},[200],[53,2093,2070],{"className":2094,"style":2095},[200,270],"margin-right:0.0315em;",[53,2097,2073],{"className":2098},[200,270],[53,2100,2076],{"className":2101},[200,270],"573k). With the typo, MSE jumps to almost ",[53,2104,2106,2151],{"className":2105},[56],[53,2107,2109],{"className":2108},[60],[62,2110,2111],{"xmlns":64},[66,2112,2113,2148],{},[69,2114,2115,2118,2120,2122,2124,2127,2129,2132,2134,2136,2138,2140,2143,2145],{},[82,2116,2117],{},"609",[86,2119,2070],{},[75,2121,136],{"separator":121},[86,2123,22],{},[86,2125,2126],{},"j",[86,2128,2076],{},[86,2130,2131],{},"l",[86,2133,2073],{},[86,2135,2076],{},[86,2137,128],{},[86,2139,2076],{},[86,2141,2142],{},"v",[86,2144,1211],{},[86,2146,2147],{},"r",[180,2149,2150],{"encoding":182},"609k, a jolt of over ",[53,2152,2154],{"className":2153,"ariaHidden":121},[187],[53,2155,2157,2161,2164,2167,2170,2173,2178,2181,2185,2188,2191,2194,2197,2200],{"className":2156},[191],[53,2158],{"className":2159,"style":2160},[195],"height:0.8889em;vertical-align:-0.1944em;",[53,2162,2117],{"className":2163},[200],[53,2165,2070],{"className":2166,"style":2095},[200,270],[53,2168,136],{"className":2169},[458],[53,2171],{"className":2172,"style":318},[208],[53,2174,2177],{"className":2175,"style":2176},[200,270],"margin-right:0.0572em;","aj",[53,2179,2076],{"className":2180},[200,270],[53,2182,2131],{"className":2183,"style":2184},[200,270],"margin-right:0.0197em;",[53,2186,2073],{"className":2187},[200,270],[53,2189,2076],{"className":2190},[200,270],[53,2192,128],{"className":2193,"style":425},[200,270],[53,2195,2076],{"className":2196},[200,270],[53,2198,2142],{"className":2199,"style":536},[200,270],[53,2201,2204],{"className":2202,"style":2203},[200,270],"margin-right:0.0278em;","er","35k just from ",[1816,2207,1818],{}," wrong row in the spreadsheet, while MAE and Huber barely move (under $3k of difference). Try it yourself, dragging the outlier:",[2210,2211],"housing-outlier-explorer",{"x-label":2212,"y-label":1836},"Size (100 sqft)",[11,2214,2215],{},"In a real database, with thousands of rows, typos happen. The choice of loss function decides whether one of those turns into your problem or not.",[2217,2218,2219],"style",{},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":1956,"searchDepth":1969,"depth":1969,"links":2221},[2222,2223,2224,2225,2226,2227,2228],{"id":35,"depth":1969,"text":36},{"id":608,"depth":1969,"text":609},{"id":1126,"depth":1969,"text":1127},{"id":1143,"depth":1969,"text":1144},{"id":1807,"depth":1969,"text":1808},{"id":1842,"depth":1969,"text":1843},{"id":1938,"depth":1969,"text":1939},null,"2026-08-18","Why we squared the error back in post 2, and what happens if you choose differently: MAE, Huber, and the real impact a single outlier has on each choice.","md",{},"\u002Fen\u002Fplaylists\u002Fmachine-learning-specialization\u002Floss-functions","machine-learning-specialization",{"title":6,"description":2231},"published","en\u002Fplaylists\u002Fmachine-learning-specialization\u002Floss-functions",[2240,2241,2242,2243,2244],"loss-function","mse","mae","huber","robustness","arTSdKZgqx98Fje6zNQdsIV1u51dSyyxETRISLcdEtQ",1787338984631]