[{"data":1,"prerenderedAt":1896},["ShallowReactive",2],{"lang-switch-post-\u002Fen\u002Fplaylists\u002Fmachine-learning-specialization\u002Fw2-lab01-numpy-vectorization":3,"post-en-machine-learning-specialization-w2-lab01-numpy-vectorization":4},"\u002Fplaylists\u002Fmachine-learning-specialization\u002Fw2-lab01-numpy-vectorization",{"id":5,"title":6,"body":7,"cover":1881,"date":1882,"description":1883,"extension":1884,"meta":1885,"navigation":326,"order":336,"path":1886,"playlist":1887,"seo":1888,"status":1889,"stem":1890,"tags":1891,"__hash__":1895},"posts\u002Fen\u002Fplaylists\u002Fmachine-learning-specialization\u002Fw2-lab01-numpy-vectorization.md","NumPy and Vectorization",{"type":8,"value":9,"toc":1867},"minimark",[10,18,26,31,39,48,51,55,305,369,376,381,403,442,446,452,486,498,504,508,526,820,823,882,886,896,902,915,926,929,932,936,1029,1063,1089,1339,1343,1564,1582,1586,1602,1653,1672,1706,1721,1764,1768,1859,1863],[11,12,13],"p",{},[14,15],"img",{"alt":16,"src":17},"The \"silent protector\" meme: a soldier labeled Linear Algebra, Statistics, Calculus, and Probability Theory takes hit after hit in place of an \"ML Newbie\" sleeping peacefully in bed, completely unaware any of it is happening","\u002Fimages\u002Fposts\u002Fmachine-learning-specialization\u002Fw2-lab01-numpy-vectorization\u002Fmeme-vectorization.jpg",[11,19,20,21,25],{},"Let's take a break from the trilogy. This post has no new model to train, no cost to compute, no hill to walk down. It's a tool post, like pulling into a gas station to swap a bald tire before it blows out on the highway. It exists because starting in Week 2 of the course the model gains several features at once (size, bedroom count, house age, all together), and at that point computing with a ",[22,23,24],"code",{},"for"," loop stops being viable. Let's understand the tool that solves this before we actually need it.",[27,28,30],"h2",{"id":29},"python-lists-work-so-why-switch","Python lists work, so why switch",[11,32,33,34,38],{},"Python already ships with lists, which store numbers just fine. So why does the entire ML community use a separate library for this? Short answer: ",[35,36,37],"strong",{},"memory",".",[11,40,41,42,47],{},"A Python list stores each number as a separate object, scattered around memory, with a bunch of extra bookkeeping stuck to it (reference count, and so on). Doing math on top of that means the processor keeps jumping from address to address, unwrapping each object to get the value inside. A ",[43,44,46],"glossary-term",{"definition":45},"a Python library specialized in numeric arrays, with every element of the same type stored side by side in memory","NumPy"," array, on the other hand, stores raw numbers of the same type, packed right next to each other in one contiguous block of memory. The processor can crunch a whole bunch of them at once, using special instructions that exist exactly for this.",[11,49,50],{},"That difference in memory layout is the real reason behind NumPy's speed. It's not magic, it's low-level engineering.",[27,52,54],{"id":53},"vectors-the-basic-building-block","Vectors: the basic building block",[11,56,57,58,106,107,111,112,172,173,202,203,249,250,304],{},"A vector is a bunch of numbers arranged in order, all of the same type. In the course's notation, a vector is a bold lowercase letter, like ",[59,60,63,87],"span",{"className":61},[62],"katex",[59,64,67],{"className":65},[66],"katex-mathml",[68,69,71],"math",{"xmlns":70},"http:\u002F\u002Fwww.w3.org\u002F1998\u002FMath\u002FMathML",[72,73,74,82],"semantics",{},[75,76,77],"mrow",{},[78,79,81],"mi",{"mathvariant":80},"bold","x",[83,84,86],"annotation",{"encoding":85},"application\u002Fx-tex","\\mathbf{x}",[59,88,92],{"className":89,"ariaHidden":91},[90],"katex-html","true",[59,93,96,101],{"className":94},[95],"base",[59,97],{"className":98,"style":100},[99],"strut","height:0.4444em;",[59,102,81],{"className":103},[104,105],"mord","mathbf",". A vector's size is called its ",[43,108,110],{"definition":109},"a tuple describing an array's layout, how many elements exist along each dimension","shape",", written ",[59,113,115,141],{"className":114},[62],[59,116,118],{"className":117},[66],[68,119,120],{"xmlns":70},[72,121,122,138],{},[75,123,124,129,132,135],{},[125,126,128],"mo",{"stretchy":127},"false","(",[78,130,131],{},"n",[125,133,134],{"separator":91},",",[125,136,137],{"stretchy":127},")",[83,139,140],{"encoding":85},"(n,)",[59,142,144],{"className":143,"ariaHidden":91},[90],[59,145,147,151,155,159,163,168],{"className":146},[95],[59,148],{"className":149,"style":150},[99],"height:1em;vertical-align:-0.25em;",[59,152,128],{"className":153},[154],"mopen",[59,156,131],{"className":157},[104,158],"mathnormal",[59,160,134],{"className":161},[162],"mpunct",[59,164],{"className":165,"style":167},[166],"mspace","margin-right:0.1667em;",[59,169,137],{"className":170},[171],"mclose"," for a vector with ",[59,174,176,189],{"className":175},[62],[59,177,179],{"className":178},[66],[68,180,181],{"xmlns":70},[72,182,183,187],{},[75,184,185],{},[78,186,131],{},[83,188,131],{"encoding":85},[59,190,192],{"className":191,"ariaHidden":91},[90],[59,193,195,199],{"className":194},[95],[59,196],{"className":197,"style":198},[99],"height:0.4306em;",[59,200,131],{"className":201},[104,158]," elements. Notice the lone comma inside the parentheses: that's Python's way of saying \"this is a one-element tuple\", and it's easy to confuse ",[59,204,206,225],{"className":205},[62],[59,207,209],{"className":208},[66],[68,210,211],{"xmlns":70},[72,212,213,223],{},[75,214,215,217,219,221],{},[125,216,128],{"stretchy":127},[78,218,131],{},[125,220,134],{"separator":91},[125,222,137],{"stretchy":127},[83,224,140],{"encoding":85},[59,226,228],{"className":227,"ariaHidden":91},[90],[59,229,231,234,237,240,243,246],{"className":230},[95],[59,232],{"className":233,"style":150},[99],[59,235,128],{"className":236},[154],[59,238,131],{"className":239},[104,158],[59,241,134],{"className":242},[162],[59,244],{"className":245,"style":167},[166],[59,247,137],{"className":248},[171]," with ",[59,251,253,277],{"className":252},[62],[59,254,256],{"className":255},[66],[68,257,258],{"xmlns":70},[72,259,260,274],{},[75,261,262,264,266,268,272],{},[125,263,128],{"stretchy":127},[78,265,131],{},[125,267,134],{"separator":91},[269,270,271],"mn",{},"1",[125,273,137],{"stretchy":127},[83,275,276],{"encoding":85},"(n,1)",[59,278,280],{"className":279,"ariaHidden":91},[90],[59,281,283,286,289,292,295,298,301],{"className":282},[95],[59,284],{"className":285,"style":150},[99],[59,287,128],{"className":288},[154],[59,290,131],{"className":291},[104,158],[59,293,134],{"className":294},[162],[59,296],{"className":297,"style":167},[166],[59,299,271],{"className":300},[104],[59,302,137],{"className":303},[171]," later on, so keep an eye on that.",[306,307,312],"pre",{"className":308,"code":309,"language":310,"meta":311,"style":311},"language-python shiki shiki-themes github-light github-dark","import numpy as np\n\na = np.zeros(4)\nprint(f\"np.zeros(4): a = {a}, shape = {a.shape}, dtype = {a.dtype}\")\n# a = [0. 0. 0. 0.], shape = (4,), dtype = float64\n\na = np.array([5, 4, 3, 2])\nprint(f\"np.array: a = {a}, shape = {a.shape}, dtype = {a.dtype}\")\n# a = [5 4 3 2], shape = (4,), dtype = int64\n","python","",[22,313,314,321,328,334,340,346,351,357,363],{"__ignoreMap":311},[59,315,318],{"class":316,"line":317},"line",1,[59,319,320],{},"import numpy as np\n",[59,322,324],{"class":316,"line":323},2,[59,325,327],{"emptyLinePlaceholder":326},true,"\n",[59,329,331],{"class":316,"line":330},3,[59,332,333],{},"a = np.zeros(4)\n",[59,335,337],{"class":316,"line":336},4,[59,338,339],{},"print(f\"np.zeros(4): a = {a}, shape = {a.shape}, dtype = {a.dtype}\")\n",[59,341,343],{"class":316,"line":342},5,[59,344,345],{},"# a = [0. 0. 0. 0.], shape = (4,), dtype = float64\n",[59,347,349],{"class":316,"line":348},6,[59,350,327],{"emptyLinePlaceholder":326},[59,352,354],{"class":316,"line":353},7,[59,355,356],{},"a = np.array([5, 4, 3, 2])\n",[59,358,360],{"class":316,"line":359},8,[59,361,362],{},"print(f\"np.array: a = {a}, shape = {a.shape}, dtype = {a.dtype}\")\n",[59,364,366],{"class":316,"line":365},9,[59,367,368],{},"# a = [5 4 3 2], shape = (4,), dtype = int64\n",[11,370,371,372,375],{},"A detail that tripped me up as a newcomer: a single decimal-point value in the list is enough for the whole array to become ",[22,373,374],{},"float64",". Makes sense, since every element of an array has to share the same type.",[377,378,380],"h3",{"id":379},"indexing-and-slicing","Indexing and slicing",[11,382,383,384,387,388,391,392,395,396,399,400,402],{},"Indexing (",[22,385,386],{},"a[2]",") grabs one element, while slicing (",[22,389,390],{},"a[2:7]",") grabs a chunk. The rules are the same as Python lists: counting starts at zero, negative indices count from the end (",[22,393,394],{},"a[-1]"," is the last element), and the end of a slice is ",[35,397,398],{},"not"," included (",[22,401,390],{}," grabs indices 2, 3, 4, 5, and 6, five elements, not six).",[306,404,406],{"className":308,"code":405,"language":310,"meta":311,"style":311},"a = np.arange(10)          # [0 1 2 3 4 5 6 7 8 9]\n\nprint(a[2])                 # 2, grabbing one element = a scalar\nprint(a[-1])                 # 9, the last element\nprint(a[2:7:1])              # [2 3 4 5 6], from index 2 through 6\nprint(a[3:])                  # [3 4 5 6 7 8 9], from index 3 to the end\nprint(a[:3])                   # [0 1 2], from the start through index 2\n",[22,407,408,413,417,422,427,432,437],{"__ignoreMap":311},[59,409,410],{"class":316,"line":317},[59,411,412],{},"a = np.arange(10)          # [0 1 2 3 4 5 6 7 8 9]\n",[59,414,415],{"class":316,"line":323},[59,416,327],{"emptyLinePlaceholder":326},[59,418,419],{"class":316,"line":330},[59,420,421],{},"print(a[2])                 # 2, grabbing one element = a scalar\n",[59,423,424],{"class":316,"line":336},[59,425,426],{},"print(a[-1])                 # 9, the last element\n",[59,428,429],{"class":316,"line":342},[59,430,431],{},"print(a[2:7:1])              # [2 3 4 5 6], from index 2 through 6\n",[59,433,434],{"class":316,"line":348},[59,435,436],{},"print(a[3:])                  # [3 4 5 6 7 8 9], from index 3 to the end\n",[59,438,439],{"class":316,"line":353},[59,440,441],{},"print(a[:3])                   # [0 1 2], from the start through index 2\n",[377,443,445],{"id":444},"operations-with-no-loop-at-all","Operations with no loop at all",[11,447,448,449,451],{},"This is where the good part lives. Vector operations run over the entire array at once, with no ",[22,450,24],{}," written by you:",[306,453,455],{"className":308,"code":454,"language":310,"meta":311,"style":311},"a = np.array([1, 2, 3, 4])\n\nprint(-a)          # [-1 -2 -3 -4], flips the sign of everyone at once\nprint(np.sum(a))   # 10, sums it all up\nprint(a ** 2)       # [1 4 9 16], squares every element\nprint(5 * a)          # [5 10 15 20], multiplies every element by 5\n",[22,456,457,462,466,471,476,481],{"__ignoreMap":311},[59,458,459],{"class":316,"line":317},[59,460,461],{},"a = np.array([1, 2, 3, 4])\n",[59,463,464],{"class":316,"line":323},[59,465,327],{"emptyLinePlaceholder":326},[59,467,468],{"class":316,"line":330},[59,469,470],{},"print(-a)          # [-1 -2 -3 -4], flips the sign of everyone at once\n",[59,472,473],{"class":316,"line":336},[59,474,475],{},"print(np.sum(a))   # 10, sums it all up\n",[59,477,478],{"class":316,"line":342},[59,479,480],{},"print(a ** 2)       # [1 4 9 16], squares every element\n",[59,482,483],{"class":316,"line":348},[59,484,485],{},"print(5 * a)          # [5 10 15 20], multiplies every element by 5\n",[11,487,488,489,492,493,497],{},"Notice that last example: the ",[22,490,491],{},"5"," was just a loose number, but NumPy \"stretched\" it on its own to match the array's 4 elements. That has a name, ",[43,494,496],{"definition":495},"when NumPy stretches a number (or a smaller array) to match the shape of a bigger array, with no loop written by you","broadcasting",", and it'll show up a lot from here on.",[11,499,500,501,503],{},"That's the whole point of this lab: you describe the operation over the entire array, and let NumPy figure out how to apply it to each element under the hood. If you catch yourself writing a ",[22,502,24],{}," to walk through an array, there's probably a vectorized equivalent waiting to be used instead.",[27,505,507],{"id":506},"the-dot-product-the-operation-linear-regression-runs-on","The dot product: the operation linear regression runs on",[11,509,510,511,515,516,519,520,525],{},"The ",[43,512,514],{"definition":513},"multiplying two vectors element by element and summing everything up, ending in a single number","dot product"," is this lab's most important operation, because it's literally what's going to replace that ",[22,517,518],{},"w * x[i] + b"," we wrote with a loop ",[521,522,524],"a",{"href":523},"\u002Fen\u002Fplaylists\u002Fmachine-learning-specialization\u002Flab02-model-representation","in the previous posts",", once the model gains several features.",[11,527,528],{},[59,529,531,592],{"className":530},[62],[59,532,534],{"className":533},[66],[68,535,536],{"xmlns":70},[72,537,538,589],{},[75,539,540,542,545,548,551,576,583],{},[78,541,521],{"mathvariant":80},[125,543,544],{},"⋅",[78,546,547],{"mathvariant":80},"b",[125,549,550],{},"=",[552,553,554,557,567],"msubsup",{},[125,555,556],{},"∑",[75,558,559,562,564],{},[78,560,561],{},"i",[125,563,550],{},[269,565,566],{},"0",[75,568,569,571,574],{},[78,570,131],{},[125,572,573],{},"−",[269,575,271],{},[577,578,579,581],"msub",{},[78,580,521],{},[78,582,561],{},[577,584,585,587],{},[78,586,547],{},[78,588,561],{},[83,590,591],{"encoding":85},"\\mathbf{a} \\cdot \\mathbf{b} = \\sum_{i=0}^{n-1} a_i b_i",[59,593,595,616,637],{"className":594,"ariaHidden":91},[90],[59,596,598,602,605,609,613],{"className":597},[95],[59,599],{"className":600,"style":601},[99],"height:0.4445em;",[59,603,521],{"className":604},[104,105],[59,606],{"className":607,"style":608},[166],"margin-right:0.2222em;",[59,610,544],{"className":611},[612],"mbin",[59,614],{"className":615,"style":608},[166],[59,617,619,623,626,630,634],{"className":618},[95],[59,620],{"className":621,"style":622},[99],"height:0.6944em;",[59,624,547],{"className":625},[104,105],[59,627],{"className":628,"style":629},[166],"margin-right:0.2778em;",[59,631,550],{"className":632},[633],"mrel",[59,635],{"className":636,"style":629},[166],[59,638,640,644,734,737,780],{"className":639},[95],[59,641],{"className":642,"style":643},[99],"height:1.2537em;vertical-align:-0.2997em;",[59,645,648,654],{"className":646},[647],"mop",[59,649,556],{"className":650,"style":653},[647,651,652],"op-symbol","small-op","position:relative;top:0em;",[59,655,658],{"className":656},[657],"msupsub",[59,659,663,725],{"className":660},[661,662],"vlist-t","vlist-t2",[59,664,667,720],{"className":665},[666],"vlist-r",[59,668,672,699],{"className":669,"style":671},[670],"vlist","height:0.954em;",[59,673,675,680],{"style":674},"top:-2.4003em;margin-left:0em;margin-right:0.05em;",[59,676],{"className":677,"style":679},[678],"pstrut","height:2.7em;",[59,681,687],{"className":682},[683,684,685,686],"sizing","reset-size6","size3","mtight",[59,688,690,693,696],{"className":689},[104,686],[59,691,561],{"className":692},[104,158,686],[59,694,550],{"className":695},[633,686],[59,697,566],{"className":698},[104,686],[59,700,702,705],{"style":701},"top:-3.2029em;margin-right:0.05em;",[59,703],{"className":704,"style":679},[678],[59,706,708],{"className":707},[683,684,685,686],[59,709,711,714,717],{"className":710},[104,686],[59,712,131],{"className":713},[104,158,686],[59,715,573],{"className":716},[612,686],[59,718,271],{"className":719},[104,686],[59,721,724],{"className":722},[723],"vlist-s","​",[59,726,728],{"className":727},[666],[59,729,732],{"className":730,"style":731},[670],"height:0.2997em;",[59,733],{},[59,735],{"className":736,"style":167},[166],[59,738,740,743],{"className":739},[104],[59,741,521],{"className":742},[104,158],[59,744,746],{"className":745},[657],[59,747,749,771],{"className":748},[661,662],[59,750,752,768],{"className":751},[666],[59,753,756],{"className":754,"style":755},[670],"height:0.3117em;",[59,757,759,762],{"style":758},"top:-2.55em;margin-left:0em;margin-right:0.05em;",[59,760],{"className":761,"style":679},[678],[59,763,765],{"className":764},[683,684,685,686],[59,766,561],{"className":767},[104,158,686],[59,769,724],{"className":770},[723],[59,772,774],{"className":773},[666],[59,775,778],{"className":776,"style":777},[670],"height:0.15em;",[59,779],{},[59,781,783,786],{"className":782},[104],[59,784,547],{"className":785},[104,158],[59,787,789],{"className":788},[657],[59,790,792,812],{"className":791},[661,662],[59,793,795,809],{"className":794},[666],[59,796,798],{"className":797,"style":755},[670],[59,799,800,803],{"style":758},[59,801],{"className":802,"style":679},[678],[59,804,806],{"className":805},[683,684,685,686],[59,807,561],{"className":808},[104,158,686],[59,810,724],{"className":811},[723],[59,813,815],{"className":814},[666],[59,816,818],{"className":817,"style":777},[670],[59,819],{},[11,821,822],{},"Multiply pairwise, sum it all up. That simple. Before reaching for NumPy's built-in version, I wrote my own, just to make what's happening under the hood explicit:",[306,824,826],{"className":308,"code":825,"language":310,"meta":311,"style":311},"def my_dot(a, b):\n    x = 0\n    for i in range(a.shape[0]):\n        x = x + a[i] * b[i]\n    return x\n\na = np.array([1, 2, 3, 4])\nb = np.array([-1, 4, 3, 2])\n\nprint(my_dot(a, b))       # 24\nprint(np.dot(a, b))       # 24, same result, NumPy's built-in function\n",[22,827,828,833,838,843,848,853,857,861,866,870,876],{"__ignoreMap":311},[59,829,830],{"class":316,"line":317},[59,831,832],{},"def my_dot(a, b):\n",[59,834,835],{"class":316,"line":323},[59,836,837],{},"    x = 0\n",[59,839,840],{"class":316,"line":330},[59,841,842],{},"    for i in range(a.shape[0]):\n",[59,844,845],{"class":316,"line":336},[59,846,847],{},"        x = x + a[i] * b[i]\n",[59,849,850],{"class":316,"line":342},[59,851,852],{},"    return x\n",[59,854,855],{"class":316,"line":348},[59,856,327],{"emptyLinePlaceholder":326},[59,858,859],{"class":316,"line":353},[59,860,461],{},[59,862,863],{"class":316,"line":359},[59,864,865],{},"b = np.array([-1, 4, 3, 2])\n",[59,867,868],{"class":316,"line":365},[59,869,327],{"emptyLinePlaceholder":326},[59,871,873],{"class":316,"line":872},10,[59,874,875],{},"print(my_dot(a, b))       # 24\n",[59,877,879],{"class":316,"line":878},11,[59,880,881],{},"print(np.dot(a, b))       # 24, same result, NumPy's built-in function\n",[27,883,885],{"id":884},"the-speed-gap-live-in-your-browser","The speed gap, live in your browser",[11,887,888,889,892,893,895],{},"The original notebook tests this with 10-million-element arrays, comparing ",[22,890,891],{},"np.dot"," (vectorized) against a hand-written ",[22,894,24],{}," loop, in Python. The gain there runs dozens to hundreds of times over.",[11,897,898,899,901],{},"Below, you can run a similar test, except in your own browser, right now, in JavaScript. I need to be honest with you about one thing first, though: the browser is ",[35,900,398],{}," going to show a 100x gain. And it's not because the test is broken.",[11,903,904,905,907,908,911,912,914],{},"Your browser's JavaScript engine (V8, if you're on Chrome or Edge) already optimizes a plain ",[22,906,24],{}," loop in a way pretty similar to what it does for built-in methods like ",[22,909,910],{},".reduce()",". When I compared \"loop\" against \"",[22,913,910],{},"\" on the same kind of array, it came out close to a tie, and that was expected, not a bug in the test.",[11,916,917,918,921,922,925],{},"The comparison that actually shows a real, honest gap is a different one: a plain JavaScript ",[22,919,920],{},"Array"," (which stores numbers somewhat \"boxed up\", similar to the Python list we talked about above) versus a ",[22,923,924],{},"Float64Array"," (contiguous memory, single type, no boxing). That's the same underlying reason NumPy is fast, just reproduced here in the browser with a smaller, honest gain (usually 2 to 5x, sometimes more), instead of me pretending I could recreate Python's giant number in an environment that wasn't built for that.",[927,928],"vectorization-benchmark",{},[11,930,931],{},"Run it a few times with different sizes. Notice that \"Array comum + laço\" (plain Array + loop) tends to be visibly slower than the other two, and that \"Float64Array + laço\" (typed array + loop) and \"Float64Array + .reduce()\" land close to each other. That's exactly the pattern the explanation above predicted.",[27,933,935],{"id":934},"matrices-when-one-vector-isnt-enough","Matrices: when one vector isn't enough",[11,937,938,939,970,971,1024,1025,1028],{},"A matrix is a two-dimensional array, written with a bold uppercase letter (",[59,940,942,957],{"className":941},[62],[59,943,945],{"className":944},[66],[68,946,947],{"xmlns":70},[72,948,949,954],{},[75,950,951],{},[78,952,953],{"mathvariant":80},"X",[83,955,956],{"encoding":85},"\\mathbf{X}",[59,958,960],{"className":959,"ariaHidden":91},[90],[59,961,963,967],{"className":962},[95],[59,964],{"className":965,"style":966},[99],"height:0.6861em;",[59,968,953],{"className":969},[104,105],"), with shape ",[59,972,974,997],{"className":973},[62],[59,975,977],{"className":976},[66],[68,978,979],{"xmlns":70},[72,980,981,994],{},[75,982,983,985,988,990,992],{},[125,984,128],{"stretchy":127},[78,986,987],{},"m",[125,989,134],{"separator":91},[78,991,131],{},[125,993,137],{"stretchy":127},[83,995,996],{"encoding":85},"(m, n)",[59,998,1000],{"className":999,"ariaHidden":91},[90],[59,1001,1003,1006,1009,1012,1015,1018,1021],{"className":1002},[95],[59,1004],{"className":1005,"style":150},[99],[59,1007,128],{"className":1008},[154],[59,1010,987],{"className":1011},[104,158],[59,1013,134],{"className":1014},[162],[59,1016],{"className":1017,"style":167},[166],[59,1019,131],{"className":1020},[104,158],[59,1022,137],{"className":1023},[171],". In the course's context the convention is always the same: ",[35,1026,1027],{},"row is a training example, column is a feature",". One house per row, one attribute (size, bedrooms, age) per column.",[306,1030,1032],{"className":308,"code":1031,"language":310,"meta":311,"style":311},"X = np.array([[1, 5], [2, 3], [3, 1]])   # shape (3, 2): 3 examples, 2 features each\n\nprint(X.shape)      # (3, 2)\nprint(X[1])          # [2 3], the entire row 1, becomes a 1-D vector\nprint(X[1, 0])         # 2, one specific element, becomes a scalar\nprint(X[:, 0])           # [1 2 3], the entire column 0\n",[22,1033,1034,1039,1043,1048,1053,1058],{"__ignoreMap":311},[59,1035,1036],{"class":316,"line":317},[59,1037,1038],{},"X = np.array([[1, 5], [2, 3], [3, 1]])   # shape (3, 2): 3 examples, 2 features each\n",[59,1040,1041],{"class":316,"line":323},[59,1042,327],{"emptyLinePlaceholder":326},[59,1044,1045],{"class":316,"line":330},[59,1046,1047],{},"print(X.shape)      # (3, 2)\n",[59,1049,1050],{"class":316,"line":336},[59,1051,1052],{},"print(X[1])          # [2 3], the entire row 1, becomes a 1-D vector\n",[59,1054,1055],{"class":316,"line":342},[59,1056,1057],{},"print(X[1, 0])         # 2, one specific element, becomes a scalar\n",[59,1059,1060],{"class":316,"line":348},[59,1061,1062],{},"print(X[:, 0])           # [1 2 3], the entire column 0\n",[11,1064,1065,1066,1069,1070,1073,1074,1077,1078,1080,1081,1084,1085,1088],{},"The detail that confuses newcomers the most here: indexing a matrix with just ",[35,1067,1068],{},"one"," index (",[22,1071,1072],{},"X[1]",") returns an array with ",[35,1075,1076],{},"one fewer dimension",", not a single-row matrix. ",[22,1079,1072],{}," has shape ",[22,1082,1083],{},"(2,)",", not ",[22,1086,1087],{},"(1, 2)",". This specific gotcha is behind a good chunk of the dimension errors you'll run into later in the course, and I've learned to flag it mentally every time I touch shapes.",[11,1090,1091,1092,1120,1121,1149,1150,1196,1197,1228,1229,1275,1276,1305,1306,1309,1310,1338],{},"And this is exactly why the dot product comes back into play: once the model has ",[59,1093,1095,1108],{"className":1094},[62],[59,1096,1098],{"className":1097},[66],[68,1099,1100],{"xmlns":70},[72,1101,1102,1106],{},[75,1103,1104],{},[78,1105,131],{},[83,1107,131],{"encoding":85},[59,1109,1111],{"className":1110,"ariaHidden":91},[90],[59,1112,1114,1117],{"className":1113},[95],[59,1115],{"className":1116,"style":198},[99],[59,1118,131],{"className":1119},[104,158]," features, each row of the matrix ",[59,1122,1124,1137],{"className":1123},[62],[59,1125,1127],{"className":1126},[66],[68,1128,1129],{"xmlns":70},[72,1130,1131,1135],{},[75,1132,1133],{},[78,1134,953],{"mathvariant":80},[83,1136,956],{"encoding":85},[59,1138,1140],{"className":1139,"ariaHidden":91},[90],[59,1141,1143,1146],{"className":1142},[95],[59,1144],{"className":1145,"style":966},[99],[59,1147,953],{"className":1148},[104,105]," becomes a vector of shape ",[59,1151,1153,1172],{"className":1152},[62],[59,1154,1156],{"className":1155},[66],[68,1157,1158],{"xmlns":70},[72,1159,1160,1170],{},[75,1161,1162,1164,1166,1168],{},[125,1163,128],{"stretchy":127},[78,1165,131],{},[125,1167,134],{"separator":91},[125,1169,137],{"stretchy":127},[83,1171,140],{"encoding":85},[59,1173,1175],{"className":1174,"ariaHidden":91},[90],[59,1176,1178,1181,1184,1187,1190,1193],{"className":1177},[95],[59,1179],{"className":1180,"style":150},[99],[59,1182,128],{"className":1183},[154],[59,1185,131],{"className":1186},[104,158],[59,1188,134],{"className":1189},[162],[59,1191],{"className":1192,"style":167},[166],[59,1194,137],{"className":1195},[171],", ready to take a dot product directly with the weight vector ",[59,1198,1200,1215],{"className":1199},[62],[59,1201,1203],{"className":1202},[66],[68,1204,1205],{"xmlns":70},[72,1206,1207,1212],{},[75,1208,1209],{},[78,1210,1211],{"mathvariant":80},"w",[83,1213,1214],{"encoding":85},"\\mathbf{w}",[59,1216,1218],{"className":1217,"ariaHidden":91},[90],[59,1219,1221,1224],{"className":1220},[95],[59,1222],{"className":1223,"style":100},[99],[59,1225,1211],{"className":1226,"style":1227},[104,105],"margin-right:0.016em;",", also of shape ",[59,1230,1232,1251],{"className":1231},[62],[59,1233,1235],{"className":1234},[66],[68,1236,1237],{"xmlns":70},[72,1238,1239,1249],{},[75,1240,1241,1243,1245,1247],{},[125,1242,128],{"stretchy":127},[78,1244,131],{},[125,1246,134],{"separator":91},[125,1248,137],{"stretchy":127},[83,1250,140],{"encoding":85},[59,1252,1254],{"className":1253,"ariaHidden":91},[90],[59,1255,1257,1260,1263,1266,1269,1272],{"className":1256},[95],[59,1258],{"className":1259,"style":150},[99],[59,1261,128],{"className":1262},[154],[59,1264,131],{"className":1265},[104,158],[59,1267,134],{"className":1268},[162],[59,1270],{"className":1271,"style":167},[166],[59,1273,137],{"className":1274},[171],". Example ",[59,1277,1279,1292],{"className":1278},[62],[59,1280,1282],{"className":1281},[66],[68,1283,1284],{"xmlns":70},[72,1285,1286,1290],{},[75,1287,1288],{},[78,1289,561],{},[83,1291,561],{"encoding":85},[59,1293,1295],{"className":1294,"ariaHidden":91},[90],[59,1296,1298,1302],{"className":1297},[95],[59,1299],{"className":1300,"style":1301},[99],"height:0.6595em;",[59,1303,561],{"className":1304},[104,158],"'s prediction becomes ",[22,1307,1308],{},"np.dot(w, X[i]) + b",", a single line of code, no loop at all, whether ",[59,1311,1313,1326],{"className":1312},[62],[59,1314,1316],{"className":1315},[66],[68,1317,1318],{"xmlns":70},[72,1319,1320,1324],{},[75,1321,1322],{},[78,1323,131],{},[83,1325,131],{"encoding":85},[59,1327,1329],{"className":1328,"ariaHidden":91},[90],[59,1330,1332,1335],{"className":1331},[95],[59,1333],{"className":1334,"style":198},[99],[59,1336,131],{"className":1337},[104,158]," is 1 or 100.",[27,1340,1342],{"id":1341},"wrapping-up","Wrapping up",[1344,1345,1346,1360],"table",{},[1347,1348,1349],"thead",{},[1350,1351,1352,1357],"tr",{},[1353,1354,1356],"th",{"align":1355},"left","Topic",[1353,1358,1359],{"align":1355},"What we established",[1361,1362,1363,1372,1526,1534,1545,1556],"tbody",{},[1350,1364,1365,1369],{},[1366,1367,1368],"td",{"align":1355},"Why NumPy",[1366,1370,1371],{"align":1355},"contiguous, typed memory with no per-element boxing is where the speed comes from",[1350,1373,1374,1377],{},[1366,1375,1376],{"align":1355},"Shape",[1366,1378,1379,1425,1426,1478,1479,1525],{"align":1355},[59,1380,1382,1401],{"className":1381},[62],[59,1383,1385],{"className":1384},[66],[68,1386,1387],{"xmlns":70},[72,1388,1389,1399],{},[75,1390,1391,1393,1395,1397],{},[125,1392,128],{"stretchy":127},[78,1394,131],{},[125,1396,134],{"separator":91},[125,1398,137],{"stretchy":127},[83,1400,140],{"encoding":85},[59,1402,1404],{"className":1403,"ariaHidden":91},[90],[59,1405,1407,1410,1413,1416,1419,1422],{"className":1406},[95],[59,1408],{"className":1409,"style":150},[99],[59,1411,128],{"className":1412},[154],[59,1414,131],{"className":1415},[104,158],[59,1417,134],{"className":1418},[162],[59,1420],{"className":1421,"style":167},[166],[59,1423,137],{"className":1424},[171]," is a vector, ",[59,1427,1429,1451],{"className":1428},[62],[59,1430,1432],{"className":1431},[66],[68,1433,1434],{"xmlns":70},[72,1435,1436,1448],{},[75,1437,1438,1440,1442,1444,1446],{},[125,1439,128],{"stretchy":127},[78,1441,987],{},[125,1443,134],{"separator":91},[78,1445,131],{},[125,1447,137],{"stretchy":127},[83,1449,1450],{"encoding":85},"(m,n)",[59,1452,1454],{"className":1453,"ariaHidden":91},[90],[59,1455,1457,1460,1463,1466,1469,1472,1475],{"className":1456},[95],[59,1458],{"className":1459,"style":150},[99],[59,1461,128],{"className":1462},[154],[59,1464,987],{"className":1465},[104,158],[59,1467,134],{"className":1468},[162],[59,1470],{"className":1471,"style":167},[166],[59,1473,131],{"className":1474},[104,158],[59,1476,137],{"className":1477},[171]," is a matrix, and the lone comma in ",[59,1480,1482,1501],{"className":1481},[62],[59,1483,1485],{"className":1484},[66],[68,1486,1487],{"xmlns":70},[72,1488,1489,1499],{},[75,1490,1491,1493,1495,1497],{},[125,1492,128],{"stretchy":127},[78,1494,131],{},[125,1496,134],{"separator":91},[125,1498,137],{"stretchy":127},[83,1500,140],{"encoding":85},[59,1502,1504],{"className":1503,"ariaHidden":91},[90],[59,1505,1507,1510,1513,1516,1519,1522],{"className":1506},[95],[59,1508],{"className":1509,"style":150},[99],[59,1511,128],{"className":1512},[154],[59,1514,131],{"className":1515},[104,158],[59,1517,134],{"className":1518},[162],[59,1520],{"className":1521,"style":167},[166],[59,1523,137],{"className":1524},[171]," matters",[1350,1527,1528,1531],{},[1366,1529,1530],{"align":1355},"Indexing",[1366,1532,1533],{"align":1355},"starts at zero, and a single index into a matrix returns one fewer dimension",[1350,1535,1536,1539],{},[1366,1537,1538],{"align":1355},"Slicing",[1366,1540,1541,1542],{"align":1355},"the end is always exclusive, same as Python's ",[22,1543,1544],{},"range()",[1350,1546,1547,1550],{},[1366,1548,1549],{"align":1355},"Dot product",[1366,1551,1552,1555],{"align":1355},[22,1553,1554],{},"np.dot(a, b)",", multiplies pairwise and sums, returns a scalar",[1350,1557,1558,1561],{},[1366,1559,1560],{"align":1355},"Vectorization",[1366,1562,1563],{"align":1355},"the real gain comes from contiguous, single-type memory, not \"library magic\"",[11,1565,1566,1569,1570,1573,1574,1577,1578,1581],{},[35,1567,1568],{},"How this connects to the rest of the playlist:"," with these tools in hand, the model you built in ",[521,1571,1572],{"href":523},"Lab 02"," with a hand-written ",[22,1575,1576],{},"w * x + b"," becomes ",[22,1579,1580],{},"np.dot(w, x) + b",", working for any number of features without rewriting anything. This exact foundation is what the course uses to extend everything you've already seen (model, cost, gradient descent) to linear regression with multiple variables.",[27,1583,1585],{"id":1584},"practical-application","Practical application",[11,1587,1588,1589,1595,1596,1598,1599,1601],{},"Same real housing dataset from the previous posts (",[521,1590,1594],{"href":1591,"rel":1592},"https:\u002F\u002Fwww.kaggle.com\u002Fdatasets\u002Fdenkuznetz\u002Fhousing-prices-regression",[1593],"nofollow","Housing Prices Regression, Kaggle","). This post doesn't train anything, so here the idea is just applying the matrix\u002Fvector mechanics on top of real column names, instead of the toy ",[22,1597,521],{}," and ",[22,1600,547],{}," examples.",[306,1603,1605],{"className":308,"code":1604,"language":310,"meta":311,"style":311},"import pandas as pd\nimport numpy as np\n\ndf = pd.read_csv(\"real_estate_dataset.csv\")\n\nX = df[[\"Square_Feet\", \"Num_Bedrooms\"]].to_numpy()[:3]   # first 3 houses, 2 features\n\nprint(\"X.shape:\", X.shape)        # (3, 2): 3 examples, 2 features\nprint(\"X[1]:\", X[1])              # the entire second house, becomes a 1-D vector\nprint(\"X[:, 0]:\", X[:, 0])        # the entire Square_Feet column, all houses\n",[22,1606,1607,1612,1616,1620,1625,1629,1634,1638,1643,1648],{"__ignoreMap":311},[59,1608,1609],{"class":316,"line":317},[59,1610,1611],{},"import pandas as pd\n",[59,1613,1614],{"class":316,"line":323},[59,1615,320],{},[59,1617,1618],{"class":316,"line":330},[59,1619,327],{"emptyLinePlaceholder":326},[59,1621,1622],{"class":316,"line":336},[59,1623,1624],{},"df = pd.read_csv(\"real_estate_dataset.csv\")\n",[59,1626,1627],{"class":316,"line":342},[59,1628,327],{"emptyLinePlaceholder":326},[59,1630,1631],{"class":316,"line":348},[59,1632,1633],{},"X = df[[\"Square_Feet\", \"Num_Bedrooms\"]].to_numpy()[:3]   # first 3 houses, 2 features\n",[59,1635,1636],{"class":316,"line":353},[59,1637,327],{"emptyLinePlaceholder":326},[59,1639,1640],{"class":316,"line":359},[59,1641,1642],{},"print(\"X.shape:\", X.shape)        # (3, 2): 3 examples, 2 features\n",[59,1644,1645],{"class":316,"line":365},[59,1646,1647],{},"print(\"X[1]:\", X[1])              # the entire second house, becomes a 1-D vector\n",[59,1649,1650],{"class":316,"line":872},[59,1651,1652],{},"print(\"X[:, 0]:\", X[:, 0])        # the entire Square_Feet column, all houses\n",[1654,1655,1656],"blockquote",{},[11,1657,1658,1661,1662,1665,1666,1665,1669],{},[35,1659,1660],{},"Output:"," ",[22,1663,1664],{},"X.shape: (3, 2)"," \u002F ",[22,1667,1668],{},"X[1]: [55.15  5.]",[22,1670,1671],{},"X[:, 0]: [143.64  55.15 202.96]",[306,1673,1675],{"className":308,"code":1674,"language":310,"meta":311,"style":311},"# A real dot product, with arbitrary weights just to illustrate the mechanics\n# (we're not training anything in this post, this isn't the \"right\" fit):\nw = np.array([2000, 10000])\n\nfor i in range(X.shape[0]):\n    print(f\"np.dot(w, X[{i}]) = {np.dot(w, X[i]):,.0f}\")\n",[22,1676,1677,1682,1687,1692,1696,1701],{"__ignoreMap":311},[59,1678,1679],{"class":316,"line":317},[59,1680,1681],{},"# A real dot product, with arbitrary weights just to illustrate the mechanics\n",[59,1683,1684],{"class":316,"line":323},[59,1685,1686],{},"# (we're not training anything in this post, this isn't the \"right\" fit):\n",[59,1688,1689],{"class":316,"line":330},[59,1690,1691],{},"w = np.array([2000, 10000])\n",[59,1693,1694],{"class":316,"line":336},[59,1695,327],{"emptyLinePlaceholder":326},[59,1697,1698],{"class":316,"line":342},[59,1699,1700],{},"for i in range(X.shape[0]):\n",[59,1702,1703],{"class":316,"line":348},[59,1704,1705],{},"    print(f\"np.dot(w, X[{i}]) = {np.dot(w, X[i]):,.0f}\")\n",[1654,1707,1708],{},[11,1709,1710,1661,1712,1665,1715,1665,1718],{},[35,1711,1660],{},[22,1713,1714],{},"np.dot(w, X[0]) = 297,280",[22,1716,1717],{},"np.dot(w, X[1]) = 160,300",[22,1719,1720],{},"np.dot(w, X[2]) = 455,920",[11,1722,1723,1724,1598,1727,1730,1731,1734,1735,1763],{},"Same math, same mechanics from the rest of the post, just running on top of real ",[22,1725,1726],{},"Square_Feet",[22,1728,1729],{},"Num_Bedrooms"," instead of ",[22,1732,1733],{},"[1, 2, 3, 4]",". Actually training these weights for real (finding the ",[59,1736,1738,1751],{"className":1737},[62],[59,1739,1741],{"className":1740},[66],[68,1742,1743],{"xmlns":70},[72,1744,1745,1749],{},[75,1746,1747],{},[78,1748,1211],{"mathvariant":80},[83,1750,1214],{"encoding":85},[59,1752,1754],{"className":1753,"ariaHidden":91},[90],[59,1755,1757,1760],{"className":1756},[95],[59,1758],{"className":1759,"style":100},[99],[59,1761,1211],{"className":1762,"style":1227},[104,105]," that makes sense for the data) is content further ahead in the course, with multiple features at once.",[377,1765,1767],{"id":1766},"picking-the-matrix-columns-live","Picking the matrix columns, live",[11,1769,1770,1771,1799,1800,1598,1828,1858],{},"Remember ",[59,1772,1774,1787],{"className":1773},[62],[59,1775,1777],{"className":1776},[66],[68,1778,1779],{"xmlns":70},[72,1780,1781,1785],{},[75,1782,1783],{},[78,1784,953],{"mathvariant":80},[83,1786,956],{"encoding":85},[59,1788,1790],{"className":1789,"ariaHidden":91},[90],[59,1791,1793,1796],{"className":1792},[95],[59,1794],{"className":1795,"style":966},[99],[59,1797,953],{"className":1798},[104,105]," is just a matrix, one row per example, one column per feature? Pick which two columns become the ",[59,1801,1803,1816],{"className":1802},[62],[59,1804,1806],{"className":1805},[66],[68,1807,1808],{"xmlns":70},[72,1809,1810,1814],{},[75,1811,1812],{},[78,1813,81],{},[83,1815,81],{"encoding":85},[59,1817,1819],{"className":1818,"ariaHidden":91},[90],[59,1820,1822,1825],{"className":1821},[95],[59,1823],{"className":1824,"style":198},[99],[59,1826,81],{"className":1827},[104,158],[59,1829,1831,1845],{"className":1830},[62],[59,1832,1834],{"className":1833},[66],[68,1835,1836],{"xmlns":70},[72,1837,1838,1843],{},[75,1839,1840],{},[78,1841,1842],{},"z",[83,1844,1842],{"encoding":85},[59,1846,1848],{"className":1847,"ariaHidden":91},[90],[59,1849,1851,1854],{"className":1850},[95],[59,1852],{"className":1853,"style":198},[99],[59,1855,1842],{"className":1856,"style":1857},[104,158],"margin-right:0.044em;"," axes below (height always stays price) and rotate to feel how each column pairing relates to the price of the 50 houses:",[1860,1861],"housing-feature-explorer3d",{"y-label":1862},"price (dollars)",[1864,1865,1866],"style",{},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":311,"searchDepth":323,"depth":323,"links":1868},[1869,1870,1874,1875,1876,1877,1878],{"id":29,"depth":323,"text":30},{"id":53,"depth":323,"text":54,"children":1871},[1872,1873],{"id":379,"depth":330,"text":380},{"id":444,"depth":330,"text":445},{"id":506,"depth":323,"text":507},{"id":884,"depth":323,"text":885},{"id":934,"depth":323,"text":935},{"id":1341,"depth":323,"text":1342},{"id":1584,"depth":323,"text":1585,"children":1879},[1880],{"id":1766,"depth":330,"text":1767},null,"2026-08-18","Why every serious ML codebase uses NumPy instead of a Python list: vectors, matrices, the dot product, and the real (not the legendary) gap between a loop and a vectorized operation.","md",{},"\u002Fen\u002Fplaylists\u002Fmachine-learning-specialization\u002Fw2-lab01-numpy-vectorization","machine-learning-specialization",{"title":6,"description":1883},"published","en\u002Fplaylists\u002Fmachine-learning-specialization\u002Fw2-lab01-numpy-vectorization",[1892,1893,1894],"numpy","vectorization","tools","qbhEIwMXkIpf4m8_kfHWuHUpB1psJgLRD8EH-d6eFko",1787338984787]