[{"data":1,"prerenderedAt":819},["ShallowReactive",2],{"lang-switch-post-\u002Fen\u002Fplaylists\u002Fpattern-recognition\u002Fnlp-intro":3,"post-en-pattern-recognition-nlp-intro":4},"\u002Fplaylists\u002Fpattern-recognition\u002Fnlp-intro",{"id":5,"title":6,"body":7,"cover":805,"date":806,"description":807,"extension":808,"meta":809,"navigation":74,"order":206,"path":810,"playlist":811,"seo":812,"status":813,"stem":814,"tags":815,"__hash__":818},"posts\u002Fen\u002Fplaylists\u002Fpattern-recognition\u002Fnlp-intro.md","Text Becomes a Vector: My First Steps in NLP",{"type":8,"value":9,"toc":795},"minimark",[10,31,36,39,99,113,121,125,228,235,250,262,266,276,291,305,309,312,398,405,420,427,431,439,511,533,542,549,555,561,565,568,619,634,638,681,688,692,709,734,751,788,791],[11,12,13,14,19,20,24,25,30],"p",{},"Lecture 14. The last one in the course, and the topic closes the entire playlist with a nice twist: every model I've used so far, ",[15,16,18],"a",{"href":17},"\u002Fen\u002Fplaylists\u002Fpattern-recognition\u002Flinear-regression-estimator","from linear regression back in the first post"," to ",[15,21,23],{"href":22},"\u002Fen\u002Fplaylists\u002Fpattern-recognition\u002Ffeature-selection","feature selection in the previous post",", expects a vector of numbers as input. A pixel is already a number. A wine's chemical measurement is already a number. But text, a movie review written by a real person, isn't a number at all. This lecture is about the first step of any ",[26,27,29],"glossary-term",{"definition":28},"Natural Language Processing, the ML field dealing with text: from Google search to your phone's keyboard autocorrect","NLP"," (Natural Language Processing) task: how to turn text into a vector without losing what matters.",[32,33,35],"h2",{"id":34},"standardize-and-tokenize-the-bare-minimum","Standardize and tokenize: the bare minimum",[11,37,38],{},"Before any counting, the professor cleans the text: everything lowercase, punctuation gone.",[40,41,46],"pre",{"className":42,"code":43,"language":44,"meta":45,"style":45},"language-python shiki shiki-themes github-light github-dark","def standardize(text):\n    text = text.lower()\n    return \"\".join(c for c in text if c not in string.punctuation)\n\ndef tokenize(text):\n    return standardize(text).split()\n\ntokenize(\"I write, erase, rewrite, erase again, and then a poppy blooms!\")\n","python","",[47,48,49,57,63,69,76,82,88,93],"code",{"__ignoreMap":45},[50,51,54],"span",{"class":52,"line":53},"line",1,[50,55,56],{},"def standardize(text):\n",[50,58,60],{"class":52,"line":59},2,[50,61,62],{},"    text = text.lower()\n",[50,64,66],{"class":52,"line":65},3,[50,67,68],{},"    return \"\".join(c for c in text if c not in string.punctuation)\n",[50,70,72],{"class":52,"line":71},4,[50,73,75],{"emptyLinePlaceholder":74},true,"\n",[50,77,79],{"class":52,"line":78},5,[50,80,81],{},"def tokenize(text):\n",[50,83,85],{"class":52,"line":84},6,[50,86,87],{},"    return standardize(text).split()\n",[50,89,91],{"class":52,"line":90},7,[50,92,75],{"emptyLinePlaceholder":74},[50,94,96],{"class":52,"line":95},8,[50,97,98],{},"tokenize(\"I write, erase, rewrite, erase again, and then a poppy blooms!\")\n",[100,101,102],"blockquote",{},[11,103,104,108,109,112],{},[105,106,107],"strong",{},"Output:"," ",[47,110,111],{},"['i', 'write', 'erase', 'rewrite', 'erase', 'again', 'and', 'then', 'a', 'poppy', 'blooms']",".",[11,114,115,116,120],{},"Notice \"Erase\" and \"erase\" are now the same word, and the comma right after the first \"erase\" is gone instead of sticking to it. Without this step, the model would treat \"erase\" and \"erase,\" (comma glued on) as two completely different words, and \"Write\" and \"write\" too, inflating the vocabulary with duplicates that shouldn't exist. It's the same spirit as the normalization I already saw ",[15,117,119],{"href":118},"\u002Fen\u002Fplaylists\u002Fpattern-recognition\u002Fknn-classifier","back in the KNN post",", just rescaling text into a canonical form instead of rescaling numbers.",[32,122,124],{"id":123},"the-homemade-vectorizer-word-becomes-index","The homemade vectorizer: word becomes index",[40,126,128],{"className":42,"code":127,"language":44,"meta":45,"style":45},"class Vectorizer:\n    def standardize(self, text):\n        ...\n    def tokenize(self, text):\n        ...\n    def make_vocabolary(self, dataset):\n        self.vocabulary = {\"\": 0, \"[UNK]\": 1}\n        for text in dataset:\n            text = self.standardize(text)\n            tokens = self.tokenize(text)\n            for token in tokens:\n                if token not in self.vocabulary:\n                    self.vocabulary[token] = len(self.vocabulary)\n\n    def encode(self, text):\n        text = self.standardize(text)\n        tokens = self.tokenize(text)\n        return [self.vocabulary.get(token, 1) for token in tokens]\n",[47,129,130,135,140,145,150,154,159,164,169,175,181,187,193,199,204,210,216,222],{"__ignoreMap":45},[50,131,132],{"class":52,"line":53},[50,133,134],{},"class Vectorizer:\n",[50,136,137],{"class":52,"line":59},[50,138,139],{},"    def standardize(self, text):\n",[50,141,142],{"class":52,"line":65},[50,143,144],{},"        ...\n",[50,146,147],{"class":52,"line":71},[50,148,149],{},"    def tokenize(self, text):\n",[50,151,152],{"class":52,"line":78},[50,153,144],{},[50,155,156],{"class":52,"line":84},[50,157,158],{},"    def make_vocabolary(self, dataset):\n",[50,160,161],{"class":52,"line":90},[50,162,163],{},"        self.vocabulary = {\"\": 0, \"[UNK]\": 1}\n",[50,165,166],{"class":52,"line":95},[50,167,168],{},"        for text in dataset:\n",[50,170,172],{"class":52,"line":171},9,[50,173,174],{},"            text = self.standardize(text)\n",[50,176,178],{"class":52,"line":177},10,[50,179,180],{},"            tokens = self.tokenize(text)\n",[50,182,184],{"class":52,"line":183},11,[50,185,186],{},"            for token in tokens:\n",[50,188,190],{"class":52,"line":189},12,[50,191,192],{},"                if token not in self.vocabulary:\n",[50,194,196],{"class":52,"line":195},13,[50,197,198],{},"                    self.vocabulary[token] = len(self.vocabulary)\n",[50,200,202],{"class":52,"line":201},14,[50,203,75],{"emptyLinePlaceholder":74},[50,205,207],{"class":52,"line":206},15,[50,208,209],{},"    def encode(self, text):\n",[50,211,213],{"class":52,"line":212},16,[50,214,215],{},"        text = self.standardize(text)\n",[50,217,219],{"class":52,"line":218},17,[50,220,221],{},"        tokens = self.tokenize(text)\n",[50,223,225],{"class":52,"line":224},18,[50,226,227],{},"        return [self.vocabulary.get(token, 1) for token in tokens]\n",[11,229,230,231,234],{},"The idea is literal: every word seen during training gets an integer, an index in a dictionary. ",[47,232,233],{},"\"[UNK]\""," (unknown) is reserved at index 1 from the start, for any new word that shows up later, in a test sentence, that wasn't in the training vocabulary. The professor tests this with a sentence using the word \"still\", absent from the 3-sentence training dataset:",[100,236,237],{},[11,238,239,241,242,245,246,249],{},[105,240,107],{}," the word \"still\" becomes index ",[47,243,244],{},"1",", the same as ",[47,247,248],{},"[UNK]",", while the other words in the sentence (already seen during training) become their real indices.",[11,251,252,253,257,258,261],{},"That's the same problem an unseen category solved ",[15,254,256],{"href":255},"\u002Fen\u002Fplaylists\u002Fpattern-recognition\u002Ftitanic","back in the Titanic post"," with ",[47,259,260],{},"OneHotEncoder",": what to do when production data brings something training never saw. There it was an embarkation category, here it's a word.",[32,263,265],{"id":264},"the-dataset-50-thousand-real-movie-reviews","The dataset: 50 thousand real movie reviews",[11,267,268,269,275],{},"The professor downloads the ",[15,270,274],{"href":271,"rel":272},"https:\u002F\u002Fai.stanford.edu\u002F~amaas\u002Fdata\u002Fsentiment\u002F",[273],"nofollow","IMDB Large Movie Review Dataset",", from Stanford, one of the most cited datasets in sentiment analysis: 50 thousand real IMDB reviews, half labeled positive, half negative, already split 25 thousand train \u002F 25 thousand test.",[40,277,279],{"className":42,"code":278,"language":44,"meta":45,"style":45},"train_df = text_dataset_from_directory('aclImdb\u002Ftrain')\ntest_df = text_dataset_from_directory('aclImdb\u002Ftest')\n",[47,280,281,286],{"__ignoreMap":45},[50,282,283],{"class":52,"line":53},[50,284,285],{},"train_df = text_dataset_from_directory('aclImdb\u002Ftrain')\n",[50,287,288],{"class":52,"line":59},[50,289,290],{},"test_df = text_dataset_from_directory('aclImdb\u002Ftest')\n",[100,292,293],{},[11,294,295,297,298,301,302,112],{},[105,296,107],{}," 25000 training rows, 25000 test rows, columns ",[47,299,300],{},"text"," and ",[47,303,304],{},"label",[32,306,308],{"id":307},"bag-of-words-every-word-is-a-vote-no-order","Bag-of-words: every word is a vote, no order",[11,310,311],{},"The first strategy for vectorizing an entire review: a vector the size of the vocabulary, with 1 in every position whose word appears in the review, 0 everywhere else. Completely ignores word order, just marks presence.",[40,313,315],{"className":42,"code":314,"language":44,"meta":45,"style":45},"class UnigramTransformer(BaseEstimator, TransformerMixin):\n    def fit(self, X, y=None):\n        self.vocabulary = {\"\": 0, \"[UNK]\": 1}\n        for text in X:\n            for token in set(self.tokenize(text)):\n                if token not in self.vocabulary and len(self.vocabulary) \u003C self.max_features:\n                    self.vocabulary[token] = len(self.vocabulary)\n        return self\n\n    def transform(self, X, y=None):\n        rows, cols, data = [], [], []\n        for row, text in enumerate(X):\n            for token in set(self.tokenize(text)):\n                rows.append(row)\n                cols.append(self.vocabulary.get(token, 1))\n                data.append(1)\n        return csr_matrix((data, (rows, cols)), shape=(len(X), len(self.vocabulary)))\n",[47,316,317,322,327,331,336,341,346,350,355,359,364,369,374,378,383,388,393],{"__ignoreMap":45},[50,318,319],{"class":52,"line":53},[50,320,321],{},"class UnigramTransformer(BaseEstimator, TransformerMixin):\n",[50,323,324],{"class":52,"line":59},[50,325,326],{},"    def fit(self, X, y=None):\n",[50,328,329],{"class":52,"line":65},[50,330,163],{},[50,332,333],{"class":52,"line":71},[50,334,335],{},"        for text in X:\n",[50,337,338],{"class":52,"line":78},[50,339,340],{},"            for token in set(self.tokenize(text)):\n",[50,342,343],{"class":52,"line":84},[50,344,345],{},"                if token not in self.vocabulary and len(self.vocabulary) \u003C self.max_features:\n",[50,347,348],{"class":52,"line":90},[50,349,198],{},[50,351,352],{"class":52,"line":95},[50,353,354],{},"        return self\n",[50,356,357],{"class":52,"line":171},[50,358,75],{"emptyLinePlaceholder":74},[50,360,361],{"class":52,"line":177},[50,362,363],{},"    def transform(self, X, y=None):\n",[50,365,366],{"class":52,"line":183},[50,367,368],{},"        rows, cols, data = [], [], []\n",[50,370,371],{"class":52,"line":189},[50,372,373],{},"        for row, text in enumerate(X):\n",[50,375,376],{"class":52,"line":195},[50,377,340],{},[50,379,380],{"class":52,"line":201},[50,381,382],{},"                rows.append(row)\n",[50,384,385],{"class":52,"line":206},[50,386,387],{},"                cols.append(self.vocabulary.get(token, 1))\n",[50,389,390],{"class":52,"line":212},[50,391,392],{},"                data.append(1)\n",[50,394,395],{"class":52,"line":218},[50,396,397],{},"        return csr_matrix((data, (rows, cols)), shape=(len(X), len(self.vocabulary)))\n",[11,399,400,401,404],{},"Notice the ",[47,402,403],{},"csr_matrix",": with a 10-thousand-word vocabulary and each review only using a few hundred of those words, the matrix is almost entirely zero. Storing a dense vector of 10 thousand positions, almost all zero, for each of the 20 thousand reviews would waste a huge amount of memory. The sparse format only stores the positions with a nonzero value.",[40,406,408],{"className":42,"code":407,"language":44,"meta":45,"style":45},"pipeline = Pipeline([(\"vectorizer\", UnigramTransformer(10000)), (\"classifier\", RandomForestClassifier(random_state=42))])\npipeline.fit(train_texts, train_labels)\n",[47,409,410,415],{"__ignoreMap":45},[50,411,412],{"class":52,"line":53},[50,413,414],{},"pipeline = Pipeline([(\"vectorizer\", UnigramTransformer(10000)), (\"classifier\", RandomForestClassifier(random_state=42))])\n",[50,416,417],{"class":52,"line":59},[50,418,419],{},"pipeline.fit(train_texts, train_labels)\n",[100,421,422],{},[11,423,424,426],{},[105,425,107],{}," 0.832 validation accuracy, just from which words show up, with no order at all.",[32,428,430],{"id":429},"tf-idf-not-every-word-carries-the-same-weight","TF-IDF: not every word carries the same weight",[11,432,433,434,438],{},"Bag-of-words treats \"the\" (shows up in almost every review) and \"wonderful\" (shows up mostly in the good ones) the same way: both count as 1 if present. But \"the\" carries no information at all about whether a review is good or bad, while \"wonderful\" carries plenty. ",[26,435,437],{"definition":436},"Term Frequency times Inverse Document Frequency: weighs each word by how often it appears in the text, multiplied by how rare it is across the rest of the corpus","TF-IDF"," fixes this with two multiplied quantities:",[40,440,442],{"className":42,"code":441,"language":44,"meta":45,"style":45},"class TfidfTransformer(BaseEstimator, TransformerMixin):\n    def fit(self, X, y=None):\n        doc_freq = {}\n        for text in X:\n            for token in set(self.tokenize(text)):\n                doc_freq[token] = doc_freq.get(token, 0) + 1\n        ...\n        self.idf = {token: math.log(len(X) \u002F freq) for token, freq in doc_freq.items()}\n        return self\n\n    def transform(self, X, y=None):\n        ...\n        for token, count in token_counts.items():\n            tf = count \u002F total_tokens_in_document\n            value = tf * self.idf[token]\n",[47,443,444,449,453,458,462,466,471,475,480,484,488,492,496,501,506],{"__ignoreMap":45},[50,445,446],{"class":52,"line":53},[50,447,448],{},"class TfidfTransformer(BaseEstimator, TransformerMixin):\n",[50,450,451],{"class":52,"line":59},[50,452,326],{},[50,454,455],{"class":52,"line":65},[50,456,457],{},"        doc_freq = {}\n",[50,459,460],{"class":52,"line":71},[50,461,335],{},[50,463,464],{"class":52,"line":78},[50,465,340],{},[50,467,468],{"class":52,"line":84},[50,469,470],{},"                doc_freq[token] = doc_freq.get(token, 0) + 1\n",[50,472,473],{"class":52,"line":90},[50,474,144],{},[50,476,477],{"class":52,"line":95},[50,478,479],{},"        self.idf = {token: math.log(len(X) \u002F freq) for token, freq in doc_freq.items()}\n",[50,481,482],{"class":52,"line":171},[50,483,354],{},[50,485,486],{"class":52,"line":177},[50,487,75],{"emptyLinePlaceholder":74},[50,489,490],{"class":52,"line":183},[50,491,363],{},[50,493,494],{"class":52,"line":189},[50,495,144],{},[50,497,498],{"class":52,"line":195},[50,499,500],{},"        for token, count in token_counts.items():\n",[50,502,503],{"class":52,"line":201},[50,504,505],{},"            tf = count \u002F total_tokens_in_document\n",[50,507,508],{"class":52,"line":206},[50,509,510],{},"            value = tf * self.idf[token]\n",[11,512,513,516,517,520,521,524,525,528,529,532],{},[105,514,515],{},"TF"," (term frequency) is simple: how many times the word appears in the review, divided by the review's total word count, the \"local\" weight. ",[105,518,519],{},"IDF"," (inverse document frequency) is the \"global\" weight: ",[47,522,523],{},"log(total documents \u002F documents containing the word)",". A word appearing in almost every document (like \"the\") has an IDF near zero, barely counting at all. A rare word, present in few documents, has a high IDF. Multiplying the two, a word only gets a high weight if it's frequent in ",[105,526,527],{},"that specific"," review ",[105,530,531],{},"and"," rare across the rest of the corpus, exactly the kind of word that helps tell one review apart from another.",[40,534,536],{"className":42,"code":535,"language":44,"meta":45,"style":45},"pipeline = Pipeline([(\"vectorizer\", TfidfTransformer(10000)), (\"classifier\", RandomForestClassifier(random_state=42))])\n",[47,537,538],{"__ignoreMap":45},[50,539,540],{"class":52,"line":53},[50,541,535],{},[100,543,544],{},[11,545,546,548],{},[105,547,107],{}," 0.8316 validation accuracy, essentially tied with bag-of-words (0.832).",[11,550,551,552,554],{},"Interactive: type a sentence below and watch the TF-IDF weight of each word, computed against a small 16-review example corpus. A common word like \"the\" or \"was\" gets a short bar, a rare, loaded word like \"wonderful\", \"terrible\" or \"boring\" gets a much taller one, and a word that this tiny example corpus never saw shows up marked as ",[47,553,248],{},", with no weight, the same problem the homemade vectorizer had above.",[556,557],"text-vectorizer-explorer",{"label":558,"readout-suffix":559,"unk-label":560},"Type a sentence (in English, same vocabulary as the example corpus)","token(s) recognized in the example vocabulary","UNK",[32,562,564],{"id":563},"on-the-full-dataset-against-the-official-tfidfvectorizer","On the full dataset, against the official TfidfVectorizer",[11,566,567],{},"With both techniques tested on validation, the professor retrains on the entire training set (25 thousand reviews, not just the 20-thousand slice) and measures on the real test set:",[569,570,571,586],"table",{},[572,573,574],"thead",{},[575,576,577,582],"tr",{},[578,579,581],"th",{"align":580},"left","Technique",[578,583,585],{"align":584},"right","Test accuracy",[587,588,589,598,608],"tbody",{},[575,590,591,595],{},[592,593,594],"td",{"align":580},"Bag-of-words (unigram)",[592,596,597],{"align":584},"0.83804",[575,599,600,603],{},[592,601,602],{"align":580},"Homemade TF-IDF",[592,604,605],{"align":584},[105,606,607],{},"0.84292",[575,609,610,616],{},[592,611,612,613],{"align":580},"scikit-learn's ",[47,614,615],{},"TfidfVectorizer",[592,617,618],{"align":584},"0.83872",[11,620,621,622,301,626,630,631,633],{},"TF-IDF wins by a small margin over bag-of-words, and more importantly: my homemade TF-IDF implementation (0.84292) lands less than half a percentage point from scikit-learn's own official implementation (0.83872), the same \"matches the professional tool, down to the decimal\" test I already saw ",[15,623,625],{"href":624},"\u002Fen\u002Fplaylists\u002Fpattern-recognition\u002Fnormal-equation","back in the normal equation post",[15,627,629],{"href":628},"\u002Fen\u002Fplaylists\u002Fpattern-recognition\u002Fpca","in the PCA post",". The real ",[47,632,615],{}," has extra optimizations and details (L2 normalization of the final vector, for instance), but the core idea, the same TF times IDF math, is identical.",[32,635,637],{"id":636},"wrapping-up","Wrapping up",[569,639,640,650],{},[572,641,642],{},[575,643,644,647],{},[578,645,646],{"align":580},"What I already knew",[578,648,649],{"align":580},"What this lecture settled",[587,651,652,660,670],{},[575,653,654,657],{},[592,655,656],{"align":580},"Every model expects a vector of numbers",[592,658,659],{"align":580},"Text has to become a vector too, and how you do it is a design choice, not an automatic detail",[575,661,662,665],{},[592,663,664],{"align":580},"An unseen category in production needs a plan",[592,666,667,669],{"align":580},[47,668,248],{}," solves for words the same problem an unknown category solved for the Titanic",[575,671,672,675],{},[592,673,674],{"align":580},"Matching scikit-learn validates a homemade implementation",[592,676,677,678,680],{"align":580},"Hand-built TF-IDF (0.84292) lands close enough to the official ",[47,679,615],{}," (0.83872) to confirm the logic is right",[11,682,683,684,687],{},"And with that, the 14 lectures in this course close out. I started ",[15,685,686],{"href":17},"fitting a line with the normal equation"," and end here, turning a movie review into a vector of weighted words. Along the way there were trees, ensembles, clusters, dimensionality reduction, imbalanced data, with Bishop holding down the theoretical side of almost all of it, except the last three lectures (DBSCAN, semi-supervised learning, and NLP), which came after his 2006 book and still fit right into the same recurring logic: data becomes a vector, a vector becomes a decision. Thanks, professor Boldt.",[32,689,691],{"id":690},"practical-application","Practical application",[11,693,694,695,697,698,701,702,301,705,708],{},"To confirm the lecture's finding (bag-of-words, homemade TF-IDF, and the official ",[47,696,615],{}," landing on similar results) on a dataset different from IMDB, I used scikit-learn's own ",[47,699,700],{},"20newsgroups",": real forum posts, on two clearly distinct topics, ",[47,703,704],{},"sci.space",[47,706,707],{},"rec.sport.baseball",", 1190 training posts and 791 test posts.",[40,710,712],{"className":42,"code":711,"language":44,"meta":45,"style":45},"from sklearn.datasets import fetch_20newsgroups\ncats = ['sci.space', 'rec.sport.baseball']\ntrain = fetch_20newsgroups(subset='train', categories=cats, remove=('headers','footers','quotes'), random_state=42)\ntest = fetch_20newsgroups(subset='test', categories=cats, remove=('headers','footers','quotes'), random_state=42)\n",[47,713,714,719,724,729],{"__ignoreMap":45},[50,715,716],{"class":52,"line":53},[50,717,718],{},"from sklearn.datasets import fetch_20newsgroups\n",[50,720,721],{"class":52,"line":59},[50,722,723],{},"cats = ['sci.space', 'rec.sport.baseball']\n",[50,725,726],{"class":52,"line":65},[50,727,728],{},"train = fetch_20newsgroups(subset='train', categories=cats, remove=('headers','footers','quotes'), random_state=42)\n",[50,730,731],{"class":52,"line":71},[50,732,733],{},"test = fetch_20newsgroups(subset='test', categories=cats, remove=('headers','footers','quotes'), random_state=42)\n",[11,735,736,737,301,740,743,744,746,747,750],{},"I reproduced all three approaches (",[47,738,739],{},"UnigramTransformer",[47,741,742],{},"TfidfTransformer"," exactly as the professor built them, plus the official ",[47,745,615],{},"), with ",[47,748,749],{},"RandomForestClassifier(random_state=42)"," on top of a 5000-word vocabulary:",[569,752,753,761],{},[572,754,755],{},[575,756,757,759],{},[578,758,581],{"align":580},[578,760,585],{"align":584},[587,762,763,770,777],{},[575,764,765,767],{},[592,766,594],{"align":580},[592,768,769],{"align":584},"0.8786",[575,771,772,774],{},[592,773,602],{"align":580},[592,775,776],{"align":584},"0.8774",[575,778,779,783],{},[592,780,612,781],{"align":580},[47,782,615],{},[592,784,785],{"align":584},[105,786,787],{},"0.8963",[11,789,790],{},"Three close numbers again, on a completely different topic and dataset from the IMDB movies, which gives confidence the lecture's pattern wasn't a coincidence of that specific dataset: any reasonable text vectorization (word presence or TF-IDF) already delivers most of the useful signal for separating two clearly distinct classes, here rocket science against baseball bats.",[792,793,794],"style",{},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":45,"searchDepth":59,"depth":59,"links":796},[797,798,799,800,801,802,803,804],{"id":34,"depth":59,"text":35},{"id":123,"depth":59,"text":124},{"id":264,"depth":59,"text":265},{"id":307,"depth":59,"text":308},{"id":429,"depth":59,"text":430},{"id":563,"depth":59,"text":564},{"id":636,"depth":59,"text":637},{"id":690,"depth":59,"text":691},null,"2026-08-20","Lecture 14, the last one in the course: the professor turns a movie review into numbers by hand, and shows a homemade TF-IDF matching scikit-learn's official TfidfVectorizer.","md",{},"\u002Fen\u002Fplaylists\u002Fpattern-recognition\u002Fnlp-intro","pattern-recognition",{"title":6,"description":807},"published","en\u002Fplaylists\u002Fpattern-recognition\u002Fnlp-intro",[816,817,300],"nlp","tfidf","RXe164tiQy1aq47XAa3jbNwnSt8Thn5i-WQZyCLxqj8",1787338984568]