\n",
"\n",
"\n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
"

\n",
"

"
],
"text/plain": [
" city1 country1 city2 country2\n",
"0 Athens Greece Bangkok Thailand\n",
"1 Athens Greece Beijing China\n",
"2 Athens Greece Berlin Germany\n",
"3 Athens Greece Bern Switzerland\n",
"4 Athens Greece Cairo Egypt"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"data = pd.read_csv('./data/capitals.txt', delimiter=' ')\n",
"data.columns = ['city1', 'country1', 'city2', 'country2']\n",
"\n",
"# print first five elements in the DataFrame\n",
"data.head(5)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"***\n",
"\n",
"### To Run This Code On Your Own Machine:\n",
"Note that because the original google news word embedding dataset is about 3.64 gigabytes,\n",
"the workspace is not able to handle the full file set. So we've downloaded the full dataset,\n",
"extracted a sample of the words that we're going to analyze in this assignment, and saved\n",
"it in a pickle file called word_embeddings_capitals.p\n",
"\n",
"If you want to download the full dataset on your own and choose your own set of word embeddings,\n",
"please see the instructions and some helper code.\n",
"\n",
"- Download the dataset from this [page](https://code.google.com/archive/p/word2vec/).\n",
"- Search in the page for 'GoogleNews-vectors-negative300.bin.gz' and click the link to download.\n",
"- You'll need to unzip the file."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Copy-paste the code below and run it on your local machine after downloading\n",
"the dataset to the same directory as the notebook.\n",
"\n",
"```python\n",
"import nltk\n",
"from gensim.models import KeyedVectors\n",
"\n",
"\n",
"embeddings = KeyedVectors.load_word2vec_format('./GoogleNews-vectors-negative300.bin', binary = True)\n",
"f = open('capitals.txt', 'r').read()\n",
"set_words = set(nltk.word_tokenize(f))\n",
"select_words = words = ['king', 'queen', 'oil', 'gas', 'happy', 'sad', 'city', 'town', 'village', 'country', 'continent', 'petroleum', 'joyful']\n",
"for w in select_words:\n",
" set_words.add(w)\n",
"\n",
"def get_word_embeddings(embeddings):\n",
"\n",
" word_embeddings = {}\n",
" for word in embeddings.vocab:\n",
" if word in set_words:\n",
" word_embeddings[word] = embeddings[word]\n",
" return word_embeddings\n",
"\n",
"\n",
"# Testing your function\n",
"word_embeddings = get_word_embeddings(embeddings)\n",
"print(len(word_embeddings))\n",
"pickle.dump( word_embeddings, open( \"word_embeddings_subset.p\", \"wb\" ) )\n",
"```\n",
"\n",
"***"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we will load the word embeddings as a [Python dictionary](https://docs.python.org/3/tutorial/datastructures.html#dictionaries).\n",
"As stated, these have already been obtained through a machine learning algorithm. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"243"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"word_embeddings = pickle.load(open(\"./data/word_embeddings_subset.p\", \"rb\"))\n",
"len(word_embeddings) # there should be 243 words that will be used in this assignment"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Each of the word embedding is a 300-dimensional vector."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"dimension: 300\n"
]
}
],
"source": [
"print(\"dimension: {}\".format(word_embeddings['Spain'].shape[0]))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Predict relationships among words\n",
"\n",
"Now you will write a function that will use the word embeddings to predict relationships among words.\n",
"* The function will take as input three words.\n",
"* The first two are related to each other.\n",
"* It will predict a 4th word which is related to the third word in a similar manner as the two first words are related to each other.\n",
"* As an example, \"Athens is to Greece as Bangkok is to ______\"?\n",
"* You will write a program that is capable of finding the fourth word.\n",
"* We will give you a hint to show you how to compute this.\n",
"\n",
"A similar analogy would be the following:\n",
"\n",
"\n",
"\n",
"You will implement a function that can tell you the capital of a country.\n",
"You should use the same methodology shown in the figure above. To do this,\n",
"you'll first compute the cosine similarity metric or the Euclidean distance."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 1.2 Cosine Similarity\n",
"\n",
"The cosine similarity function is:\n",
"\n",
"$$\\cos (\\theta)=\\frac{\\mathbf{A} \\cdot \\mathbf{B}}{\\|\\mathbf{A}\\|\\|\\mathbf{B}\\|}=\\frac{\\sum_{i=1}^{n} A_{i} B_{i}}{\\sqrt{\\sum_{i=1}^{n} A_{i}^{2}} \\sqrt{\\sum_{i=1}^{n} B_{i}^{2}}}\\tag{1}$$\n",
"\n",
"$A$ and $B$ represent the word vectors and $A_i$ or $B_i$ represent index i of that vector. Note that if A and B are identical, you will get $cos(\\theta) = 1$.\n",
"* Otherwise, if they are the total opposite, meaning, $A= -B$, then you would get $cos(\\theta) = -1$.\n",
"* If you get $cos(\\theta) =0$, that means that they are orthogonal (or perpendicular).\n",
"* Numbers between 0 and 1 indicate a similarity score.\n",
"* Numbers between -1 and 0 indicate a dissimilarity score.\n",
"\n",
"**Instructions**: Implement a function that takes in two word vectors and computes the cosine distance."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"city1 | country1 | city2 | country2 | |
---|---|---|---|---|

0 | Athens | Greece | Bangkok | Thailand |

1 | Athens | Greece | Beijing | China |

2 | Athens | Greece | Berlin | Germany |

3 | Athens | Greece | Bern | Switzerland |

4 | Athens | Greece | Cairo | Egypt |

\n",
"## \n",
" **Hints**\n",
"

\n",
"

\n", "

- \n",
"
- Python's NumPy library adds support for linear algebra operations (e.g., dot product, vector norm ...). \n", "
- Use numpy.dot . \n", "
- Use numpy.linalg.norm . \n", "

\n",
"## \n",
" **Hints**\n",
"

\n",
"

\n", "

- \n",
"
- Use numpy.linalg.norm . \n", "

\n",
"## \n",
" **Hints**\n",
"

\n",
"

\n", "

- \n",
"
- Use pandas.DataFrame.iterrows . \n", "

\n",
"## \n",
" **Hints**\n",
"

\n",
"

\n",
"\n",
"Now you will use your pca function to plot a few words we have chosen for you.\n",
"You will see that similar words tend to be clustered near each other.\n",
"Sometimes, even antonyms tend to be clustered near each other. Antonyms\n",
"describe the same thing but just tend to be on the other end of the scale\n",
"They are usually found in the same location of a sentence,\n",
"have the same parts of speech, and thus when\n",
"learning the word vectors, you end up getting similar weights. In the next week\n",
"we will go over how you learn them, but for now let's just enjoy using them.\n",
"\n",
"**Instructions:** Run the cell below."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"You have 11 words each of 300 dimensions thus X.shape is: (11, 300)\n"
]
}
],
"source": [
"words = ['oil', 'gas', 'happy', 'sad', 'city', 'town',\n",
" 'village', 'country', 'continent', 'petroleum', 'joyful']\n",
"\n",
"# given a list of words and the embeddings, it returns a matrix with all the embeddings\n",
"X = get_vectors(word_embeddings, words)\n",
"\n",
"print('You have 11 words each of 300 dimensions thus X.shape is:', X.shape)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAZwAAAD8CAYAAABDwhLXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAnt0lEQVR4nO3deXRV1fn/8fdjQIhCjQICQSQOlDJlkDAUGlAQUEtlLiIog4q0VTt8m29x0VVRcWml3zq0VItVUKGCBEjR2uKA/ATFSiIJo1S0UUkQIghCCRXh+f2RmzRgQgK5Ofcm+bzWuosz7HP2cy5ZebL32Wcfc3dERERq2hmRDkBEROoHJRwREQmEEo6IiARCCUdERAKhhCMiIoFQwhERkUDU+4RjZglmtinScYiI1HX1PuGIiEgwGkQ6gHAxs7OB54ELgBjgXqAD8D0gFngLuNXd3cy6AU+FDn05AuGKiNQ7Fs0zDTRv3twTEhKqVPbzzz/niy++oF27dgAcPXoUd6dBg+Kc+q9//Ytzzz2XuLg4tmzZQtu2bWnatCk7duxg//79dO7cuaYuQ0QkMNnZ2Z+5e4tIx1GeqG7hJCQkkJWVVaWy//znPxk0aBADBgxgyJAhpKWlsWTJEh588EEOHTpE48aNufnmm5k6dSqJiYls27YNgA0bNnD99ddXuR4RkWhmZh9FOoaKRHXCORXf/OY3effdd3nppZf45S9/yYABA5g9ezZZWVm0bduWGTNmcPjw4UiHKSJSb9WZQQMFBQWcddZZjB8/nvT0dN59910AmjdvzsGDB8nIyAAgLi6OuLg41qxZA8CCBQsiFrOISH1SZ1o4GzduJD09nTPOOIOGDRvy2GOPkZmZSZcuXWjVqhXdu3cvLTt37lwmT56MmTFo0KAIRi0iUn9E9aCB1NRU170VEZGqM7Nsd0+NdBzlqTNdaiIiEt3qTJdaicz1+cxasY2CfUXEx8WSPrgDw1LaRDosEZF6r04lnMz1+dy5dCNFR44CkL+viDuXbgRQ0hERibA61aU2a8W20mRToujIUWat2BahiEREpESdSjgF+4pOabuIiASnTiWc+LjYU9ouIiLBqVMJJ31wB2Ibxhy3LbZhDOmDO0QoIhERKVGnBg2UDAzQKDURkehTpxIOFCcdJRgRkehTp7rUREQkeinhiIhIIJRwREQkEPUy4fTu3fu0jhs7diyJiYk89NBDFZZZtWoVQ4YMOd3QRETqrDo3aKAq3nrrrVM+5tNPP2XdunVs3769BiISEan76mULp0mTJrg76enpdOnSha5du7Jo0SIAbrzxRjIzM0vLjhs3jr/85S8MGjSI/Px8kpOTWb16NZdffnnpa6k/++wzEhISInAlIiK1R71s4QAsXbqUnJwccnNz+eyzz+jevTt9+/blpptu4qGHHmLYsGHs37+ft956i6effpqkpCSGDBlCTk5OpEMXEamVwtLCMbOnzGy3mW2qYP/lZrbfzHJCn1+Fo97qWLNmDWPHjiUmJoaWLVvSr18/1q1bR79+/Xj//fcpLCzkueeeY+TIkTRoUG/zsohI2ITrN+k84PfAMycps9rda8Xd9BtvvJH58+ezcOFC5s6dW26ZBg0acOzYMQAOHz4cZHgiIrVSWFo47v4GsDcc5wpKWloaixYt4ujRoxQWFvLGG2/Qo0cPACZOnMjDDz8MQKdOnco9PiEhgezsbAAyMjICiVlEpDYLctDAt80s18z+ZmadA6z3a8yM4cOHk5iYSFJSEv379+fBBx+kVatWALRs2ZKOHTsyadKkCs/x85//nMcee4yUlBQ+++yzoEIXEam1zN3DcyKzBOBFd+9Szr5vAMfc/aCZXQM84u7tKzjPFGAKwIUXXtjto48+Ckt8Jfbs2cNll11GyXnz8vIYMmQImzb99/bToUOH6Nq1K++++y7nnHNOWOsXEalJZpbt7qmRjqM8gbRw3P0Ldz8YWn4JaGhmzSsoO8fdU909tUWLFmGNo6CggG9/+9v8/Oc/r7DMq6++SseOHbn99tuVbEREwiiQ4Vdm1grY5e5uZj0oTnR7gqi7rPj4eP75z38CkLk+n1krtvHRR3ns3X2AgSOup2BbLm3atOG9995j/vz5dO/enS+//JJLL72UZ599lrPOOouJEyfSuHFjsrKy+OKLL/jtb3/LkCFDmDdvHsuWLWP//v3k5+czfvx47rrrLn71q19x3nnn8ZOf/ASA6dOnc/755/PjH/846MsXEYmocA2Lfg5YC3Qwsx1mdpOZTTWzqaEio4BNZpYLPApc5+HqyzsNmevzuXPpRvJDr54u+mwH25v14b75LxMXF8eSJUsYMWIE69atIzc3l44dO/Lkk0+WHp+Xl8c777zDX//6V6ZOnVo6Su2dd95hyZIlbNiwgcWLF5OVlcXkyZN55pniwXvHjh1j4cKFjB8/PviLFhGJsLC0cNx9bCX7f0/xsOmoMGvFNoqOHC1dbxDXEm+WwKwV27i2Wzfy8vLYtGkTv/zlL9m3bx8HDx5k8ODBpeW///3vc8YZZ9C+fXsuvvhi3nvvPQAGDhxIs2bNABgxYgRr1qzhJz/5Cc2aNWP9+vXs2rWLlJSU0jIiIvVJvXyisSDUsilhMQ1Lt8c0j6GoqIiJEyeSmZlJUlIS8+bNY9WqVf8tb3b88aH1irbffPPNzJs3j08//ZTJkyeH+3JERGqFejmXWnxcbKXbDxw4QOvWrTly5AgLFiw4rtzixYs5duwYH3zwAR9++CEdOnQA4JVXXmHv3r0UFRWRmZlJnz59ABg+fDh///vfWbdu3XEtJRGR+qRetnDSB3fgzqUbj+tWi20YQ/rgDmx/7V0A7r33Xnr27EmLFi3o2bMnBw4cKC174YUX0qNHD7744gsef/xxGjduDECPHj0YOXIkO3bsYPz48aSmFo9MPPPMM7niiiuIi4sjJiYmwCsVEYke9TLhDEtpAxTfyymgJd3/Zy7pgzsUb0/575DpH/zgB+Uef+WVV/L4449/bfsFF1xw3EzTJY4dO8bbb7/N4sWLw3MBIiK1UL1MOFCcdEoST03asmULQ4YMYfjw4bRvX+6zriIi9ULYZhqoCampqV7yzpmaVPJMTsG+IuLjYv/b2hERqWWieaaBetvCKVHyTE7J/Zz8fUXcuXQjgJKOiEgY1ctRamWd+EwOQNGRo8xasS1CEYmI1E31PuGc+ExOZdtFROT01PuEU5VnckREpPrqfcJJH9yB2IbHPxtT8kyOiIiET70fNHDcMzkapSYiUmPqfcKB4J7JERGpz+p9l5qIiARDCUdERAKhhCMiIoFQwhERkUAo4YiISCCUcEREJBBKOCIiEgglHBERCURYEo6ZPWVmu81sUwX7zcweNbPtZrbBzC4LR70iIlJ7hKuFMw+46iT7rwbahz5TgMfCVK+IiNQSYUk47v4GsPckRYYCz3ixt4E4M2sdjrpFRKR2COoeThvgkzLrO0LbvsbMpphZlpllFRYWBhKciIjUvKgbNODuc9w91d1TW7RoEelwREQkTIJKOPlA2zLrF4S2iYhIPRFUwlkO3BgardYL2O/uOwOqW0REokBY3odjZs8BlwPNzWwHcBfQEMDdHwdeAq4BtgOHgEnhqFdERGqPsCQcdx9byX4HfhSOukREpHaKukEDIiJSNynhiIhIIJRwREQkEEo4IiISCCUcEREJhBKOiIgEQglHREQCoYQjIiKBUMIREZFAKOGIiEgglHBERCQQSjgiIhIIJRwREQmEEo6IiARCCUdERAKhhCMiIoFQwhERkUAo4YiISCCUcEREJBBKOCIiEoiwJBwzu8rMtpnZdjObVs7+iWZWaGY5oc/N4ahXRERqjwbVPYGZxQCzgYHADmCdmS139y0nFF3k7rdVtz4REamdwtHC6QFsd/cP3f1LYCEwNAznFRGRCsybN4+CgoJTPs7M8syseQ2EVKlwJJw2wCdl1neEtp1opJltMLMMM2tb0cnMbIqZZZlZVmFhYRjCExGpe06WcEI9T1EnqEEDLwAJ7p4IvAI8XVFBd5/j7qnuntqiRYuAwhMRiay8vDy+9a1vMW7cODp27MioUaM4dOgQ2dnZ9OvXj27dujF48GB27txJRkYGWVlZjBs3juTkZIqKikhISOAXv/gFQEdgtJmNNbONZrbJzH5dXp1mNt7M3gndW/9jSaIys4Nlyowys3mh5Xlm9piZvW1mH5rZ5Wb2lJltLSlzMuFIOPlA2RbLBaFtpdx9j7v/J7T6J6BbGOoVEalTtm3bxg9/+EO2bt3KN77xDWbPns3tt99ORkYG2dnZTJ48menTpzNq1ChSU1NZsGABOTk5xMbGAtCsWTOArcAbwK+B/kAy0N3MhpWty8w6AmOAPu6eDBwFxlUhzHOBbwM/BZYDDwGdga5mlnyyA8ORcNYB7c3sIjM7E7guFEQpM2tdZvVair8QEZE6o3fv3kBxS6VLly6ndY62bdvSp08fAMaPH8+KFSvYtGkTAwcOJDk5mZkzZ7Jjx44Kjx8zZkzJYndglbsXuvtXwAKg7wnFB1D8x/86M8sJrV9chTBfcHcHNgK73H2jux8DNgMJJzuw2qPU3P0rM7sNWAHEAE+5+2YzuwfIcvflwB1mdi3wFbAXmFjdekVEoslbb71V7XOY2XHrTZs2pXPnzqxdu7ZKx5999tmnVB3wtLvfWc4+L7Pc+IR9Jb1Vx8osl6yfNKeE5R6Ou7/k7t9090vc/b7Qtl+Fkg3ufqe7d3b3JHe/wt3fC0e9IiKR8Nvf/pYuXbrQpUsXHn74YQCaNGlS7fN+/PHHpcnlz3/+M7169aKwsLB025EjR9i8eTNQnIwOHDhQ0aneAfqZWfPQfZmxwP87ocxrwCgzOx/AzM4zs3ahfbvMrKOZnQEMr/aFhWimARGRU5Cdnc3cuXP5xz/+wdtvv80TTzzB+vXrw3LuDh06MHv2bDp27Mjnn39eev/mF7/4BUlJSSQnJ5e2pCZOnMjUqVNLBw2U5e47gWnA60AukO3ufzmhzBbgl8DLZraB4gFdJbc/pgEvAm8BO8NycYShS01EpD5Zs2YNw4cPL+2+GjFiBKtXrw7LuRs0aMD8+fOP25acnMwbb7zxtbIjR45k5MiRpet5eXnH7Xf354DnTjzO3RPKLC8CFpVTJgPIKGf7xDLLeUCX8vZVRC0cEREJhBKOiMgpSEtLIzMzk0OHDvHvf/+bZcuWkZaWVu3zJiQkMPPZFfR5YCUXTfsrfR5YSeb6/MoPrEXUpSYicgouu+wyJk6cSI8ePQC4+eabSUlJqfZ5M9fnc+fSjRQdOQpA/r4i7ly6EYBhKeVN3lL7WPFw6uiUmprqWVlZkQ5DRKTG9XlgJfn7ir62vU1cLG9O61/l85hZtrunhjO2cFGXmohIFCgoJ9mcbHttpC41EZHTkLk+n1krtlGwr4j4uFjSB3eoVtdXfFxsuS2c+LjY6oQZVdTCERE5RSX3W/L3FeH8935LdW7ypw/uQGzD4yd5jm0YQ/rgDtWMNnoo4YiInKJZK7aV3twvUXTkKLNWbDvtcw5LacP9I7rSJi4Wo/jezf0jutaZAQOgLjURqQfuvfde5s+fT4sWLWjbti3dunXjnHPOYc6cOXz55ZdceumlPPvss5x11lksXryYu+++m5iYGM4555xyH7qsqfstw1La1KkEcyK1cESkTlu3bh1LliwhNzeXv/3tb5SMfB0xYgTr1q0jNzeXjh078uSTTwJwzz33sGLFCnJzc1m+fHm556zovkpdut9SE5RwRKROe/PNNxk6dCiNGzemadOmfO973wNg06ZNpKWl0bVrVxYsWFA6KWafPn2YOHEiTzzxBEePHi33nPXhfktNUMIRkXpp4sSJ/P73v2fjxo3cddddHD58GIDHH3+cmTNn8sknn9CtWzf27NnztWPrw/2WmqCEIyJ1Wp8+fXjhhRc4fPgwBw8e5MUXXwTgwIEDtG7dmiNHjrBgwYLS8h988AE9e/bknnvuoUWLFnzyySflnndYShvenNaffz3wXd6c1l/Jpgo0aEBE6rTu3btz7bXXkpiYSMuWLenatSvnnHMO9957Lz179qRFixb07Nmz9N0y6enpvP/++7g7AwYMICkpKcJXUHdoahsRqfMOHjxIkyZNOHToEH379mXOnDlcdtllkQ6rRkTz1DZq4YhInTdlyhS2bNnC4cOHmTBhQp1NNtFOLRwRqRfCPRVNtFILR0QkgurD1P+1QVhGqZnZVWa2zcy2m9m0cvY3MrNFof3/MLOEcNQrIlIVNTEVjZy6aiccM4sBZgNXA52AsWbW6YRiNwGfu/ulwEPAr6tbr4hIVdWHqf9rg3C0cHoA2939Q3f/ElgIDD2hzFDg6dByBjDAzCwMdYuIVEpT0USHcCScNkDZJ6N2hLaVW8bdvwL2A83KO5mZTTGzLDPLKiwsDEN4IlLfaSqa6BB1Mw24+xx3T3X31BYtWkQ6HBGpAzQVTXQIxyi1fKBtmfULQtvKK7PDzBoA5wBfn6BIRKSG1PWp/2uDcLRw1gHtzewiMzsTuA44cU7v5cCE0PIoYKVH8wNAIiISdtVu4bj7V2Z2G7ACiAGecvfNZnYPkOXuy4EngWfNbDuwl+KkJCIi9UhYHvx095eAl07Y9qsyy4eB0eGoS0REaqeoGzQgIiJ1kxKOiIgEQglHREQCoYQjIiKBUMIREZFAKOGIiEgglHAk7B5++GEOHToU6TBEJMoo4UjYnSzhHD16tNztIlL3KeHUU8888wyJiYkkJSVxww03kJeXR//+/UlMTGTAgAF8/PHHAEycOJGMjIzS45o0aQLAqlWruPzyyxk1ahTf+ta3GDduHO7Oo48+SkFBAVdccQVXXHFF6TH/8z//Q1JSEvfddx/Dhg0rPd8rr7zC8OHDg7twEYkcd4/aT7du3VzCb9OmTd6+fXsvLCx0d/c9e/b4kCFDfN68ee7u/uSTT/rQoUPd3X3ChAm+ePHi0mPPPvtsd3d//fXX/Rvf+IZ/8sknfvToUe/Vq5evXr3a3d3btWtXem53d8AXLVrk7u7Hjh3zDh06+O7du93dfezYsb58+fKavWCReoTiKcUi/vu7vI9aOPXQypUrGT16NM2bNwfgvPPOY+3atVx//fUA3HDDDaxZs6bS8/To0YMLLriAM844g+TkZPLy8sotFxMTw8iRIwEwM2644Qbmz5/Pvn37WLt2LVdffXV4LkxEolpY5lKTuqtBgwYcO3YMgGPHjvHll1+W7mvUqFHpckxMDF999VW552jcuDExMf99+dWkSZP43ve+R+PGjRk9ejQNGujHUKQ+UAunHurfvz+LFy9mz57iVxLt3buX3r17s3DhQgAWLFhAWloaAAkJCWRnZwOwfPlyjhw5Uun5mzZtyoEDByrcHx8fT3x8PDNnzmTSpEnVvRwRqSX0p2U91LlzZ6ZPn06/fv2IiYkhJSWF3/3ud0yaNIlZs2bRokUL5s6dC8Att9zC0KFDSUpK4qqrruLss8+u9PxTpkzhqquuIj4+ntdff73cMuPGjaOwsJCOHTuG9dpEJHqZR/F70FJTUz0rKyvSYUgNuO2220hJSeGmm26KdCgidYqZZbt7aqTjKI+61KTG5OXl8ec//7l0PSsrizvuuINu3bqxYcMGxo8fH5Z6MjMz2bJlS1jOJSI1R11q9Vjm+nxmrdhGwb4i4uNiSR/cIazvfC9JOCWj31JTU0lNDf8fXpmZmQwZMoROnTqF/dwiEj5q4dRTmevzuXPpRvL3FeFA/r4i7ly6kcz1+aVlTuXh0DvuuIPevXtz8cUXlz4oOm3aNFavXk1ycjIPPfQQq1atYsiQIQDMmDGDyZMnc/nll3PxxRfz6KOPltY7f/58evToQXJyMrfeemvp7ARNmjRh+vTpJCUl0atXL3bt2sVbb73F8uXLSU9PJzk5mQ8++CCgb1BETpUSTj01a8U2io4cP81M0ZGjzFqxDYDNmzczc+ZMVq5cSW5uLo888gi33347EyZMYMOGDYwbN4477rij9NidO3eyZs0aXnzxRaZNmwbAAw88QFpaGjk5Ofz0pz/9WgzvvfceK1as4J133uHuu+/myJEjbN26lUWLFvHmm2+Sk5NDTEwMCxYsAODf//43vXr1Ijc3l759+/LEE0/Qu3dvrr32WmbNmkVOTg6XXHJJTX1lIlJN6lKrpwr2FZ10e0UPhy5duhQofjj0f//3f0uPGzZsGGeccQadOnVi165dVYrhu9/9Lo0aNaJRo0acf/757Nq1i9dee43s7Gy6d+8OQFFREeeffz4AZ555ZmkLqVu3brzyyiunceUiEilKOPVUfFws+eUknfi42NM6X9mHQKs68rG8B0fdnQkTJnD//fd/rXzDhg0xs+PKi0jtUa0uNTM7z8xeMbP3Q/+eW0G5o2aWE/osr06dEh7pgzsQ2zDmuG2xDWNIH9wBOLWHQytS2QOg5RkwYAAZGRns3r27tN6PPvoo7PWISPCqew9nGvCau7cHXgutl6fI3ZNDn2urWaeEwbCUNtw/oitt4mIxoE1cLPeP6Fo6Sq3sw6FJSUn87Gc/43e/+x1z584lMTGRZ599lkceeeSkdSQmJhITE0NSUhIPPfRQleLq1KkTM2fOZNCgQSQmJjJw4EB27tx50mOuu+46Zs2aRUpKigYNiESxaj34aWbbgMvdfaeZtQZWuXuHcsoddPcmp3p+PfgpInJq6vKDny3dveTPz0+BlhWUa2xmWWb2tpkNO9kJzWxKqGxWYWFhNcMTEZFoUemgATN7FWhVzq7pZVfc3c2souZSO3fPN7OLgZVmttHdy+37cPc5wBwobuFUFp/UrJp+OFRE6o9KE467X1nRPjPbZWaty3Sp7a7gHPmhfz80s1VACqDO9ihX8nBoyfM6JQ+HAko6InLKqtulthyYEFqeAPzlxAJmdq6ZNQotNwf6AJr4qhao7OFQEZFTUd2E8wAw0MzeB64MrWNmqWb2p1CZjkCWmeUCrwMPuLsSTi1Q2cOhIiKnoloPfrr7HmBAOduzgJtDy28BXatTj0RGuB8OFZH6TXOpSYUqezhURORUaGobqVDJwACNUhORcFDCkZMaltJGCUZEwkJdaiIiEgglHBERCYQSjoiIBEIJR0REAqGEIyIigVDCERGRQCjhiIhIIJRwREQkEEo4IiISCCUcEREJhBKOiIgEQglHREQCoYRTCzz++OM888wzAMybN4+CgoIIRyQicuo0W3QtMHXq1NLlefPm0aVLF+Lj4yMYkYjIqVPCiULPPPMMv/nNbzAzEhMTueSSS2jSpAkJCQlkZWUxbtw4YmNjue+++3jiiSfIzMwE4JVXXuEPf/gDy5Yti+wFiIiUQ11qUWbz5s3MnDmTlStXkpubyyOPPFK6b9SoUaSmprJgwQJycnK45ppreO+99ygsLARg7ty5TJ48OVKhi4iclBJOlFm5ciWjR4+mefPmAJx33nkVljUzbrjhBubPn8++fftYu3YtV199dVChioickmolHDMbbWabzeyYmaWepNxVZrbNzLab2bTq1CnHmzRpEvPnz+e5555j9OjRNGigXlIRiU7VbeFsAkYAb1RUwMxigNnA1UAnYKyZdapmvXVW//79Wbx4MXv27AFg7969x+1v2rQpBw4cKF2Pj48nPj6emTNnMmnSpEBjFRE5FdX6c9jdt0Jx185J9AC2u/uHobILgaHAlurUXVd17tyZ6dOn069fP2JiYkhJSSEhIaF0/8SJE5k6dSqxsbGsXbuW2NhYxo0bR2FhIR07doxc4CIilQii/6UN8EmZ9R1Az4oKm9kUYArAhRdeWLORRakJEyYwYcKEcveNHDmSkSNHHrdtzZo13HLLLUGEJiJy2ipNOGb2KtCqnF3T3f0v4Q7I3ecAcwBSU1M93Oeva7p168bZZ5/N//3f/0U6FBGRk6o04bj7ldWsIx9oW2b9gtA2CYPs7OxIhyAiUiVBdKmtA9qb2UUUJ5rrgOsDqLdWy1yfz6wV2yjYV0R8XCzpgzswLKVNpMMSETlt1R0WPdzMdgDfBv5qZitC2+PN7CUAd/8KuA1YAWwFnnf3zdULO/L27dvHH/7whxo5d+b6fO5cupH8fUU4kL+viDuXbiRzvRqGIlJ7VSvhuPsyd7/A3Ru5e0t3HxzaXuDu15Qp95K7f9PdL3H3+6obdDSoyYQza8U2io4cPW5b0ZGjzFqxrUbqExEJgmYaOE3Tpk3jgw8+IDk5mfT0dNLT0+nSpQtdu3Zl0aJFAPzoRz9i+fLlAAwfPrx02pmnnnqK6dOnk5eXR8eOHbnlllvo3LkzgwYNoqioiIJ9ReXWWdF2EZHaQAnnND3wwANccskl5OTk0KtXL3JycsjNzeXVV18lPT2dnTt3kpaWxurVqwHIz89ny5biR49Wr15N3759AXj//ff50Y9+xObNm4mLi2PJkiXEx8WWW2dF20VEagMlnDBYs2YNY8eOJSYmhpYtW9KvXz/WrVtXmnC2bNlCp06daNmyJTt37mTt2rX07t0bgIsuuojk5GSgeIhzXl4e6YM7ENsw5rg6YhvGkD64Q9CXJiISNpp4qwa1adOGffv28fe//52+ffuyd+9enn/+eZo0aULTpk3Zs2cPjRo1Ki0fExNDUVFR6Wg0jVITkbpECec0lZ3TLC0tjT/+8Y9MmDCBvXv38sYbbzBr1iwAevXqxcMPP8zKlSvZs2cPo0aNYtSoUZWef1hKGyUYEalTlHBOU7NmzejTpw9dunTh6quvJjExkaSkJMyMBx98kFatiidnSEtL4+WXX+bSSy+lXbt27N27l7S0tAhHLyISPHOP3tljUlNTPSsrK9JhiIjUGmaW7e4Vvi4mkjRoQEREAqEutWrSFDQiIlWjhFMNJVPQlMwKUDIFDaCkIyJyAnWpVYOmoBERqTolnGrQFDQiIlWnhFMNmoJGRKTqlHCqQVPQiIhUnQYNVIOmoBERqTolnGrSFDQiIlWjLjUREQmEEs5JFBQUlE60uWrVKoYMGQLAvHnzuO222yIZmohIraOEcxLx8fFkZGREOgwRkTpBCSdk2rRpzJ49u3R9xowZ/OY3v6FLly4nPe6FF16gZ8+epKSkcOWVV7Jr1y4ACgsLGThwIJ07d+bmm2+mXbt2fPbZZwDMnz+fHj16kJyczK233srRo0dPVoWISJ1QrYRjZqPNbLOZHTOzCmcnNbM8M9toZjlmFpXTP48ZM4bnn3++dP3555+nZ8+elR73ne98h7fffpv169dz3XXX8eCDDwJw9913079/fzZv3syoUaP4+OOPAdi6dSuLFi3izTffJCcnh5iYGBYsWFAzFyUiEkWqO0ptEzAC+GMVyl7h7p9Vs74ak5KSwu7duykoKKCwsJBzzz2Xtm3bVnrcjh07GDNmDDt37uTLL7/koosuAopfO71s2TIArrrqKs4991wAXnvtNbKzs+nevTsARUVFnH/++TV0VSIi0aNaCcfdtwKYWXiiibDRo0eTkZHBp59+ypgxY6p0zO23387PfvYzrr32WlatWsWMGTNOWt7dmTBhAvfff38YIhYRqT2CuofjwMtmlm1mU05W0MymmFmWmWUVFhYGFF6xMWPGsHDhQjIyMhg9enSVjtm/fz9t2hQ/h/P000+Xbu/Tp09pF93LL7/M559/DsCAAQPIyMhg9+7dAOzdu5ePPvoonJchIhKVKk04ZvaqmW0q5zP0FOr5jrtfBlwN/MjM+lZU0N3nuHuqu6e2aNHiFKqovs6dO3PgwAHatGlD69atq3TMjBkzGD16NN26daN58+al2++66y5efvllunTpwuLFi2nVqhVNmzalU6dOzJw5k0GDBpGYmMjAgQPZuXNnTV2SiEjUCMsrps1sFfBzd690QICZzQAOuvtvKitbm18x/Z///IeYmBgaNGjA2rVr+cEPfkBOTk6kwxKROi6aXzFd41PbmNnZwBnufiC0PAi4p6brjbSPP/6Y73//+xw7dowzzzyTJ554ItIhiYhEVLUSjpkNB34HtAD+amY57j7YzOKBP7n7NUBLYFloYEED4M/u/vdqxl1jwvXK6Pbt27N+/foaiFBEpHaq7ii1ZcCycrYXANeElj8EkqpTT1D0ymgRkZqjmQbK0CujRURqjhJOGXpltIhIzVHCKUOvjBYRqTlKOGXoldEiIjVHb/wsQ6+MFhGpOUo4J9Aro0VEaoa61EREJBBKOCIiEgglHBERCYQSjoiIBEIJR0REAhGW1xPUFDMrBCLxdrLmQNS+DrsctSne2hQrKN6aVJtihdoTbzt3D/ZlYlUU1QknUswsK1rfJ1Ge2hRvbYoVFG9Nqk2xQu2LNxqpS01ERAKhhCMiIoFQwinfnEgHcIpqU7y1KVZQvDWpNsUKtS/eqKN7OCIiEgi1cEREJBBKOCIiEgglHMDMRpvZZjM7ZmYVDns0s6vMbJuZbTezaUHGeEIc55nZK2b2fujfcysod9TMckKf5QHHeNLvyswamdmi0P5/mFlCkPGVE09l8U40s8Iy3+fNkYgzFMtTZrbbzDZVsN/M7NHQtWwws8uCjrFMLJXFermZ7S/zvf4q6BhPiKetmb1uZltCvxN+XE6ZqPl+ax13r/cfoCPQAVgFpFZQJgb4ALgYOBPIBTpFKN4HgWmh5WnArysodzBC8VX6XQE/BB4PLV8HLIrg/39V4p0I/D5SMZ4QS1/gMmBTBfuvAf4GGNAL+EcUx3o58GKkv9My8bQGLgstNwX+Wc7PQtR8v7XtoxYO4O5b3X1bJcV6ANvd/UN3/xJYCAyt+ejKNRR4OrT8NDAsQnFUpCrfVdlryAAGmJkFGGNZ0fR/Wyl3fwPYe5IiQ4FnvNjbQJyZtQ4muuNVIdao4u473f3d0PIBYCtw4guyoub7rW2UcKquDfBJmfUdfP0HMSgt3X1naPlToGUF5RqbWZaZvW1mw4IJDajad1Vaxt2/AvYDzQKJ7uuq+n87MtSFkmFmbYMJ7bRE089qVXzbzHLN7G9m1jnSwZQIdfOmAP84YVdt+36jRr1546eZvQq0KmfXdHf/S9DxVOZk8ZZdcXc3s4rGtrdz93wzuxhYaWYb3f2DcMdaT7wAPOfu/zGzWylunfWPcEx1wbsU/5weNLNrgEygfWRDAjNrAiwBfuLuX0Q6nrqi3iQcd7+ymqfIB8r+VXtBaFuNOFm8ZrbLzFq7+85QU353BefID/37oZmtovivtSASTlW+q5IyO8ysAXAOsCeA2MpTabzuXja2P1F8Hy1aBfqzWh1lf5m7+0tm9gcza+7uEZsk08waUpxsFrj70nKK1JrvN9qoS63q1gHtzewiMzuT4hvdgY78KmM5MCG0PAH4WgvNzM41s0ah5eZAH2BLQPFV5bsqew2jgJUeuiMbAZXGe0If/bUU9+1Hq+XAjaHRVL2A/WW6YKOKmbUquXdnZj0o/p0UqT88CMXyJLDV3X9bQbFa8/1GnUiPWoiGDzCc4n7Y/wC7gBWh7fHAS2XKXUPxqJUPKO6Ki1S8zYDXgPeBV4HzQttTgT+FlnsDGykecbURuCngGL/2XQH3ANeGlhsDi4HtwDvAxRH+Gags3vuBzaHv83XgWxGM9TlgJ3Ak9HN7EzAVmBrab8Ds0LVspIKRl1ES621lvte3gd4R/jn4DuDABiAn9LkmWr/f2vbR1DYiIhIIdamJiEgglHBERCQQSjgiIhIIJRwREQmEEo6IiARCCUdERAKhhCMiIoH4/1img1fgibz1AAAAAElFTkSuQmCC\n",
"text/plain": [
"

\n", "

- \n",
"
- Use numpy.mean(a,axis=None) : If you set
`axis = 0`

, you take the mean for each column. If you set`axis = 1`

, you take the mean for each row. Remember that each row is a word vector, and the number of columns are the number of dimensions in a word vector. \n",
" - Use numpy.cov(m, rowvar=True) . This calculates the covariance matrix. By default
`rowvar`

is`True`

. From the documentation: \"If rowvar is True (default), then each row represents a variable, with observations in the columns.\" In our case, each row is a word vector observation, and each column is a feature (variable). \n",
" - Use numpy.linalg.eigh(a, UPLO='L') \n", "
- Use numpy.argsort sorts the values in an array from smallest to largest, then returns the indices from this sort. \n", "
- In order to reverse the order of a list, you can use:
`x[::-1]`

. \n",
" - To apply the sorted indices to eigenvalues, you can use this format
`x[indices_sorted]`

. \n",
" - When applying the sorted indices to eigen vectors, note that each column represents an eigenvector. In order to preserve the rows but sort on the columns, you can use this format
`x[:,indices_sorted]`

\n",
" - To transform the data using a subset of the most relevant principle components, take the matrix multiplication of the eigenvectors with the original data. \n", "
- The data is of shape
`(n_observations, n_features)`

. \n",
" - The subset of eigenvectors are in a matrix of shape
`(n_features, n_components)`

. \n",
" - To multiply these together, take the transposes of both the eigenvectors
`(n_components, n_features)`

and the data (n_features, n_observations). \n",
" - The product of these two has dimensions
`(n_components,n_observations)`

. Take its transpose to get the shape`(n_observations, n_components)`

. \n",
"

\n", " 0.43437323\n", " | \n", " 0.49820384\n", " |

\n", " 0.42077249\n", " | \n", " -0.50351448\n", " |

\n", " -0.85514571\n", " | \n", " 0.00531064\n", " |