RDF2Vec: RDF Graph Embeddings for Data Mining

Editor's Notes

• #3: igure~\ref{fig:lodKDDpipeline} gives an overview of a general LOD-enabled knowledge discovery process. Given a set of local data (such as a relational database), the first step is to link the data to the corresponding LOD concepts from the chosen LOD dataset. After the links are set, outgoing links to external LOD datasets can be explored. In the next step, various techniques for data consolidation and cleansing are applied. Next, transformations on the collected data need to be performed in order to represent the data in a way that it can be processed with any arbitrary data analysis algorithms. After the data transformation is done, a suitable data mining algorithm is applied on the data. In the final step, the results of the data mining process are presented to the user.
• #9: Skip gram softmax is actually logistic regression It is unsupervised method
• #10: On the input layer we get the context words and in the output layer we are trying to calculate the target word. For each of the input words the vector is retrieved from the input->hiden matrix and averaged which is represented in the hidden layer. This implies that the link (activation) function of the hidden layer units is simply linear r (i.e., directly passing its weighted sum of inputs to the next layer). Using the weights from the hidden-output layer weights, we can compute a score uj for each word in the vocabulary. Then the objective of model is to maximize the average log probability. Where the posterior probability is defined using the softmax function. The averaged vector representation from the context is computed as. With the softmax function we calculate the probabilyt for the target given the context, and we use the value to calculate the error (true is 1 or 0) and then we calculate the loss function using gradient descent and backpropagate through the network We take the input vectors, but you can also take the output vectors, or use sum of both, or empiracal evaluation has shown that concatattion is better
• #11: If we represent the words in a low dimensional feature space we expect words semanticly similar words to be close to each other.
• #12: Skip gram softmax is actually logistic regression It is unsupervised method
• #13: The target word is now at the input layer, and the context words are on the output layer. The input is a one-hot encoded vector. which means h is simply copying a row of the input→hidden weight matrix The objective of the Skip-gram model is to maximize the average log probability. where vw and v ′ w are the “input” and “output” vector representations of w, and W is the number of words in the vocabulary. This formulation is impractical because the cost of computing ∇ log p(wO|wI ) is proportional to W, which is often large (105–107 terms). There are hierarchilcal softmax and negative sampling This inversion might seem like an arbitrary choice, but statistically it has the effect that CBOW smoothes over a lot of the distributional information (by treating an entire context as one observation). For the most part, this turns out to be a useful thing for smaller datasets. However, skip-gram treats each context-target pair as a new observation, and this tends to do better when we have larger datasets. We take the input vectors, but you can also take the output vectors, or use sum of both, or empiracal evaluation has shown that concatattion is better
• #17: If we represent the words in a low dimensional feature space we expect words semantically similar words to be close to each other.