Knowledge graphs are useful for providing structured sources of information for many downstream tasks. Hence, it is an interesting problem to build large knowledge graphs (KG) from a large text corpus. Being able to learn a KG from web-scale corpora means that we could leverage the large amount of unstructured information on websites (e.g. TechCrunch) and build structured knowledge bases. At a large scale, a KG is hard to maintain as it is not easy to keep track of issues like fact coverage, freshness and correctness. This blog post serves as a short introduction to the techniques used in building a simple KG.
This post provides a primer on the Transformer model architecture. It is extremely adept at sequence modelling tasks such as language modelling, where the elements in the sequences exhibit temporal correlations with each other.