Text networks, part three: Dependency networks

This post is part of a series of posts about text network analysis. Each post will discuss one approach to algorithmically creating networks from texts. See the primer for an introduction and the list of posts.

In the previous post, we introduced the basics of text-network analysis using the example of co-occurrence networks. Any application of text-network analysis evolves around two questions: 1. What are my nodes? and 2. what are the relations (edges) between nodes? The answer to the first question is to a large extend the same with different text-network creation methods. The main differences concern the way of identifying edges. Therefore, this post will focus on the second question and only highlight relevant differences in the selection of nodes.

Co-occurrence networks construct edges between nodes based on their proximity in the text. This is a quite rough criterion and error prone: Words close to each other will not always have a meaningful relation to each other, so one gets more edges than desired (false positives). On the other hand, some syntactic arrangements like subordinate clauses can have the effect that related words appear relatively far from each other, and the methods misses relevant edges (false negatives). A better approach thus might take the syntactic structure of the text into account.

There are two main formalisms that describe syntax, both of which model sentence structures as a tree. The constituency approach separates a sentence into a hierarchy of smaller phrases, with individual words being the terminal nodes (leaves) of the tree. The dependency approach always uses words as nodes in the tree. Usually starting with the central verb in a sentence, words are linked to each other based on a concept of “dependency,” meaning that e.g. subject and object “depend” on the verb, while articles “depend” on the noun, and so on.

From a network perspective, the latter is closer to the structure of a text network: Since only words, and not larger units like phrases, are used as nodes, the dependency tree already constitutes a kind of per-sentence proto-network. By transforming the tree structure (e.g., by using lemmas instead of inflected word forms) and merging the per-sentence graphs, one can create a text network.

In the literature, there are mainly two approaches to dependency networks: From a linguistic perspective, one can merge the almost unaltered dependency tree to receive a network model of language (Ferrer i Cancho, Solé, and Köhler 2004). This allows for example to study differences and similarities between languages. But due to the inclusion of semantically meaningless words (stop words), it is hard to interpret the content of such a network. A different approach is to extract the semantic relations expressed through syntax: By searching for specific dependency structures (e.g., phrases expressing quotation), one can query a dependency modes of a text to construct a network including only those relations that are of interest for the research question (see e.g. Atteveldt 2008, 91 et seq.) However, this requires to define specific dependency patterns that are of interest and does not allow to generate a generic network representation of a text.

We tried to go for some kind of middle ground: A dependency network mechanism that is generic enough to be able to model any text without specifying relevant relations, but that also tries to model only semantically meaningful nodes and edges, leaving out stop words. The main steps involved in this process are:

  1. Get a dependency tree for each sentence.
  2. Generalize the nodes, e.g. by using lemmas instead of inflected word forms. (This is the same as with co-occurrence networks.)
  3. Filter nodes, e.g. by word class and/or a stop word list. In order to keep the relevant relations intact, this requires to bridge the gaps that would occur when one removes nodes from the original tree representation.
  4. (Optionally) replace verb nodes by edges between the subject and object(s). This creates a different kind of network which takes the “bridging” function of verbs into account.
  5. Merge a sentence-level networks into a text network.
Step 1: A dependency tree.

Step 1: A dependency tree.

Step 2: A full dependency lemma network.

Step 2: A full dependency lemma network.

Step 3: A filtered dependency network.

Step 3: A filtered dependency network.

Step 4: A dependency network with verbs as edges.

Step 4: A dependency network with verbs as edges.

This kind of dependency network is much more sparse as a co-occurrence network. With 366 nodes, it has 370 edges. A comparable co-occurrence network of the text (379 nodes) has 1886 edges. The number of meaningless edges in the dependency network is much lower compared to co-occurrence networks. But this very strict method also might leave out some relevant relations that are not expressed through direct syntactic relations. In order to include those, we loosened our method a bit, trying to find some kind of middle ground between a strict dependency network and a co-occurrence network. In an additional step between (4) and (5), we add additional edges between all nodes that have at most one node between them (or formally speaking, if their path distance is two or less). The resulting network then has 696 edges.

Dependency network of Moby Dick, chapter 1

Dependency network of Moby Dick, chapter 1.

Comparison: A co-occurrence network of the same text.

Comparison: A co-occurrence network of the same text.

The resulting network is semantically more coherent than the co-occurrence network. However, it also requires more pre-processing: In addition to part-of-speech tagging and lemmatization, one also needs to perform dependency parsing. Since dependency parsers are not available for all languages, co-occurrence networks can often still serve as a useful approximation.

References

Atteveldt, Wouter van. 2008. “Semantic Network Analysis: Techniques for Extracting, Representing and Querying Media Content.” PhD thesis, Amsterdam: Vrije Universiteit. http://vanatteveldt.com/pub/vanatteveldt_semanticnetworkanalysis.pdf.

Ferrer i Cancho, Ramon, Ricard V. Solé, and Reinhard Köhler. 2004. “Patterns in Syntactic Dependency Networks.” Physical Review E 69 (5): 051915. doi:10.1103/PhysRevE.69.051915.


3 Gedanken zu „Text networks, part three: Dependency networks

  1. Hi Frederik,

    thank you very much for this interesting blog on text network analysis.
    I was wondering if you know Quid (please see: https://quid.com/feed/decoding-the-bible OR https://quid.com/quid-in-action)?
    This company analyzes a document corpus, where each node in the network represents a single document. Their approach seems to do a fairly good job at visualizing document similarity based on topic and semantic similarity between documents. Moreover, the network clusters are coherent with the topic clusters in the document corpus.

    I was wondering if you could help me with some of my questions:
    1) How would I combine topic models (e.g. LDA) with semantic networks?
    2) How would I compute the edges for such document networks to account for topic similarity and semantic similarity between documents?

    Thank you for sharing your thoughts on this.

    Kind regards,
    Johannes

Schreibe einen Kommentar

Deine E-Mail-Adresse wird nicht veröffentlicht. Erforderliche Felder sind mit * markiert.