A method for the quantitative mapping of change

By Eric Cruet

The problem of change[1].  Much of mankind’s preoccupation has been with changes in the sciences, technology, sociology and economics.  More recently, we seem to be concerned about variations in climate change, global financial states, the effect of technology on society, and the increasing use of unlawful violence intended to coerce or to intimidate governments or societies i.e terrorism.

Traditionally, network, graph, and cluster analysis are the mathematical tools used to understand specific instances of the data generated by these scenarios at a given point in time. But without methods to distinguish between real patterns and statistical error, which can be significant in large data sets, these approaches may not be ideal for studying change.  By assigning weights to individual networks, we can determine meaningful structural differences vs. random fluctuations [3].

Alternatively a bootstrap technique [2] can be used when there are multiple networks to arrive at an accurate estimate by resampling the empirical distribution of observations for each network.  In the case of a single network, resampling can be accomplished by using a parametric model to fit the link weights without undermining the individual characteristics of the nodes.  Using this technique, we can determine cluster significance and also estimate the accuracy of the summary statistics (μ, σ. ρ) based on the proportion of bootstrap networks that support the observation. 

Statistically:

 

Diagramatically:

The standard procedure to cluster networks is to minimize an objective function over probable partitions (left side of diagram).  By resampling the weighted links of the original network, a bootstrap world is created of resampled networks.  Next, these are clustered and compared to the clustering of the original network (2nd row, right side).  This provides an estimate of the probability that a node belongs to a specific cluster.  The result is a “significant clustering” [3].  For example, in the diagram above, the darker nodes (bottom of the diagram) are clustered together in at least 95% of the 1000 bootstrap networks.  Several algorithms in the public domain exist to automate the majority of these tasks.  

Finally, once a significance cluster has been generated for the network at each point in time,  an alluvial diagram is used to reveal the trends in the data. An alluvial diagram (bottom of the picture) orders the cluster by size and reveals changes in network structures over time [3]. Please refer to the diagram below:

As you can see from the alluvial diagram, from time 1 to time 2, the condition scenario represented by ORANGE clustered with the condition scenario represented by PINK.  This clustering was a result of some underlying change, and was not obvious at time 1.  As a result, the bootstrap/cluster analysis allowed the quantitative mapping of the change to take place.

The model can be used in a variety of scenarios:  to map the changes in global weather patterns, US emigration flows from state to state based on various factors (employment, housing prices, education, income per capita), variations in federal funds market in response to major events [3], and track global targets of terrorism activity.

But my main area of interest is illustrating the method by applying to map the change in the structure of science [4].  Stay tuned.  I conclude with a rather lengthy but appropriate and relevant quote.

From Michael Focault’s “The Order of Things”

The problem of change.  It has been said that this work denies the very possibility of change. And yet my main concern has been with changes. In fact, two things in particular struck me: the suddenness and thorough­ness with which certain sciences were sometimes reorganized; and the fact that at the same time similar changes occurred in apparently very different disciplines. Within a few years (around 1800), the tradition of general grammar was replaced by an essentially historical philology; natural classifications were ordered according to the analyses of comparative anatomy; and a political economy was founded whose main themes were labour and production. Confronted by such a curious combination of phenomena, it occurred to me that these changes should be examined more closely, without being reduced, in the name of continuity, in either abruptness or scope. It seemed to me at the outset that different kinds of change were taking place in scientific discourse – changes that did not occur at the same level, proceed at the same pace, or obey the same laws; the way in which, within a particular science, new propositions were pro­duced, new facts isolated, or new concepts built up (the events that make up the everyday life of a science) did not, in all probability, follow the same model as the appearance of new fields of study (and the frequently corresponding disappearance of old ones); but the appearance of new fields of study must not, in turn, be confused with those overall re-dis­tributions that alter not only the general form of a science, but also its relations with other areas of knowledge. It seemed to me, therefore, that all these changes should not be treated at the same level, or be made to culminate at a single point, as is sometimes done, or be attributed to the genius of an individual, or a new collective spirit, or even to the fecundity of a single discovery; that it would be better to respect such differences, and even to try to grasp them in their specificity. In this way I tried to describe the combination of corresponding transformations that char­acterized the appearance of biology, political economy, philology, a number of human sciences, and a new type of philosophy, at the threshold of the nineteenth century.

References: 

[1] Foucault, M. (2002). The order of things. Routledge.

[2] Palla, G., Derényi, I., Farkas, I., & Vicsek, T. (2005). Uncovering the overlapping community structure of complex networks in nature and society. Nature,435(7043), 814-818.

[3] Rosvall, M., & Bergstrom, C. T. (2010). Mapping change in large networks. PloS one5(1), e8694.

[4] de Solla Price DJ (1965) Networks of scientific papers. Science 149: 510–515. doi:10.1126/science.149.3683.510.

Note: Dragon Dictate is used as a speech to text transcriber for a portion of this document.  Although I make every effort to proofread the postings, any unusual syntax, lexicon or semantic error in language is attributed to my lack of attention and the immaturity of this technology.