- The top German politicians don't have a party-specific style when they're campaigning for elections
- After the election some winning parties (CDU & AfD) exhibit similar tweet styles. They align in a cluster together.
- The members of the SPD strongly change their tweeting style after their bad election result. They decluster with other parties.
The algorithm described by Kohonen (1990) works as follows:
-
Define a two- or three-dimensional of neurons i and assign every neuron a coordinate in the feature space (also called weight vector). We can assign the weight vector randomly in the cube spanned by the min and max value of all features. Alternatively, we can span the grid in the space spanned by the largest two eigenvectors (principal components).
-
Choose one data point at a time -- either systematically or randomly. If the number of observations is really small, we can use bootstrapping to make sure we reach the necessary number of iterations
-
Compute the Euclidean distances from the data point to each neuron.
-
Identify the neuron with the smallest distance to the data point. This is the best matching unit (BMU).
-
Update the coordinates of the BMU and the neurons using a neighborhood function (f.e. gaussian, bubble, etc.) The learning rate in the neighborhood function determines how much the position of neurons change iteration and decreases with time.
-
Start again with 1. until you reached the desired amount of iterations.
An SOM possesses 7 tuning parameters:
-
Number of neurons: The larger the better. Our choice is 30x30 neurons. Alternatively use a growing SOM (GSOM).
-
Dimension of grid: No clear rule. If a lot is known about the data distribution, the dimension can be adjusted to follow the distribution more closely. This speeds up the training process.
-
Grid shape: Hexagonal or rectangular. If the data distribution is known, also a more specific shape is possible (circle, ring etc.).
-
Initial position of grid in space: span subspace by the first to d-th principal components (= eigenvectors corresponding to the d largest eigenvalues), initialize weights randomly or by sample points. The initializiation with principal components is supposed to speed up computation
-
Neighborhood function: Typically a Gaussian common choice, radial, triangle, bubble, etc. The most simple choice is a function that only updates the n closest neurons of the BMU
-
Learning rate: Typically decreases with time (lineary, logarithmic, etc.)
-
Distance measure: The most common distance measure is the Euclidean distance. However, also other measures could be used (Jaccard, Manhattan, etc.)
Alternative calibration with bubble neighborhood function and learning rate = 0.25
Cluster formation becomes more quadratic but our results from before still hold. For a mexican hat neighborhood function we found it quite difficult to find clusters at all. This is because a mexican hat neighborhood function skips some neurons in the neighborhood of the BMU and excites neurons that are 'behind' them. This does not really have a justification in our context.
- Original SOM: Teuvo Kohonen, Self-Organized Formation of Topologically Correct Feature Maps. Biological Cybernetics (1982)
- Minisom package for python with a range of tuning parameter options and examples https://github.com/JustGlowing/minisom
- Parameter options: http://www.cis.hut.fi/somtoolbox/documentation/somalg.shtml
- Doc2Vec model for numeric representation of facebook posts from German politicians: https://humboldt-wi.github.io/blog/research/information_systems_1718/04topicmodels/