Gephi and Digital Humanities Networks

[elijah]

I've been using Gephi for a couple months now, but more seriously lately. I've been posting a few problems, so I figured I'd also post a note about how excited and happy I am with the tool. One of the projects I've taken part in at Stanford was to transition the Mapping the Republic of Letters database from a relational to a graph model. The result is an 86,000 node/695,000 edge database of letters, individuals, locations, sources and routes of the 17th-19th century. While my initial efforts to ingest the entire database and "see what happens" have made me dream of xgrid implementations of Gephi and bemoan my paltry 8GB of RAM, I've been taking individual, highly-connected actors and envisioning their network in part and in whole. Gephi's intuitive interface and the explicitness of its settings has helped tremendously in my ability to understand network data and have created a number of sophisticated visualizations. In fact, it's so far outstripped my understanding of what I'm actually producing that I feel both invigorated and intimidated. Still, that hasn't stopped me from showing off one of the more interesting visualizations, a portrait of Benjamin Franklin:

https://dhs.stanford.edu/spatial-humani ... ing-gephi/

Thanks so much for producing such a professional, powerful tool. Keep up the good work!

[admin]

Hi,

Thank you so much for this positive feedback!

I like the way you sum up the importance of aesthetics in the blog post:

Gephi provides such arresting visuals that it makes me feel like I need to spend the rest of the year figuring out just what, exactly those visuals are.

That's why we care about the beauty of the visualization: graphs are generally ugly and don't invite people to mine inside the data. Aesthetics enables this desire! then knowledge can start flowing after hours of hard work

Btw, it's weird that you need so much time to apply a layout. I have a 2-year laptop with 2BG RAM, and I usually get a satisfactory visual of a 5k nodes and 50k edges in less than 4 hours. Do you apply the layout just after having loaded and filtered the data?

Maybe these tips can help:
* apply the Yifan Hu's multilevel algorithm before using the ForceAtlas. It will explode the graph well.
* speed-up the ForceAtlas by increasing the "speed" and "max displacement" options. The gain is very impressive, and you'll decrease these values after that to recover precision.

[elijah]

Thanks for the tips. I feel like my current work with Gephi mirrors my initial experiences with ArcGIS and I'm sure, given that experience, that the manner in which I'm approaching my dasasets leaves a lot to be desired in the realm of sophisticated and efficient use of the tool. I haven't had much success with the multilevel Yifan Hu in this dataset, but the other Yifan Hu algorithm explodes things well (and provides meaningful visualization of document-location or document-person networks, but tends to be illegible when looking at document-person-location networks, whereas ForceAtlas produces recognizable regions). I confess that I enjoy the Overtaxed Machine theme, but I'm sure it's more related to the lack of sophistication on my part and less the complexity of the data I'm examining. I tend to filter the data in MySQL first and then ingest and apply the layout but I'm not doing much more than selecting one or two top level options and then letting it run.

[mbastian]

Great feedback, very happy the tool is fulfilling your needs. It also shows how Gephi enabled you to go deeper in the dataset, bringing new questions.

If your "Yifan Hu Multilevel" algorithm was stopping to quickly, I suggest to increase the step ratio from 0.97 to 0.99 or 0.995. The algorithm is stopping by itself and increasing this number would let the algorithm keep going longer. However, "Yifan Hu" is good also and will return quite similar results.

[elijah]

I tried multilevel again with up to .999 and the dataset remained a square the entire time. I don't think you should worry too much about it--I've got quite a bit more studying to do before I move beyond just messing around.