Source Code Visualization Plug-in

[Nellistos]

I stumbled into gephi while researching various visualization tools.

Since i am interested in sourcecode visualizations and just recently
had finished the migration of our company's repo from svn 1.3 to 1.6
i decided to experiment with gephi and subversion's logs.

So i wrote an exporter from svn log format to gexf after performing some analysis
and extracting relationships information among developers from the log.

And...voila!
Accurate and pleasant to the eye results!



Although our team is relatively small gephi succesfully identified and clustered our 3 separate groups
placing co-commiters in the edges.

Next i'll try to extract developer -> file information etc.

Another scenario i'd like to run would be to run gephi some very large data sets we have ( millions of records )
in an effort to better understand user behavior and identify domains' intricacies especially
for a newcomer to a project.

Tell me what you think!
Keep up the good work and thank you for Gephi!

[admin]

Hey that's amazing! Love it!

How did you converted the logs to gexf? It would be cool if an import plug-in would make Gephi able to produce this graph directly, don't it?

Keep us knowing your progresses.

Also if you want to communicate on your studies, feel free to propose an article for the Gephi blog

[mbastian]

That is great !!! You'll find help here to design such a plug-in. If you can generate a GEXF file it's not difficult to directly import the date from a plugin. The ImportAPI provides what you need. Possible improvements is to use dynamic features to create such a graph within time.

[Nellistos]

First of all, thank you very much for your commentary.
I am very glad you liked the visualization.

How did you converted the logs to gexf?

I wrote a subversion-log api to read and analyze the xml subversion's log files.

It would be cool if an import plug-in would make Gephi able to produce this graph directly, don't it?

Yes it would! I'd like to undertake it if you do not mind.
I plan to rewrite code to leverage gephi's import-api and refactor subversion-api to be more readable and extendable.
Well ... i used DOM to parse subversion log files because i wanted zero dependencies although i know it is mermory consuming and ... ugly. But if this is going to be included as a plugin in gephi i'd preger to go with StAX.

feel free to propose an article for the Gephi blog

Definetely i will I just need some time to improve the subversion logs' analysis and validate the results against unknown repositories.

Possible improvements is to use dynamic features to create such a graph within time.

Exactly! What i have in mind is an import plugin for subversion's log files exporting a network including optianally:

• Dev to Dev relationships

• Dev to File

• File to File - not from a dependencies perspective

• Dynamic or not
  • Thank you again for your feedback you'll probably hear from me again after Monday [took some days off of work ]
    probably asking for help

[admin]

You're welcome

[pbittner]

Very nice work Nellistos! It's a funny coincidence that you bring that idea right now. As part of the Software Visualization Plug-in that I am currently writing, I have also added the functionality to import data from SVN and Bazaar log files. I associate to each file in the repository the last developer to make a change, the main developer (the one that changes the file the most often), the number of changes and the last change (revision number). As I target source code visualization, I don’t currently extract a developer-developer as you are doing.

Regarding the parsing of the XML files, I use Stax which is more efficient than Dom as you state it.

Since my last post, I have added more functionalities to my plug-in. It is possible to get a call graph or a dependency graph of a java application with metrics (number of lines, cyclomatic complexity, maintainability index, cohesion, coupling, number of bugs…) attached to each method. It is also possible to visualize a software graph at a package and class level.

And here is an example: Class Dependency Graph of my Gephi branch with the different developers (color) and the total number of changes made in the repository (size).

[Nellistos]

Very nice!

Very nice work Nellistos! It's a funny coincidence

Indeed!

I am currently trying to visualize a social perspective of the repository to depict bonds among developers and files - not by their dependencies but their tendency to get modified simultaneously.

Number of changes separated by type - Add, Merge etc - are of course supported as node attributes and in the mean time i plan to improve and implement a deeper analysis - ( does a developer Modifies frequently files that someone else Added? adjust edge attribute accordigly etc ) - and provide more accurate dev to dev graphs.

On the other hand i do not plan to support metrics' visualization which is a very interesting visual of course but currently out of my scope. Rolling eyes on the graph you posted .

Sure thing is we have an overlap here
The question is do we merge or continue separately?

My opinion is since we have already implemented some features a merging would slow us both down.
Should we let code mature, materialize as two separate plugins and maybe join forces later?

What do you think? Any advise from the main devs?

[admin]

You should continue separately, not because of some kind of competition, but because you might have two different approaches and way of doing. After your respective releases, it will be fulfilling to discuss about your solutions of common challenges, if a way of doing is more efficient than an other...

Another good point is to visualize the same SVN repos and see the different results and figure out why: bug? design? etc.

[pbittner]

My plugin is not devoted to study repositories, it is only a small feature I included. So we only have a small overlap and I think that merging is not really an option right now. But when your plugin will be mature enough to be released, I will definitely take advantage of it in my own plugin. I still have a lot of work to do on it and will not release it before a couple of weeks.

Regarding repositories, the next step would be to use the Java API provided by SVN or CVS to connect directly to a repository and have access to more functionalities. But that would require a lot more work

[jbilcke]

This topic's starting to become exciting

I'd be happy to test your two plugins once they will be available