low memory usage on 64bits windows

Once it's running
Post Reply
paketecuento
Posts: 4
Joined: 17 Jan 2018 22:57

low memory usage on 64bits windows

Post by paketecuento » 17 Jan 2018 23:27

Hi,

I set up a nice install of 64 bit gephi 0.9.2 (also tried 0.9.3-snapshot) on a 64 bit windows 10 machine with 256 GB RAM and 2 x Intel Xeon E5-2660 v4 (total of 24 cores and 56 threads). I modified gephi.conf to get -J-XMS64m and several -J-XMX values starting from -J-XMX2048m to -J-XMX102400m. Gephi starts normally and import successfully a graph of 140 000 nodes and 2 221 000 edges.

The issue arises when I check the memory usage. It is always about 5-10GB of RAM and cannot get it higher changing XMX value. So i experiment a very low run for my tests.

Any guess on how can I force a higher RAM usage?

User avatar
eduramiba
Gephi Code Manager
Posts: 976
Joined: 22 Mar 2010 15:30
Location: Madrid, Spain

Re: low memory usage on 64bits windows

Post by eduramiba » 18 Jan 2018 08:58

Hi,

This is normal memory usage by Gephi and it cannot benefit from using more. You probably are hitting a OpenGL rendering bottleneck or a CPU bottleneck depending on the algorithm you are running (complexity might be too high or not parallelized enough).

paketecuento
Posts: 4
Joined: 17 Jan 2018 22:57

Re: low memory usage on 64bits windows

Post by paketecuento » 18 Jan 2018 09:32

eduramiba wrote:
18 Jan 2018 08:58
Hi,

This is normal memory usage by Gephi and it cannot benefit from using more. You probably are hitting a OpenGL rendering bottleneck or a CPU bottleneck depending on the algorithm you are running (complexity might be too high or not parallelized enough).
Interesting... I always use ForceAtlas2 because of its nice layout representation, but in this case, I started with OpenOrd and 55 threads because OpenOrd is reported to deal with parallelization efficiently.

Also I notice some instabilities using gephi itself (mostly related with freezing screen).

I have the latest NVIDIA drivers for my Quadro P5000, including OpenGL v4.6.
Image


Anyway I will try to make some more tests.

thank you

broorg
Posts: 8
Joined: 20 Jun 2017 02:15

Re: low memory usage on 64bits windows

Post by broorg » 18 Jan 2018 22:55

140 000 nodes and 2 221 000 edges
That's not that much, I have 1,2 million nodes and 1,8 million edges and my gephi uses ~3-4gb RAM, a bit more while OrderOrd runs.

paketecuento
Posts: 4
Joined: 17 Jan 2018 22:57

Re: low memory usage on 64bits windows

Post by paketecuento » 19 Jan 2018 11:31

broorg wrote:
18 Jan 2018 22:55
140 000 nodes and 2 221 000 edges
That's not that much, I have 1,2 million nodes and 1,8 million edges and my gephi uses ~3-4gb RAM, a bit more while OrderOrd runs.
can you please post your gephi.conf and workstation specs? I use 64 bit for java and gephi. Also can you tell me about your OpenGL and graphic card?
I suffer continuos freezing of the GUI...

paketecuento
Posts: 4
Joined: 17 Jan 2018 22:57

Re: low memory usage on 64bits windows

Post by paketecuento » 24 Jan 2018 17:16

I'm trying other versions of gephi in order to find some clue about the weird problem I'm experiencing. After install 0.9.1, and open my network of about 140,000 nodes en 2,221,000 edges, the GUI reports this error:

Code: Select all

java.lang.RuntimeException: Waited 5000ms for: <98aef38, 82bc536>[count 3, qsz 0, owner <AWT-EventQueue-0-Display-.windows_nil-1-EDT-1>] - <DisplayAnimator>
	at jogamp.common.util.locks.RecursiveLockImpl01Unfairish.lock(RecursiveLockImpl01Unfairish.java:198)
	at com.jogamp.newt.opengl.GLWindow.display(GLWindow.java:755)
	at org.gephi.visualization.swing.GLAbstractListener.display(GLAbstractListener.java:322)
	at org.gephi.visualization.scheduler.CompatibilityScheduler$1.run(CompatibilityScheduler.java:97)
[catch] at org.gephi.visualization.scheduler.BasicFPSAnimator.run(BasicFPSAnimator.java:34)
So it looks like opengl is behind of this bottleneck. So any idea on how to debug opengl in my system?

It's a shame that 256GB of RAM and 55 threads are unusable with this wonderful piece of software :(

Post Reply