[phpBB Debug] PHP Warning: in file [ROOT]/phpbb/session.php on line 583: sizeof(): Parameter must be an array or an object that implements Countable
[phpBB Debug] PHP Warning: in file [ROOT]/phpbb/session.php on line 639: sizeof(): Parameter must be an array or an object that implements Countable
[phpBB Debug] PHP Warning: in file [ROOT]/includes/functions.php on line 4516: Cannot modify header information - headers already sent by (output started at [ROOT]/includes/functions.php:3262)
[phpBB Debug] PHP Warning: in file [ROOT]/includes/functions.php on line 4516: Cannot modify header information - headers already sent by (output started at [ROOT]/includes/functions.php:3262)
[phpBB Debug] PHP Warning: in file [ROOT]/includes/functions.php on line 4516: Cannot modify header information - headers already sent by (output started at [ROOT]/includes/functions.php:3262)
Gephi forums •low memory usage on 64bits windows
Page 1 of 1

low memory usage on 64bits windows

Posted: 17 Jan 2018 23:27
by paketecuento
Hi,

I set up a nice install of 64 bit gephi 0.9.2 (also tried 0.9.3-snapshot) on a 64 bit windows 10 machine with 256 GB RAM and 2 x Intel Xeon E5-2660 v4 (total of 24 cores and 56 threads). I modified gephi.conf to get -J-XMS64m and several -J-XMX values starting from -J-XMX2048m to -J-XMX102400m. Gephi starts normally and import successfully a graph of 140 000 nodes and 2 221 000 edges.

The issue arises when I check the memory usage. It is always about 5-10GB of RAM and cannot get it higher changing XMX value. So i experiment a very low run for my tests.

Any guess on how can I force a higher RAM usage?

Re: low memory usage on 64bits windows

Posted: 18 Jan 2018 08:58
by eduramiba
Hi,

This is normal memory usage by Gephi and it cannot benefit from using more. You probably are hitting a OpenGL rendering bottleneck or a CPU bottleneck depending on the algorithm you are running (complexity might be too high or not parallelized enough).

Re: low memory usage on 64bits windows

Posted: 18 Jan 2018 09:32
by paketecuento
eduramiba wrote:
18 Jan 2018 08:58
Hi,

This is normal memory usage by Gephi and it cannot benefit from using more. You probably are hitting a OpenGL rendering bottleneck or a CPU bottleneck depending on the algorithm you are running (complexity might be too high or not parallelized enough).
Interesting... I always use ForceAtlas2 because of its nice layout representation, but in this case, I started with OpenOrd and 55 threads because OpenOrd is reported to deal with parallelization efficiently.

Also I notice some instabilities using gephi itself (mostly related with freezing screen).

I have the latest NVIDIA drivers for my Quadro P5000, including OpenGL v4.6.
Image


Anyway I will try to make some more tests.

thank you

Re: low memory usage on 64bits windows

Posted: 18 Jan 2018 22:55
by broorg
140 000 nodes and 2 221 000 edges
That's not that much, I have 1,2 million nodes and 1,8 million edges and my gephi uses ~3-4gb RAM, a bit more while OrderOrd runs.

Re: low memory usage on 64bits windows

Posted: 19 Jan 2018 11:31
by paketecuento
broorg wrote:
18 Jan 2018 22:55
140 000 nodes and 2 221 000 edges
That's not that much, I have 1,2 million nodes and 1,8 million edges and my gephi uses ~3-4gb RAM, a bit more while OrderOrd runs.
can you please post your gephi.conf and workstation specs? I use 64 bit for java and gephi. Also can you tell me about your OpenGL and graphic card?
I suffer continuos freezing of the GUI...

Re: low memory usage on 64bits windows

Posted: 24 Jan 2018 17:16
by paketecuento
I'm trying other versions of gephi in order to find some clue about the weird problem I'm experiencing. After install 0.9.1, and open my network of about 140,000 nodes en 2,221,000 edges, the GUI reports this error:

Code: Select all

java.lang.RuntimeException: Waited 5000ms for: <98aef38, 82bc536>[count 3, qsz 0, owner <AWT-EventQueue-0-Display-.windows_nil-1-EDT-1>] - <DisplayAnimator>
	at jogamp.common.util.locks.RecursiveLockImpl01Unfairish.lock(RecursiveLockImpl01Unfairish.java:198)
	at com.jogamp.newt.opengl.GLWindow.display(GLWindow.java:755)
	at org.gephi.visualization.swing.GLAbstractListener.display(GLAbstractListener.java:322)
	at org.gephi.visualization.scheduler.CompatibilityScheduler$1.run(CompatibilityScheduler.java:97)
[catch] at org.gephi.visualization.scheduler.BasicFPSAnimator.run(BasicFPSAnimator.java:34)
So it looks like opengl is behind of this bottleneck. So any idea on how to debug opengl in my system?

It's a shame that 256GB of RAM and 55 threads are unusable with this wonderful piece of software :(

Re: low memory usage on 64bits windows

Posted: 03 Jun 2018 23:48
by broorg
paketecuento wrote:
19 Jan 2018 11:31
can you please post your gephi.conf and workstation specs? I use 64 bit for java and gephi. Also can you tell me about your OpenGL and graphic card?
I suffer continuos freezing of the GUI...
Sorry, wasn't in the forum till now .... hope it still helps.

Gephi Config:

Code: Select all

# ${HOME} will be replaced by user home directory according to platform
default_userdir="${HOME}/.${APPNAME}/0.9.2/dev"
default_mac_userdir="${HOME}/Library/Application Support/${APPNAME}/0.9.2/dev"

# options used by the launcher by default, can be overridden by explicit
# command line switches
default_options="--branding gephi -J-Xms64m -J-Xmx12000m -J-Xverify:none -J-Dsun.java2d.noddraw=true -J-Dsun.awt.noerasebackground=true -J-Dnetbeans.indexing.noFileRefresh=true -J-Dplugin.manager.check.interval=EVERY_DAY -J-Duser.language=en"
# for development purposes you may wish to append: -J-Dnetbeans.logger.console=true -J-ea

# default location of JDK/JRE, can be overridden by using --jdkhome <dir> switch
#jdkhome="/path/to/jdk"

# clusters' paths separated by path.separator (semicolon on Windows, colon on Unices)
#extra_clusters=
Workstation:
CPU: Ryzen 5 1600, 6 cores, 12 threads
RAM: 16GB DDR4
SSD: Samsung 850 Evo 500GB
GPU: GTX960 3GB, actual Nvidia driver
JVM: always up to date due to auto updater, atm java version "1.8.0_171"