StatSVN performance is bad on my system

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

StatSVN performance is bad on my system

chavarahul
Hi,
 
     When I tried StatSVN on my project, in console it is showing approximate complete time as 9hours. Is there any way to decrease this time?

By reading console log, I understood that it is using 20 threads to perform this task. Is there any way to configure this number of threads to improve performance?


-
Rahul
Reply | Threaded
Open this post in threaded view
|

Re: StatSVN performance is bad on my system

jkealey@lavablast.com
StatSVN uses threads automatically to speed things up.  It starts with a single thread and, if it looks slow, launches other threads. The threads option (http://wiki.statsvn.org/User%20Manual.ashx) lets you configure how many threads are used. 

The main reason why StatSVN would be slow on its first run (other than having a huge repository) is because your server is throttling your connections, is slow, or your internet connection is simply slow.  Many servers have denial of service protection that end up slowing down StatSVN. Other servers are simply slow: I find SourceForge is horribly slow compared to my own servers, for example. 

Thanks,
---
Jason Kealey - [hidden email]
LavaBlast Franchise Software - http://www.lavablast.com
Simplifying day-to-day franchise operations


On Sat, Feb 13, 2010 at 2:32 AM, chavarahul <[hidden email]> wrote:

Hi,

    When I tried StatSVN on my project, in console it is showing
approximate complete time as 9hours. Is there any way to decrease this time?

By reading console log, I understood that it is using 20 threads to perform
this task. Is there any way to configure this number of threads to improve
performance?


-
Rahul

--
View this message in context: http://n2.nabble.com/StatSVN-performance-is-bad-on-my-system-tp4565304p4565304.html
Sent from the StatSVN Users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
SOLARIS 10 is the OS for Data Centers - provides features such as DTrace,
Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW
http://p.sf.net/sfu/solaris-dev2dev
_______________________________________________
Statsvn-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/statsvn-users



------------------------------------------------------------------------------
SOLARIS 10 is the OS for Data Centers - provides features such as DTrace,
Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW
http://p.sf.net/sfu/solaris-dev2dev
_______________________________________________
Statsvn-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/statsvn-users
Reply | Threaded
Open this post in threaded view
|

Re: StatSVN performance is bad on my system

Quix0r
This post was updated on .
The "problem" arises with 0.7, version 0.5/0.6 where both fine and worked flawless here. :) I just need to increase a little the reserved RAM and all was working again. But with 0.7 it tooks more than 12 hours (!) and still it wasn't complete after that long time.

Hope you can take a look at this.

For instance, please use this SVN repos:
http://mxchange.org/repos/mailer/branches/0.2.1-FINAL/

... and create a verbose logfile. :)

Edit: URL has changed, no www.
Reply | Threaded
Open this post in threaded view
|

Re: StatSVN performance is bad on my system

jkealey@lavablast.com
If you run version 0.6 today on the same server, does it still run fast? (Assuming you haven't cached all the results) The most common reason for it running slow are server-side firewalls. 

Otherwise are you using the svnkit command line option? Technically, between 0.6 and 0.7 we integrated a prototype for svnkit but left the original code as the default. In some contexts svn-kit was faster, in others it is slower. Therefore, nothing should have changed in the way we query in the default 0.7. 

---
Jason Kealey - [hidden email]
LavaBlast Franchise Software - http://www.lavablast.com
Simplifying day-to-day franchise operations


On Sun, Mar 7, 2010 at 9:28 PM, Quix0r <[hidden email]> wrote:

The "problem" arises with 0.7, version 0.5/0.6 where both fine and worked
flawless here. :) I just need to increase a little the reserved RAM and all
was working again. But with 0.7 it tooks more than 12 hours (!) and still it
wasn't complete after that long time.

Hope you can take a look at this.

For instance, please use this SVN repos:
http://www.mxchange.org/repos/mailer/branches/0.2.1-FINAL/

... and create a verbose logfile. :)
--
View this message in context: http://n2.nabble.com/StatSVN-performance-is-bad-on-my-system-tp4565304p4693059.html
Sent from the StatSVN Users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Statsvn-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/statsvn-users



------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Statsvn-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/statsvn-users
Reply | Threaded
Open this post in threaded view
|

Re: StatSVN performance is bad on my system

Quix0r
Yes, the same applies to 0.6. :) But unfortunally I replaced it with 0.7 and did not backup it. :( Is there somewhere a copy around?
Reply | Threaded
Open this post in threaded view
|

Re: StatSVN performance is bad on my system

jkealey@lavablast.com
I made a mistake: 0.6 never existed. We went from 0.5 to 0.7 to keep in synch with StatCVS. 

Here are the links for downloads: 

Please confirm that 0.5 runs fast and 0.7 runs slow, on the same repository, if you specify an empty cache directory. 

Thanks,
---
Jason Kealey - [hidden email]
LavaBlast Franchise Software - http://www.lavablast.com
Simplifying day-to-day franchise operations


On Tue, Mar 9, 2010 at 5:13 PM, Quix0r <[hidden email]> wrote:

Yes, the same applies to 0.6. :) But unfortunally I replaced it with 0.7 and
did not backup it. :( Is there somewhere a copy around?
--
View this message in context: http://n2.nabble.com/StatSVN-performance-is-bad-on-my-system-tp4565304p4705505.html
Sent from the StatSVN Users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Statsvn-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/statsvn-users



------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Statsvn-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/statsvn-users
Reply | Threaded
Open this post in threaded view
|

Re: StatSVN performance is bad on my system

Quix0r
Here is my script for automated stats generation for instance: (I know, running things as root is not good. :( )
#!/bin/sh

cd /root
echo "$0: Checking out branch 0.2.1-FINAL ..."
nice --adjustment=19 svn co file:///var/www/some-repos/path/branches/0.2.1-FINAL/ /root/co > /dev/null 2>&1
echo "$0: Generating logfile ..."
nice --adjustment=19 svn log --xml -vvv /root/co/ > /root/svn-stats.xml
cd /var/www/some/long/path
rm -f *
echo "$0: Generating statistics ..."
nice --adjustment=19 java -mx272m -jar /root/statsvn.jar /root/svn-stats.xml /root/co/ > /dev/null 2>&1
echo "$0: chown ..."
chown -c foo.bar * > /dev/null 2>&1
echo "$0: Cleaning up ..."
rm -rf /root/co/ /root/svn-stats.xml
I killed some paths/user names due to security concerns. 0.5 runs flawless, but 0.7 runs very slow but only if I set -mx256m. But for that repos I need more than 256m.
Reply | Threaded
Open this post in threaded view
|

Re: StatSVN performance is bad on my system

jkealey@lavablast.com
Hey,

Looking at my Eclipse settings, -mx256m means a maximum of 256MB of ram...not a minimum or starting point. Therefore, you are limiting it's memory usage, which is causing your problem. If you used -ms256m, it would start at 256MB. 

It also looks like -mx is the old way of setting the option. The newer way is -Xmx.  I am not sure if you should say M or m to mean a megabyte. 

I therefore suggest you try something like this:
-Xms512M -Xmx1024M

Thanks,
---
Jason Kealey - [hidden email]
LavaBlast Franchise Software - http://www.lavablast.com
Simplifying day-to-day franchise operations


On Tue, Mar 9, 2010 at 8:28 PM, Quix0r <[hidden email]> wrote:

Here is my script for automated stats generation for instance: (I know,
running things as root is not good. :( )

#!/bin/sh
>
> cd /root
> echo "$0: Checking out branch 0.2.1-FINAL ..."
> nice --adjustment=19 svn co
> file:///var/www/some-repos/path/branches/0.2.1-FINAL/ /root/co > /dev/null
> 2>&1
> echo "$0: Generating logfile ..."
> nice --adjustment=19 svn log --xml -vvv /root/co/ > /root/svn-stats.xml
> cd /var/www/some/long/path
> rm -f *
> echo "$0: Generating statistics ..."
> nice --adjustment=19 java -mx272m -jar /root/statsvn.jar
> /root/svn-stats.xml /root/co/ > /dev/null 2>&1
> echo "$0: chown ..."
> chown -c foo.bar * > /dev/null 2>&1
> echo "$0: Cleaning up ..."
> rm -rf /root/co/ /root/svn-stats.xml
I killed some paths/user names due to security concerns. 0.5 runs flawless,
but 0.7 runs very slow but only if I set -mx256m. But for that repos I need
more than 256m.
--
View this message in context: http://n2.nabble.com/StatSVN-performance-is-bad-on-my-system-tp4565304p4706436.html
Sent from the StatSVN Users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Statsvn-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/statsvn-users



------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Statsvn-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/statsvn-users
Reply | Threaded
Open this post in threaded view
|

Re: StatSVN performance is bad on my system

Quix0r
I have now a stronger server box (quad-core, 4 GB RAM). The performance is now back as usual:

runtime: 123.215 seconds
memory usage: 832448.0 kb


And another one:
runtime: 114.169 seconds
memory usage: 393472.0 kb


So the trick was to increase memory. I still would recommend to rewrite this to a less memory-consuming approach.