running boinc on a ram drive

Message boards : Number crunching : running boinc on a ram drive
Message board moderation

To post messages, you must log in.

AuthorMessage
Rainz3

Send message
Joined: 2 Apr 05
Posts: 10
Credit: 23,310
RAC: 0
United States
Message 222633 - Posted: 29 Dec 2005, 5:39:11 UTC

As the title states I was wondering what kind of impact it would have on boinc to run it from a ram drive? I am thinking of setting one up on both my computers to run various little programs like winamp and stuff like that. anyone got any idea if it would help/hurt?
ID: 222633 · Report as offensive
Profile AlecStaar
Avatar

Send message
Joined: 16 Dec 05
Posts: 260
Credit: 44,472
RAC: 0
United States
Message 222635 - Posted: 29 Dec 2005, 5:45:31 UTC - in response to Message 222633.  
Last modified: 29 Dec 2005, 6:02:44 UTC

As the title states I was wondering what kind of impact it would have on boinc to run it from a ram drive? I am thinking of setting one up on both my computers to run various little programs like winamp and stuff like that. anyone got any idea if it would help/hurt?


I do it, but on a Solid-State unit by CENATEK.

There's "advantages" to it, but they're not 'huge' imo, unless you reset how often the client portions write to disk.

Where ramdrives rule, imo, the most? File seeks/access times...

On ramdrives (software OR solid-state units), those access times are just many, MANY orders of magnitude faster than even the fastest ATA/EIDE or UltraScSi disks out there (10k-15k rpm, notwithstanding).

The latency in seek time (which iirc, is 2ms you subtract off the avg. access time) is when the disk read/write heads have to find the file on disk first, & mechanical parts introduce this - they just do not move as fast as memory accesses period.

Others may add to this (or, correct me, as I MAY be 'off' in my definitions in that last paragraph - but, DO correct me if needed, thanks), but that's my take on it, I use it...

So, bottom-line - Whenever this project's exe's need to hit disk/files?

It's just dead-up as fast as it can be here (unless there is faster solid-state drives out there now, & their might be, mine's 2++ years old now & things in THIS field? Change for the better (usually) every year imo)...

APK

P.S.=> There's a review I did for CENATEK in my signature, feel free to read it, it's decent enough to still be featured on that company's front/main page of their website in fact.

(In my review's content, I also cited tests others did as well in it, iirc, Maximum PC mag did one as well, also Windows IT Pro mag also)

It may give you an idea WHERE using ramdisks/ramdrives can help you out here on this project, & others...

Also in my signature is a decent "tuning guide" (started out as NTCompatible.com's "Article #1" back circa 1997-1998 when I first posted it & was the "original" out there afaik, & entire sites grew out of that concept alone over time... anyhow, it grew from there for both security AND speed optimization of your OS + software) you might be able to get something out of for better performance (and, additionally, security of a Win32 based OS @ least, specifically NT-based (NT/2000/XP/Server 2003))

It mentions other uses of them as well (like putting your pagefile.sys onto one, but I wouldn't with a software-based one on paging files, using it for your system's environment %TEMP% variable, webpage caching, & more).

The reason I mention this last part (tuning guide)? Well, speedup how your OS reacts/acts, & all else usually gets a boost too...

IMO, it never hurts to learn how to speedup the OS and your software... after all - it's ALL part of a whole machine acting in synergy.

(Only problem I have EVER noted in tuning/optmizing is, you generally seem to "pull" from one area, to give gain & "push" it to another area)

Anyhow - The costs of the solid-state ones though? That's their downside - megacoins! apk
http://torry.net/authorsmore.php?id=1781

"The object's hull is made of SOLID neutronium: A single StarShip cannot combat it!" quote Mr. Spock, Star Trek original series, episode title: "The Doomsday Machine"
ID: 222635 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13746
Credit: 208,696,464
RAC: 304
Australia
Message 222697 - Posted: 29 Dec 2005, 9:22:51 UTC - in response to Message 222633.  

As the title states I was wondering what kind of impact it would have on boinc to run it from a ram drive? I am thinking of setting one up on both my computers to run various little programs like winamp and stuff like that. anyone got any idea if it would help/hurt?

For programmes where you've got lots of very random accesses of large or mutliple files or large streaming files it would give a considerable benefit- except for the fact you would loose everything when the power goes off.

I wouldn't think it would have that much effect with Seti as although there are many frequent writes (and i assume reads) to disk, due to the size of the file in question it would all be taken care of by the system's HD cache as well as the drive's own hardwarde cache.
Grant
Darwin NT
ID: 222697 · Report as offensive
Profile AlecStaar
Avatar

Send message
Joined: 16 Dec 05
Posts: 260
Credit: 44,472
RAC: 0
United States
Message 222764 - Posted: 29 Dec 2005, 13:17:32 UTC - in response to Message 222697.  
Last modified: 29 Dec 2005, 13:34:44 UTC

except for the fact you would loose everything when the power goes off.


Which is PART of the reason I switched from using software-based ramdrives (I even designed one YEARS ago, nearly a decade now based off the MS-DDK prototype): You lose "state" when the power goes out.

The CENATEK unit I mention? Comes with a backing power supply, & if you couple that with a UPS (unininterruptible power supply)? As the saying goes:

"Baby got back"

(LOL, but not the same meaning, of course)

Also, when you create a software-based ramdisk, you are chopping out portions of system RAM... portions the diskcache of the OS (or 3rd party, e.g.-> SuperCache-NT/SuperCache-II by EEC Systems/SuperSpeed.com) could potentially/theoretically use, or your applications for their datasets.

BUT, out of today's typical setups of 256mb-1gb system RAM onboard? 32mb might be considered negligible by some but not by others - something to consider.

Could you use a software-based one? Sure imo. The folder here for SETI@Home is of a size of 22.4mb per Explorer.exe's properties reading of it.

Thus, even the older 32mb maximum-size MS-DDK model for NT-based OS (one model is for NT, the other works for 2000/XP/Server 2003 afaik) can work here, assuming YOUR folder for SETI@Home #2 is not larger than 32mb in size, & if so?

However - for software-based ramdisks? There are alternatives & ones that DON'T have a 32mb maximum size-limit...

In fact, I mention a GREAT free one below in fact next, read on:

And, the 'best' software-based RamDisk (because it's free & has theoretically unlimited size) is ArSoft's model.

(The developer once told me how to take the one I built years ago & have it beat its 32mb size-limits even, that's how cool the guy was... I never got around to it, & instead utilized his here, prior to switching to a hardware-based one in the CENATEK "rocketdrive" here though).

I wouldn't think it would have that much effect with Seti as although there are many frequent writes (and i assume reads) to disk, due to the size of the file in question it would all be taken care of by the system's HD cache as well as the drive's own hardwarde cache.


The HD cache is really a buffer - afaik, there's no algorithm governing it like true caches have (e.g.-> FIFO/LRU & 4-way set associative), & it's really a delayed write mechanism in that it's an area of memory accumulating data until it fills & POW- down to disk it goes. NO "intelligence" really.

And, diskcaches take time to build up efficacy, whereas ramdrives (of any kind) ALWAYS operate @ the speed of the memory they reside or are created on - always, from the get-go, to the finish & if I wish?

Since the CENATEK 'rocketdrive' I use here (w/ 2gb onboard, 4gb max per board & 4 can be spanned/striped into 1 LARGE 16gb unit no less) can be 'cached' by the OS diskcache as well, but I don't use it that way here myself.

APK

P.S.=> Above all - If you guys have a better definition of it (hd buffer in particular above) that contradicts mine, let 'er rip - I can stand correction when I'm "off", to gain in the long run, like anyone else... apk
http://torry.net/authorsmore.php?id=1781

"The object's hull is made of SOLID neutronium: A single StarShip cannot combat it!" quote Mr. Spock, Star Trek original series, episode title: "The Doomsday Machine"
ID: 222764 · Report as offensive
Profile Mr.Pernod
Volunteer tester
Avatar

Send message
Joined: 8 Feb 04
Posts: 350
Credit: 1,015,988
RAC: 0
Netherlands
Message 222771 - Posted: 29 Dec 2005, 13:41:49 UTC

True ramdrives (in system memory) are faster, but with the read/write behaviour of BOINC/SETI your gains may be counted in seconds per day. Deduct the time it takes to load/unload from harddisk when rebooting and your gains are gone.

Solid state drives like the CENATEK's and Gigabyte's iRam are just too expensive for the amount of performancegain they give for "regular" users.
For the same amount of money as a fully loaded iRam, I can buy a SCSI-controller and a couple of SCSI-disks that will max out the PCI-bus, pretty much matching the iRam in performance, but giving a lot more diskspace.
ID: 222771 · Report as offensive
Profile AlecStaar
Avatar

Send message
Joined: 16 Dec 05
Posts: 260
Credit: 44,472
RAC: 0
United States
Message 222777 - Posted: 29 Dec 2005, 14:11:32 UTC - in response to Message 222771.  
Last modified: 29 Dec 2005, 14:17:42 UTC

True ramdrives (in system memory) are faster, but with the read/write behaviour of BOINC/SETI your gains may be counted in seconds per day. Deduct the time it takes to load/unload from harddisk when rebooting and your gains are gone.


I reboot once every GREAT while (usually for driver installs or Diskeeper 9.x boottime directory consolidations only). That's not an issue for myself @ least... not normally, on reboots.

HOWEVER, I'll take the gains it gives which you mention, but... you failed to mention another (that I did in my earlier replies):

Superior seek times/access of file times...

It's VERY evident, & most of the tests in the analysis I wrote up for CENATEK show it, in my analysis & also those of others cited in mine (I put in URL's to other tests of the CENATEK unit as well into my analysis so others had more reference than just my own).

It's also evident in process' running speed, those that are diskspeed dependent - see my review, & note the zip/unzip tests for instance (& others).

If a process is diskbound (& seti is to an extent, but perhaps not as much as a zip/unzip would be - which is WHY I mentioned the fact you can alter how often the SETI@Home2 process uses the disk in fact in its preferences iirc so you CAN get the "bennies" of using a ramdisk of any type really), & does File I/O, a ramdisk can be very helpful for extra speed.

Solid state drives like the CENATEK's and Gigabyte's iRam are just too expensive for the amount of performancegain they give for "regular" users.


Yes, that is their 'downside': cost, & I mention it freely now here in this reply & earlier as well - no avoiding it.

In fact, & I will admit this too: I got a large discount on mine because of the review I did in fact, when I mentioned I would like to have one, but could not afford it (after I submitted my review)...

They gave me the time I put into the article & tests off as work done basically for them, more or less, as a discount on its asking price.

For the same amount of money as a fully loaded iRam, I can buy a SCSI-controller and a couple of SCSI-disks that will max out the PCI-bus, pretty much matching the iRam in performance, but giving a lot more diskspace.


More diskspace is true, but seek times/access times would not be NEARLY as fast as a ramdisk yields, not by a long-shot (RAID or not even). Also, if a process is very diskbound in File I/O??

I just cannot see std. electro-mechanical hdd's being as fast as memory based disks are... software based ones will be faster (no travel over the PCI bus as my CENATEK one does) but still, no head movement either... thus, greater latency, slower seek/access times to files as well.

E.G.-> Memory is what? 1000x as fast as hdd's are, due to mechanical latencies of read/write head movement??

APK
http://torry.net/authorsmore.php?id=1781

"The object's hull is made of SOLID neutronium: A single StarShip cannot combat it!" quote Mr. Spock, Star Trek original series, episode title: "The Doomsday Machine"
ID: 222777 · Report as offensive
SteveB

Send message
Joined: 1 Jul 99
Posts: 5
Credit: 16,465,234
RAC: 62
United Kingdom
Message 222778 - Posted: 29 Dec 2005, 14:14:51 UTC - in response to Message 222633.  

As the title states I was wondering what kind of impact it would have on boinc to run it from a ram drive? I am thinking of setting one up on both my computers to run various little programs like winamp and stuff like that. anyone got any idea if it would help/hurt?


You can stick a Compact Flash card into an IDE adaptor and get much the same effect (only use it for 'read only' and 'write not very often' files = CF's will 'wear out' after a few 100,000 write operations :-) )

As for effect on SETI etc., well since SETI is CPU bound (and only writes a few bytes to disk every 10 mins. or so) 'a few seconds a day' is probably about right.

NB> REAL advantage of RAM disks is when you want to run a SETI Farm !
ID: 222778 · Report as offensive
Profile AlecStaar
Avatar

Send message
Joined: 16 Dec 05
Posts: 260
Credit: 44,472
RAC: 0
United States
Message 222780 - Posted: 29 Dec 2005, 14:19:16 UTC - in response to Message 222778.  
Last modified: 29 Dec 2005, 14:25:15 UTC

As for effect on SETI etc., well since SETI is CPU bound (and only writes a few bytes to disk every 10 mins. or so) 'a few seconds a day' is probably about right.


Note that I mention "stepping up/increasing" the amount of times that SETI writes to disk in my replies?

That's WHY I do it - to hopefully gain here by using a Ramdisk in hardware.

It's configurable in your SETI@Home 2 settings.

You can stick a Compact Flash card into an IDE adaptor and get much the same effect (only use it for 'read only' and 'write not very often' files = CF's will 'wear out' after a few 100,000 write operations :-) )


The 'wear out' factor definitely comes into play, but another I'd like to mention since I tested it (albeit 2-3 years ago for comparison' sake):

Compact Flash cards, when I tested them @ least via a Sandisk reader, don't NEARLY run as fast as the CENATEK unit I have, and especially not as fast as software-based Ramdrives do... not by a longshot & ESPECIALLY on writes... it was VERY slow here, & not very fast, imo @ least, on read operations either.

I tested this 2-3 years ago & found that... have Compact Flash cards sped up since then?

APK

P.S.=> Also, why I earlier mentioned using the settings in SETI@Home 2 for reconfigging the amount of times it hits disk? That IS configurable as a setting in SETI@Home #2, how often the process hits disk, & if you step it up, you get more out of using a ramdrive for this project's file I/O stages... apk
http://torry.net/authorsmore.php?id=1781

"The object's hull is made of SOLID neutronium: A single StarShip cannot combat it!" quote Mr. Spock, Star Trek original series, episode title: "The Doomsday Machine"
ID: 222780 · Report as offensive
Profile Mr.Pernod
Volunteer tester
Avatar

Send message
Joined: 8 Feb 04
Posts: 350
Credit: 1,015,988
RAC: 0
Netherlands
Message 222789 - Posted: 29 Dec 2005, 14:40:20 UTC

I have a machine at home that has been up for a couple of days, running BOINC.
I'll check the I/O-info in taskmanager and post the numbers here.
In the mean time, here's some numbers from my laptop (Write to disk at most every: 300 seconds), which has now been running BOINC/SETI for 6h30m.
BOINC:
reads: 340 operations / 1.156.360 bytes (+70% at startup)
writes: 1906 operations / 2.166.549 bytes
BOINCManager:
reads: 103 operations / 287.712 bytes (at startup)
writes: 0 operations / 0 bytes
SETI (optimized app, running 1h30m):
reads: 101 operations / 377.891 bytes (at startup)
writes: 20 operations / 53.629 bytes

The only point where I can see any true disk intensive behaviour right now is the BOINC writes, but those are small enough to fit in the disk cache, so these can be performed at full interface burst speed.
Other projects, like CPDN, SIMAP, uFluids, BURP or Predictor are bound to gain more from ramdrives or solid state disks because they are using larger files and perform file(de)compression before starting/uploading the result.

However, for other applications, which are disk intensive, there can be huge performance gains by using ramdrives or solid state disks instead of harddrives, so I agree with you on that point.
ID: 222789 · Report as offensive
Profile AlecStaar
Avatar

Send message
Joined: 16 Dec 05
Posts: 260
Credit: 44,472
RAC: 0
United States
Message 223017 - Posted: 29 Dec 2005, 23:23:05 UTC - in response to Message 222789.  
Last modified: 29 Dec 2005, 23:46:41 UTC

I have a machine at home that has been up for a couple of days, running BOINC. I'll check the I/O-info in taskmanager and post the numbers here.


Sure, sounds good (especially if you run a software or hardware based ramdisk, to experiment with this, running the project folders from it).

Make sure you look @ the possibility of doing what I discussed above, & that's ALTERING the project's default number of times per whatever period (minutes?) it hits (writes) to disk... again, this IS configurable & I'd LOVE to see the results of "upping/increasing" the number of writes IF run from a ramdisk & comparing it to what you see now.

Again - best FREE ramdisk I know of (even though I built one myself years ago off the MS-DDK template with a nice GUI front to it for tuning/resizing it)?

ArSoft!

Nothing like "real-world" testing... though, I must admit, I quite often operate on pure theory, but most times? It works out as theorized for more performance (not all though)...

In the mean time, here's some numbers from my laptop (Write to disk at most every: 300 seconds), which has now been running BOINC/SETI for 6h30m.
BOINC:
reads: 340 operations / 1.156.360 bytes (+70% at startup)
writes: 1906 operations / 2.166.549 bytes
BOINCManager:
reads: 103 operations / 287.712 bytes (at startup)
writes: 0 operations / 0 bytes
SETI (optimized app, running 1h30m):
reads: 101 operations / 377.891 bytes (at startup)
writes: 20 operations / 53.629 bytes


Right... but, that's the "stock/oem" disk access setup in this program suite for this project, though, isn't it?

The only point where I can see any true disk intensive behaviour right now is the BOINC writes, but those are small enough to fit in the disk cache, so these can be performed at full interface burst speed.


My point here, if you noted it in my posts above? I literally can tell the OS to cache the Ramdisk (solid-state type) here, but don't bother wasting the RAM on that... the project is already running from RAM, albeit NOT quite as fast as system RAM, because it crosses the PCI bus, & that maxes out @ what?

133mb/sec, in theory @ least??

SO, I don't waste the RAM caching it (Windows Server 2003 allows you to set caching on disks by the by, I am not 100% sure if Windows XP or 2000 let you though, it's been 2 years since I ran/used either of those older MS NT-based OS builds here is why I cannot recall if it does that or not).

Other projects, like CPDN, SIMAP, uFluids, BURP or Predictor are bound to gain more from ramdrives or solid state disks because they are using larger files and perform file(de)compression before starting/uploading the result.


I don't know about that - not a "multi-project BOINC man" here, just SETI@Home!

Still, I think you ought to play with the write config of the SETI project, & try a software based Ramdisk, & compare results - cannot hurt imo @ least to compare them!

However, for other applications, which are disk intensive, there can be huge performance gains by using ramdrives or solid state disks instead of harddrives, so I agree with you on that point.


Yup, like I wrote about placing the pagefile.sys onto one here (since it is solid-state & independent of system RAM, avoiding useless paging RAM-to-RAM here like doing pagefile.sys in RAM on a software-based ramdisk can be - foolhardy imo)

WELL, unless you have over 4gb of system RAM to start with (since 32-bit Windows only can access 4gb @ a time (2gb system/2gb user process split evenly by default) per user process, w/out using the boot.ini /3gb per process memory access possible switch that is).

Other stuff too, like putting your webpage cache, or %TMP% - %TEMP% (both system-wide & current user wide) for apps & the OS, & even putting your %COMSPEC% (usually cmd.exe) there as well...

IMO, speedup your OS + apps via tuning such as this (and many more, I have a page full of this in my signature I have been building since 1996-1997 or so) & ALL ELSE speeds up too!

(The limit of the imagination really, including running specific apps from it, like I do for SETI@Home here & have since day #1 of the last project, on a ramdisk)

:)

* Let me know what you find, if you go & try a ramdrive (upping SETI's default write intervals to disk especially), & compare your results to what you posted above...

APK

P.S.=> It would be interesting imo @ least, to see if 'theory' holds up to actual real-world results on THIS type of topic & an application of ramdrives for this project, potentially for BETTER/FASTER performance (software OR hardware based ramdisks used. I wish you had a solid-state unit like I do here + Windows Server 2003, since I KNOW it allows you to set caching on/off on disks you choose to do so on)!

Anyhow - Sense alone (just based on seek/access times alone being 1000x faster on Ramdrives than on std. mechanical hdd's, 10-15k rpm notwithstanding or EIDE/SATA + UltraScSi too) tell me I gain here!

However, admittedly, I have NEVER tested SETI@Home outright for this either (but, I have many other tasks, such as in the review I did for CENATEK & their solid-state rocketdrive I use here running SETI from it & many other things I note above)... I operated on pure theory only & it would be NICE to see how this project holds up operating on that alone...! apk
http://torry.net/authorsmore.php?id=1781

"The object's hull is made of SOLID neutronium: A single StarShip cannot combat it!" quote Mr. Spock, Star Trek original series, episode title: "The Doomsday Machine"
ID: 223017 · Report as offensive
Jack Gulley

Send message
Joined: 4 Mar 03
Posts: 423
Credit: 526,566
RAC: 0
United States
Message 223216 - Posted: 30 Dec 2005, 7:00:10 UTC - in response to Message 222780.  

Note that I mention "stepping up/increasing" the amount of times that SETI writes to disk in my replies?

That's WHY I do it - to hopefully gain here by using a Ramdisk in hardware.

It's configurable in your SETI@Home 2 settings.

Not sure why you would be doing that?
Increasing the frequency of updates does nothing but add more processor overhead by the setiathome client and reduce the processor time crunching. Plus the overhead handling the writes. On Windows 9x/ME/NT/XP platforms, disk I/O is buffered anyway through the windows disk cache, so BOINC/Setiathome are not really slowed down much by the writes, reads maybe slow it down.

The only reason for stepping up/increasing the frequency of writes is if have a program monitoring the current progress through the disk file and you like to see frequent updates (watch the numbers roll). But accessing that information takes processor time time, again reducing what might be available for crunching. You can only justify it if you have more than one processor and one is not running setiathome and setting idle part of the time.

ID: 223216 · Report as offensive
Profile AlecStaar
Avatar

Send message
Joined: 16 Dec 05
Posts: 260
Credit: 44,472
RAC: 0
United States
Message 223265 - Posted: 30 Dec 2005, 12:36:29 UTC - in response to Message 223216.  
Last modified: 30 Dec 2005, 12:53:41 UTC

Not sure why you would be doing that?


Well, to hopefully get more out of running this process off of a solid-state ramdisk & experiment with that setting, seeing if this aids (or conversely harms) my average credit given & what-not.

Ramdrives/ramdisks are after all, many orders of magnitude faster than std. mechanical harddisks (of any kind (eide/sata or ultrascsi even @ 10-15k rpm rotational rates)) because memory is 1000's of times faster, & that is what ramdrives are - memory buffers seen by the OS as a harddrive.

They especially excel in file seeks & access times.

Increasing the frequency of updates does nothing but add more processor overhead by the setiathome client and reduce the processor time crunching.


That makes sense, but the weird thing is (and perhaps this is because I don't FULLY understand how 'average credit granted' works) that after doing it, in little over a week I went from average credit granted of around the 200's mark, up to over the 400's as of today...

Plus the overhead handling the writes. On Windows 9x/ME/NT/XP platforms, disk I/O is buffered anyway through the windows disk cache


On Windows Server 2003 SP #1 (which I use here), you can SELECTIVELY choose if a disk gets disk-caching, or not!

Since I run this from a solid state disk (which is a ramdrive in hardware), and that is just RAM chips really, seen by the OS (via its driver & settings + the OS & its "HAL" (hardware abstraction layer)) as a diskdrive (like any other HDD is)?

I don't allow it to be cached on this OS version, so the diskcache can function on the slower diskdrives here (2 WD 10,000 rpm + 8mb buffered 'raptors') using diskcache memory for those, rather than the CENATEK 'rocketdrive' solid-state drive I have, which is RAM anyhow!

so BOINC/Setiathome are not really slowed down much by the writes, reads maybe slow it down.


Interesting - MAYBE that's how it is benefitting me the most, since access & seek times on ramdrives of ANY kind (solid-state hardware OR software based ones) FAR exceed that of std. mechanical HDD's of any kind (SATA/EIDE or UltraScSi even, 10-15k notwithstanding & RAID's as well - ramdrives tend to blow away other kinds of disk speed-wise in those areas).

The only reason for stepping up/increasing the frequency of writes is if have a program monitoring the current progress through the disk file and you like to see frequent updates (watch the numbers roll).


Heh, we all do here on this project, don't we in Boincmgr.exe? I (perhaps incorrectly) assume it access any 'state.sah' (from SETI@Home project round #1, since it had the current state-of-affairs in it progress-wise) type of files on disk this project utilizes for boincmgr.exe to report progress to us as its end-users with.

That is unless boinc.exe or even seti 4.18.exe DIRECTLY messages the boingmgr.exe, which IS programmatically (& pretty easily) possible.

But accessing that information takes processor time time, again reducing what might be available for crunching.


Imo, that's offset here @ least (because of using a ramdrive) by the speed of access & seek time on ramdrives.

Memory is, after all, 1000's of times faster/many orders of magnitude quicker than mechanical hard disk bound access.

You can only justify it if you have more than one processor and one is not running setiathome and setting idle part of the time.


I do, technically: A Pentium 4 3.2ghz with "HyperThreading" turned on... & I don't do manually "Processor-Affinity" resetting here (say, via taskmgr.exe or some of the modified reoptimized clients, which Trux's CAN do).

* :)

(I just let the OS' process scheduler determine where the code's threads (since boincmgr.exe & seti 4.18.exe (the real workhorse that processes units here in this project) BOTH have multiple threads of execution)...

APK
http://torry.net/authorsmore.php?id=1781

"The object's hull is made of SOLID neutronium: A single StarShip cannot combat it!" quote Mr. Spock, Star Trek original series, episode title: "The Doomsday Machine"
ID: 223265 · Report as offensive
Profile Mr.Pernod
Volunteer tester
Avatar

Send message
Joined: 8 Feb 04
Posts: 350
Credit: 1,015,988
RAC: 0
Netherlands
Message 223266 - Posted: 30 Dec 2005, 12:50:43 UTC
Last modified: 30 Dec 2005, 13:09:20 UTC

AlecStaar,

I suggest you first read up on RAC and credit in the BOINC Wiki ;)

I pulled some numbers from the other machine last night and they are showing the same trend as the laptop.

The majority of disk I/O is handled by boinc.exe (including the uploads/downloads and state-checking), which is running independantly from the actual science application.

boincmgr.exe loads itself at startup and then communicates with boinc.exe through a networkconnection on localhost, not through disk-I/O.

The science application itself (which does the actual processing) seems to only read from disk when starting, not during the actual processing. Total kilobytes read depend on the size of the result it has to process, usually 350~380 kilobytes. Disk writes during processing seem to be around 75 kilobytes per hour during processing, which will be handled by the hardware cache on the harddisk, so the actual writespeed from the point of view of the science application is at interface burstspeed (66MB/s ~ 150 MB/s depending on the interface).

Now for some numbers.
Lets assume the harddisk has been formatted with NTFS, 4 kilobyte clusters.
Reading 1 cluster takes 25 milliseconds (access + seek + read) (ultra heavy fragmentation).
1 result = 400 kilobyte = 100 clusters = 100 read operations.
100 * 25ms = 2.5 seconds.
processing a result takes 4000 seconds.
total readingtime = 0,0625% of the processingtime.
ID: 223266 · Report as offensive
Profile AlecStaar
Avatar

Send message
Joined: 16 Dec 05
Posts: 260
Credit: 44,472
RAC: 0
United States
Message 223268 - Posted: 30 Dec 2005, 13:01:33 UTC - in response to Message 223266.  
Last modified: 30 Dec 2005, 13:46:19 UTC

AlecStaar,

I suggest you first read up on RAC and credit in the BOINC Wiki ;)


Thanks for the links, I just did & see WHY my 'recent avg. credit' has nearly doubled in 1 week & taken me to 5th place on my team (out of 575 users total, with around 200 actively contributing still to SETI). Just "time-served" it appears!

Anyhow - Still, I KNOW I run this project faster from the solid-state ramdisk I have here because of the FAR superior speed of fileseeks & file accesses ramdrives afford when you use them & that is literally, 1000's of times faster than std. hdd's are speed-wise!

That said? So, depending on how much/how often I use the disk here for this project in particular??

Everytime I perform a read/write op by running this project from a solid-state ramdrive, sense just tells me I am running it in that portion of its operations 1000's of times faster than std. hdd's can perform it on other systems.

(Also, I use the solid-state drive here for MANY other things which I mention below/earlier in this thread (and now here in this reply) for greater overall performance from my OS + programs)

IMO @ least, by speeding the OS up alone all else gains imo. By using this solid-state ramdrive not just for SETI@Home 2 but also what I mentioned above just now in this reply?

Especially tuning the OS by:

----------------------------------------------------------------------------

1.) Placing the pagefile.sys onto the solid-state ramdrive

2.) Setting the %temp%/%tmp% environmental value ops onto the ramdrive

(Thus OS and program temp ops (@ BOTH the current user & OS level) overall are sped-up since the 'environmental values' are more-or-less, an "in memory .ini file" each program uses to an extent & is given a copy of, everytime they are instanced)

3.) Putting my webbrowser page caching onto the ramdrive here

4.) Plus, lastly, placing other things like setting the %COMSPEC% environmental value for the command interpreter cmd.exe (copied to it) set to it as well!

5.) Placing the System EVENT LOGS onto the ramdisk as well via registry hacks for the FILE value for them & where they are located.

----------------------------------------------------------------------------

ALL for faster access - all contributing to faster overall operations here... in addition to running SETI@Home 2, directly from the solid-state drive here by placing its folders on it (as well as installing it directly to it, so any registry values it may have had point to it also).

* Doing those 5-6 items (in addition to running SETI@Home 2 from it by putting its folders onto the ramdrive here) I just make EVERYTHING run faster, as well as this project specifically!

Especially again since disk read/write ops are 1000's of times faster in access & seek times alone, especially via the techniques I note above & use here.

E.G. - I am running SETI@Home 2 via placing BOINC's folders onto it & running all of it from it, & since there is NO 'slowdown' due to mechanical drive read/write head latency?

So, ANY read/write ops gain here that SETI@Home 2 performs speed up.

EVEN using caching on HDD's or not, you still have to deal with eventually moving & positioning those read/write heads & THAT head movement & positioning IS latency & a slowdown (especially in seeks/file accesses), & that does not happen here... That level of 'slowdown' isn't one I have to deal with running this project from a ramdrive.

This all depends on how often I access the disk for this particular project though I imagine, which is WHY I 'stepped up' the write frequency of SETI to experiment with that here...

Just to see if it helps or hinders. Even though this means some CPU time, imo the overall gain I get from read/write & file seek/access times on a ramdrive offset this.

If it proves to hinder my speed? I can/will go the "reverse" direction, & delay the writes more.

Anways - RAM which composes what this disk is made of again, is 1000's of times slower than memory speed is, & memory is what ramdisks are.

ALL that alone above (and common-sense) tell me I have to be getting gains here... how "huge" they are, I don't know, but the gains are there just based on fileseek/access alone.

APK

P.S.=> Remember also - I don't allow the ramdrive (solid-state one by CENATEK called a "rocketdrive") to be cached by the OS diskcache here @ all period. This is simply imo because it accesses & seeks files SO much faster anyhow, it just didn't make sense to me to allow the OS caching it, since it is made of RAM already! I let the diskcache cache the mechanical disks here (2 WD raptors, 10,000 rpm + 8mb buffers onboard them).

(Again: Windows Server 2003 allows you to set if the OS uses its diskcache on a drive, or not - I am NOT sure if Windows 2000/XP have that feature or not, I haven't used them in over 2++ years now, & details start to fade on earlier OS as I am sure you all know & have had happen to you as well)

Also again, since the solid-state ramdisk I use here has NO 'moving parts' & the latency those mechanical parts introduce which IS a 'slowdown', diskcaches & hdd buffers notwithstanding... AND, sooner or later on a std. mechanical drive, cached or not?

You have to move those heads to find the file & position the heads for access to it for the File Open, Read/Write, Close cycle, cached by the diskcache or not & THAT introduces latencies which are far slower than what occurs on ramdisks!

In fact, anything done on ANYTHING in a Win32 based OS, or other modern OS like Linux that performs I/O technically, (even device contexts like your screen believe-it-or-not) really are files & accessed in this manner @ an abstract level (via the API) & go thru that File Open, Read/Write, Close cycle... some trivia there! apk
http://torry.net/authorsmore.php?id=1781

"The object's hull is made of SOLID neutronium: A single StarShip cannot combat it!" quote Mr. Spock, Star Trek original series, episode title: "The Doomsday Machine"
ID: 223268 · Report as offensive

Message boards : Number crunching : running boinc on a ram drive


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.