More tasks in cache

Message boards : Number crunching : More tasks in cache
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Michel Makhlouta
Volunteer tester
Avatar

Send message
Joined: 21 Dec 03
Posts: 169
Credit: 41,799,743
RAC: 0
Lebanon
Message 1933763 - Posted: 5 May 2018, 8:14:08 UTC

Hello fellow crunchers,
I've got a rig with 3x1070 on linux running 24/7, and lately discovered that it is consuming a good chunk of my humble monthly quota. I do get unlimited quota between 1AM and 5AM and I've set the scheduler to download/upload during that time only. And the cache to 10 days.

Well whatever I am getting during those hours is usually processed by 10AM. Is there a way around this?
ID: 1933763 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13736
Credit: 208,696,464
RAC: 304
Australia
Message 1933767 - Posted: 5 May 2018, 8:26:50 UTC - in response to Message 1933763.  
Last modified: 5 May 2018, 8:40:23 UTC

Is there a way around this?

Not really.
There is a limit of 100 WUs per GPU, and 100WUs for all CPUs in a given system and any given time. And the simple fact of the matter is your GPUs would appear to be capable of processing more WUs than your data allowance will allow- so even if you could download enough to keep them busy during your non download hours, you will still run out of data allowance.

The next best option would be in the Computing preferences, instead of limiting the hours you can download (unless you get downloads that don't count against your allowance during certain hours?) would be under Usage limits and "Limit usage to --- MB every --- days"
Figure out how much of your monthly allowance isn't Seti related, minus that from the allowance, and put what's left in the ---MB section and 30.2 in the --- days section. That way you can download as much work as possible for Seti, leaving some room for other internet activity, and not run in to your data allowance limit.

You will still run out of work towards the end of the month, but at least you won't come up against your data allowance.

Edit-
Or it may be possible if you put 1 in the --- days section, and divide your estimated Seti usage by 30.2 and put that value in the ---MB section. That way you will run out of work each day, but will have work to do every day of the month. The other way will result in work all day, each day, till you hit your set limit for Seti work.
Either way, there will be periods without work (either for a part of each day or for several days towards the end of the 30 day period).
Either that, or remove a video card (or 2) to reduce your data usage, or upgrade to a better plan (probably painfully expensive?)
Grant
Darwin NT
ID: 1933767 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 1933778 - Posted: 5 May 2018, 9:22:15 UTC - in response to Message 1933763.  

Is there a way around this?
The more political correct answer is: at this project only, no. See Grant's answer about limitations for that.
But nothing stands in your way to not add a second or third project. Add the daily/weekly/monthly allowance here and you're set.
ID: 1933778 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13736
Credit: 208,696,464
RAC: 304
Australia
Message 1933779 - Posted: 5 May 2018, 9:24:31 UTC - in response to Message 1933778.  

But nothing stands in your way to not add a second or third project. Add the daily/weekly/monthly allowance here and you're set.

But even then, they will still run in to their ISP data allowance limit, unless the other project has small WUs, that take ages to process.
Grant
Darwin NT
ID: 1933779 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 1933783 - Posted: 5 May 2018, 9:47:55 UTC - in response to Message 1933779.  

Best do Einstein@Home then. Their tasks run quite a while, and you download one or more large data files once, then slice tasks off of those for the duration of several days/weeks. During that time the only data being moved is upload files and reports.
ID: 1933783 · Report as offensive
Profile Cactus Bob
Avatar

Send message
Joined: 19 May 99
Posts: 209
Credit: 10,924,287
RAC: 29
Canada
Message 1933840 - Posted: 5 May 2018, 18:03:47 UTC

The part most are missing I think is this.

. I do get unlimited quota between 1AM and 5AM and I've set the scheduler to download/upload during that time only.


So if he had a way to Download more WU's during this time it would not cut into his monthly quota. Ageless idea seems to be the most responsible answer and would work if he wants to run other projects. I have seen suggestions in other threads of have more than I instance of BOINC although that may not work as he wants, I assume it would mean he would have to run 2 accounts (or would making it look like 2 computers be viable). The other issue how to do schedule DLs/ULs with 2 instances while sleeping.

Just a thought, I have never has this issue but it is intriguing and there must be a way around this archaic limit of 100 units that worked well 10 years ago but not today.

Bob
Sometimes I wonder, what happened to all the people I gave directions to?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
SETI@home classic workunits 4,321
SETI@home classic CPU time 22,169 hours
ID: 1933840 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 22200
Credit: 416,307,556
RAC: 380
United Kingdom
Message 1933842 - Posted: 5 May 2018, 18:15:42 UTC

The limit has only been in a few years, not ten, and is there to "protect the servers". It also helps (but doesn't stop) those who grab many thousands of tasks which all time-out because.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1933842 · Report as offensive
Profile Cactus Bob
Avatar

Send message
Joined: 19 May 99
Posts: 209
Credit: 10,924,287
RAC: 29
Canada
Message 1933845 - Posted: 5 May 2018, 18:32:33 UTC
Last modified: 5 May 2018, 18:49:44 UTC

Thanx for the clarification Bob. I thought it was set a long time ago. Protecting the server with a one size fits all method isn't probably the best option as viewed in hindsight.

Bob

Edit : Here is a thread about this issue from June 2010
Message boards : Number crunching : Message from server: (reached daily quota of 100 tasks)
https://setiathome.berkeley.edu/forum_thread.php?id=60285#1002284 from SciManStev
Sometimes I wonder, what happened to all the people I gave directions to?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
SETI@home classic workunits 4,321
SETI@home classic CPU time 22,169 hours
ID: 1933845 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 22200
Credit: 416,307,556
RAC: 380
United Kingdom
Message 1933847 - Posted: 5 May 2018, 19:19:21 UTC

Daily quota is not the same as the cache size limit. If they were the same then all the top 20 (or maybe top 1000?) crunchers would struggle to get work.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1933847 · Report as offensive
mmonnin
Volunteer tester

Send message
Joined: 8 Jun 17
Posts: 58
Credit: 10,176,849
RAC: 0
United States
Message 1933854 - Posted: 5 May 2018, 20:19:55 UTC
Last modified: 5 May 2018, 20:20:03 UTC

You can try multiple clients and some scripting to start the project on client2 when client1 has no more work. Or maybe just suspend client1 after 4 hours, client2 4 hours later, etc to get you through the day then allow networking after 1am.
https://boinc.berkeley.edu/wiki/Boinccmd_tool
ID: 1933854 · Report as offensive
Profile Cactus Bob
Avatar

Send message
Joined: 19 May 99
Posts: 209
Credit: 10,924,287
RAC: 29
Canada
Message 1933910 - Posted: 6 May 2018, 2:40:48 UTC - in response to Message 1933847.  

Daily quota is not the same as the cache size limit. If they were the same then all the top 20 (or maybe top 1000?) crunchers would struggle to get work.


lol, gonna try and remove my foot without hurting my mouth too much.

Bob
Sometimes I wonder, what happened to all the people I gave directions to?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
SETI@home classic workunits 4,321
SETI@home classic CPU time 22,169 hours
ID: 1933910 · Report as offensive
Profile Michel Makhlouta
Volunteer tester
Avatar

Send message
Joined: 21 Dec 03
Posts: 169
Credit: 41,799,743
RAC: 0
Lebanon
Message 1933947 - Posted: 6 May 2018, 8:07:36 UTC

anything downloaded or uploaded between 1AM and 5AM does not count towards my quota, those are "free" hours. that is why I was wondering if someone somewhere would do something about this. at least give us a 36 hours worth of tasks based on our few days of crunching.

the only feasible option it seems is to have a separate ISP for this machine with terrible but open quota package... not sure this is a cost I can justify on top of electricity spent running this rig.
ID: 1933947 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13736
Credit: 208,696,464
RAC: 304
Australia
Message 1933951 - Posted: 6 May 2018, 8:30:20 UTC - in response to Message 1933947.  

anything downloaded or uploaded between 1AM and 5AM does not count towards my quota, those are "free" hours. that is why I was wondering if someone somewhere would do something about this.

I would suggest a combination of what I posted about limiting data for Seti to so many MB per day or month, and combine that with the "Free from Quota" time period.
Check your ISP account for just how much your are downloading from 1:00 to 5:00. Figure out as in my previous posts how much per month (or day) that allows you to download. Add that value to the amount your are downloading between 1:00 and 5:00, and use that value to set how much you can download each day or month, and then set the period you can download for however many hours per day that that you can download where it affects your Quota either before, after or broken up & spread either side of the Quota free period.

Even so, the simple fact is that your turn around times are so short that even then I expect you will run out of work.
Grant
Darwin NT
ID: 1933951 · Report as offensive

Message boards : Number crunching : More tasks in cache


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.