Rescheduling Hosts - Bad Practice

Message boards : Number crunching : Rescheduling Hosts - Bad Practice
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 6 · 7 · 8 · 9 · 10 · 11 · Next

AuthorMessage
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1469246 - Posted: 26 Jan 2014, 19:00:19 UTC
Last modified: 26 Jan 2014, 19:00:54 UTC

When the awarded credits for AP drop to the point where it is not an advantage to crunch them VS MB work, this discussion shall all be a moot point.

I crunch 24/7, and as long as the servers shall send me something to work on, the kitties shall be content to know that they are participating in what I consider to be one of the most significant attempts by mankind to find out just where we stand in the makings of the universe.

Succeed or not, at least I can say that I did, indeed, make the attempt.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1469246 · Report as offensive
Profile James Sotherden
Avatar

Send message
Joined: 16 May 99
Posts: 10436
Credit: 110,373,059
RAC: 54
United States
Message 1469383 - Posted: 27 Jan 2014, 4:32:35 UTC

I can see both sides of this. I still have no idea what my two I7's are capable of. When we have AP's the RAC climbs. When no AP it goes south fast. Before ver7 came out I never saw swings of what we see now. I see see swings of close tp 5,000 RAC on my machines. And I just have below middle of the range machines.

I can see why its called credit screw. The name fits 100%.

That being said. I agree with Mark I will run what I get untill this project closes the door.
[/quote]

Old James
ID: 1469383 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1469518 - Posted: 27 Jan 2014, 13:30:37 UTC - in response to Message 1469383.  

I can see both sides of this. I still have no idea what my two I7's are capable of. When we have AP's the RAC climbs. When no AP it goes south fast. Before ver7 came out I never saw swings of what we see now. I see see swings of close tp 5,000 RAC on my machines. And I just have below middle of the range machines.

I can see why its called credit screw. The name fits 100%.

That being said. I agree with Mark I will run what I get untill this project closes the door.

Before we saw v7 of MB I would see swings of 10-15K of my total RAC. I still see the same swing. I wouldn't say anything has changed there. The only difference is that the numbers are a bit lower now.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1469518 · Report as offensive
Profile SongBird
Volunteer tester

Send message
Joined: 23 Oct 01
Posts: 104
Credit: 164,826,157
RAC: 297
Bulgaria
Message 1469950 - Posted: 28 Jan 2014, 15:53:39 UTC

Is this machine rescheduling?
ID: 1469950 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14649
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1469958 - Posted: 28 Jan 2014, 16:47:06 UTC - in response to Message 1469950.  

Is this machine rescheduling?

I see no sign of it. Just running optimised applications for CPU and Intel GPU, each on the advertised resource. No harm in that.
ID: 1469958 · Report as offensive
Profile SongBird
Volunteer tester

Send message
Joined: 23 Oct 01
Posts: 104
Credit: 164,826,157
RAC: 297
Bulgaria
Message 1470156 - Posted: 29 Jan 2014, 7:36:22 UTC - in response to Message 1469958.  

It has 109 tasks In progress. I thought the whole point is that there is a maximum of a 100.

Anyway. This is actually my machine. I was doing some stuff I do not fully understand in order to use a beta Intel_GPU app and was just wondering if I've been in fact rescheduling... As you might have picked up I'm not much of a specialist in all this :D

Thanks!
ID: 1470156 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 34744
Credit: 261,360,520
RAC: 489
Australia
Message 1470166 - Posted: 29 Jan 2014, 8:00:58 UTC - in response to Message 1470156.  

It has 109 tasks In progress. I thought the whole point is that there is a maximum of a 100.

Anyway. This is actually my machine. I was doing some stuff I do not fully understand in order to use a beta Intel_GPU app and was just wondering if I've been in fact rescheduling... As you might have picked up I'm not much of a specialist in all this :D

Thanks!

Actually you are allowed 100 for CPU and 100 for GPU.

Cheers.
ID: 1470166 · Report as offensive
Profile ausymark

Send message
Joined: 9 Aug 99
Posts: 95
Credit: 10,175,128
RAC: 0
Australia
Message 1470198 - Posted: 29 Jan 2014, 9:24:24 UTC

To the developers, just a thought on how to handle credit calculation/allocation that should solve everyones issues - assuming its possible.

Here is my thought. When Boinc starts it knows the processing Power of the CPU and GPU's of its computer. There does not need to be any estimates of how long a work unit will take, what we do need is something like an Odometer within the seti apps(s) that calculates and tallies how much processing power is being used at that point on that work unit (calculated every second for example), then tallies up the total processing power used to fully process that work unit. (Hence my Odometer analogy - or even 'total trip fuel used' meter - whatever analogy works best for you)

Also by doing it this way fixes any issues we have of diminishing credits due to more efficient ways of processing the work units in software.

So for example, lets say we have a work unit that takes 3 seconds to complete and the processor can process at 600 MIPS per core, and we are using one core (for the sake of simplicity in the example).

Second SETI CPU Loading MIPS Total Mips
1 50% 300 300
2 100% 600 900
3 60 360 1260

Total Work Unit Processing: 1260 MI (Million Instructions)
Allocate Credits for 1260 MI (Whatever that conversion rate may be)

So thats my idea, it could be used for all BOINC projects. Thoughts?

Cheers

Mark
ID: 1470198 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14649
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1470202 - Posted: 29 Jan 2014, 9:52:18 UTC - in response to Message 1470198.  

That is, in fact, exactly how the previous credit system used to work. If you look at one of my tasks at random, like task 3358432916, you'll see a line in the middle of the output:

Flopcounter: 16102518492340.953000

That's exactly your odometer (though I'm not sure what the significance of one-thousandth of a floating point operation is).

But if you look at your own recent AP results, you won't see that line (I think it might have been there once, but was taken out because there was already more than enough clutter).

And thereby hangs the problem. The scheme only works if every programmer, at every project, adds in the odometer mechanism. I'm told it isn't as easy as it sounds. You don't want to actually 'count' flops, because that would take almost as much computing power as running the job - which would be horribly wasteful. Instead, you have to go through all the significant chunks of computer code, and assign each one a realistic 'count' you can add up to make the overall total.

The present scheme was designed to save that extra programming work. It would be a nice idea if it worked, but unfortunately it doesn't, quite - yet.
ID: 1470202 · Report as offensive
Profile ausymark

Send message
Joined: 9 Aug 99
Posts: 95
Credit: 10,175,128
RAC: 0
Australia
Message 1470206 - Posted: 29 Jan 2014, 10:12:57 UTC - in response to Message 1470202.  
Last modified: 29 Jan 2014, 10:14:15 UTC

Hi Richard

Well thats the thing, we wouldnt need to calculate the flops at any instant as they are calculated when boinc first starts and runs the benchmarks on the PC. Then all one has to do is work out the % load that the seti program is having on the core(s), which Im guessing can be done fairly easily, and if not use the total core load, less precise but possibly still more accurate than what is currently being used. That load on the CPU to calculate the 'odometer'should be very minimal in the scheme of things these days. (Linux and Windows system monitors do that with negligible load.)

And to reduce manipulation of the system have BOINC run the benchmark program randomly again every 24 to 40 hours.

Perhaps im over simplifying it and dont know the complexities involved, but i guess the question is - would it be a more accurate system than what we currently have assuming it doesnt adversely affect affect crunching ability by more than half of one percent?

Cheers

Mark
ID: 1470206 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 34744
Credit: 261,360,520
RAC: 489
Australia
Message 1470207 - Posted: 29 Jan 2014, 10:15:57 UTC
Last modified: 29 Jan 2014, 10:16:13 UTC

And here goes our friend again who dumps MB work as soon as AP's are bring split.

7114519
5745362

So it's left to some of us to clean up his refuse. :-(

No Cheers.
ID: 1470207 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14649
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1470210 - Posted: 29 Jan 2014, 10:41:18 UTC - in response to Message 1470206.  

Now you're asking for something slightly different.

Have a look at http://boinc.berkeley.edu/trac/wiki/CreditNew

Your first suggestion - the odometer analogy - is what is described as "The second credit system" - equal pay for equal work done.

Your second suggestion - relying on the benchmark - goes back to "The first credit system". The problem we found with that was that it makes no allowance for the efficiency of the programming. If you install an optimised application, the benchmark remains the same, and the processing time goes down, so you are awarded less credit per task. That breaks the rule of "equal pay for equal work". The benchmark is a speedometer, not an odometer - and it's not a very good one.
ID: 1470210 · Report as offensive
Profile ausymark

Send message
Joined: 9 Aug 99
Posts: 95
Credit: 10,175,128
RAC: 0
Australia
Message 1470236 - Posted: 29 Jan 2014, 12:16:51 UTC - in response to Message 1470210.  

Hi Richard

I glanced over the link and see that its nowhere near as simple as I had assumed... I will have a good look at it tomorrow to see if I get any flashes of brilliance, or just zombie eyes lol

Cheers :)

Mark
ID: 1470236 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 1471227 - Posted: 31 Jan 2014, 18:06:54 UTC

And another host for Wiggo's list, one with 1250+ tasks and most of them returning as error: https://setiathome.berkeley.edu/results.php?hostid=7090798
ID: 1471227 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1471232 - Posted: 31 Jan 2014, 18:13:48 UTC - in response to Message 1471227.  
Last modified: 31 Jan 2014, 18:19:25 UTC

And another host for Wiggo's list, one with 1250+ tasks and most of them returning as error: https://setiathome.berkeley.edu/results.php?hostid=7090798

That host has In progress (33) & does not have any aborts. Just a lot of errors. While 20% of their tasks are errors it is better than I have seen on other machines.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1471232 · Report as offensive
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 1478340 - Posted: 17 Feb 2014, 15:51:27 UTC
Last modified: 17 Feb 2014, 15:58:47 UTC

We all are out of AP WU to DL then i see this hosts:

http://setiathome.berkeley.edu/results.php?hostid=6637324 with 846 in progress AP WU

http://setiathome.berkeley.edu/results.php?hostid=6016862 with 1180.

http://setiathome.berkeley.edu/results.php?hostid=6794209 with 696.

Hard to belive that could be right, even with all hosts producing few errors.

All are anonymous.
ID: 1478340 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 1478390 - Posted: 17 Feb 2014, 18:37:33 UTC
Last modified: 17 Feb 2014, 18:37:51 UTC

Then one compute number of tasks that gone to plain stock CPU hosts that just waste of electricity and (in case of Linux) just trashing tasks... one would thank these 3 for saving at least some tasks ;)
SETI apps news
We're not gonna fight them. We're gonna transcend them.
ID: 1478390 · Report as offensive
Batter Up
Avatar

Send message
Joined: 5 May 99
Posts: 1946
Credit: 24,860,347
RAC: 0
United States
Message 1478396 - Posted: 17 Feb 2014, 18:49:29 UTC

It is obvious that the powers that be condone this type of behavior so it must still get the job done for them. I found the script that reassigns CPU to GPU but cobblestones are worth less than crypto-currency so why bother.
ID: 1478396 · Report as offensive
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1478407 - Posted: 17 Feb 2014, 19:14:45 UTC - in response to Message 1478396.  
Last modified: 17 Feb 2014, 19:16:30 UTC

The limits were imposed due to some of the faster hosts having a "State: All" number of around 10,000 or more. Those sort of numbers really affect the database. The previously mentioned hosts are just above 1,000, well below some of the other hosts that are 'following the limits'. Until someone approaches around the 5k mark, there really isn't an issue. Unless there is some actual need to hoard that many tasks, it's just a waste of time & energy on their part.
ID: 1478407 · Report as offensive
Batter Up
Avatar

Send message
Joined: 5 May 99
Posts: 1946
Credit: 24,860,347
RAC: 0
United States
Message 1478416 - Posted: 17 Feb 2014, 19:42:47 UTC - in response to Message 1478407.  

Unless there is some actual need to hoard that many tasks, it's just a waste of time & energy on their part.

If cobblestones are the goal reassigning and hoarding AP WU so one never runs out is as good as running another GPU.
ID: 1478416 · Report as offensive
Previous · 1 . . . 6 · 7 · 8 · 9 · 10 · 11 · Next

Message boards : Number crunching : Rescheduling Hosts - Bad Practice


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.