Friday, July 9, server start

Message boards : Number crunching : Friday, July 9, server start
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3

AuthorMessage
Profile Geek@Play
Volunteer tester
Avatar

Send message
Joined: 31 Jul 01
Posts: 2467
Credit: 86,146,931
RAC: 0
United States
Message 1014096 - Posted: 10 Jul 2010, 2:41:56 UTC
Last modified: 10 Jul 2010, 2:42:19 UTC

Same thing here........finally got some CUDA work assigned to 2 computers but unable to download them. Cricket graph up over 90.
Boinc....Boinc....Boinc....Boinc....
ID: 1014096 · Report as offensive
Profile Uli
Volunteer tester
Avatar

Send message
Joined: 6 Feb 00
Posts: 10923
Credit: 5,996,015
RAC: 1
Germany
Message 1014109 - Posted: 10 Jul 2010, 3:23:31 UTC - in response to Message 1014067.  

I observe that the limit has been lifted (or increased) shortly after 6pm PDT.


Looks like they upped it to 6 per logical CPU core now.


Ok, thanks, that explains why I now have 12 WU, well 11 one hasn't made it quite yet.
Pluto will always be a planet to me.

Seti Ambassador
Not to late to order an Anni Shirt
ID: 1014109 · Report as offensive
Profile BANZAI56
Volunteer tester

Send message
Joined: 17 May 00
Posts: 139
Credit: 47,299,948
RAC: 2
United States
Message 1014141 - Posted: 10 Jul 2010, 4:37:02 UTC - in response to Message 1013656.  

I had to hide both the jobs limit thread and the new outage schedule thread. Queries for those threads were clobbering the boinc database.



Thanks for all you are doing Jeff!

Very very interesting info about the threads you mention.

Seemed like a no brainer that huge arse threads were a bad idea.
(esp. for those of us stuck with dial-up.)

This as some even went out of their way to create them...

Even had that (bad idea) conversation here with someone who was NEVER ever wrong about anything..........ever. lol

Too bad he was burned out on this place, otherwise he could tell us how we're still wrong... ;)
ID: 1014141 · Report as offensive
Profile Odan

Send message
Joined: 8 May 03
Posts: 91
Credit: 15,331,177
RAC: 0
United Kingdom
Message 1014280 - Posted: 10 Jul 2010, 12:45:02 UTC - in response to Message 1013936.  

I am starting the servers up.

I had to hide both the jobs limit thread and the new outage schedule thread. Queries for those threads were clobbering the boinc database. I will put something more about this up later...

Slow down the creation/sending of AP units. Those are really bandwidth hogs. At least until the worst rush is over.

I'm not sure I agree with the logic, an AP task has about 23 times the bits of an S@H Enhanced task, but the estimated runtime may have a higher ratio. For many hosts, one AP WU fulfills the cache setting so they would stop requesting work. <snip>

                                                               Joe


The runtime is longer but not 23 times longer, more like 12 times. That means they take somewhere in the region of twice the download bandwidth per unit crunch time. I see that as supporting the "bandwidth hog" description but I wouldn't go quite so far as to call them that :)

When AP were in much readier supply last year, in fact they were produced to fully satisfy demand & we had a reservoir of units ready to send, we did run for quite a while where the download bandwidth was maxed out. IIRC we were running with approx 400,000 AP in progress as the stable demand. Most of this year we have averaged about 100,000 in progress which is well below the demand if distribution were less restricted.
ID: 1014280 · Report as offensive
Profile hiamps
Volunteer tester
Avatar

Send message
Joined: 23 May 99
Posts: 4292
Credit: 72,971,319
RAC: 0
United States
Message 1014340 - Posted: 10 Jul 2010, 16:20:09 UTC - in response to Message 1013656.  

I am starting the servers up.

I had to hide both the jobs limit thread and the new outage schedule thread. Queries for those threads were clobbering the boinc database. I will put something more about this up later...

What I don't understand is...If a couple of threads needed to be hidden because they were pounding the servers...What is 100,000 computers constantly asking for work but not getting any doing? Guess I don't understand servers at all.
Official Abuser of Boinc Buttons...
And no good credit hound!
ID: 1014340 · Report as offensive
Profile Hellsheep
Volunteer tester

Send message
Joined: 12 Sep 08
Posts: 428
Credit: 784,780
RAC: 0
Australia
Message 1014346 - Posted: 10 Jul 2010, 16:27:21 UTC - in response to Message 1014340.  

I am starting the servers up.

I had to hide both the jobs limit thread and the new outage schedule thread. Queries for those threads were clobbering the boinc database. I will put something more about this up later...

What I don't understand is...If a couple of threads needed to be hidden because they were pounding the servers...What is 100,000 computers constantly asking for work but not getting any doing? Guess I don't understand servers at all.


It's not 2 threads that caused the problem, it's the number of times those threads were being queried from the database.

Each time someone visits a thread on the forums, their browser sends a message to the server requesting the page, the server says "I need to query the database to get the information for this page" and it talks to the database asking for it, the database retrieves it and sends the data back to the server in a message and then the server outputs it to the page and your browser renders that page (or something like that anyway).

So imagine 100,000 forum users all reading the same thread and refreshing over and over. ;)
- Jarryd
ID: 1014346 · Report as offensive
Profile hiamps
Volunteer tester
Avatar

Send message
Joined: 23 May 99
Posts: 4292
Credit: 72,971,319
RAC: 0
United States
Message 1014351 - Posted: 10 Jul 2010, 16:29:55 UTC - in response to Message 1014346.  

I am starting the servers up.

I had to hide both the jobs limit thread and the new outage schedule thread. Queries for those threads were clobbering the boinc database. I will put something more about this up later...

What I don't understand is...If a couple of threads needed to be hidden because they were pounding the servers...What is 100,000 computers constantly asking for work but not getting any doing? Guess I don't understand servers at all.


It's not 2 threads that caused the problem, it's the number of times those threads were being queried from the database.

Each time someone visits a thread on the forums, their browser sends a message to the server requesting the page, the server says "I need to query the database to get the information for this page" and it talks to the database asking for it, the database retrieves it and sends the data back to the server in a message and then the server outputs it to the page and your browser renders that page (or something like that anyway).

So imagine 100,000 forum users all reading the same thread and refreshing over and over. ;)

Is that because of the data in our nametags?
Official Abuser of Boinc Buttons...
And no good credit hound!
ID: 1014351 · Report as offensive
Profile zoom3+1=4
Volunteer tester
Avatar

Send message
Joined: 30 Nov 03
Posts: 65709
Credit: 55,293,173
RAC: 49
United States
Message 1014361 - Posted: 10 Jul 2010, 16:39:11 UTC - in response to Message 1014340.  

I am starting the servers up.

I had to hide both the jobs limit thread and the new outage schedule thread. Queries for those threads were clobbering the boinc database. I will put something more about this up later...

What I don't understand is...If a couple of threads needed to be hidden because they were pounding the servers...What is 100,000 computers constantly asking for work but not getting any doing? Guess I don't understand servers at all.

I think the forums server is able to handle lots of threads, While the server where the News thread is at wasn't intended for that much traffic.
The T1 Trust, PRR T1 Class 4-4-4-4 #5550, 1 of America's First HST's
ID: 1014361 · Report as offensive
Profile Hellsheep
Volunteer tester

Send message
Joined: 12 Sep 08
Posts: 428
Credit: 784,780
RAC: 0
Australia
Message 1014362 - Posted: 10 Jul 2010, 16:39:40 UTC - in response to Message 1014351.  

I am starting the servers up.

I had to hide both the jobs limit thread and the new outage schedule thread. Queries for those threads were clobbering the boinc database. I will put something more about this up later...

What I don't understand is...If a couple of threads needed to be hidden because they were pounding the servers...What is 100,000 computers constantly asking for work but not getting any doing? Guess I don't understand servers at all.


It's not 2 threads that caused the problem, it's the number of times those threads were being queried from the database.

Each time someone visits a thread on the forums, their browser sends a message to the server requesting the page, the server says "I need to query the database to get the information for this page" and it talks to the database asking for it, the database retrieves it and sends the data back to the server in a message and then the server outputs it to the page and your browser renders that page (or something like that anyway).

So imagine 100,000 forum users all reading the same thread and refreshing over and over. ;)

Is that because of the data in our nametags?


Well, that all contributes to it, sure.

The thing is, each thread requires a query to the database containing the information relevant to where the thread is stored, how it's stored and how many posts and where those posts can be located in the database (post numbers etc).

Then, each post in each thread requires another query to retrieve the actual post itself.

So if there are 500 posts in a thread on a single page and one user loads that thread, essentially there are 501 queries to the database just to retrieve the thread and it's posts. Not including the user data and whatever else it needs to get i guess (unless that's tied in with the post data in the database.)

So small math, if 400 users were to read a thread with 500 posts. Essentially that would require 200,000 queries just to get it for all of them.

I may actually be wrong with this, but as far as i know that's how forums work.

It's also possible that rather than queries for each post, each thread inside the database also contains the post data, meaning instead of 501 queries it'd be 1 query per page load, still when 400 users load that same thread, 400 queries.

So even if i'm wrong about whichever way it does it, if 50,000 people view the same topic, there was at least 50,000 queries to that same area of the database.

- Jarryd
ID: 1014362 · Report as offensive
Profile hiamps
Volunteer tester
Avatar

Send message
Joined: 23 May 99
Posts: 4292
Credit: 72,971,319
RAC: 0
United States
Message 1014364 - Posted: 10 Jul 2010, 16:50:48 UTC - in response to Message 1014362.  

So putting a limit on the thread count would help the servers do more work? Say a thread has to be restarted at 200 posts?
Official Abuser of Boinc Buttons...
And no good credit hound!
ID: 1014364 · Report as offensive
Profile hiamps
Volunteer tester
Avatar

Send message
Joined: 23 May 99
Posts: 4292
Credit: 72,971,319
RAC: 0
United States
Message 1014370 - Posted: 10 Jul 2010, 17:31:21 UTC - in response to Message 1014369.  

I think I'm beginning to see the light why certain Seti staff wanted to ditch the forums a few years ago .......

I bet some of the threads in the Cafe just kill the servers...
Official Abuser of Boinc Buttons...
And no good credit hound!
ID: 1014370 · Report as offensive
Profile soft^spirit
Avatar

Send message
Joined: 18 May 99
Posts: 6497
Credit: 34,134,168
RAC: 0
United States
Message 1014397 - Posted: 10 Jul 2010, 18:33:24 UTC

The CPU and GPU limits have gone very well. A lot of people have had a lot of trouble with the total limit. Raising it is helping, elimination of it would be prudent.

Just an opinion.
Janice
ID: 1014397 · Report as offensive
Profile Hellsheep
Volunteer tester

Send message
Joined: 12 Sep 08
Posts: 428
Credit: 784,780
RAC: 0
Australia
Message 1014469 - Posted: 10 Jul 2010, 21:23:43 UTC - in response to Message 1014364.  

So putting a limit on the thread count would help the servers do more work? Say a thread has to be restarted at 200 posts?


There is already a limit for the NC forum i believe. Also the staff have actually made it so only the first post and the last 75 posts show and you have to press a button to display all posts if the thread is larger than 75 posts.

It wouldn't help the servers do more work, but help them be more stable to prevent database issues.

- Jarryd
ID: 1014469 · Report as offensive
Profile arkayn
Volunteer tester
Avatar

Send message
Joined: 14 May 99
Posts: 4438
Credit: 55,006,323
RAC: 0
United States
Message 1014479 - Posted: 10 Jul 2010, 22:19:34 UTC

It is not an official limit, just one that we try to keep at a reasonable limit because some people are still on dial-up.

ID: 1014479 · Report as offensive
Profile zoom3+1=4
Volunteer tester
Avatar

Send message
Joined: 30 Nov 03
Posts: 65709
Credit: 55,293,173
RAC: 49
United States
Message 1014496 - Posted: 10 Jul 2010, 23:18:58 UTC - in response to Message 1014479.  

It is not an official limit, just one that we try to keep at a reasonable limit because some people are still on dial-up.

Although some ISPs are I've read, dropping dial-up, If It becomes a trend, Some are going to have to get some sort of alternate, Like either DSL, Cable or Satellite.
The T1 Trust, PRR T1 Class 4-4-4-4 #5550, 1 of America's First HST's
ID: 1014496 · Report as offensive
Josef W. Segur
Volunteer developer
Volunteer tester

Send message
Joined: 30 Oct 99
Posts: 4504
Credit: 1,414,761
RAC: 0
United States
Message 1014566 - Posted: 11 Jul 2010, 5:00:10 UTC - in response to Message 1014280.  


Slow down the creation/sending of AP units. Those are really bandwidth hogs. At least until the worst rush is over.

I'm not sure I agree with the logic, an AP task has about 23 times the bits of an S@H Enhanced task, but the estimated runtime may have a higher ratio. For many hosts, one AP WU fulfills the cache setting so they would stop requesting work. <snip>

                                                               Joe

The runtime is longer but not 23 times longer, more like 12 times. That means they take somewhere in the region of twice the download bandwidth per unit crunch time. I see that as supporting the "bandwidth hog" description but I wouldn't go quite so far as to call them that :)

When AP were in much readier supply last year, in fact they were produced to fully satisfy demand & we had a reservoir of units ready to send, we did run for quite a while where the download bandwidth was maxed out. IIRC we were running with approx 400,000 AP in progress as the stable demand. Most of this year we have averaged about 100,000 in progress which is well below the demand if distribution were less restricted.

Your facts are right, since the chirp resolution change S@H Enhanced raw task estimates produced by the splitters are about 1/12 the raw AP estimates. It is of course the estimates which affect work fetch much more than actual runtimes, though.

There's another factor now in the server-side estimate adjustments. My guess is there are 40000 or more hosts running stock applications which are eligible for AP work. Most if not all of those hosts will have more than 10 completed Enhanced tasks so the estimate adjustments will be in effect and the host DCF will have been forced up by about a factor of 5 (0.2 -> 1.0). Very few will have 10 completed AP tasks (and the BOINC code doesn't seem able to count them anyhow), so no adjustment will be in effect for those. The 12 ratio becomes 60. Completion of an AP task in 1/5 the estimated time will temporarily reduce DCF 8% or so, giving a ~55 ratio, but Enhanced tasks will again force it up. If David again turns on server-side adjustment for anonymous platform hosts, the same effects will apply for another 30000 or so hosts.

I don't think D.A.'s adjustments are going to be all that stable for individual cases, but may have about that effect across the set of 40000 hosts. When he gets the counting of AP tasks working, and enough time has passed that most hosts are getting adjusted estimates for all kinds of work, then I agree that reduced AP delivery during the initial recovery after the weekly outage would definitely be a good idea. Perhaps then the best approach would be simply not pre-splitting any AP work during the outage and not enabling AP splitting until late Friday afternoon Berkeley time.
                                                                Joe
ID: 1014566 · Report as offensive
Profile zoom3+1=4
Volunteer tester
Avatar

Send message
Joined: 30 Nov 03
Posts: 65709
Credit: 55,293,173
RAC: 49
United States
Message 1014572 - Posted: 11 Jul 2010, 5:49:36 UTC - in response to Message 1014566.  
Last modified: 11 Jul 2010, 5:50:12 UTC

Well I'd that hope D.A. could soon get Anonymous DCF running like It should be, When He's able to of course.
The T1 Trust, PRR T1 Class 4-4-4-4 #5550, 1 of America's First HST's
ID: 1014572 · Report as offensive
Tony Li

Send message
Joined: 21 May 01
Posts: 6
Credit: 1,337,747
RAC: 0
United States
Message 1019913 - Posted: 27 Jul 2010, 1:19:14 UTC - in response to Message 1013908.  

Just FYI: Due to personal circumstances, I'm not able to participate on weekends. As a result, I'm effectively now only getting work two days a week.

So far, this is going ok, but if I run out of work, I'm going to give my cycles to another project...


No problem! By contributing your cycles, you are helping science in one way or another. Good luck with what ever you choose! :)


Well, given the lack of WUs, my CPU is now off contributing to other projects. I'm surprised that there isn't more concern about retaining the general case of retaining compute cycles.


ID: 1019913 · Report as offensive
Previous · 1 · 2 · 3

Message boards : Number crunching : Friday, July 9, server start


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.