Furniture Shortage (Jan 03 2008)

Message boards : Technical News : Furniture Shortage (Jan 03 2008)
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 697086 - Posted: 3 Jan 2008, 20:54:14 UTC

Spreading the workunit creation over several files at once seems to be helping create a healthier mix of fast/slow workunits. However, adding a second download server seems to have confirmed a suspicion of mine (key word: "seems"): that somewhere down the pike we're being capped at 60 Mbits/sec. For a while there we had two download servers and a workunit storage server with plenty I/O capacity to spare, but still we were hitting a hard 60 Mbit ceiling outbound. Inquiries are being drafted/sent to the appropriate parties. It still could be a local problem, but we're not sure what else to try (given our current hardware).

We are in the middle of building another helpful index on the science database. Looks like Bob's magic informix incantations are working - we can keep the project running simultaneously (though the assimilators might back up a bit). It is always happier around here when work is flowing. To be safe we increased the ready-to-send queue size to one million - we have the disk space now to keep more workunits around. The only downside is that this inflates the result table in the database by approximately 5-10%, which may exercise the RAM on the BOINC database server that much more.

There is another problem Dave and I were poking at today: excessive "out of range" failures on our public web sites. Here's the deal: BOINC clients have a nice GUI which shows you icons, pictures, etc. from different projects as you select which to run on your computer. Where does it get these files? From the project's web servers. This is all well and good, but there are several (hundreds? thousands?) older clients out there making such requests but are being met with 416 "range not satisfiable" errors. Why? Because they have already downloaded the image file, but are making requests for more bytes beyond the file boundaries as if there was more to download. Obviously a bug somewhere, or a change in the way apache handles such things, but there's not much we can do about it. Even though this activity is creating bursts of heavy load on our web servers, this is a fire we're going to let burn for now.

The official press release about multi-beam is finally out. This should help on many levels (though I'll be busier making sure the servers can handle any significant load increase). I guess I'll also be shaving every morning in case there is interest from the national television news media.

I guess this is "technical" news: Our desks/chairs/furniture are mostly ancient hand-me-downs, some pieces older than I. We did get some new chair donations recently, but one of them broke - it came loose from its base, causing unsuspecting sitters to suddenly fall forward if their balance wasn't particularly keen. It's been lurking in our lab way too long, coaxing uninformed standers with tired legs to rest upon its comfortable and seemingly stable cushion base. I came to the lab this morning and that evil chair was by my desk with a note taped to it: "Matt - can you please toss this chair?" I guess enough was enough. I dragged it to the dumpster and sent it back to the dark void from whence it came.

- Matt

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 697086 · Report as offensive
Profile Robert M Swift

Send message
Joined: 6 Jul 05
Posts: 5
Credit: 3,676,248
RAC: 5
Spain
Message 697098 - Posted: 3 Jan 2008, 22:07:51 UTC - in response to Message 697086.  

Thanks for the post Matt and for the rest of the news throughout the year.
I wish all at SETI a very Happy and Peacefull New Year.
Kindest Regards to all

Bob Swift

Bob Swift
ID: 697098 · Report as offensive
Ingleside
Volunteer developer

Send message
Joined: 4 Feb 03
Posts: 1546
Credit: 15,832,022
RAC: 13
Norway
Message 697106 - Posted: 3 Jan 2008, 22:23:15 UTC - in response to Message 697086.  
Last modified: 3 Jan 2008, 22:23:58 UTC

There is another problem Dave and I were poking at today: excessive "out of range" failures on our public web sites. Here's the deal: BOINC clients have a nice GUI which shows you icons, pictures, etc. from different projects as you select which to run on your computer. Where does it get these files? From the project's web servers. This is all well and good, but there are several (hundreds? thousands?) older clients out there making such requests but are being met with 416 "range not satisfiable" errors. Why? Because they have already downloaded the image file, but are making requests for more bytes beyond the file boundaries as if there was more to download. Obviously a bug somewhere, or a change in the way apache handles such things, but there's not much we can do about it. Even though this activity is creating bursts of heavy load on our web servers, this is a fire we're going to let burn for now.


The gui-image-files doesn't include <nbytes> as they should, client therefore has no idea how large these apparently zero-byte-files really are.
"I make so many mistakes. But then just think of all the mistakes I don't make, although I might."
ID: 697106 · Report as offensive
Profile Dr. C.E.T.I.
Avatar

Send message
Joined: 29 Feb 00
Posts: 16019
Credit: 794,685
RAC: 0
United States
Message 697117 - Posted: 3 Jan 2008, 22:45:49 UTC


. . . most likely Matt, the Best Story Told by You thus far - Well done Sir & Thank You for the Update

suggestion: You should publish these accounts (Posts) and go into it all a bit deeper with other details - it would make for a most interestin' read for a Public that doesn't know about SETI / Berkeley, possibly generating some capital for the Project per se . . .


BOINC Wiki . . .

Science Status Page . . .
ID: 697117 · Report as offensive
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 697118 - Posted: 3 Jan 2008, 22:48:11 UTC - in response to Message 697106.  
Last modified: 3 Jan 2008, 22:48:28 UTC

The gui-image-files doesn't include as they should, client therefore has no idea how large these apparently zero-byte-files really are.


..interesting. I just added those fields to our project_files.xml and restarted all the servers. No immediate effect.

- Matt
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 697118 · Report as offensive
Ingleside
Volunteer developer

Send message
Joined: 4 Feb 03
Posts: 1546
Credit: 15,832,022
RAC: 13
Norway
Message 697134 - Posted: 3 Jan 2008, 23:43:49 UTC - in response to Message 697118.  

..interesting. I just added those fields to our project_files.xml and restarted all the servers. No immediate effect.


Now <nbytes> is included in the Scheduler-reply, clients will get the updated info in their next Scheduler-request, and this should over time lead to fewer clients asking for gui-files after their actual file-size.

But, if not mistaken, really old BOINC-clients doesn't update file-info even gets other info in a later scheduler-reply. Still, downloads should error-out if not finished after 2 weeks or something...


"I make so many mistakes. But then just think of all the mistakes I don't make, although I might."
ID: 697134 · Report as offensive
John Twohy
Volunteer tester

Send message
Joined: 21 Dec 07
Posts: 9
Credit: 2,058,932
RAC: 0
United States
Message 697182 - Posted: 4 Jan 2008, 3:01:41 UTC

Matt I know things get old I set in a chair that sounds just the same as you had there, thanks for telling errors where coming from your end, after I just spent 500 bucks on new ram, And another 1500 bucks for a new processor thinking that it was coming my unit.. I Could have gotten a new chair for thee butt.
ID: 697182 · Report as offensive
seti@elrcastor.com
Volunteer tester

Send message
Joined: 30 Jan 00
Posts: 35
Credit: 4,879,559
RAC: 0
United States
Message 697219 - Posted: 4 Jan 2008, 5:16:49 UTC

looks like the Technical news page has php issues

http://setiathome.berkeley.edu/tech_news.php



Parse error: syntax error, unexpected T_STRING, expecting ')' in /disks/setifiler1/home/boincadm/projects/sah/html/project/flat_tech_news.inc on line 5
ID: 697219 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 697222 - Posted: 4 Jan 2008, 5:24:39 UTC

I'd guess the science status page is cosmetically acting up too -- Due to the science database table juggling going on maybe?
-256,129,382 Gaussians inserted in the last 24 hours. (for a grand total of 0)

Jason

"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 697222 · Report as offensive
djmotiska

Send message
Joined: 26 Jul 01
Posts: 20
Credit: 29,378,647
RAC: 105
Finland
Message 697234 - Posted: 4 Jan 2008, 6:33:01 UTC - in response to Message 697086.  

For a while there we had two download servers and a workunit storage server with plenty I/O capacity to spare, but still we were hitting a hard 60 Mbit ceiling outbound.


I'd suspect firewall(s) and/or router(s), especially it they're running on old hardware.
ID: 697234 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 697247 - Posted: 4 Jan 2008, 7:15:46 UTC

Matt...very much appreciate your updates,
But can you explain why the forum response has been so poor lately, even though the total bandwidth as displayed by the Cricket Graphs seems back down to reasonable levels?
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 697247 · Report as offensive
Profile Sharkbait

Send message
Joined: 2 Jun 99
Posts: 11
Credit: 6,333,145
RAC: 0
United States
Message 697254 - Posted: 4 Jan 2008, 7:25:05 UTC - in response to Message 697247.  

Can somebody please explain the overflow rate?

Thanks Matt and Seti again for the hard work!

ID: 697254 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 30608
Credit: 53,134,872
RAC: 32
United States
Message 697259 - Posted: 4 Jan 2008, 7:58:54 UTC - in response to Message 697086.  

So off the home page I hit the link for technical news and
"Parse error: syntax error, unexpected T_STRING, expecting ')' in /disks/setifiler1/home/boincadm/projects/sah/html/project/flat_tech_news.inc on line 5"

As for that data rate throttle, I think elsewhere you indicated that the fiber to the lab was shared by others up on the hill. Could be something is being held in reserve for other users even if they aren't using it, which someone should know about. Or there could be campus equipment that simply is maxing out somewhere on the path, which no one knows has either failed or wasn't spec'd right to begin with. The only way to know is test. You do have dead time to test while you do the weekly pipe cleanup.

Gary

Spreading the workunit creation over several files at once seems to be helping create a healthier mix of fast/slow workunits. However, adding a second download server seems to have confirmed a suspicion of mine (key word: "seems"): that somewhere down the pike we're being capped at 60 Mbits/sec. For a while there we had two download servers and a workunit storage server with plenty I/O capacity to spare, but still we were hitting a hard 60 Mbit ceiling outbound. Inquiries are being drafted/sent to the appropriate parties. It still could be a local problem, but we're not sure what else to try (given our current hardware).

We are in the middle of building another helpful index on the science database. Looks like Bob's magic informix incantations are working - we can keep the project running simultaneously (though the assimilators might back up a bit). It is always happier around here when work is flowing. To be safe we increased the ready-to-send queue size to one million - we have the disk space now to keep more workunits around. The only downside is that this inflates the result table in the database by approximately 5-10%, which may exercise the RAM on the BOINC database server that much more.

There is another problem Dave and I were poking at today: excessive "out of range" failures on our public web sites. Here's the deal: BOINC clients have a nice GUI which shows you icons, pictures, etc. from different projects as you select which to run on your computer. Where does it get these files? From the project's web servers. This is all well and good, but there are several (hundreds? thousands?) older clients out there making such requests but are being met with 416 "range not satisfiable" errors. Why? Because they have already downloaded the image file, but are making requests for more bytes beyond the file boundaries as if there was more to download. Obviously a bug somewhere, or a change in the way apache handles such things, but there's not much we can do about it. Even though this activity is creating bursts of heavy load on our web servers, this is a fire we're going to let burn for now.

The official press release about multi-beam is finally out. This should help on many levels (though I'll be busier making sure the servers can handle any significant load increase). I guess I'll also be shaving every morning in case there is interest from the national television news media.

I guess this is "technical" news: Our desks/chairs/furniture are mostly ancient hand-me-downs, some pieces older than I. We did get some new chair donations recently, but one of them broke - it came loose from its base, causing unsuspecting sitters to suddenly fall forward if their balance wasn't particularly keen. It's been lurking in our lab way too long, coaxing uninformed standers with tired legs to rest upon its comfortable and seemingly stable cushion base. I came to the lab this morning and that evil chair was by my desk with a note taped to it: "Matt - can you please toss this chair?" I guess enough was enough. I dragged it to the dumpster and sent it back to the dark void from whence it came.

- Matt


ID: 697259 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1639
Credit: 12,921,799
RAC: 89
New Zealand
Message 697260 - Posted: 4 Jan 2008, 8:08:33 UTC - in response to Message 697222.  

I'd guess the science status page is cosmetically acting up too -- Due to the science database table juggling going on maybe?
-256,129,382 Gaussians inserted in the last 24 hours. (for a grand total of 0)

Jason

it apear's to be ok now
Results received in last hour 45,385
ID: 697260 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1639
Credit: 12,921,799
RAC: 89
New Zealand
Message 697263 - Posted: 4 Jan 2008, 8:16:45 UTC - in response to Message 697254.  

Can somebody please explain the overflow rate?

Thanks Matt and Seti again for the hard work!

I think Overflow** rate 3.4% (inserted during last 10 minutes) mean's that the amount of data has been inserted in the last 10 minutes is 3.4% then the data that is been taken from the server's by us cruncher's. Hope's this help's
ID: 697263 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1639
Credit: 12,921,799
RAC: 89
New Zealand
Message 697265 - Posted: 4 Jan 2008, 8:21:29 UTC - in response to Message 697234.  
Last modified: 4 Jan 2008, 8:47:15 UTC

[quote]For a while there we had two download servers and a workunit storage server with plenty I/O capacity to spare, but still we were hitting a hard 60 Mbit ceiling outbound.

How much data dose Seti go through in a month to feed all of it's eager cruncher's?
ID: 697265 · Report as offensive
John Twohy
Volunteer tester

Send message
Joined: 21 Dec 07
Posts: 9
Credit: 2,058,932
RAC: 0
United States
Message 697307 - Posted: 4 Jan 2008, 13:13:53 UTC

I think I have an answer to some problems and a solution, I using a quad chip and ran the ting using all of it to run in Boinc for points as the heat want higher in this room too I got errors in my stats I did find out that if I tuned Boinc to use only two of the Processors I got no stats errors, its cooler in this room, and the computer run smoother, the only thing here is I have 1000kbps net and piping all of the information back through a bottle neck 10Mbps set to 768Kbps upload system. I can’t wait tell they put Fiber wire here. if all of us clents looked in this matter we would stop loading bad infomation back into your drives.
ID: 697307 · Report as offensive
Profile Henk Haneveld
Volunteer tester

Send message
Joined: 16 May 99
Posts: 154
Credit: 1,577,293
RAC: 1
Netherlands
Message 697331 - Posted: 4 Jan 2008, 15:19:26 UTC

On the server status page the progress display of the splitter queue has changed to a solid bar from the previous seperate block for each beam pair done.

This makes it very difficult to view the current state of progress on a data file.

If possible please return to the old display format.
ID: 697331 · Report as offensive
Josef W. Segur
Volunteer developer
Volunteer tester

Send message
Joined: 30 Oct 99
Posts: 4504
Credit: 1,414,761
RAC: 0
United States
Message 697336 - Posted: 4 Jan 2008, 16:28:07 UTC - in response to Message 697254.  

Can somebody please explain the overflow rate?

Thanks Matt and Seti again for the hard work!

The "** results that exited early due to excessive noise" explanation is rather terse. The SETI science application is trying to extract possible E.T. signals from a background of noise which is fairly constant. Signal detection has thresholds which roughly correspond to a 50/50 chance of finding a signal of each type in a WU due to the normal noise level to ensure that weak signals will be reported. But sometimes the noise is more than expected and might produce a huge number of false signals, so the application quits when 31 signals have been found and puts the result_overflow informational message in the stderr text reported to the Scheduler. When the signals are entered into the Master Science Database that overflow indication is included.

All the multibeam data we're processing now was recorded before the radar interference problem was understood, and although there's some filtering in the splitter which reduces that noise source it cannot eliminate it entirely.
                                                               Joe
ID: 697336 · Report as offensive
PhonAcq

Send message
Joined: 14 Apr 01
Posts: 1656
Credit: 30,658,217
RAC: 1
United States
Message 697345 - Posted: 4 Jan 2008, 17:03:01 UTC

Does your explanation mean that all the wu's we have processed thus far are, in one way or another, corrupted due to unexpected noise and possibly not valid? And if so, is this why nothing has been done with all the work we have contributed over the last couple of years? Are we, in effect, starting over?
ID: 697345 · Report as offensive
1 · 2 · Next

Message boards : Technical News : Furniture Shortage (Jan 03 2008)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.