Blitzed Again (Jul 02 2009)

Message boards : Technical News : Blitzed Again (Jul 02 2009)
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · Next

AuthorMessage
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13715
Credit: 208,696,464
RAC: 304
Australia
Message 913889 - Posted: 4 Jul 2009, 5:06:37 UTC - in response to Message 913854.  

... and if there isn't a good clean line-of-sight from the lab to the right building(s) on Campus, then RF isn't a good choice.

That would be the biggest impediment IMHO.

Grant
Darwin NT
ID: 913889 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 30591
Credit: 53,134,872
RAC: 32
United States
Message 913905 - Posted: 4 Jul 2009, 5:58:06 UTC - in response to Message 913854.  


No need for a trencher- go wireless.

Since connectivity on campus is provided by the IST department (the folks who brought you Cricket) they have to maintain it.

... and if there isn't a good clean line-of-sight from the lab to the right building(s) on Campus, then RF isn't a good choice.

Screw LOS between the Lab and a campus building, is there LOS between the LAB and their ISP's node? (Calren?) Remove campus entirely. Get them an OC-768 link!

ID: 913905 · Report as offensive
Cameron
Avatar

Send message
Joined: 27 Nov 02
Posts: 110
Credit: 5,082,471
RAC: 17
Australia
Message 913939 - Posted: 4 Jul 2009, 9:16:55 UTC

All the talks were great Matt.

There was a bit of a bad pickup from the microphone on the camera during your pipeline talk [popping and hissing] that didn't appear to be noticable, or at least as bad during the other talks. A good effort by everyone though.

I assume you meant there isn't the money for another set of hands for technical assistance to help you and Jeff. Or perhaps you meant someone to field the frequently asked setup questions.

I say this because you and Jeff seem to be the Tech support currently administring and kicking (when needed) the servers. And you do a very good job at it.
ID: 913939 · Report as offensive
insomnia

Send message
Joined: 27 Nov 04
Posts: 1
Credit: 226,439
RAC: 0
Germany
Message 913949 - Posted: 4 Jul 2009, 11:10:12 UTC

Storm? What storm? The last few days I get (on Windows and Linux):
"Message from server: (Project has no jobs available)". I've got BOINC 6.6.36 under Windows 2000 and if I remember correctly 6.4.xyz under Ubuntu 9.04 x86_64.
I guessed there's allways something to compute until LGM have been found ;-)
ID: 913949 · Report as offensive
zpm
Volunteer tester
Avatar

Send message
Joined: 25 Apr 08
Posts: 284
Credit: 1,659,024
RAC: 0
United States
Message 913977 - Posted: 4 Jul 2009, 14:16:42 UTC - in response to Message 913949.  

ugh... my computers are dry.

I recommend Secunia PSI: http://secunia.com/vulnerability_scanning/personal/
Go Georgia Tech.
ID: 913977 · Report as offensive
zpm
Volunteer tester
Avatar

Send message
Joined: 25 Apr 08
Posts: 284
Credit: 1,659,024
RAC: 0
United States
Message 913983 - Posted: 4 Jul 2009, 14:40:29 UTC - in response to Message 913978.  

it's not the fact that the cpu is dry, it's the fact that the gpu's are dry..... 2 9400 gt's.... 32 sp each and they can run but probably won't finish a gpugrid wu.... cpu's are running, WCG,MW,ROSSETA, and a few others to keep them going.... i had those cpu do vtu@home but it(project leaders and servers have been unresponive as to why they were down for 2 weeks and i'm not crunching for them as i fear they will go down again....

I recommend Secunia PSI: http://secunia.com/vulnerability_scanning/personal/
Go Georgia Tech.
ID: 913983 · Report as offensive
zpm
Volunteer tester
Avatar

Send message
Joined: 25 Apr 08
Posts: 284
Credit: 1,659,024
RAC: 0
United States
Message 913988 - Posted: 4 Jul 2009, 14:47:09 UTC - in response to Message 913984.  

i'll just go to the beta seti as their are 217,000 wu's waiting to be sent.....

I recommend Secunia PSI: http://secunia.com/vulnerability_scanning/personal/
Go Georgia Tech.
ID: 913988 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 914013 - Posted: 4 Jul 2009, 17:05:17 UTC - in response to Message 913905.  


No need for a trencher- go wireless.

Since connectivity on campus is provided by the IST department (the folks who brought you Cricket) they have to maintain it.

... and if there isn't a good clean line-of-sight from the lab to the right building(s) on Campus, then RF isn't a good choice.

Screw LOS between the Lab and a campus building, is there LOS between the LAB and their ISP's node? (Calren?) Remove campus entirely. Get them an OC-768 link!

Range (and background noise) becomes an issue.

Not saying it isn't possible, just suggesting that it might cost more than the fiber.

... and being on campus means that they have to use what IST will support.
ID: 914013 · Report as offensive
Profile Jon Golding
Avatar

Send message
Joined: 20 Apr 00
Posts: 105
Credit: 841,861
RAC: 0
United Kingdom
Message 914029 - Posted: 4 Jul 2009, 18:03:26 UTC - in response to Message 913743.  

I see what you mean, but how about a system in which download servers are distributed around various universities (just split data and download to clients), whilst the ONLY functions that the Berkeley servers do is to receive all uploaded completed tasks, do validation, and result storage/archiving. Wouldn't that ease the bandwidth/server pain? Of course, there may be logistical problems in that the raw data disks from Aricebo would need to be sent out the different participating universities, but maybe this happens anyway for some observing projects.

ID: 914029 · Report as offensive
OzzFan Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Apr 02
Posts: 15691
Credit: 84,761,841
RAC: 28
United States
Message 914048 - Posted: 4 Jul 2009, 19:33:19 UTC - in response to Message 914029.  

I see what you mean, but how about a system in which download servers are distributed around various universities (just split data and download to clients), whilst the ONLY functions that the Berkeley servers do is to receive all uploaded completed tasks, do validation, and result storage/archiving. Wouldn't that ease the bandwidth/server pain? Of course, there may be logistical problems in that the raw data disks from Aricebo would need to be sent out the different participating universities, but maybe this happens anyway for some observing projects.


Its a good idea in theory, but there's still some problems with it.

There's still a cost issue there. SETI@Home leases space from the University, and has to work within the constraints set forth by the University's dictates, which include power requirements for both the servers and the air conditioning to cool the servers. Then there's the issue of staff wages (they do try to get paid for their work).

I mention this because if the plan is to get other universities involved, you are effectively doubling the financial strain on the project. Other scientists at other universities will have to purchase servers (or look for donations), lease space, pay for their power usage, pay for the connection to the internet, etc. And of course those scientists will want to get paid as well.

Its like a company with a poor business model that's extra-busy due to their popularity. Suggesting that the company branch out when there's no finances to do so will only make things worse. Unfortunate for SETI, they do not have a "business model" because they are not a business - they are a science project.
ID: 914048 · Report as offensive
zpm
Volunteer tester
Avatar

Send message
Joined: 25 Apr 08
Posts: 284
Credit: 1,659,024
RAC: 0
United States
Message 914077 - Posted: 4 Jul 2009, 20:58:49 UTC - in response to Message 914048.  

Unfortunate for SETI, they do not have a "business model" because they are not a business - they are a science project.



actually that makes me happy that it's a science project b/c we have seen what happens to company's that run on a bussiness model and has a spot on wallstreet.......
Bankruptcy



I recommend Secunia PSI: http://secunia.com/vulnerability_scanning/personal/
Go Georgia Tech.
ID: 914077 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 914105 - Posted: 4 Jul 2009, 22:21:22 UTC - in response to Message 913784.  
Last modified: 4 Jul 2009, 22:21:42 UTC

I think there is merit to the idea of sharing the SETI@Home database to data servers across the Internet and leveraging the distributed servers' bandwidth. P2P networks are very good at demonstrating the power of collective bandwidth for spreading information from a single source :)

The thing that makes P2P really work is that each file (each bootleg copy of the latest DVD) is uploaded once, and basically spreads: each downloader also becomes a new upload site.

That doesn't work for SETI because each result goes to ONE user.

Sure, you could come up with a "nano-splitter" that took one result and turned it into two (one for each wingman) but even that wouldn't help that much.

... and not at all for uploads.
ID: 914105 · Report as offensive
zpm
Volunteer tester
Avatar

Send message
Joined: 25 Apr 08
Posts: 284
Credit: 1,659,024
RAC: 0
United States
Message 914139 - Posted: 4 Jul 2009, 23:36:50 UTC - in response to Message 914105.  
Last modified: 4 Jul 2009, 23:37:46 UTC

maybe, we should explore what drugdiscovery(A) and hydrogen(A) have done...... use 7zip to send files.... it makes 10 mb into 4-5 mb or less.

7zip works ok with boinc.... it will reduce the bandwidth a little. and may help with the congestion....

gamers also use it.... alot better at compressing than windows zip.

I recommend Secunia PSI: http://secunia.com/vulnerability_scanning/personal/
Go Georgia Tech.
ID: 914139 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13715
Credit: 208,696,464
RAC: 304
Australia
Message 914152 - Posted: 4 Jul 2009, 23:53:36 UTC - in response to Message 914139.  

maybe, we should explore what drugdiscovery(A) and hydrogen(A) have done...... use 7zip to send files.... it makes 10 mb into 4-5 mb or less.

Compression as been suggested before. But given the nature of the data, it's not very compressible.
Grant
Darwin NT
ID: 914152 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 914156 - Posted: 5 Jul 2009, 0:28:59 UTC - in response to Message 914152.  

maybe, we should explore what drugdiscovery(A) and hydrogen(A) have done...... use 7zip to send files.... it makes 10 mb into 4-5 mb or less.

Compression as been suggested before. But given the nature of the data, it's not very compressible.

It ought to compress fairly nicely, but there is a trade off with CPU time on the server to do the compression vs the transfer time to transfer the uncompressed bytes.


BOINC WIKI
ID: 914156 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 30591
Credit: 53,134,872
RAC: 32
United States
Message 914159 - Posted: 5 Jul 2009, 0:48:23 UTC - in response to Message 914156.  

maybe, we should explore what drugdiscovery(A) and hydrogen(A) have done...... use 7zip to send files.... it makes 10 mb into 4-5 mb or less.

Compression as been suggested before. But given the nature of the data, it's not very compressible.

It ought to compress fairly nicely, but there is a trade off with CPU time on the server to do the compression vs the transfer time to transfer the uncompressed bytes.

I'm not so sure about compression. My understanding was that random noise wouldn't compress very much at all as there are no patterns in it to exploit to compress. Now a WU with a signal might compress because of the pattern of the signal.

ID: 914159 · Report as offensive
zpm
Volunteer tester
Avatar

Send message
Joined: 25 Apr 08
Posts: 284
Credit: 1,659,024
RAC: 0
United States
Message 914163 - Posted: 5 Jul 2009, 1:20:02 UTC - in response to Message 914159.  
Last modified: 5 Jul 2009, 1:20:40 UTC

perhaps it should be beta tested on the seti beta...... just a 1-2 week trial... it couldn't hurt.... may loose a little time if it doesn't pan out, but it would be an option explored...

I recommend Secunia PSI: http://secunia.com/vulnerability_scanning/personal/
Go Georgia Tech.
ID: 914163 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 914165 - Posted: 5 Jul 2009, 1:25:57 UTC - in response to Message 914159.  

maybe, we should explore what drugdiscovery(A) and hydrogen(A) have done...... use 7zip to send files.... it makes 10 mb into 4-5 mb or less.

Compression as been suggested before. But given the nature of the data, it's not very compressible.

It ought to compress fairly nicely, but there is a trade off with CPU time on the server to do the compression vs the transfer time to transfer the uncompressed bytes.

I'm not so sure about compression. My understanding was that random noise wouldn't compress very much at all as there are no patterns in it to exploit to compress. Now a WU with a signal might compress because of the pattern of the signal.

Correct me if I am wrong, but isn't it a text file that is sent each way?


BOINC WIKI
ID: 914165 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13715
Credit: 208,696,464
RAC: 304
Australia
Message 914176 - Posted: 5 Jul 2009, 1:45:31 UTC - in response to Message 914165.  


Lossless_data_compression
By operation of the pigeonhole principle, no lossless compression algorithm can efficiently compress all possible data, and completely random data streams cannot be compressed.
I would expect that most of the data downloaded would be mostly ramdom data, hence unlikely to compress much at all.
The result data would probably be more compressable, but how much more? And given how small it already is any savings in bandwidth could well be offset by the prcessing required at the other end to expand it all again.
Grant
Darwin NT
ID: 914176 · Report as offensive
Josef W. Segur
Volunteer developer
Volunteer tester

Send message
Joined: 30 Oct 99
Posts: 4504
Credit: 1,414,761
RAC: 0
United States
Message 914192 - Posted: 5 Jul 2009, 2:11:36 UTC
Last modified: 5 Jul 2009, 2:18:39 UTC

Rather than speculating in a vacuum, I suggest any of you could simply try compressing a WU or two. You'd find that setiathome_enhanced WUs are moderately compressible, astropulse_v505 only slightly. If compressed downloads were implemented, it would be akin to adding another 25 MBits/second to the download bandwidth; enough to help short term, but hardly a permanent fix.

BOINC 5.x and later have libCurl which is prepared to decompress files sent with gzip compression. The download servers run Apache, which can be configured to gzip data being sent. All that would be needed is to set the warning that all users must upgrade to BOINC 5.x or later, give it long enough to be seen and adopted, then reconfigure the download servers.

The real issue is whether the download servers can handle the extra load of doing the compression; an earlier suggestion for a trial at SETI Beta wouldn't test that effectively. And although the 7zip format does compress better than gzip, it takes more memory and time to do the compression. It would also take project-specific coding to implement, rather than using the features BOINC already has.

Edit:
Correct me if I am wrong, but isn't it a text file that is sent each way?

The payload in Enhanced work is 256KB of nearly random data encoded in uue fashion, so can be sent as text. The payload in AP work is 8MB of pure 8 bit nearly random data. In both cases, it's the task of the science applications to ferret out the few cases where the data deviates from random noise.

                                                           Joe
ID: 914192 · Report as offensive
Previous · 1 · 2 · 3 · 4 · 5 · Next

Message boards : Technical News : Blitzed Again (Jul 02 2009)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.