Blitzed Again (Jul 02 2009)


log in

Advanced search

Message boards : Technical News : Blitzed Again (Jul 02 2009)

Previous · 1 · 2 · 3 · 4 · 5
Author Message
1mp0£173
Volunteer tester
Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 914957 - Posted: 6 Jul 2009, 21:30:12 UTC - in response to Message 914953.

theoretically speaking, if we were to get another server that only compressed and uncompressed the data, would that help?.....

It doesn't make sense to me to take uncompressed, mostly-random binary data, encode it to text (six bits per character) and then compress that.

It makes more sense to see if we can eliminate the text encoding.
____________

1mp0£173
Volunteer tester
Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 914958 - Posted: 6 Jul 2009, 21:31:13 UTC - in response to Message 914911.


As to compression of random noise, that is after all what every work unit is unless ET is out there, if you understand the math behind compression, you will know that the result is likely to be bigger than what you started with. True I'm sure there is some non-random noise in the work unit, like the time, frequency, and location of the work unit, but it is such a small fraction of the data it won't make a difference.

... and it suddenly dawned on me that if Astropulse can send work in raw binary, multibeam should be able to as well.
____________

Profile Gary Charpentier
Volunteer tester
Avatar
Send message
Joined: 25 Dec 00
Posts: 12130
Credit: 6,412,888
RAC: 8,138
United States
Message 914968 - Posted: 6 Jul 2009, 22:03:26 UTC - in response to Message 914958.


As to compression of random noise, that is after all what every work unit is unless ET is out there, if you understand the math behind compression, you will know that the result is likely to be bigger than what you started with. True I'm sure there is some non-random noise in the work unit, like the time, frequency, and location of the work unit, but it is such a small fraction of the data it won't make a difference.

... and it suddenly dawned on me that if Astropulse can send work in raw binary, multibeam should be able to as well.

Yes, that is one artifact from the Classic days than can and should be pulled out of the code. Might even free up a few machine cycles and disk space as the server won't have to encode the data before transmission.

____________

Profile Borgholio
Avatar
Send message
Joined: 2 Aug 99
Posts: 651
Credit: 11,868,834
RAC: 4,127
United States
Message 914988 - Posted: 6 Jul 2009, 22:39:05 UTC - in response to Message 914954.

But there is nothing to compress...


Now who is having trouble reading the thread?
____________


You will be assimilated...bunghole!

John McLeod VII
Volunteer developer
Volunteer tester
Avatar
Send message
Joined: 15 Jul 99
Posts: 24093
Credit: 517,832
RAC: 152
United States
Message 915009 - Posted: 6 Jul 2009, 23:10:40 UTC - in response to Message 914930.


HTTP is 8 bit, you can transmit binary.

The SETI@HOME logo on this page was not sent as encoded text, it was sent binary.

And a considerable number of corporate gateways/virus/firewalls won't transmit binary through them if it isn't a jpeg. Or at least wouldn't when the original Seti@home was released. Today many more do and I agree that the MB work units would be much better transmitted as binary.

As to compression of random noise, that is after all what every work unit is unless ET is out there, if you understand the math behind compression, you will know that the result is likely to be bigger than what you started with. True I'm sure there is some non-random noise in the work unit, like the time, frequency, and location of the work unit, but it is such a small fraction of the data it won't make a difference.

Not only do I understand that random binary does not compress, you'll probably find a post on this thread where I point that out.

... and there is the (server) CPU time to do the compression, which probably isn't worth it.

I wouldn't be surprised to find corporate firewalls that block .zip files as well, because they can be used to mask trojans coming in to the network.

Matt commented in a related post that work was being sent with a strange MIME type (suitable for MAN pages and that's about all) and he changed to to application/text, which is pretty harmless.

It seems to follow that BOINC doesn't use the (HTTP) MIME type at all, and they could just as easily tag .gif or .jpg on the end, send the right MIME type for an image, and that'd be that.

Bad Idea. Some ISPs re-compress jpg files to save transmission space. jpg is NOT a lossless compression mechanism. Any packet inspection that tries to reassamble them as either a jpg or gif to see if there were a trojan burried (deep packed inspection) would fail to generate a valid file, and pitch the result.
____________


BOINC WIKI

Profile Gary Charpentier
Volunteer tester
Avatar
Send message
Joined: 25 Dec 00
Posts: 12130
Credit: 6,412,888
RAC: 8,138
United States
Message 915031 - Posted: 6 Jul 2009, 23:35:19 UTC - in response to Message 914988.

But there is nothing to compress...


Now who is having trouble reading the thread?

Not going to happen anyway.

And it is official, too many CUDA crunchers. Demand now exceeds supply. Put those backup projects online for the foreseeable future. Lower your SETI share.

____________

Profile Borgholio
Avatar
Send message
Joined: 2 Aug 99
Posts: 651
Credit: 11,868,834
RAC: 4,127
United States
Message 915053 - Posted: 6 Jul 2009, 23:59:39 UTC - in response to Message 915031.


Put those backup projects online for the foreseeable future.


Isn't that something most people should have done since day one? :)
____________


You will be assimilated...bunghole!

1mp0£173
Volunteer tester
Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 915063 - Posted: 7 Jul 2009, 0:10:15 UTC - in response to Message 915031.


And it is official, too many CUDA crunchers. Demand now exceeds supply. Put those backup projects online for the foreseeable future. Lower your SETI share.

... or leave everything as-is and don't sweat it.

Frankly, looking at it from the "backup project" point of view, why would I (as the manager of a project) want a whole bunch of fickle users who only show up when one of the big projects is down.

I like SETI, I don't do other projects, and I accept that things won't always be great, but when I have crunched multiple projects, I've always given the others a decent share.

If the cricket graphs are any indication, things will level off.
____________

Profile Borgholio
Avatar
Send message
Joined: 2 Aug 99
Posts: 651
Credit: 11,868,834
RAC: 4,127
United States
Message 915074 - Posted: 7 Jul 2009, 0:19:32 UTC - in response to Message 915063.


Frankly, looking at it from the "backup project" point of view, why would I (as the manager of a project) want a whole bunch of fickle users who only show up when one of the big projects is down.


TBH if I were running a volunteer-based DC project, I'd be happy with whatever contributions I got, regardless of why I got them. That's just me though.
____________


You will be assimilated...bunghole!

1mp0£173
Volunteer tester
Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 915081 - Posted: 7 Jul 2009, 0:28:40 UTC - in response to Message 915074.


Frankly, looking at it from the "backup project" point of view, why would I (as the manager of a project) want a whole bunch of fickle users who only show up when one of the big projects is down.


TBH if I were running a volunteer-based DC project, I'd be happy with whatever contributions I got, regardless of why I got them. That's just me though.

It depends. I read somewhere that Einstein was "taken down" by the influx from a recent SETI outage.

If the cost of those occasional crunchers is angry loyal users, that may be an easy choice.
____________

zpm
Volunteer tester
Avatar
Send message
Joined: 25 Apr 08
Posts: 284
Credit: 1,478,204
RAC: 3,149
United States
Message 915084 - Posted: 7 Jul 2009, 0:32:19 UTC - in response to Message 915081.
Last modified: 7 Jul 2009, 0:42:38 UTC

some systems get strained by others going down....


seti should have a sign-up list for back-up projects so that no one project get strained...

say like i would say that im going to wcg...

other go to milky,

to ross,

all those with a 50+ sp cuda card, go to gpugrid...

VTU@home could use another 20,000 users/host.....

edit:

to me, seti troubles arn't a biggy as i let a lot of others to run on cpu, and let gpu run ggrid, and seti.... but it more recntly has been a problem as i bought 2 nvid cards to let seti cuda run on those old p4s.. my p4's and P-M are running milky, and have vtu as a back-up(now that the server is back and beter maintained.) for cpu...
____________

I recommend Secunia PSI: http://secunia.com/vulnerability_scanning/personal/
Go Georgia Tech.

Profile Borgholio
Avatar
Send message
Joined: 2 Aug 99
Posts: 651
Credit: 11,868,834
RAC: 4,127
United States
Message 915109 - Posted: 7 Jul 2009, 0:49:46 UTC - in response to Message 915081.


If the cost of those occasional crunchers is angry loyal users, that may be an easy choice.


Yeah that's a good point. I wonder why everybody chose Einstein though? World Community Grid is a much better funded project currently seeking a cure for AIDS, Muscular Dystrophy, Dengue Fever, and other stuff. Perhaps it would be wise to put a list of alternative projects such as WCG on the main Seti site, since Matt admitted that the current issues aren't going away anytime soon. If people spread the backup load amongst WCG, Einstein, Milkyway, GPUGrid, etc...then it wouldn't be a problem.

____________


You will be assimilated...bunghole!

zpm
Volunteer tester
Avatar
Send message
Joined: 25 Apr 08
Posts: 284
Credit: 1,478,204
RAC: 3,149
United States
Message 915117 - Posted: 7 Jul 2009, 0:57:34 UTC - in response to Message 915109.

hmmm, lets see..

vtu@home
Einstein
Comso
Aqua
WCG
Rosseta
GGrid
Docking(oops i forgot it in the org. post.)
Milky
CPDN
Other math projects.... which i don't part in but would be willingly if i needed too...

____________

I recommend Secunia PSI: http://secunia.com/vulnerability_scanning/personal/
Go Georgia Tech.

1mp0£173
Volunteer tester
Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 915208 - Posted: 7 Jul 2009, 4:13:52 UTC - in response to Message 915109.


If the cost of those occasional crunchers is angry loyal users, that may be an easy choice.


Yeah that's a good point. I wonder why everybody chose Einstein though? World Community Grid is a much better funded project currently seeking a cure for AIDS, Muscular Dystrophy, Dengue Fever, and other stuff. Perhaps it would be wise to put a list of alternative projects such as WCG on the main Seti site, since Matt admitted that the current issues aren't going away anytime soon. If people spread the backup load amongst WCG, Einstein, Milkyway, GPUGrid, etc...then it wouldn't be a problem.

For some reason, I can't get excited about Biomedical research. Sure, it's worthy and all that, but learning that we're not alone would be life-changing, while medicine is just life-extending. Important, but it doesn't stir the imagination.

Same with gravity waves. Unless someone figures out how to modulate them and ride their equivalent of "I Love Lucy" on the g-waves, I don't feel a deep need to tune in.

So, I crunch here, using computers that are on anyway, and if the cache runs dry, it runs dry. There will be more work another day.

I don't have access to what Matt can see, but I look at the bandwidth graphs, and it looks like things were really good from the 27th to the 1st. Nearly pegged at the beginning of the day on the 27th. The heavy load hit on the 2nd, and the glitch that killed the raw storage (mentioned in Matt's post) probably also hurt, but I expect things will settle SOME once we get past this rush.
____________

rob smith
Volunteer tester
Send message
Joined: 7 Mar 03
Posts: 8135
Credit: 52,674,668
RAC: 74,652
United Kingdom
Message 919455 - Posted: 19 Jul 2009, 19:34:55 UTC - in response to Message 914318.

...///... Has anyone noticed that the splitter working on 05mr09af appears to be 'stuck'? Its been at the same point since the servers came back up on Tuesday...


19th July, and 05mr09af is still stuck - something amiss me thinks as others have been split and gone in the last two weeks.....

____________
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

Josef W. Segur
Volunteer developer
Volunteer tester
Send message
Joined: 30 Oct 99
Posts: 4203
Credit: 1,030,288
RAC: 267
United States
Message 922205 - Posted: 29 Jul 2009, 20:13:35 UTC - in response to Message 919455.

...///... Has anyone noticed that the splitter working on 05mr09af appears to be 'stuck'? Its been at the same point since the servers came back up on Tuesday...

19th July, and 05mr09af is still stuck - something amiss me thinks as others have been split and gone in the last two weeks.....

29th July, going for a full month. I have visions of all the other files going (done) and disappearing before new data arrives from Arecibo....
Joe

SteveL
Volunteer tester
Send message
Joined: 3 Jun 99
Posts: 10
Credit: 149,026
RAC: 0
Brazil
Message 926884 - Posted: 18 Aug 2009, 1:19:31 UTC - in response to Message 922205.

...///... Has anyone noticed that the splitter working on 05mr09af appears to be 'stuck'? Its been at the same point since the servers came back up on Tuesday...

19th July, and 05mr09af is still stuck - something amiss me thinks as others have been split and gone in the last two weeks.....

29th July, going for a full month. I have visions of all the other files going (done) and disappearing before new data arrives from Arecibo....


I've been following this unit for some time wondering if it would ever disappear... Guess what, it's been stuck again for some days
____________

Sten-Arne
Volunteer tester
Send message
Joined: 1 Nov 08
Posts: 3334
Credit: 19,034,721
RAC: 20,128
Sweden
Message 928060 - Posted: 22 Aug 2009, 19:25:31 UTC - in response to Message 926884.

...///... Has anyone noticed that the splitter working on 05mr09af appears to be 'stuck'? Its been at the same point since the servers came back up on Tuesday...

19th July, and 05mr09af is still stuck - something amiss me thinks as others have been split and gone in the last two weeks.....

29th July, going for a full month. I have visions of all the other files going (done) and disappearing before new data arrives from Arecibo....


I've been following this unit for some time wondering if it would ever disappear... Guess what, it's been stuck again for some days


It's still there, and still stuck. Never gets past 1 channel in MB. After some time it resets and start on 1 channel again, gets stuck and....endless loop.

Sten-Arne

Richard Haselgrove
Volunteer tester
Send message
Joined: 4 Jul 99
Posts: 8375
Credit: 46,704,513
RAC: 20,104
United Kingdom
Message 929903 - Posted: 31 Aug 2009, 8:36:16 UTC - in response to Message 922205.

...///... Has anyone noticed that the splitter working on 05mr09af appears to be 'stuck'? Its been at the same point since the servers came back up on Tuesday...

19th July, and 05mr09af is still stuck - something amiss me thinks as others have been split and gone in the last two weeks.....

29th July, going for a full month. I have visions of all the other files going (done) and disappearing before new data arrives from Arecibo....
Joe

31st August, two full months, and the other files have indeed gone (done). So 05mr09af gets the undivided attention of a splitter - and breaks it. I wonder whether 16jl09ab, 27fe08ac, and 30jl09aa will become long-term fixtures too, or whether by drawing attention to themselves like this, they'll trigger a bout of spring-cleaning when the boyz get in.

rob smith
Volunteer tester
Send message
Joined: 7 Mar 03
Posts: 8135
Credit: 52,674,668
RAC: 74,652
United Kingdom
Message 930019 - Posted: 31 Aug 2009, 20:38:16 UTC

Of your Three Amigos for 05mr09af two have departed for the time being, leaving 27fe08ac sitting with all its APs done, and one MB in progress, with no more MB in progress.
Just a thought - are the Berkeley Boyz trying to flush out the badies?
____________
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

Previous · 1 · 2 · 3 · 4 · 5

Message boards : Technical News : Blitzed Again (Jul 02 2009)

Copyright © 2014 University of California