UPLOADING II --- I is TOOOOOO big

Message boards : Number crunching : UPLOADING II --- I is TOOOOOO big
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 4 · 5 · 6 · 7 · 8 · Next

AuthorMessage
Bart Barenbrug

Send message
Joined: 7 Jul 04
Posts: 52
Credit: 337,401
RAC: 0
Netherlands
Message 138513 - Posted: 18 Jul 2005, 20:57:07 UTC - in response to Message 138484.  

So is the Philmor server being reconfigured to serve as either upload or download server (so Kryten's load is divided)? I assume some gateway or router can be configured to split the traffic appropriately? By the looks of it, the Kosh server could be another candidate for re-deployment if really need be (and validation can keep up on the remaining two validation processes). I can imagine that moving tasks from one machine to another is far from trivial in a running environment, with many scripts and programs running in concert. So to the people at seti: stay cool, despite the sometimes heated discussions here.

ID: 138513 · Report as offensive
Profile ksnash

Send message
Joined: 28 Nov 99
Posts: 402
Credit: 528,725
RAC: 0
United States
Message 138484 - Posted: 18 Jul 2005, 20:34:31 UTC

Maybe it would be good if they set up a protocol so setiqueue can work again. Just like in seti classic, the main servers can't handle all the load at many times. It was determined that Setiqueue was helpful in lowering connect problems with main servers.
ID: 138484 · Report as offensive
Profile Tigher
Volunteer tester

Send message
Joined: 18 Mar 04
Posts: 1547
Credit: 760,577
RAC: 0
United Kingdom
Message 138479 - Posted: 18 Jul 2005, 20:30:35 UTC - in response to Message 138478.  

Ya just think we would have heard something by now.
Maybe they are home reading Harry Potter.


LOL!

ID: 138479 · Report as offensive
SeaEagle

Send message
Joined: 14 Jun 99
Posts: 12
Credit: 3,291,985
RAC: 4
United States
Message 138478 - Posted: 18 Jul 2005, 20:29:49 UTC

Ya just think we would have heard something by now.
Maybe they are home reading Harry Potter.
ID: 138478 · Report as offensive
Profile Tigher
Volunteer tester

Send message
Joined: 18 Mar 04
Posts: 1547
Credit: 760,577
RAC: 0
United Kingdom
Message 138467 - Posted: 18 Jul 2005, 20:20:29 UTC

Waiting to upload:

62 - waiting to go up now.

Uploads completed:

15th 10 went up
16th 10 went up
17th 35 went up
18th (so far) 17 went up

Downloads completed:

suspended new work for some days 15th -17th

18th 86 downloaded.

seti was suspended overall for periods as I thought about what to do and gave the time to lhc and cpdn. All back now though.

ID: 138467 · Report as offensive
Profile spacemeat
Avatar

Send message
Joined: 4 Oct 99
Posts: 239
Credit: 8,425,288
RAC: 0
United States
Message 138461 - Posted: 18 Jul 2005, 20:06:44 UTC - in response to Message 138451.  

Back to topic...

on the 15th, I uploaded 32
16th: 34
17th: 29
18th: 16 (partial day)...

I grant that I have 53 pending upload success another 41 in flight, and 4 pendig download... but... Paul says that work is getting back ...

This was a "hand tally" so, the results may be off a bit. As I suspended more work download I may be able to know more tomorrow ... we shall see ...


on my top machine which completes about 24 WU/day:

15th: 20 uploaded
16th: 16 uploaded
17th: 18 uploaded

my last one today went through 6 hours ago. backload went from about 12 on friday to 40 right now. boinc is averaging less than 2 mins between retries without me provoking it. downloads do not seem to be a problem. my first deadline passes on the 25th (even on the slower machines) so nothing at this end is critical yet.
ID: 138461 · Report as offensive
timethief

Send message
Joined: 1 Jan 04
Posts: 25
Credit: 545,474
RAC: 0
Germany
Message 138458 - Posted: 18 Jul 2005, 20:04:30 UTC

Finally I loaded up most outstanding results today (yes!) by using a proxy see previous posting http://setiweb.ssl.berkeley.edu/forum_thread.php?id=16939#138257 and increased the priority of the other projects. Hope its getting better in the next days, trusting in the seti-team or a miracle, whichever comes first ;-)

Have a nice day and good luck!
ID: 138458 · Report as offensive
Profile Paul D. Buck
Volunteer tester

Send message
Joined: 19 Jul 00
Posts: 3898
Credit: 1,158,042
RAC: 0
United States
Message 138451 - Posted: 18 Jul 2005, 19:52:35 UTC

Back to topic...

on the 15th, I uploaded 32
16th: 34
17th: 29
18th: 16 (partial day)...

I grant that I have 53 pending upload success another 41 in flight, and 4 pendig download... but... Paul says that work is getting back ...

This was a "hand tally" so, the results may be off a bit. As I suspended more work download I may be able to know more tomorrow ... we shall see ...
ID: 138451 · Report as offensive
Profile KWSN - MajorKong
Volunteer tester
Avatar

Send message
Joined: 5 Jan 00
Posts: 2892
Credit: 1,499,890
RAC: 0
United States
Message 138446 - Posted: 18 Jul 2005, 19:32:17 UTC - in response to Message 138439.  

Each block is split into 256 bands. Each of these is sent out 4 times so the answer is n * 256 * 4 where n is the number of blocks on a tape.


i believe you forgot that each block is only 1.7 seconds long.... shouldn't the formula be

n*(256/48)*4 ??


ID: 138446 · Report as offensive
Profile Tigher
Volunteer tester

Send message
Joined: 18 Mar 04
Posts: 1547
Credit: 760,577
RAC: 0
United Kingdom
Message 138441 - Posted: 18 Jul 2005, 19:28:45 UTC - in response to Message 138439.  
Last modified: 18 Jul 2005, 19:40:29 UTC

Each block is split into 256 bands. Each of these is sent out 4 times so the answer is n * 256 * 4 where n is the number of blocks on a tape.


Thanks for the explanation. John......do you know exactly what has happened/ is happening. Its a pretty hot topic here right now and most of what we discuss is based on little (actually NO) information.

Thanks.
Ian


ID: 138441 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 138439 - Posted: 18 Jul 2005, 19:25:37 UTC

Each block is split into 256 bands. Each of these is sent out 4 times so the answer is n * 256 * 4 where n is the number of blocks on a tape.


BOINC WIKI
ID: 138439 · Report as offensive
Profile spacemeat
Avatar

Send message
Joined: 4 Oct 99
Posts: 239
Credit: 8,425,288
RAC: 0
United States
Message 138438 - Posted: 18 Jul 2005, 19:25:12 UTC

where are we at now, figuring out if the problem is being 'repaired' by limiting new work? that's silly, they would have shut off downloads. even that's silly, people have up to 10 days cache, there would be 2-3 weeks of users being completely idle to catch up at this rate
ID: 138438 · Report as offensive
Ingleside
Volunteer developer

Send message
Joined: 4 Feb 03
Posts: 1546
Credit: 15,832,022
RAC: 29
Norway
Message 138430 - Posted: 18 Jul 2005, 19:09:00 UTC - in response to Message 138421.  
Last modified: 18 Jul 2005, 19:09:27 UTC

After reading and re-reading, i'm not sure if I can answer your question. I'm not sure if the splitter divides the 48 blocks into 256 WU or if each block is divided up into 256 WU. The first time I read it I interpreted it as each block has 256 WU. The second time, however, i'm not so sure.

Either way, there is a lot of blocks to be split and WU's to be sent out.


Each wu is roughly 107 seconds long, but there is some overlap between wu to make sure all possible signals is detected. Since each block is 1.7 seconds, it's fairly apparent this means 48 blocks is used to generate 256 wu.

A quick calculation reveals it can be roughly 48k blocks left to split, this means 256k wu, and this again means 1M results when finished splitting.

With 216k ready now, and 4h transitioner-backlog, maybe upto 1.3M results, but can also be much less...
ID: 138430 · Report as offensive
Profile Tigher
Volunteer tester

Send message
Joined: 18 Mar 04
Posts: 1547
Credit: 760,577
RAC: 0
United Kingdom
Message 138429 - Posted: 18 Jul 2005, 19:04:40 UTC - in response to Message 138426.  

After reading and re-reading, i'm not sure if I can answer your question. I'm not sure if the splitter divides the 48 blocks into 256 WU or if each block is divided up into 256 WU. The first time I read it I interpreted it as each block has 256 WU. The second time, however, i'm not so sure.

Either way, there is a lot of blocks to be split and WU's to be sent out.

Jim


Just check my maths but guess there are about 1.13M WUs on the tapes still being processed?

EDIT: Hmmm....might be more as the terminology is not leaping out as clear to me...perhaps that * 256?



48 blocks => 256 work units => 1024 results (what we download)



AH ok thanks. I got lost twice reading it as I interpretted a couple of ways. Thanks for that clarity.

ID: 138429 · Report as offensive
Profile KWSN - MajorKong
Volunteer tester
Avatar

Send message
Joined: 5 Jan 00
Posts: 2892
Credit: 1,499,890
RAC: 0
United States
Message 138426 - Posted: 18 Jul 2005, 19:02:46 UTC - in response to Message 138421.  

After reading and re-reading, i'm not sure if I can answer your question. I'm not sure if the splitter divides the 48 blocks into 256 WU or if each block is divided up into 256 WU. The first time I read it I interpreted it as each block has 256 WU. The second time, however, i'm not so sure.

Either way, there is a lot of blocks to be split and WU's to be sent out.

Jim


Just check my maths but guess there are about 1.13M WUs on the tapes still being processed?

EDIT: Hmmm....might be more as the terminology is not leaping out as clear to me...perhaps that * 256?



48 blocks => 256 work units => 1024 results (what we download)


https://youtu.be/iY57ErBkFFE

#Texit

Don't blame me, I voted for Johnson(L) in 2016.

Truth is dangerous... especially when it challenges those in power.
ID: 138426 · Report as offensive
Profile Jim Baize
Volunteer tester

Send message
Joined: 6 May 00
Posts: 758
Credit: 149,536
RAC: 0
United States
Message 138421 - Posted: 18 Jul 2005, 18:48:28 UTC - in response to Message 138417.  

After reading and re-reading, i'm not sure if I can answer your question. I'm not sure if the splitter divides the 48 blocks into 256 WU or if each block is divided up into 256 WU. The first time I read it I interpreted it as each block has 256 WU. The second time, however, i'm not so sure.

Either way, there is a lot of blocks to be split and WU's to be sent out.

Jim


Just check my maths but guess there are about 1.13M WUs on the tapes still being processed?

EDIT: Hmmm....might be more as the terminology is not leaping out as clear to me...perhaps that * 256?


ID: 138421 · Report as offensive
Profile Tigher
Volunteer tester

Send message
Joined: 18 Mar 04
Posts: 1547
Credit: 760,577
RAC: 0
United Kingdom
Message 138417 - Posted: 18 Jul 2005, 18:39:26 UTC - in response to Message 138414.  
Last modified: 18 Jul 2005, 18:42:09 UTC

I think we are going to start seeing this problem correct itself. After reading someone else's post, I took a look at the server status page. There are no more tapes qued to be split. One splitter is already shut down, so only 4 are left running. I can only assume that they have done this intentionally.

Jim

Can I add a further refinement.

If each boinc client indicated how low it was on work the ucb dl server could say yes or no to requests quite quickly. This would see cpu time and bandwidth reserved for upload and only give work to nearly dry clients.

I guess the same scheme could be applied to upload also. That way there is a balancing of the ul/dl server to meet demands plus urgent running dry situations.



Just check my maths but guess there are about 1.13M WUs on the tapes still being processed?

EDIT: Hmmm....might be more as the terminology is not leaping out as clear to me...perhaps that * 256?


ID: 138417 · Report as offensive
timethief

Send message
Joined: 1 Jan 04
Posts: 25
Credit: 545,474
RAC: 0
Germany
Message 138415 - Posted: 18 Jul 2005, 18:35:51 UTC

Some results are still uploaded to the server, but the rate of sucessfull is lousy.

By the way, I have to revise my statement from ago: the network seems to be clear, so the packet drops must caused by the server.

ID: 138415 · Report as offensive
Profile Jim Baize
Volunteer tester

Send message
Joined: 6 May 00
Posts: 758
Credit: 149,536
RAC: 0
United States
Message 138414 - Posted: 18 Jul 2005, 18:34:19 UTC - in response to Message 138408.  

I think we are going to start seeing this problem correct itself. After reading someone else's post, I took a look at the server status page. There are no more tapes qued to be split. One splitter is already shut down, so only 4 are left running. I can only assume that they have done this intentionally.

Jim

Can I add a further refinement.

If each boinc client indicated how low it was on work the ucb dl server could say yes or no to requests quite quickly. This would see cpu time and bandwidth reserved for upload and only give work to nearly dry clients.

I guess the same scheme could be applied to upload also. That way there is a balancing of the ul/dl server to meet demands plus urgent running dry situations.


ID: 138414 · Report as offensive
KB7RZF
Volunteer tester
Avatar

Send message
Joined: 15 Aug 99
Posts: 9549
Credit: 3,308,926
RAC: 4
United States
Message 138413 - Posted: 18 Jul 2005, 18:33:47 UTC
Last modified: 18 Jul 2005, 18:37:36 UTC

I have not gotten any results uploaded yet, but that don't mean anything. The people at Berkley are probably currently working on the issue at hand and trying to get it working again. Just gotta sit back and wait. All the more reason so just let the computer do its work, and eventually they will get through. Looking at this shows that the data is going in and out, so it appears to be working, but probably like others have said, it may be that the servers have reached capacity. Never know till Berkley tells us. :-)

Jeremy
ID: 138413 · Report as offensive
Previous · 1 . . . 4 · 5 · 6 · 7 · 8 · Next

Message boards : Number crunching : UPLOADING II --- I is TOOOOOO big


 
©2020 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.