Uploading

Message boards : Number crunching : Uploading
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 12 · 13 · 14 · 15

AuthorMessage
timethief

Send message
Joined: 1 Jan 04
Posts: 25
Credit: 545,474
RAC: 0
Germany
Message 138338 - Posted: 18 Jul 2005, 17:01:14 UTC

Even if some have modified the client source, I think their number would not be as big to harm the project server at all.

Meanwhile I have traced some client-server communication. It's right, the server drops some connections, but the reason seems to be the tcp/ip protocol. The client is unable to pass a SYN or ACK packet to the server, cause the network is simply overcharged with requests.
:-(
ID: 138338 · Report as offensive
Profile spacemeat
Avatar

Send message
Joined: 4 Oct 99
Posts: 239
Credit: 8,425,288
RAC: 0
United States
Message 138349 - Posted: 18 Jul 2005, 17:24:11 UTC

who cares if anyone modified the source to expedite retries? BOINC only attemts to transfer two files at a time anyway so there's no great advantage. the modified client isn't hammering the server any more than an unmodified client with a large backlog. and once they are all uploaded, it stops. if the server could keep up with demand, it's a non-issue.
ID: 138349 · Report as offensive
Profile Jim Baize
Volunteer tester

Send message
Joined: 6 May 00
Posts: 758
Credit: 149,536
RAC: 0
United States
Message 138357 - Posted: 18 Jul 2005, 17:34:41 UTC - in response to Message 138349.  

But if this hypothetical modified client only has one or two results then it is causeing network connection, just as those with large result ques are doing. The only difference is those with large result ques are legitimate. So, the more people who use this hypothetical client the more the network congestion and the bigger the problem. If it was only one person doing it then the added network congestion would be negligible.

Jim

who cares if anyone modified the source to expedite retries? BOINC only attemts to transfer two files at a time anyway so there's no great advantage. the modified client isn't hammering the server any more than an unmodified client with a large backlog. and once they are all uploaded, it stops. if the server could keep up with demand, it's a non-issue.


ID: 138357 · Report as offensive
Profile spacemeat
Avatar

Send message
Joined: 4 Oct 99
Posts: 239
Credit: 8,425,288
RAC: 0
United States
Message 138374 - Posted: 18 Jul 2005, 17:53:49 UTC

bleh

network congestion does not even appear to be a problem here, all indications point to server capacity.

if the server was keeping up, a 1 minute retry would finish and stop. you would have to wait maybe, what, 30 minutes for your next upload..? are you that impatient?

anyone with more than 6 WU's in cache sees no benefit from 1 minute retries. boinc has to wait for all the previous attempts to time out first.

i think you have more to worry about from people constantly hitting 'retry' than modified clients (if there are even any out there).
ID: 138374 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 138380 - Posted: 18 Jul 2005, 18:00:29 UTC - in response to Message 138374.  
Last modified: 18 Jul 2005, 18:02:35 UTC

If you move your mousepointer over the retry button and leave something heavy (like a stapler, or use sticky tape to tape down that key) lying on the Enter key on the numerical keyboard, it will retry every second.

If a lot of people are doing it this way...

(mind: before you try, the backoff time will grow exponentially large within a minute. ;))
i think you have more to worry about from people constantly hitting 'retry' than modified clients (if there are even any out there).


ID: 138380 · Report as offensive
timethief

Send message
Joined: 1 Jan 04
Posts: 25
Credit: 545,474
RAC: 0
Germany
Message 138381 - Posted: 18 Jul 2005, 18:01:24 UTC

Anyway there might be such clients or not, it doesn't really matter at the moment. The IS a situation like in a DOS-attack and it must be handeled NOW and fast.

I see only one practicable option to get out quickly: Modify the firewall and block some IP address ranges in round robin manner. The first priority should be to lower the amount of request by collecting the outstanding results.
This might be hard, but if nothing is done, I assume a complete breakdown of the network connections within the next hours.

If there anybody has a better idea, spit it out!
ID: 138381 · Report as offensive
Profile Jim Baize
Volunteer tester

Send message
Joined: 6 May 00
Posts: 758
Credit: 149,536
RAC: 0
United States
Message 138382 - Posted: 18 Jul 2005, 18:02:24 UTC - in response to Message 138374.  

Network congestion, server capacity... either way, each request adds to the problem.

Jim

bleh

network congestion does not even appear to be a problem here, all indications point to server capacity.

if the server was keeping up, a 1 minute retry would finish and stop. you would have to wait maybe, what, 30 minutes for your next upload..? are you that impatient?

anyone with more than 6 WU's in cache sees no benefit from 1 minute retries. boinc has to wait for all the previous attempts to time out first.

i think you have more to worry about from people constantly hitting 'retry' than modified clients (if there are even any out there).


ID: 138382 · Report as offensive
Profile Jim Baize
Volunteer tester

Send message
Joined: 6 May 00
Posts: 758
Credit: 149,536
RAC: 0
United States
Message 138386 - Posted: 18 Jul 2005, 18:08:53 UTC - in response to Message 138381.  

Sounds like a decent idea to me.

Another idea that was discussed was for the servers to disable downloads for a small period of time at different times during the day and accept only uploads during this time then go back to the normal ul / dl routine for the rest of the day. One idea was 1 hour 4 times a day. another idea was like 5 minutes each hour, or whatever time scale works.

But again, these are all just ideas. I'm sure the DEV team has a very good plan in place to diagnose and treat the problems as they see them.

Jim

Anyway there might be such clients or not, it doesn't really matter at the moment. The IS a situation like in a DOS-attack and it must be handeled NOW and fast.

I see only one practicable option to get out quickly: Modify the firewall and block some IP address ranges in round robin manner. The first priority should be to lower the amount of request by collecting the outstanding results.
This might be hard, but if nothing is done, I assume a complete breakdown of the network connections within the next hours.

If there anybody has a better idea, spit it out!


ID: 138386 · Report as offensive
TPR_Mojo
Volunteer tester

Send message
Joined: 18 Apr 00
Posts: 323
Credit: 7,001,052
RAC: 0
United Kingdom
Message 138394 - Posted: 18 Jul 2005, 18:16:27 UTC

I've disallowed all new SETI work to the farm for the short term. Wherever the problem lies, adding to it on a daily basis can't help.
ID: 138394 · Report as offensive
timethief

Send message
Joined: 1 Jan 04
Posts: 25
Credit: 545,474
RAC: 0
Germany
Message 138396 - Posted: 18 Jul 2005, 18:17:01 UTC - in response to Message 138386.  

The idea sounds better than mine, the only rub in it seems to be that the number of requests, which blocks the server will not be reduced at the moment.
But it is a good and practicable idea to handle the first time after an outtake.


Sounds like a decent idea to me.

Another idea that was discussed was for the servers to disable downloads for a small period of time at different times during the day and accept only uploads during this time then go back to the normal ul / dl routine for the rest of the day. One idea was 1 hour 4 times a day. another idea was like 5 minutes each hour, or whatever time scale works.

But again, these are all just ideas. I'm sure the DEV team has a very good plan in place to diagnose and treat the problems as they see them.

Jim


ID: 138396 · Report as offensive
Profile Jim Baize
Volunteer tester

Send message
Joined: 6 May 00
Posts: 758
Credit: 149,536
RAC: 0
United States
Message 138398 - Posted: 18 Jul 2005, 18:19:45 UTC - in response to Message 138396.  
Last modified: 18 Jul 2005, 18:22:53 UTC

duplicate post
ID: 138398 · Report as offensive
Profile Jim Baize
Volunteer tester

Send message
Joined: 6 May 00
Posts: 758
Credit: 149,536
RAC: 0
United States
Message 138399 - Posted: 18 Jul 2005, 18:20:02 UTC - in response to Message 138396.  
Last modified: 18 Jul 2005, 18:21:09 UTC

I'm moving my reply over to the UPLOADING II thread.

Jim

The idea sounds better than mine, the only rub in it seems to be that the number of requests, which blocks the server will not be reduced at the moment.
But it is a good and practicable idea to handle the first time after an outtake.


Sounds like a decent idea to me.

Another idea that was discussed was for the servers to disable downloads for a small period of time at different times during the day and accept only uploads during this time then go back to the normal ul / dl routine for the rest of the day. One idea was 1 hour 4 times a day. another idea was like 5 minutes each hour, or whatever time scale works.

But again, these are all just ideas. I'm sure the DEV team has a very good plan in place to diagnose and treat the problems as they see them.

Jim



ID: 138399 · Report as offensive
Profile spacemeat
Avatar

Send message
Joined: 4 Oct 99
Posts: 239
Credit: 8,425,288
RAC: 0
United States
Message 138424 - Posted: 18 Jul 2005, 18:58:49 UTC - in response to Message 138382.  

Network congestion, server capacity... either way, each request adds to the problem.

Jim



no it doesnt. it creates network congestion issues only. if the server can receive work units at a rate greater than they are being completed, the backlog decreases over time for everyone. historically, 2-3 days

more frequent requests probably improves the situation capacity-wise. if the server is ready to accept a new connection, it shouldn't have to wait. more frequent requests will empty the backlog more quickly.

however, if the bandwidth is limiting the server's ability to receive uploads, then the increased requests will make the problem worse. but, as mentioned before, this does not appear to be the problem now. a week after the last downtime, people's caches are getting much larger, indicating the server cannot keep up with production for whatever reason. it drops requests it can't handle. dropping requests due to capacity does not make it run any slower doing what it can handle.
ID: 138424 · Report as offensive
Profile Tigher
Volunteer tester

Send message
Joined: 18 Mar 04
Posts: 1547
Credit: 760,577
RAC: 0
United Kingdom
Message 138425 - Posted: 18 Jul 2005, 19:01:19 UTC
Last modified: 18 Jul 2005, 19:24:00 UTC

Hi spacemeat. We moved over to UPLOADING II btw as this thread is just to long for dial up users.


THANKS to FFEMTcJ for providing and interesting topic!

Cheers

ID: 138425 · Report as offensive
chrisjohnston
Volunteer tester

Send message
Joined: 31 Aug 99
Posts: 385
Credit: 91,410
RAC: 0
United States
Message 138434 - Posted: 18 Jul 2005, 19:14:35 UTC

PLEASE NOTE: THIS THREAD IS CLOSED. UPLOADING II HAS BEEN OPENED.
- cJ

ID: 138434 · Report as offensive
Previous · 1 . . . 12 · 13 · 14 · 15

Message boards : Number crunching : Uploading


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.