Slowly Uploading

Message boards : Number crunching : Slowly Uploading
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · 6 · Next

AuthorMessage
W-K 666 Project Donor
Volunteer tester

Send message
Joined: 18 May 99
Posts: 19048
Credit: 40,757,560
RAC: 67
United Kingdom
Message 161203 - Posted: 1 Sep 2005, 4:45:10 UTC

Yup, I just uploaded 12 in 9 mins, its a good tweak, even if not logical.
Andy
ID: 161203 · Report as offensive
Profile Pappa
Volunteer tester
Avatar

Send message
Joined: 9 Jan 00
Posts: 2562
Credit: 12,301,681
RAC: 0
United States
Message 161206 - Posted: 1 Sep 2005, 4:47:12 UTC - in response to Message 161169.  

ROM

So many people have replied that "this works," it is not nefarious... It is just that there are not enough process/threads/child processes that can be spawned... They are caught in the 300 second network timeout and hand off to the other server that also has a 300 second network timeout... The Proxy has additional threads that can be spawned... So it is not nefarious... It is configration, it is the same IP address that is only handed off...

I don't think it is as nefarious as you all seem to think it is.

Basically the proxy headers are ignored by the web server and the request is processed just like it was any other web request. It just so happens that the cgi directory is mapped to the same location on both servers. So when an HTTP request is made for the file_upload_handler it executes the file_upload_handler on galileo instead of kryten.

The upload/download directories are also mapped to the same locations as well. So things should work.

I have sent mail to Matt and Jeff letting them know what is going on.

Now whether they let it stay that way is another thing.


Please consider a Donation to the Seti Project.

ID: 161206 · Report as offensive
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 161212 - Posted: 1 Sep 2005, 5:05:46 UTC

This works because the scheduler and the upload/download server used to be on the same machine. Now they are split into two. Even though both have the same functionality, one server is doing one thing while the other is doing the other. What decides? DNS. Both can still act as either scheduler or upload/download server...

...until tonight.

Yeah I just switched it off on galileo. People trying this proxy are now getting 404 errors. Not that I'm mad or nothing - This is hardly nefarious activity as it isn't a security risk. But it's putting more traffic on the UCB LAN when it should be on Cogent. Sorry the Cogent pipe is clogged. It'll clear.

- Matt
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 161212 · Report as offensive
W-K 666 Project Donor
Volunteer tester

Send message
Joined: 18 May 99
Posts: 19048
Credit: 40,757,560
RAC: 67
United Kingdom
Message 161216 - Posted: 1 Sep 2005, 5:13:10 UTC

Probably more worrying is that the waiting for validation queue is growing at ~4000/hour and at a time when the Cogent pipe is blocked.
Andy
ID: 161216 · Report as offensive
Profile Rom Walton (BOINC)
Volunteer tester
Avatar

Send message
Joined: 28 Apr 00
Posts: 579
Credit: 130,733
RAC: 0
United States
Message 161219 - Posted: 1 Sep 2005, 5:23:35 UTC - in response to Message 161206.  

ROM

So many people have replied that "this works," it is not nefarious... It is just that there are not enough process/threads/child processes that can be spawned... They are caught in the 300 second network timeout and hand off to the other server that also has a 300 second network timeout... The Proxy has additional threads that can be spawned... So it is not nefarious... It is configration, it is the same IP address that is only handed off...



Others in the thread were making statements to the effect that it was a security issue or bug in the code. It was just how things were configured.

----- Rom
BOINC Development Team, U.C. Berkeley
My Blog
ID: 161219 · Report as offensive
ABT Chuck P
Volunteer tester
Avatar

Send message
Joined: 15 May 99
Posts: 91
Credit: 316,669
RAC: 0
United States
Message 161220 - Posted: 1 Sep 2005, 5:25:39 UTC - in response to Message 161212.  

This works because the scheduler and the upload/download server used to be on the same machine. Now they are split into two. Even though both have the same functionality, one server is doing one thing while the other is doing the other. What decides? DNS. Both can still act as either scheduler or upload/download server...

...until tonight.

Yeah I just switched it off on galileo. People trying this proxy are now getting 404 errors. Not that I'm mad or nothing - This is hardly nefarious activity as it isn't a security risk. But it's putting more traffic on the UCB LAN when it should be on Cogent. Sorry the Cogent pipe is clogged. It'll clear.

- Matt

=====================
Problem is (with very limited exceptions), the Cogent pipe looks like it did for the week we were down.
ID: 161220 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13727
Credit: 208,696,464
RAC: 304
Australia
Message 161222 - Posted: 1 Sep 2005, 5:26:04 UTC


It doesn't look very blocked, or am i looking at the wrong graphs?


gigabitethernet0_0: 66.250.4.74: ucb-cogent p2p



gigabitethernet0_1: 66.28.250.121: cogent ssl net
Grant
Darwin NT
ID: 161222 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13727
Credit: 208,696,464
RAC: 304
Australia
Message 161227 - Posted: 1 Sep 2005, 5:40:31 UTC


Normally after an outage i can't even contact the Schedulers, let alone upload or download work.
At present i've got no problems contacting the Schedulers, i'm just not able to return results (or close enough to not being aable to- only 5% of my results ready to go have been returned in the last 12 hours) & taking lots of retries to download new Work Units to process.
Grant
Darwin NT
ID: 161227 · Report as offensive
W-K 666 Project Donor
Volunteer tester

Send message
Joined: 18 May 99
Posts: 19048
Credit: 40,757,560
RAC: 67
United Kingdom
Message 161228 - Posted: 1 Sep 2005, 5:44:20 UTC - in response to Message 161222.  


It doesn't look very blocked, or am i looking at the wrong graphs?....


I think you're looking at the right graphs, but if my memory is correct, when Boinc/Seti ans Seti Classic both ran normally, wasn't the average closer to 50 M. I believe Classic has been temporarily disconnected to give us more room in the pipe, but I don't believe it is happening if my memory is correct.

I know that you can never get the full amount, gigabit, down the pipe but isn't 30 M ave, 85 M peak a little bit low?

I'm electronics engineer so know nothing about networks, or so I keep getting told.

Andy
ID: 161228 · Report as offensive
Profile StokeyBob
Avatar

Send message
Joined: 31 Aug 03
Posts: 848
Credit: 2,218,691
RAC: 0
United States
Message 161229 - Posted: 1 Sep 2005, 5:44:36 UTC - in response to Message 161222.  
Last modified: 1 Sep 2005, 5:46:16 UTC


It doesn't look very blocked, or am i looking at the wrong graphs?



It looks like something is wrong to me but I don't know much about these things.

See how it is at "30 M" when we were shut down. Now it is up and running and it is back down to "30 M".
ID: 161229 · Report as offensive
Profile StokeyBob
Avatar

Send message
Joined: 31 Aug 03
Posts: 848
Credit: 2,218,691
RAC: 0
United States
Message 161231 - Posted: 1 Sep 2005, 5:50:51 UTC

This may show it better.


ID: 161231 · Report as offensive
Profile Jammy
Avatar

Send message
Joined: 31 May 99
Posts: 16
Credit: 92,828
RAC: 0
Vanuatu
Message 161235 - Posted: 1 Sep 2005, 6:11:34 UTC

I too tried the proxy server ip address and it does not work for me here in San Diego, CA.

Jammy



Team SETI.USA
ID: 161235 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13727
Credit: 208,696,464
RAC: 304
Australia
Message 161236 - Posted: 1 Sep 2005, 6:16:20 UTC - in response to Message 161235.  

I too tried the proxy server ip address and it does not work for me here in San Diego, CA.

If you check half a dozen or so posts up you'll see where Matt says he's disabled that option.

Grant
Darwin NT
ID: 161236 · Report as offensive
Profile Jammy
Avatar

Send message
Joined: 31 May 99
Posts: 16
Credit: 92,828
RAC: 0
Vanuatu
Message 161237 - Posted: 1 Sep 2005, 6:17:44 UTC - in response to Message 161236.  

I too tried the proxy server ip address and it does not work for me here in San Diego, CA.

If you check half a dozen or so posts up you'll see where Matt says he's disabled that option.


Oooo . . .sorry, my faux pas!


Team SETI.USA
ID: 161237 · Report as offensive
Profile Misfit
Volunteer tester
Avatar

Send message
Joined: 21 Jun 01
Posts: 21804
Credit: 2,815,091
RAC: 0
United States
Message 161238 - Posted: 1 Sep 2005, 6:18:31 UTC - in response to Message 161236.  

I too tried the proxy server ip address and it does not work for me here in San Diego, CA.

If you check half a dozen or so posts up you'll see where Matt says he's disabled that option.

Even before it was disabled it still didn't work here. I got 2 thru on upload, but they won't report.
ID: 161238 · Report as offensive
Profile StokeyBob
Avatar

Send message
Joined: 31 Aug 03
Posts: 848
Credit: 2,218,691
RAC: 0
United States
Message 161240 - Posted: 1 Sep 2005, 6:21:41 UTC

Yoinked

ID: 161240 · Report as offensive
Swibby Bear

Send message
Joined: 1 Aug 01
Posts: 246
Credit: 7,945,093
RAC: 0
United States
Message 161242 - Posted: 1 Sep 2005, 6:29:03 UTC

According to the graph, Cogent is clearly not clogged. It obviously can get up to 80 Mbps.

See the deep dip in the graph at 1600 - that's probably when Classic got shut off. Then they opened the Seti-Boinc floodgates right after 1600, and use went up to almost 50. Then the automount problem kicked in, and use dropped at 1800. They got that fixed around 21:30 and use shot up to 85. Then something filled up again, and use came down to 40-45 range and stayed there. I bet Classic was turned back on about midnight, so the level stays around 35-40, even while Boinc is down for Wednesday's backup. When the backup is finished at 11:30, use goes back up to 70 until that elusive something fills up again, and use drops to lowest point and stays there. It appears that Boinc is using only a few mbps on top of Classic because something is filled up and is only letting a dribble through.

That's my armchair analysis of what's happening. Good luck, Matt, finding the bottleneck. By the look of the graph, it sure isn't a clogged Cogent line. When I used the proxy, U/L and D/L went swell. There was no delay when we got through on the camput net, and the WUs were processed right away, suggesting no problems getting WUs through the processes. Could there possibly be another slow switch/router or NIC? Keep smilin' - we are all rooting for you.

Whit
ID: 161242 · Report as offensive
Profile The Gas Giant
Volunteer tester
Avatar

Send message
Joined: 22 Nov 01
Posts: 1904
Credit: 2,646,654
RAC: 0
Australia
Message 161246 - Posted: 1 Sep 2005, 7:06:23 UTC - in response to Message 161242.  

According to the graph, Cogent is clearly not clogged. It obviously can get up to 80 Mbps.

See the deep dip in the graph at 1600 - that's probably when Classic got shut off. Then they opened the Seti-Boinc floodgates right after 1600, and use went up to almost 50. Then the automount problem kicked in, and use dropped at 1800. They got that fixed around 21:30 and use shot up to 85. Then something filled up again, and use came down to 40-45 range and stayed there. I bet Classic was turned back on about midnight, so the level stays around 35-40, even while Boinc is down for Wednesday's backup. When the backup is finished at 11:30, use goes back up to 70 until that elusive something fills up again, and use drops to lowest point and stays there. It appears that Boinc is using only a few mbps on top of Classic because something is filled up and is only letting a dribble through.

That's my armchair analysis of what's happening. Good luck, Matt, finding the bottleneck. By the look of the graph, it sure isn't a clogged Cogent line. When I used the proxy, U/L and D/L went swell. There was no delay when we got through on the camput net, and the WUs were processed right away, suggesting no problems getting WUs through the processes. Could there possibly be another slow switch/router or NIC? Keep smilin' - we are all rooting for you.

Whit

I like your armchair analysis. Wish Matt would look at the graph and come to the same conclusion that the cogent link is not all blocked up. Clearly it ain't. Clearly something in SETI/BOINC land is fubar!
ID: 161246 · Report as offensive
Profile cjsoftuk
Volunteer tester

Send message
Joined: 3 Sep 04
Posts: 248
Credit: 183,721
RAC: 0
United Kingdom
Message 161249 - Posted: 1 Sep 2005, 7:17:43 UTC - in response to Message 161246.  

According to the graph, Cogent is clearly not clogged. It obviously can get up to 80 Mbps.

See the deep dip in the graph at 1600 - that's probably when Classic got shut off. Then they opened the Seti-Boinc floodgates right after 1600, and use went up to almost 50. Then the automount problem kicked in, and use dropped at 1800. They got that fixed around 21:30 and use shot up to 85. Then something filled up again, and use came down to 40-45 range and stayed there. I bet Classic was turned back on about midnight, so the level stays around 35-40, even while Boinc is down for Wednesday's backup. When the backup is finished at 11:30, use goes back up to 70 until that elusive something fills up again, and use drops to lowest point and stays there. It appears that Boinc is using only a few mbps on top of Classic because something is filled up and is only letting a dribble through.

That's my armchair analysis of what's happening. Good luck, Matt, finding the bottleneck. By the look of the graph, it sure isn't a clogged Cogent line. When I used the proxy, U/L and D/L went swell. There was no delay when we got through on the camput net, and the WUs were processed right away, suggesting no problems getting WUs through the processes. Could there possibly be another slow switch/router or NIC? Keep smilin' - we are all rooting for you.

Whit

I like your armchair analysis. Wish Matt would look at the graph and come to the same conclusion that the cogent link is not all blocked up. Clearly it ain't. Clearly something in SETI/BOINC land is fubar!


Like the validator queue again? It's on the rise!

ID: 161249 · Report as offensive
Profile The Gas Giant
Volunteer tester
Avatar

Send message
Joined: 22 Nov 01
Posts: 1904
Credit: 2,646,654
RAC: 0
Australia
Message 161297 - Posted: 1 Sep 2005, 11:03:42 UTC

I noticed that my home machines sometimes start the upload, the whole data file is uploaded but then stalls, times out and goes in to retry mode. Damn almost there.
ID: 161297 · Report as offensive
Previous · 1 · 2 · 3 · 4 · 5 · 6 · Next

Message boards : Number crunching : Slowly Uploading


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.