QXP-13 Space Modulator (Apr 05 2011)

Message boards : Technical News : QXP-13 Space Modulator (Apr 05 2011)
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 1093673 - Posted: 5 Apr 2011, 22:43:45 UTC

Happy Tuesday! We had our usual outage today (mysql cleanup/backup). The replica mysql server is still having issues. Over the weekend while it was catching up after being rebuilt it hit some corrupted relay log data. This is a bit troublesome - either the logs were corrputed on carolyn (the master) or they got corrupted during to transfer to the replica, or there are still fibre channel issues on this system causing random storage corruption (even after swapping out the entire disk cage, cable, and gbic). I'm rebuilding the replica yet again (with today's backup) and we'll go from there...

Some good news: The entire lab recently upgraded to a gigabit connection to the rest of the campus (and to the world). Actually that was months ago. We weren't seeing much help from this for some reason. Well today we found the bottleneck (one 100Mbit switch) that was constraining the traffic from our server closet. Yay! So now the web site is seeing 1000Mbit to the world instead of a meager 100Mbit. Does it seem snappier? Even more important is our raw data transfers to the offsite archives are vastly sped up, which means less opportunities for the data pipeline to get jammed (and therefore running low on raw data to split). Note this doesn't change our 100MBit limit through Hurricane Electric, which handles are result uploads/workunit downloads. We need to buy some hardware to make that happen, but we may very well eventually move our traffic onto the SSL LAN - this is a political problem more than a technical one at this point.

Over the weekend we had some web servers croak here and there, affecting the home page, workunit downloads, and scheduling. We think this was all due to removing ptolemy from our server mix (the system is powered off and its name and IP address are back in the general pool). Many machines/scripts still have references to ptolemy (or files on ptolemy). We did our best to clean this up before shutting it off, but we knew there would be some minor aches and pains. In this case, a generic web log rotation script was having fits and not killing/restarting apache very well.

- Matt

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 1093673 · Report as offensive
dan
Avatar

Send message
Joined: 18 Oct 02
Posts: 392
Credit: 25,046,383
RAC: 0
United States
Message 1093676 - Posted: 5 Apr 2011, 22:51:10 UTC - in response to Message 1093673.  

Thanks for the update and glad the lab is on the larger pipe.
Dan
ID: 1093676 · Report as offensive
Profile SciManStev Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Jun 99
Posts: 6658
Credit: 121,090,076
RAC: 0
United States
Message 1093677 - Posted: 5 Apr 2011, 22:51:30 UTC
Last modified: 5 Apr 2011, 23:44:30 UTC

It seems you are making excellent progress! Thank you for the update, and I share your joy for getting rid of that 100 Mbit bottleneck. The web pages are fast indeed!

Steve
Warning, addicted to SETI crunching!
Crunching as a member of GPU Users Group.
GPUUG Website
ID: 1093677 · Report as offensive
Swibby Bear

Send message
Joined: 1 Aug 01
Posts: 246
Credit: 7,945,093
RAC: 0
United States
Message 1093684 - Posted: 5 Apr 2011, 23:10:21 UTC - in response to Message 1093673.  
Last modified: 5 Apr 2011, 23:12:34 UTC

Note this doesn't change our 100MBit limit through Hurricane Electric, which handles our result uploads/workunit downloads. We need to buy some hardware to make that happen, but we may very well eventually move our traffic onto the SSL LAN - this is a political problem more than a technical one at this point.

Suggestion: As a way to smooth the political problems and to not totally saturate the gigabit line, should you ever get connected to it, consider sending downloads from one download server thru Hurricane, and the downloads from the other download server (and all the uploads) thru the gigabit connection. That way, the other members of the SSL won't have a major complaint.
Whit
ID: 1093684 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1093704 - Posted: 5 Apr 2011, 23:42:11 UTC - in response to Message 1093673.  

Thanks for the update Matt,

Claggy
ID: 1093704 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51478
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1093713 - Posted: 6 Apr 2011, 0:48:37 UTC
Last modified: 6 Apr 2011, 0:49:46 UTC

Thank you for the good news, Matt.
Swibby Bear's suggestion of splitting the download servers over 2 links is not a bad idea. Both to share the load a bit, and perhaps to provide some redundancy in case one link or the other goes down.
And yup...the forums seem pretty zippy right now!!

Or, if it takes hardware to upgrade the Hurricane link, perhaps you could advise what would be required to make it happen.
The Seti faithful have shown a great response when presented with a specific goal or problem.

Best of luck in sorting the other nagging problems...
I have the best of faith that you will keep playing whack a drive with them until you beat them into submission.

Meow meow!
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 1093713 · Report as offensive
Profile Bill Walker
Avatar

Send message
Joined: 4 Sep 99
Posts: 3868
Credit: 2,697,267
RAC: 0
Canada
Message 1093728 - Posted: 6 Apr 2011, 2:02:16 UTC

Thanks for that Matt. I was always big fan of the QXP-13.

ID: 1093728 · Report as offensive
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 1093952 - Posted: 6 Apr 2011, 16:08:07 UTC

Honestly I'm not exactly sure what the remaining red tape is, but historically UC was paying not by the pipe, but by the bit, which is why they forced us to buy our own connection in the first place (when our solitary upload/download server was accounting for 33% of the entire campus internet traffic). In any case this whole project is moving forward, albeit as time/effort allows.

- Matt
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 1093952 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51478
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1093973 - Posted: 6 Apr 2011, 16:51:40 UTC - in response to Message 1093952.  

Honestly I'm not exactly sure what the remaining red tape is, but historically UC was paying not by the pipe, but by the bit, which is why they forced us to buy our own connection in the first place (when our solitary upload/download server was accounting for 33% of the entire campus internet traffic).

- Matt

I would be amazed if that is still the case....
I wouldn't think in this day and age a Gbit link would come with a cap or be paid for 'by the bit'.
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 1093973 · Report as offensive
Profile perryjay
Volunteer tester
Avatar

Send message
Joined: 20 Aug 02
Posts: 3377
Credit: 20,676,751
RAC: 0
United States
Message 1093997 - Posted: 6 Apr 2011, 17:26:05 UTC - in response to Message 1093673.  

Note this doesn't change our 100MBit limit through Hurricane Electric, which handles are result uploads/workunit downloads. We need to buy some hardware to make that happen, but we may very well eventually move our traffic onto the SSL LAN - this is a political problem more than a technical one at this point.


As Mark already said, just let us know what needs to be bought and about how much you need. I'm sure we will be able to come up with it.


PROUD MEMBER OF Team Starfire World BOINC
ID: 1093997 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1094106 - Posted: 6 Apr 2011, 22:25:06 UTC - in response to Message 1093973.  

Honestly I'm not exactly sure what the remaining red tape is, but historically UC was paying not by the pipe, but by the bit, which is why they forced us to buy our own connection in the first place (when our solitary upload/download server was accounting for 33% of the entire campus internet traffic).

- Matt

I would be amazed if that is still the case....
I wouldn't think in this day and age a Gbit link would come with a cap or be paid for 'by the bit'.

Our 45Gb pipe at work has a fixed monthly amount then we pay by the Gb we go over. Our 10Gb link to our home office in Japan we pay by the Mb. It is done that was as those are the most cost effective solutions for us.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1094106 · Report as offensive
Jesse Viviano

Send message
Joined: 27 Feb 00
Posts: 100
Credit: 3,949,583
RAC: 0
United States
Message 1097137 - Posted: 14 Apr 2011, 16:06:48 UTC

I think that dropping Hurricane Electric is a bad idea because it has been one of the few ISPs that are IPv6 ready for a long time. I cannot find information on how ready UC Berkeley is concerning IPv6, so I think there might be problems coming during the big IPv6 switchover if you dump Hurricane Electric. Also, if you do manage to upgrade the Hurricane Electric connection to gigabit, I would bet that IST would want to use the connection as a failover in case other connections to the internet fail or get saturated, or to start making money by routing traffic between Hurricane Electric and other ISPs that UC Berkeley connects to.
ID: 1097137 · Report as offensive

Message boards : Technical News : QXP-13 Space Modulator (Apr 05 2011)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.