QXP-13 Space Modulator (Apr 05 2011)


log in

Advanced search

Message boards : Technical News : QXP-13 Space Modulator (Apr 05 2011)

Author Message
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar
Send message
Joined: 1 Mar 99
Posts: 1390
Credit: 74,079
RAC: 0
United States
Message 1093673 - Posted: 5 Apr 2011, 22:43:45 UTC

Happy Tuesday! We had our usual outage today (mysql cleanup/backup). The replica mysql server is still having issues. Over the weekend while it was catching up after being rebuilt it hit some corrupted relay log data. This is a bit troublesome - either the logs were corrputed on carolyn (the master) or they got corrupted during to transfer to the replica, or there are still fibre channel issues on this system causing random storage corruption (even after swapping out the entire disk cage, cable, and gbic). I'm rebuilding the replica yet again (with today's backup) and we'll go from there...

Some good news: The entire lab recently upgraded to a gigabit connection to the rest of the campus (and to the world). Actually that was months ago. We weren't seeing much help from this for some reason. Well today we found the bottleneck (one 100Mbit switch) that was constraining the traffic from our server closet. Yay! So now the web site is seeing 1000Mbit to the world instead of a meager 100Mbit. Does it seem snappier? Even more important is our raw data transfers to the offsite archives are vastly sped up, which means less opportunities for the data pipeline to get jammed (and therefore running low on raw data to split). Note this doesn't change our 100MBit limit through Hurricane Electric, which handles are result uploads/workunit downloads. We need to buy some hardware to make that happen, but we may very well eventually move our traffic onto the SSL LAN - this is a political problem more than a technical one at this point.

Over the weekend we had some web servers croak here and there, affecting the home page, workunit downloads, and scheduling. We think this was all due to removing ptolemy from our server mix (the system is powered off and its name and IP address are back in the general pool). Many machines/scripts still have references to ptolemy (or files on ptolemy). We did our best to clean this up before shutting it off, but we knew there would be some minor aches and pains. In this case, a generic web log rotation script was having fits and not killing/restarting apache very well.

- Matt

____________
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

dan
Avatar
Send message
Joined: 18 Oct 02
Posts: 392
Credit: 18,735,978
RAC: 2,579
United States
Message 1093676 - Posted: 5 Apr 2011, 22:51:10 UTC - in response to Message 1093673.

Thanks for the update and glad the lab is on the larger pipe.
Dan
____________

Profile SciManStevProject donor
Volunteer tester
Avatar
Send message
Joined: 20 Jun 99
Posts: 4897
Credit: 83,867,876
RAC: 10,996
United States
Message 1093677 - Posted: 5 Apr 2011, 22:51:30 UTC
Last modified: 5 Apr 2011, 23:44:30 UTC

It seems you are making excellent progress! Thank you for the update, and I share your joy for getting rid of that 100 Mbit bottleneck. The web pages are fast indeed!

Steve
____________
Warning, addicted to SETI crunching!
Crunching as a member of GPU Users Group.
GPUUG Website

Swibby Bear
Send message
Joined: 1 Aug 01
Posts: 236
Credit: 7,276,504
RAC: 1
United States
Message 1093684 - Posted: 5 Apr 2011, 23:10:21 UTC - in response to Message 1093673.
Last modified: 5 Apr 2011, 23:12:34 UTC

Note this doesn't change our 100MBit limit through Hurricane Electric, which handles our result uploads/workunit downloads. We need to buy some hardware to make that happen, but we may very well eventually move our traffic onto the SSL LAN - this is a political problem more than a technical one at this point.

Suggestion: As a way to smooth the political problems and to not totally saturate the gigabit line, should you ever get connected to it, consider sending downloads from one download server thru Hurricane, and the downloads from the other download server (and all the uploads) thru the gigabit connection. That way, the other members of the SSL won't have a major complaint.
Whit

ClaggyProject donor
Volunteer tester
Send message
Joined: 5 Jul 99
Posts: 4216
Credit: 34,480,029
RAC: 12,783
United Kingdom
Message 1093704 - Posted: 5 Apr 2011, 23:42:11 UTC - in response to Message 1093673.

Thanks for the update Matt,

Claggy

Profile Bill Walker
Avatar
Send message
Joined: 4 Sep 99
Posts: 3440
Credit: 2,196,367
RAC: 763
Canada
Message 1093728 - Posted: 6 Apr 2011, 2:02:16 UTC

Thanks for that Matt. I was always big fan of the QXP-13.
____________

Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar
Send message
Joined: 1 Mar 99
Posts: 1390
Credit: 74,079
RAC: 0
United States
Message 1093952 - Posted: 6 Apr 2011, 16:08:07 UTC

Honestly I'm not exactly sure what the remaining red tape is, but historically UC was paying not by the pipe, but by the bit, which is why they forced us to buy our own connection in the first place (when our solitary upload/download server was accounting for 33% of the entire campus internet traffic). In any case this whole project is moving forward, albeit as time/effort allows.

- Matt
____________
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

Profile perryjay
Volunteer tester
Avatar
Send message
Joined: 20 Aug 02
Posts: 3377
Credit: 16,212,512
RAC: 4,710
United States
Message 1093997 - Posted: 6 Apr 2011, 17:26:05 UTC - in response to Message 1093673.

Note this doesn't change our 100MBit limit through Hurricane Electric, which handles are result uploads/workunit downloads. We need to buy some hardware to make that happen, but we may very well eventually move our traffic onto the SSL LAN - this is a political problem more than a technical one at this point.


As Mark already said, just let us know what needs to be bought and about how much you need. I'm sure we will be able to come up with it.
____________


PROUD MEMBER OF Team Starfire World BOINC

Profile HAL9000
Volunteer tester
Avatar
Send message
Joined: 11 Sep 99
Posts: 4603
Credit: 121,653,372
RAC: 37,894
United States
Message 1094106 - Posted: 6 Apr 2011, 22:25:06 UTC - in response to Message 1093973.

Honestly I'm not exactly sure what the remaining red tape is, but historically UC was paying not by the pipe, but by the bit, which is why they forced us to buy our own connection in the first place (when our solitary upload/download server was accounting for 33% of the entire campus internet traffic).

- Matt

I would be amazed if that is still the case....
I wouldn't think in this day and age a Gbit link would come with a cap or be paid for 'by the bit'.

Our 45Gb pipe at work has a fixed monthly amount then we pay by the Gb we go over. Our 10Gb link to our home office in Japan we pay by the Mb. It is done that was as those are the most cost effective solutions for us.
____________
SETI@home classic workunits: 93,865 CPU time: 863,447 hours

Join the BP6/VP6 User Group today!

Jesse Viviano
Send message
Joined: 27 Feb 00
Posts: 95
Credit: 474,230
RAC: 0
United States
Message 1097137 - Posted: 14 Apr 2011, 16:06:48 UTC

I think that dropping Hurricane Electric is a bad idea because it has been one of the few ISPs that are IPv6 ready for a long time. I cannot find information on how ready UC Berkeley is concerning IPv6, so I think there might be problems coming during the big IPv6 switchover if you dump Hurricane Electric. Also, if you do manage to upgrade the Hurricane Electric connection to gigabit, I would bet that IST would want to use the connection as a failover in case other connections to the internet fail or get saturated, or to start making money by routing traffic between Hurricane Electric and other ISPs that UC Berkeley connects to.

Message boards : Technical News : QXP-13 Space Modulator (Apr 05 2011)

Copyright © 2014 University of California