Working as Expected (Jul 13 2009)


log in

Advanced search

Message boards : Technical News : Working as Expected (Jul 13 2009)

Previous · 1 . . . 8 · 9 · 10 · 11
Author Message
Anthony Liggins
Send message
Joined: 23 Aug 99
Posts: 14
Credit: 454,120
RAC: 0
United Kingdom
Message 920487 - Posted: 22 Jul 2009, 23:23:28 UTC - in response to Message 920450.

CUDA is the big culprit in all of this, limiting CUDA would be the best option, untill they can get a Gbit network upgrade, but that could be a long way off.

I do get your point, I am in it for the science not the RAC, and I do agree with you, they are not slow by any means, but sadly most 64 bit processors are running on a 32 bit opperating system; which is a bit of a waste.

A 64 bit processor running in 32 bit mode is still a 64 bit processor, and todays CPU's are uber fast. My old Athlon 64 3700 crunched an average wu in about 3hrs; the OS was 64 bit optomised Linux. Go forward 5 years to today, a DP or QP does the same size wu in arround 30 to 45 mins, include the GPU's and it doen't take long to realise that demand will continue to out strip supply of wu's, and continue to add to the network woe's of S&H.

The servers will continue to struggle to keep up, as demand is high, S@H are going to have to do something pretty radical soon, because the whole system is creaking and is constanly falling over, I have been cruching for 10 years, and I have never ever seen it this bad before.

My servers have 4Gb DDR 266, I am a bit surprised that your file server is not faster than that, considering fsb 800's were the next step up from mine.
____________

OzzFan
Volunteer tester
Avatar
Send message
Joined: 9 Apr 02
Posts: 13583
Credit: 29,963,895
RAC: 16,365
United States
Message 920502 - Posted: 23 Jul 2009, 0:23:02 UTC - in response to Message 920487.

CUDA is the big culprit in all of this, limiting CUDA would be the best option, untill they can get a Gbit network upgrade, but that could be a long way off.


Agreed.

I do get your point, I am in it for the science not the RAC, and I do agree with you, they are not slow by any means, but sadly most 64 bit processors are running on a 32 bit opperating system; which is a bit of a waste.


Perhaps. I think any system needing 64bit OSes have them. 32bit OSes are still put on consumer machines for compatibility reasons, but I'm sure that 64bit will become the norm eventually.

A 64 bit processor running in 32 bit mode is still a 64 bit processor, and todays CPU's are uber fast.


True, but when running in 32bit mode, you lose half the registers in the CPU.

The servers will continue to struggle to keep up, as demand is high, S@H are going to have to do something pretty radical soon, because the whole system is creaking and is constanly falling over, I have been cruching for 10 years, and I have never ever seen it this bad before.


This is the worst I've seen it in my seven years, but I'm told that there were worse outages back with SETI Classic, its only that many didn't notice because WUs back then took so long to crunch on the average machine that the server could be down for nearly a week before people would notice.

My servers have 4Gb DDR 266, I am a bit surprised that your file server is not faster than that, considering fsb 800's were the next step up from mine.


Now that Eric has increased the sensitivity in the applications, its hard to make a direct comparison, but you're right, my times were wrong. Smaller WUs took only 45 min to crunch, while larger ones can take as long as 4-6 hours. It also helps that my Xeons have the 2MB L2 cache too.
____________

Nicolas
Avatar
Send message
Joined: 30 Mar 05
Posts: 160
Credit: 10,335
RAC: 0
Argentina
Message 920556 - Posted: 23 Jul 2009, 3:42:35 UTC - in response to Message 920086.

I agree that the MySQL BOINC database is a serious bottleneck. I wonder how much work it would be to update the code to use a different database system, such as an object-oriented database? Or, perhaps as a first step, could we update the code to use a difference RDBMS such as Oracle or MS SQL?

Or, perhaps, it could use DB transactions correctly, and reach at least 1NF level of table normalization?
____________

Contribute to the Wiki!

Profile HAL9000
Volunteer tester
Avatar
Send message
Joined: 11 Sep 99
Posts: 4087
Credit: 111,868,703
RAC: 148,003
United States
Message 920560 - Posted: 23 Jul 2009, 3:55:20 UTC - in response to Message 920439.

Well that simple, the database will know if a processor is a true 32 or true 64 bit and or 64 bit running in 32 bit mode, this would still be marked as fast, as it displays the processor information and the operating system that a participant is using.

How long does it take your machine to crunch an average wu?

I have 2 retired supermicro servers each one has (2) 3.2Ghz/533 DP xeons, on average it takes about 8hrs to crunch a normal wu, 11hrs for the larger ones, and about 3hrs for the short ones.

Anthony.


I've been running 5 machines in my test lab at work with 32 and 64 bit windows OS's and so far it is looking like 32 bit vista is the fastest OS's to run seti on. This includes Intel, & AMD systems of varying speeds. The AMD systems being slower with their smaller L2 cache. I still have some more checking to do before I finally collect all my data and post my results in another thread about this.

____________
SETI@home classic workunits: 93,865 CPU time: 863,447 hours

Join the BP6/VP6 User Group today!

Profile HAL9000
Volunteer tester
Avatar
Send message
Joined: 11 Sep 99
Posts: 4087
Credit: 111,868,703
RAC: 148,003
United States
Message 920569 - Posted: 23 Jul 2009, 4:23:49 UTC - in response to Message 920502.
Last modified: 23 Jul 2009, 4:24:55 UTC

This is the worst I've seen it in my seven years, but I'm told that there were worse outages back with SETI Classic, its only that many didn't notice because WUs back then took so long to crunch on the average machine that the server could be down for nearly a week before people would notice.


Yeah, The outages back in the classic days were in fact EPIC! Back then I use to run a lot more machines, but there were advantages to how that application worked back then to ride out outages. I had at one time estimated that if it had not been for outages I could have be able to actually do 1,000,000 hours of processing time instead of just around 800,000 something.

I do miss some of those old stats. Where you could see your total CPU time and average CPU time for all the wu's you had done.
____________
SETI@home classic workunits: 93,865 CPU time: 863,447 hours

Join the BP6/VP6 User Group today!

1mp0£173
Volunteer tester
Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 920572 - Posted: 23 Jul 2009, 4:38:02 UTC - in response to Message 920487.

The servers will continue to struggle to keep up, as demand is high, S@H are going to have to do something pretty radical soon, because the whole system is creaking and is constanly falling over, I have been cruching for 10 years, and I have never ever seen it this bad before.

Worth reading.

This isn't the first time they turned up the sensitivity.

I think you're forgetting a pretty epic outage from a few years back -- where things weren't merely overloaded but down hard.
____________

Profile ML1
Volunteer tester
Send message
Joined: 25 Nov 01
Posts: 8377
Credit: 4,107,135
RAC: 1,005
United Kingdom
Message 920642 - Posted: 23 Jul 2009, 12:21:12 UTC - in response to Message 920137.
Last modified: 23 Jul 2009, 12:22:10 UTC

I believe this was a current hot-button topic recently, and I believe Eric himself chimed in and said that it wasn't an issue linking to the cricket graph.

Yikes... My web log has just been swamped by all the hits on the copies of the Cricket graphs I've taken for this thread.

(Though, a simple 'filter out the s@h stuff' is easy enough :-) )

These threads are more popular than you might think...

I still hope that we aren't abusing the Cricket servers with too many hits. Hate to lose that useful resource.

Happy crunchin',
Martin
____________
See new freedom: Mageia4
Linux Voice See & try out your OS Freedom!
The Future is what We make IT (GPLv3)

Anthony Liggins
Send message
Joined: 23 Aug 99
Posts: 14
Credit: 454,120
RAC: 0
United Kingdom
Message 920695 - Posted: 23 Jul 2009, 16:06:12 UTC - in response to Message 920572.

If i remember rightly that was a raid hardware failure that caused that outage, unlike this one which is, more participant lead.
____________

Josef W. SegurProject donor
Volunteer developer
Volunteer tester
Send message
Joined: 30 Oct 99
Posts: 4230
Credit: 1,043,255
RAC: 311
United States
Message 920728 - Posted: 23 Jul 2009, 17:28:33 UTC

On the issue of host speed, IMO there's only one measurement which makes any sense. If the project issues a task and completion is reported 5 days and 2 hours later, it doesn't matter if the host let it sit for 5 days and crunched it in 2 hours or let it sit for 2 hours and took 5 days to crunch it. IOW, if a host's average turnaround time is longer than the project average it's a slow host, otherwise it's fast.

Joe

Nick Fox
Send message
Joined: 5 Jan 04
Posts: 46
Credit: 2,068,813
RAC: 1,052
United Kingdom
Message 920787 - Posted: 23 Jul 2009, 21:29:37 UTC - in response to Message 920728.

Exactly Joe!
____________

clive G1FYE
Volunteer moderator
Send message
Joined: 4 Nov 04
Posts: 1300
Credit: 23,054,144
RAC: 0
United Kingdom
Message 920827 - Posted: 23 Jul 2009, 22:57:38 UTC
Last modified: 23 Jul 2009, 23:04:05 UTC

So the AWUTT value has to be set.
never mind inventing a new acronym :)
Average work unit turnaround time - per host

Though there is the overall unit time on the server page.

Profile ML1
Volunteer tester
Send message
Joined: 25 Nov 01
Posts: 8377
Credit: 4,107,135
RAC: 1,005
United Kingdom
Message 927693 - Posted: 21 Aug 2009, 12:44:59 UTC
Last modified: 21 Aug 2009, 13:03:52 UTC

Could this little device be the solution to the s@h campus fibre woes?

No new fibre required, likely no new routers or any other equipment required. Easy.


Too good to be true?

See:

D-Link 1000BaseT to 1000BaseSX Multimode Media Converter $351 1000BASE-T Gigabit Twisted-pair to 1000BASE-SX Gigabit Fiber Multi-mode Fiber (550m, SC) Media Converter Module • UTP to Multimode Fibre Media Converter • SC Fibre connector • Includes Power Supply for standalone use

DMC-700SC


D-Link 1000BaseT to 1000BaseLX Singlemode Media Converter $1,117 1000BASE-T Gigabit Twisted-pair to 1000BASE-LX Gigabit Fiber Single-mode Fiber (10km, SC) Media Converter Module. • UTP to Singlemode Fibre Media Converter • SC Fibre connector • Distance up to 10km • Includes Power Supply for standalone use

DMC-810SC


One of them at each end gets you 1Gbit/s over 2km of multimode fibre.

That looks to be a drop-in replacement and an awful lot less than $100k!

Happy crunchin',
Martin


Also posted in Number Crunching
____________
See new freedom: Mageia4
Linux Voice See & try out your OS Freedom!
The Future is what We make IT (GPLv3)

Richard HaselgroveProject donor
Volunteer tester
Send message
Joined: 4 Jul 99
Posts: 8466
Credit: 49,000,671
RAC: 73,267
United Kingdom
Message 927713 - Posted: 21 Aug 2009, 14:30:08 UTC - in response to Message 927693.

Gigabit Fiber Multi-mode Fiber (550m, SC) Media Converter Module
Gigabit Fiber Single-mode Fiber (10km, SC) Media Converter Module.

One of them at each end gets you 1Gbit/s over 2km of multimode fibre.

Martin,

I'm not quite sure how you get 2km reach over multi-mode fibre with either of those versions: and I couldn't find the 810SC available in America (your links are ftp transfers of high-res graphics images of the Australian version, which don't play too well with BOINC forum code).

But searching for an alternative led me to Versitron, who do have the GB2MM GBIC advertised for 2km @ 1310nm MM.

Matt,

Is it a trade secret, or could you post sometime and tell us what the technical characteristics of the existing "link down the hill" are? Are we correct to be assuming that it's multi-mode fibre? If so, might it be possible that technical advances since installation could include extended-range media converters like this? Or are Campus specifying a full upgrade to single-mode fibre?

1mp0£173
Volunteer tester
Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 927784 - Posted: 21 Aug 2009, 18:33:34 UTC - in response to Message 927713.

Is it a trade secret, or could you post sometime and tell us what the technical characteristics of the existing "link down the hill" are? Are we correct to be assuming that it's multi-mode fibre? If so, might it be possible that technical advances since installation could include extended-range media converters like this? Or are Campus specifying a full upgrade to single-mode fibre?

I doubt that this is "trade secret" but ultimately, this is a service that SETI@Home receives from campus, and how they do it is up to campus telecom.

IST (the campus department that does this) might see this as an opportunity, especially if they have other users on slower fiber that can benefit.

It sounds like IST is generally supportive, they just have a lot of users to help and priorities that are set from above.
____________

Profile ML1
Volunteer tester
Send message
Joined: 25 Nov 01
Posts: 8377
Credit: 4,107,135
RAC: 1,005
United Kingdom
Message 927818 - Posted: 21 Aug 2009, 21:06:18 UTC - in response to Message 927713.
Last modified: 21 Aug 2009, 21:09:21 UTC

Gigabit Fiber Multi-mode Fiber (550m, SC) Media Converter Module
Gigabit Fiber Single-mode Fiber (10km, SC) Media Converter Module.

One of them at each end gets you 1Gbit/s over 2km of multimode fibre.

Martin,

... I couldn't find the 810SC available in America (your links are ftp transfers of high-res graphics images of the Australian version, which don't play too well with BOINC forum code). ...

(Makes note: Must check any posted links!)

You can find the details on:

D-Link Media Converters

Scroll down the page until you find the Gbit stuff. I only glanced at the prices in a great rush earlier. Never thought that "$" might be Australian!

Even so, the great USA must have them available at the price or less.

Regardless, that solution is a long long long way down from the $100k posted elsewhere previously.

Regards,
Martin


[edit]

The image links are:

ftp://files.dlink.com.au/Products/DMC-700SC/Images/DMC-700SC_(High_Res).jpg

ftp://files.dlink.com.au/Products/DMC-810SC/Images/DMC-810SC_(High_Res).jpg

So how do you put ftp links into these forums?!

[/edit]
____________
See new freedom: Mageia4
Linux Voice See & try out your OS Freedom!
The Future is what We make IT (GPLv3)

DJStarfox
Send message
Joined: 23 May 01
Posts: 1040
Credit: 544,863
RAC: 241
United States
Message 927870 - Posted: 22 Aug 2009, 0:31:20 UTC - in response to Message 927818.

Scroll down the page until you find the Gbit stuff. I only glanced at the prices in a great rush earlier. Never thought that "$" might be Australian!


1.00 AUD = 0.834376 USD as of today's market close. Still that's less than 10% of the original proposed cost if a solution can be implemented.

1mp0£173
Volunteer tester
Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 927914 - Posted: 22 Aug 2009, 3:31:23 UTC - in response to Message 927870.

Scroll down the page until you find the Gbit stuff. I only glanced at the prices in a great rush earlier. Never thought that "$" might be Australian!


1.00 AUD = 0.834376 USD as of today's market close. Still that's less than 10% of the original proposed cost if a solution can be implemented.

It's probably worth referring to this post where Matt explains what they can do, and how much is up to IST (who must incidentally support this later).

The idea that they might be able to squeeze more bits into the same fiber is exciting. Hopefully, someone can pass this on to IST.

IST is pretty SETI-friendly, after all, they did a bunch of work to get the "cheap" bandwidth from Hurricane Electric to the lab through a whole bunch of University facilities -- and they might even be receptive to a "pilot program" to try something new and experimental.
____________

SockGap
Send message
Joined: 16 Apr 07
Posts: 13
Credit: 5,819,564
RAC: 2,777
Australia
Message 927977 - Posted: 22 Aug 2009, 12:55:30 UTC - in response to Message 927713.

Is it a trade secret, or could you post sometime and tell us what the technical characteristics of the existing "link down the hill" are? Are we correct to be assuming that it's multi-mode fibre? If so, might it be possible that technical advances since installation could include extended-range media converters like this? Or are Campus specifying a full upgrade to single-mode fibre?


If it is Multi-Mode then as per Wikipedia "Multi-mode fibre typical transmission speed/distance limits are 100 Mbit/s for distances up to 2 km (100BASE-FX), 1 Gbit/s to 500–600 m (1000BASE-SX)". So at 2km we can only really hope for 100Mbps.

And according to this Fibre Optic Cable Tutorial "in long cable runs (greater than 3000 feet [914.4 meters), multiple paths of light can cause signal distortion at the receiving end, resulting in an unclear and incomplete data transmission so designers now call for single mode fiber in new applications using Gigabit and beyond."

To add confusion there are three types of multi-mode fibre - OM1, OM2 and OM3. If the cable is an older cable it's likely to be OM1 or OM2. OM3 is made from higher quality "glass" whereas OM1 and OM2 are more likely to cause interference. So even if it is possible to push 1Gbps over multi-mode (as per Richard's post) you'd have to have an OM3 cable - and I'd be surprised if this cable will do it.

Jeff

____________

Previous · 1 . . . 8 · 9 · 10 · 11

Message boards : Technical News : Working as Expected (Jul 13 2009)

Copyright © 2014 University of California