Working as Expected (Jul 13 2009)

Message boards : Technical News : Working as Expected (Jul 13 2009)
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 8 · 9 · 10 · 11

AuthorMessage
Nicolas
Avatar

Send message
Joined: 30 Mar 05
Posts: 161
Credit: 12,985
RAC: 0
Argentina
Message 920556 - Posted: 23 Jul 2009, 3:42:35 UTC - in response to Message 920086.  

I agree that the MySQL BOINC database is a serious bottleneck. I wonder how much work it would be to update the code to use a different database system, such as an object-oriented database? Or, perhaps as a first step, could we update the code to use a difference RDBMS such as Oracle or MS SQL?

Or, perhaps, it could use DB transactions correctly, and reach at least 1NF level of table normalization?

Contribute to the Wiki!
ID: 920556 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 920560 - Posted: 23 Jul 2009, 3:55:20 UTC - in response to Message 920439.  

Well that simple, the database will know if a processor is a true 32 or true 64 bit and or 64 bit running in 32 bit mode, this would still be marked as fast, as it displays the processor information and the operating system that a participant is using.

How long does it take your machine to crunch an average wu?

I have 2 retired supermicro servers each one has (2) 3.2Ghz/533 DP xeons, on average it takes about 8hrs to crunch a normal wu, 11hrs for the larger ones, and about 3hrs for the short ones.

Anthony.


I've been running 5 machines in my test lab at work with 32 and 64 bit windows OS's and so far it is looking like 32 bit vista is the fastest OS's to run seti on. This includes Intel, & AMD systems of varying speeds. The AMD systems being slower with their smaller L2 cache. I still have some more checking to do before I finally collect all my data and post my results in another thread about this.

SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 920560 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 920569 - Posted: 23 Jul 2009, 4:23:49 UTC - in response to Message 920502.  
Last modified: 23 Jul 2009, 4:24:55 UTC

This is the worst I've seen it in my seven years, but I'm told that there were worse outages back with SETI Classic, its only that many didn't notice because WUs back then took so long to crunch on the average machine that the server could be down for nearly a week before people would notice.


Yeah, The outages back in the classic days were in fact EPIC! Back then I use to run a lot more machines, but there were advantages to how that application worked back then to ride out outages. I had at one time estimated that if it had not been for outages I could have be able to actually do 1,000,000 hours of processing time instead of just around 800,000 something.

I do miss some of those old stats. Where you could see your total CPU time and average CPU time for all the wu's you had done.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 920569 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 920572 - Posted: 23 Jul 2009, 4:38:02 UTC - in response to Message 920487.  

The servers will continue to struggle to keep up, as demand is high, S@H are going to have to do something pretty radical soon, because the whole system is creaking and is constanly falling over, I have been cruching for 10 years, and I have never ever seen it this bad before.

Worth reading.

This isn't the first time they turned up the sensitivity.

I think you're forgetting a pretty epic outage from a few years back -- where things weren't merely overloaded but down hard.
ID: 920572 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 20084
Credit: 7,508,002
RAC: 20
United Kingdom
Message 920642 - Posted: 23 Jul 2009, 12:21:12 UTC - in response to Message 920137.  
Last modified: 23 Jul 2009, 12:22:10 UTC

I believe this was a current hot-button topic recently, and I believe Eric himself chimed in and said that it wasn't an issue linking to the cricket graph.

Yikes... My web log has just been swamped by all the hits on the copies of the Cricket graphs I've taken for this thread.

(Though, a simple 'filter out the s@h stuff' is easy enough :-) )

These threads are more popular than you might think...

I still hope that we aren't abusing the Cricket servers with too many hits. Hate to lose that useful resource.

Happy crunchin',
Martin
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 920642 · Report as offensive
Anthony Liggins

Send message
Joined: 23 Aug 99
Posts: 14
Credit: 609,816
RAC: 0
United Kingdom
Message 920695 - Posted: 23 Jul 2009, 16:06:12 UTC - in response to Message 920572.  

If i remember rightly that was a raid hardware failure that caused that outage, unlike this one which is, more participant lead.
ID: 920695 · Report as offensive
Josef W. Segur
Volunteer developer
Volunteer tester

Send message
Joined: 30 Oct 99
Posts: 4504
Credit: 1,414,761
RAC: 0
United States
Message 920728 - Posted: 23 Jul 2009, 17:28:33 UTC

On the issue of host speed, IMO there's only one measurement which makes any sense. If the project issues a task and completion is reported 5 days and 2 hours later, it doesn't matter if the host let it sit for 5 days and crunched it in 2 hours or let it sit for 2 hours and took 5 days to crunch it. IOW, if a host's average turnaround time is longer than the project average it's a slow host, otherwise it's fast.
                                                                Joe
ID: 920728 · Report as offensive
Nick Fox

Send message
Joined: 5 Jan 04
Posts: 46
Credit: 2,834,922
RAC: 0
United Kingdom
Message 920787 - Posted: 23 Jul 2009, 21:29:37 UTC - in response to Message 920728.  

Exactly Joe!
ID: 920787 · Report as offensive
.clair.

Send message
Joined: 4 Nov 04
Posts: 1300
Credit: 55,390,408
RAC: 69
United Kingdom
Message 920827 - Posted: 23 Jul 2009, 22:57:38 UTC
Last modified: 23 Jul 2009, 23:04:05 UTC

So the AWUTT value has to be set.
never mind inventing a new acronym :)
Average work unit turnaround time - per host

Though there is the overall unit time on the server page.
ID: 920827 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 20084
Credit: 7,508,002
RAC: 20
United Kingdom
Message 927693 - Posted: 21 Aug 2009, 12:44:59 UTC
Last modified: 21 Aug 2009, 13:03:52 UTC

Could this little device be the solution to the s@h campus fibre woes?

No new fibre required, likely no new routers or any other equipment required. Easy.


Too good to be true?

See:

D-Link 1000BaseT  to 1000BaseSX Multimode Media Converter  	 	$351

	1000BASE-T Gigabit Twisted-pair to 1000BASE-SX
	Gigabit Fiber Multi-mode Fiber (550m, SC) Media Converter Module 	

• UTP to Multimode Fibre Media Converter
• SC Fibre connector 	• Includes Power Supply for standalone use

DMC-700SC


D-Link 1000BaseT  to 1000BaseLX Singlemode Media Converter  	 	$1,117

	1000BASE-T Gigabit Twisted-pair to 1000BASE-LX
	Gigabit Fiber Single-mode Fiber (10km, SC) Media Converter Module. 	

• UTP to Singlemode Fibre Media Converter
• SC Fibre connector
• Distance up to 10km 	• Includes Power Supply for standalone use

DMC-810SC


One of them at each end gets you 1Gbit/s over 2km of multimode fibre.

That looks to be a drop-in replacement and an awful lot less than $100k!

Happy crunchin',
Martin


Also posted in Number Crunching
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 927693 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14644
Credit: 200,643,578
RAC: 874
United Kingdom
Message 927713 - Posted: 21 Aug 2009, 14:30:08 UTC - in response to Message 927693.  

Gigabit Fiber Multi-mode Fiber (550m, SC) Media Converter Module
Gigabit Fiber Single-mode Fiber (10km, SC) Media Converter Module.

One of them at each end gets you 1Gbit/s over 2km of multimode fibre.

Martin,

I'm not quite sure how you get 2km reach over multi-mode fibre with either of those versions: and I couldn't find the 810SC available in America (your links are ftp transfers of high-res graphics images of the Australian version, which don't play too well with BOINC forum code).

But searching for an alternative led me to Versitron, who do have the GB2MM GBIC advertised for 2km @ 1310nm MM.

Matt,

Is it a trade secret, or could you post sometime and tell us what the technical characteristics of the existing "link down the hill" are? Are we correct to be assuming that it's multi-mode fibre? If so, might it be possible that technical advances since installation could include extended-range media converters like this? Or are Campus specifying a full upgrade to single-mode fibre?
ID: 927713 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 927784 - Posted: 21 Aug 2009, 18:33:34 UTC - in response to Message 927713.  

Is it a trade secret, or could you post sometime and tell us what the technical characteristics of the existing "link down the hill" are? Are we correct to be assuming that it's multi-mode fibre? If so, might it be possible that technical advances since installation could include extended-range media converters like this? Or are Campus specifying a full upgrade to single-mode fibre?

I doubt that this is "trade secret" but ultimately, this is a service that SETI@Home receives from campus, and how they do it is up to campus telecom.

IST (the campus department that does this) might see this as an opportunity, especially if they have other users on slower fiber that can benefit.

It sounds like IST is generally supportive, they just have a lot of users to help and priorities that are set from above.
ID: 927784 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 20084
Credit: 7,508,002
RAC: 20
United Kingdom
Message 927818 - Posted: 21 Aug 2009, 21:06:18 UTC - in response to Message 927713.  
Last modified: 21 Aug 2009, 21:09:21 UTC

Gigabit Fiber Multi-mode Fiber (550m, SC) Media Converter Module
Gigabit Fiber Single-mode Fiber (10km, SC) Media Converter Module.

One of them at each end gets you 1Gbit/s over 2km of multimode fibre.

Martin,

... I couldn't find the 810SC available in America (your links are ftp transfers of high-res graphics images of the Australian version, which don't play too well with BOINC forum code). ...

(Makes note: Must check any posted links!)

You can find the details on:

D-Link Media Converters

Scroll down the page until you find the Gbit stuff. I only glanced at the prices in a great rush earlier. Never thought that "$" might be Australian!

Even so, the great USA must have them available at the price or less.

Regardless, that solution is a long long long way down from the $100k posted elsewhere previously.

Regards,
Martin


[edit]

The image links are:

ftp://files.dlink.com.au/Products/DMC-700SC/Images/DMC-700SC_(High_Res).jpg

ftp://files.dlink.com.au/Products/DMC-810SC/Images/DMC-810SC_(High_Res).jpg

So how do you put ftp links into these forums?!

[/edit]
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 927818 · Report as offensive
DJStarfox

Send message
Joined: 23 May 01
Posts: 1066
Credit: 1,226,053
RAC: 2
United States
Message 927870 - Posted: 22 Aug 2009, 0:31:20 UTC - in response to Message 927818.  

Scroll down the page until you find the Gbit stuff. I only glanced at the prices in a great rush earlier. Never thought that "$" might be Australian!


1.00 AUD = 0.834376 USD as of today's market close. Still that's less than 10% of the original proposed cost if a solution can be implemented.
ID: 927870 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 927914 - Posted: 22 Aug 2009, 3:31:23 UTC - in response to Message 927870.  

Scroll down the page until you find the Gbit stuff. I only glanced at the prices in a great rush earlier. Never thought that "$" might be Australian!


1.00 AUD = 0.834376 USD as of today's market close. Still that's less than 10% of the original proposed cost if a solution can be implemented.

It's probably worth referring to this post where Matt explains what they can do, and how much is up to IST (who must incidentally support this later).

The idea that they might be able to squeeze more bits into the same fiber is exciting. Hopefully, someone can pass this on to IST.

IST is pretty SETI-friendly, after all, they did a bunch of work to get the "cheap" bandwidth from Hurricane Electric to the lab through a whole bunch of University facilities -- and they might even be receptive to a "pilot program" to try something new and experimental.
ID: 927914 · Report as offensive
SockGap

Send message
Joined: 16 Apr 07
Posts: 14
Credit: 7,700,416
RAC: 0
Australia
Message 927977 - Posted: 22 Aug 2009, 12:55:30 UTC - in response to Message 927713.  

Is it a trade secret, or could you post sometime and tell us what the technical characteristics of the existing "link down the hill" are? Are we correct to be assuming that it's multi-mode fibre? If so, might it be possible that technical advances since installation could include extended-range media converters like this? Or are Campus specifying a full upgrade to single-mode fibre?


If it is Multi-Mode then as per Wikipedia "Multi-mode fibre typical transmission speed/distance limits are 100 Mbit/s for distances up to 2 km (100BASE-FX), 1 Gbit/s to 500–600 m (1000BASE-SX)". So at 2km we can only really hope for 100Mbps.

And according to this Fibre Optic Cable Tutorial "in long cable runs (greater than 3000 feet [914.4 meters), multiple paths of light can cause signal distortion at the receiving end, resulting in an unclear and incomplete data transmission so designers now call for single mode fiber in new applications using Gigabit and beyond."

To add confusion there are three types of multi-mode fibre - OM1, OM2 and OM3. If the cable is an older cable it's likely to be OM1 or OM2. OM3 is made from higher quality "glass" whereas OM1 and OM2 are more likely to cause interference. So even if it is possible to push 1Gbps over multi-mode (as per Richard's post) you'd have to have an OM3 cable - and I'd be surprised if this cable will do it.

Jeff

ID: 927977 · Report as offensive
Previous · 1 . . . 8 · 9 · 10 · 11

Message boards : Technical News : Working as Expected (Jul 13 2009)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.