Codex Alimentarius (Nov 09 2011)


log in

Advanced search

Message boards : Technical News : Codex Alimentarius (Nov 09 2011)

1 · 2 · Next
Author Message
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar
Send message
Joined: 1 Mar 99
Posts: 1391
Credit: 74,079
RAC: 10
United States
Message 1169564 - Posted: 9 Nov 2011, 20:53:50 UTC

Funny story. About 3 years ago I realized that the BOINC database has result ids stored a integers, which are 4 bytes long and signed by default. The sign takes up one bit, thus leaving 31 bits remaining for the value. That means the maximum value is 2^31 (2 to the power of 31, or 2147483648). I mentioned this at this time, noting we were well on our way towards this maximum value, and put it on the "things we'll need to fix eventually" list.

Nobody has been really watching this (I've been pretty much out for over two months until this week), and sure enough we hit that limit yesterday, and the whole BOINC backend pretty much barfed. We tried to implement a "quick fix" by changing the result id signed integer to an unsigned integer (both in mysql and the C code), thus giving us an extra bit for the value. Now that means the maximum value is 2^32 (2 to the power of 32, or 4294967296). That should have bought us a couple more years.

However, this quick fix didn't really work. There's all kinds of code in BOINC that needs to be changed to get unsigned integers to work. Dave made some of these changes and Jeff tested them this morning, but still to no avail. More necessary fixes were found. We seem to be once again creating and sending out work at the moment. However the hood is wide open on BOINC now, so we're watching things carefully over the next day or so.

We're certainly not done - there are tons of cosmetic fixes that need to be made (our logs are full of entries containing negative result ids). In the long term we'll have to do the same for workunit ids, and at that point we'll probably go ahead and make them long longs (which are always 8 bytes, as opposed to longs, which are 4 bytes on 32-bit systems and 8 bytes on 64-bit systems) in the C code and bigints in mysql. At that point our id space will max out at 2305843009213693952, which should probably be enough. That's a million results a day for 6.3 billion years. If we're still running SETI@home 6.3 billion years from now there's probably nobody out there. Agreed?

We've been bitten by this long ago in informix, and have since been storing larger numbers there as int8's (8 byte integers) or doubles.

Warning: since we didn't come across this problem in advance and solve is gracefully, there may be some ugliness in the form of blocked results in weird states - these will most likely time out on their own and get resent. Sorry if this causes any confusion in the coming weeks.

By the way, it should be mentioned there were some random download server issues over this past weekend. No big deal - usual stuff regarding linux kernel hangs. We kicked the servers on monday morning and they went back to work.

- Matt

____________
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

ClaggyProject donor
Volunteer tester
Send message
Joined: 5 Jul 99
Posts: 4249
Credit: 34,988,542
RAC: 21,020
United Kingdom
Message 1169565 - Posted: 9 Nov 2011, 21:02:18 UTC - in response to Message 1169564.

Thanks for the update Matt,

Claggy

Richard HaselgroveProject donor
Volunteer tester
Send message
Joined: 4 Jul 99
Posts: 8830
Credit: 53,651,244
RAC: 49,135
United Kingdom
Message 1169567 - Posted: 9 Nov 2011, 21:13:41 UTC - in response to Message 1169564.

Warning: since we didn't come across this problem in advance and solve is gracefully, there may be some ugliness in the form of blocked results in weird states - these will most likely time out on their own and get resent. Sorry if this causes any confusion in the coming weeks.

Matt, sorry you came back off tour and had to wade into this so soon - but in a way it was good that it bit while there was an extra pair of hands to help bail, and at least the bug had the good grace to bite at a reasonably civilised hour and day of the week.

Re the ugly blocked results: we get periodic reports on the boards of workunits in unusual states. Typically, it's two results returned, but both still waiting for validation. Or three results, two of them valid and the third still waiting. Sometimes these clear of their own accord when the original deadline day comes round, sometimes they hang around much longer than that.

Once the string has been tied off, and the sealing wax has set, would it be a good idea - would it be possible - to run the transitioner over the whole database and try to pick off the uglies?

Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar
Send message
Joined: 20 Dec 05
Posts: 2008
Credit: 11,260,693
RAC: 15,486
United States
Message 1169606 - Posted: 9 Nov 2011, 23:55:52 UTC

How soon before the scheduler comes back up? - I can't report with the scheduler down - I've got about 18 results waiting on the scheduler... (would'a been more, but I was way down on SETI WU's before this... (except on two computers [out of 4])
____________
.

Profile Gary CharpentierProject donor
Volunteer tester
Avatar
Send message
Joined: 25 Dec 00
Posts: 13201
Credit: 7,980,007
RAC: 16,206
United States
Message 1169607 - Posted: 10 Nov 2011, 0:01:04 UTC

Thanks for the update. Let us know when the hood goes back down and gets latched.

____________

Swibby Bear
Send message
Joined: 1 Aug 01
Posts: 236
Credit: 7,276,504
RAC: 1
United States
Message 1169624 - Posted: 10 Nov 2011, 0:55:58 UTC

Will clients (on our crunchers) need to be updated, also?

zoom314Project donor
Volunteer tester
Avatar
Send message
Joined: 30 Nov 03
Posts: 47162
Credit: 37,091,117
RAC: 4,577
United States
Message 1169662 - Posted: 10 Nov 2011, 4:51:15 UTC

Well I liked the update, but I won't be posting here soon I think, at least with a RAC that keeps on falling, elsewhere like at Einstein My RAC is going up, and Einstein is set to 0% and S@H to 100%, It seems Seti at Home does not like 3-GTX295 cards in one case, and 6 like cards would be worse more likely, so I may be going elsewhere, like It or not and I don't. And I've done everything I could short of using a PCXT to crunch with, Yes I know what one of those is capable of, I've been around that long.
____________
My Facebook, War Commander, 2015

DrFoo
Send message
Joined: 17 Jul 99
Posts: 26
Credit: 26,164,638
RAC: 1,427
United States
Message 1169675 - Posted: 10 Nov 2011, 5:57:40 UTC - in response to Message 1169662.

...short of using a PCXT to crunch with, Yes I know what one of those is capable of, I've been around that long.

You should upgrade that slow Intel chip with a NEC V20!

Sorry, couldn't resist. I've been around a while too. In fact, most of my early work was on the even more ancient Z80A. I rolled my own 32bit multiply and divide routines from scratch on that puppy. Those were the days ...
;)
____________

zoom314Project donor
Volunteer tester
Avatar
Send message
Joined: 30 Nov 03
Posts: 47162
Credit: 37,091,117
RAC: 4,577
United States
Message 1169681 - Posted: 10 Nov 2011, 6:40:38 UTC - in response to Message 1169675.

...short of using a PCXT to crunch with, Yes I know what one of those is capable of, I've been around that long.

You should upgrade that slow Intel chip with a NEC V20!

Sorry, couldn't resist. I've been around a while too. In fact, most of my early work was on the even more ancient Z80A. I rolled my own 32bit multiply and divide routines from scratch on that puppy. Those were the days ...
;)

Well I have an even slower computer, an Atari 1200XL, Only 100,000 were ever made and @ 1.78MHz their the slowest short of an Apple II that could display 256 colors on screen, My 1st computer was an Atari 400 w/48K back in 1980.

Seriously though the DCF here is making Me sink fast and It's like I'm being pulled down by a tractor beam while the Warp Drive is on full reverse.
____________
My Facebook, War Commander, 2015

Richard
Avatar
Send message
Joined: 10 Jul 99
Posts: 19
Credit: 9,483,311
RAC: 2,793
Argentina
Message 1169739 - Posted: 10 Nov 2011, 14:39:48 UTC

Sorry. I know this is not the correct place to put this comment, but after read the posts of DrFoo and VW Bobier I couldn't resist to comment that I have an old ATARI 800, big as a typewritter, and one ATARI Mega ST, both still working fine. With these machines and a cartidge of Atari Basic began my sohn his informatics carrier. Today he is Master programmer, CCNT, and several titles more I don't remember...
In fact today I still playing with my three 7 to 13 y.o. Grandchildrens those old games like Karateka, Barbarian Fighter, Xenon and others, hand to hand, one joystich each...
And, want you now? Despite his own Core 4 or I3 or I7 machines both 3 are glad to play those old games, also with his Oldfather!
____________

Profile Gary CharpentierProject donor
Volunteer tester
Avatar
Send message
Joined: 25 Dec 00
Posts: 13201
Credit: 7,980,007
RAC: 16,206
United States
Message 1169742 - Posted: 10 Nov 2011, 14:49:22 UTC

Just remember when you sort out those negative numbers, two's complement.

____________

John McLeod VII
Volunteer developer
Volunteer tester
Avatar
Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 531,285
RAC: 347
United States
Message 1169926 - Posted: 11 Nov 2011, 1:33:37 UTC - in response to Message 1169739.

Sorry. I know this is not the correct place to put this comment, but after read the posts of DrFoo and VW Bobier I couldn't resist to comment that I have an old ATARI 800, big as a typewritter, and one ATARI Mega ST, both still working fine. With these machines and a cartidge of Atari Basic began my sohn his informatics carrier. Today he is Master programmer, CCNT, and several titles more I don't remember...
In fact today I still playing with my three 7 to 13 y.o. Grandchildrens those old games like Karateka, Barbarian Fighter, Xenon and others, hand to hand, one joystich each...
And, want you now? Despite his own Core 4 or I3 or I7 machines both 3 are glad to play those old games, also with his Oldfather!

Do you use it for crunching?
____________


BOINC WIKI

Richard
Avatar
Send message
Joined: 10 Jul 99
Posts: 19
Credit: 9,483,311
RAC: 2,793
Argentina
Message 1170111 - Posted: 11 Nov 2011, 15:43:14 UTC - in response to Message 1169926.

quote]
Do you use it for crunching?[/quote]

No,no... I use them only to play with my Grandchildren.

I think if that were possible, crunching a single WU would take several weeks, not to say with Astropulse...

Profile rebestProject donor
Volunteer tester
Avatar
Send message
Joined: 16 Apr 00
Posts: 1296
Credit: 33,515,321
RAC: 13,034
United States
Message 1170596 - Posted: 12 Nov 2011, 21:20:03 UTC - in response to Message 1169675.

...short of using a PCXT to crunch with, Yes I know what one of those is capable of, I've been around that long.

You should upgrade that slow Intel chip with a NEC V20!


LOL! My office was once stocked with IBM PCs. I swapped out the 8088s with V20s. The staff was amazed how I made their machines so fast.

Damn, I'm getting old.
____________

Join the PACK!

Profile Chris SProject donor
Volunteer tester
Avatar
Send message
Joined: 19 Nov 00
Posts: 32656
Credit: 14,556,014
RAC: 15,082
United Kingdom
Message 1170716 - Posted: 13 Nov 2011, 12:32:48 UTC

Thanks for the update Matt.

If we're still running SETI@home 6.3 billion years from now there's probably nobody out there. Agreed?


Agreed :-)

Clarke J Lipscomb
Send message
Joined: 20 Nov 05
Posts: 6
Credit: 4,884,080
RAC: 32
United States
Message 1171338 - Posted: 16 Nov 2011, 6:32:29 UTC - in response to Message 1169564.

How can 1970,1980 technology still remain at the front end of modern technogy? 7474's 7476's went by the wayside as the 8-bit bus was developed.
____________

Profile donaldjj
Send message
Joined: 30 Nov 00
Posts: 19
Credit: 1,429,202
RAC: 66
United States
Message 1171375 - Posted: 16 Nov 2011, 13:53:49 UTC - in response to Message 1171339.

I had a mitts 680B but never had a monitor or keyboard on it to play games with it as only had 512 BYTES of memory in it. 24 switches and leds for output. It did work for about 20 years though as a house alarm.
____________

Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar
Send message
Joined: 20 Dec 05
Posts: 2008
Credit: 11,260,693
RAC: 15,486
United States
Message 1171389 - Posted: 16 Nov 2011, 15:51:23 UTC

Getting back to SETI technical matters, I'm having trouble requesting new tasks, both on production and Beta, getting: "11/16/2011 7:46:34 AM SETI@home Beta Test Scheduler request failed: Failure when receiving data from the peer" almost every time I try: occasionally a Beta request will get through...


____________
.

Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar
Send message
Joined: 20 Dec 05
Posts: 2008
Credit: 11,260,693
RAC: 15,486
United States
Message 1171448 - Posted: 16 Nov 2011, 18:37:54 UTC - in response to Message 1171406.

Yes, but I thought the staff should be made aware...
____________
.

1 · 2 · Next

Message boards : Technical News : Codex Alimentarius (Nov 09 2011)

Copyright © 2014 University of California