Codex Alimentarius (Nov 09 2011)

Message boards : Technical News : Codex Alimentarius (Nov 09 2011)
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 1169564 - Posted: 9 Nov 2011, 20:53:50 UTC

Funny story. About 3 years ago I realized that the BOINC database has result ids stored a integers, which are 4 bytes long and signed by default. The sign takes up one bit, thus leaving 31 bits remaining for the value. That means the maximum value is 2^31 (2 to the power of 31, or 2147483648). I mentioned this at this time, noting we were well on our way towards this maximum value, and put it on the "things we'll need to fix eventually" list.

Nobody has been really watching this (I've been pretty much out for over two months until this week), and sure enough we hit that limit yesterday, and the whole BOINC backend pretty much barfed. We tried to implement a "quick fix" by changing the result id signed integer to an unsigned integer (both in mysql and the C code), thus giving us an extra bit for the value. Now that means the maximum value is 2^32 (2 to the power of 32, or 4294967296). That should have bought us a couple more years.

However, this quick fix didn't really work. There's all kinds of code in BOINC that needs to be changed to get unsigned integers to work. Dave made some of these changes and Jeff tested them this morning, but still to no avail. More necessary fixes were found. We seem to be once again creating and sending out work at the moment. However the hood is wide open on BOINC now, so we're watching things carefully over the next day or so.

We're certainly not done - there are tons of cosmetic fixes that need to be made (our logs are full of entries containing negative result ids). In the long term we'll have to do the same for workunit ids, and at that point we'll probably go ahead and make them long longs (which are always 8 bytes, as opposed to longs, which are 4 bytes on 32-bit systems and 8 bytes on 64-bit systems) in the C code and bigints in mysql. At that point our id space will max out at 2305843009213693952, which should probably be enough. That's a million results a day for 6.3 billion years. If we're still running SETI@home 6.3 billion years from now there's probably nobody out there. Agreed?

We've been bitten by this long ago in informix, and have since been storing larger numbers there as int8's (8 byte integers) or doubles.

Warning: since we didn't come across this problem in advance and solve is gracefully, there may be some ugliness in the form of blocked results in weird states - these will most likely time out on their own and get resent. Sorry if this causes any confusion in the coming weeks.

By the way, it should be mentioned there were some random download server issues over this past weekend. No big deal - usual stuff regarding linux kernel hangs. We kicked the servers on monday morning and they went back to work.

- Matt

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 1169564 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1169565 - Posted: 9 Nov 2011, 21:02:18 UTC - in response to Message 1169564.  

Thanks for the update Matt,

Claggy
ID: 1169565 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14672
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1169567 - Posted: 9 Nov 2011, 21:13:41 UTC - in response to Message 1169564.  

Warning: since we didn't come across this problem in advance and solve is gracefully, there may be some ugliness in the form of blocked results in weird states - these will most likely time out on their own and get resent. Sorry if this causes any confusion in the coming weeks.

Matt, sorry you came back off tour and had to wade into this so soon - but in a way it was good that it bit while there was an extra pair of hands to help bail, and at least the bug had the good grace to bite at a reasonably civilised hour and day of the week.

Re the ugly blocked results: we get periodic reports on the boards of workunits in unusual states. Typically, it's two results returned, but both still waiting for validation. Or three results, two of them valid and the third still waiting. Sometimes these clear of their own accord when the original deadline day comes round, sometimes they hang around much longer than that.

Once the string has been tied off, and the sealing wax has set, would it be a good idea - would it be possible - to run the transitioner over the whole database and try to pick off the uglies?
ID: 1169567 · Report as offensive
Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar

Send message
Joined: 20 Dec 05
Posts: 3187
Credit: 57,163,290
RAC: 0
United States
Message 1169606 - Posted: 9 Nov 2011, 23:55:52 UTC

How soon before the scheduler comes back up? - I can't report with the scheduler down - I've got about 18 results waiting on the scheduler... (would'a been more, but I was way down on SETI WU's before this... (except on two computers [out of 4])
.

Hello, from Albany, CA!...
ID: 1169606 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 30922
Credit: 53,134,872
RAC: 32
United States
Message 1169607 - Posted: 10 Nov 2011, 0:01:04 UTC

Thanks for the update. Let us know when the hood goes back down and gets latched.

ID: 1169607 · Report as offensive
Swibby Bear

Send message
Joined: 1 Aug 01
Posts: 246
Credit: 7,945,093
RAC: 0
United States
Message 1169624 - Posted: 10 Nov 2011, 0:55:58 UTC

Will clients (on our crunchers) need to be updated, also?
ID: 1169624 · Report as offensive
Profile zoom3+1=4
Volunteer tester
Avatar

Send message
Joined: 30 Nov 03
Posts: 66201
Credit: 55,293,173
RAC: 49
United States
Message 1169662 - Posted: 10 Nov 2011, 4:51:15 UTC

Well I liked the update, but I won't be posting here soon I think, at least with a RAC that keeps on falling, elsewhere like at Einstein My RAC is going up, and Einstein is set to 0% and S@H to 100%, It seems Seti at Home does not like 3-GTX295 cards in one case, and 6 like cards would be worse more likely, so I may be going elsewhere, like It or not and I don't. And I've done everything I could short of using a PCXT to crunch with, Yes I know what one of those is capable of, I've been around that long.
Savoir-Faire is everywhere!
The T1 Trust, T1 Class 4-4-4-4 #5550, America's First HST

ID: 1169662 · Report as offensive
DrFoo

Send message
Joined: 17 Jul 99
Posts: 26
Credit: 28,975,189
RAC: 0
United States
Message 1169675 - Posted: 10 Nov 2011, 5:57:40 UTC - in response to Message 1169662.  

...short of using a PCXT to crunch with, Yes I know what one of those is capable of, I've been around that long.

You should upgrade that slow Intel chip with a NEC V20!

Sorry, couldn't resist. I've been around a while too. In fact, most of my early work was on the even more ancient Z80A. I rolled my own 32bit multiply and divide routines from scratch on that puppy. Those were the days ...
;)
ID: 1169675 · Report as offensive
Profile zoom3+1=4
Volunteer tester
Avatar

Send message
Joined: 30 Nov 03
Posts: 66201
Credit: 55,293,173
RAC: 49
United States
Message 1169681 - Posted: 10 Nov 2011, 6:40:38 UTC - in response to Message 1169675.  

...short of using a PCXT to crunch with, Yes I know what one of those is capable of, I've been around that long.

You should upgrade that slow Intel chip with a NEC V20!

Sorry, couldn't resist. I've been around a while too. In fact, most of my early work was on the even more ancient Z80A. I rolled my own 32bit multiply and divide routines from scratch on that puppy. Those were the days ...
;)

Well I have an even slower computer, an Atari 1200XL, Only 100,000 were ever made and @ 1.78MHz their the slowest short of an Apple II that could display 256 colors on screen, My 1st computer was an Atari 400 w/48K back in 1980.

Seriously though the DCF here is making Me sink fast and It's like I'm being pulled down by a tractor beam while the Warp Drive is on full reverse.
Savoir-Faire is everywhere!
The T1 Trust, T1 Class 4-4-4-4 #5550, America's First HST

ID: 1169681 · Report as offensive
Richard
Avatar

Send message
Joined: 10 Jul 99
Posts: 19
Credit: 17,341,684
RAC: 0
Argentina
Message 1169739 - Posted: 10 Nov 2011, 14:39:48 UTC

Sorry. I know this is not the correct place to put this comment, but after read the posts of DrFoo and VW Bobier I couldn't resist to comment that I have an old ATARI 800, big as a typewritter, and one ATARI Mega ST, both still working fine. With these machines and a cartidge of Atari Basic began my sohn his informatics carrier. Today he is Master programmer, CCNT, and several titles more I don't remember...
In fact today I still playing with my three 7 to 13 y.o. Grandchildrens those old games like Karateka, Barbarian Fighter, Xenon and others, hand to hand, one joystich each...
And, want you now? Despite his own Core 4 or I3 or I7 machines both 3 are glad to play those old games, also with his Oldfather!
ID: 1169739 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 30922
Credit: 53,134,872
RAC: 32
United States
Message 1169742 - Posted: 10 Nov 2011, 14:49:22 UTC

Just remember when you sort out those negative numbers, two's complement.

ID: 1169742 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 1169926 - Posted: 11 Nov 2011, 1:33:37 UTC - in response to Message 1169739.  

Sorry. I know this is not the correct place to put this comment, but after read the posts of DrFoo and VW Bobier I couldn't resist to comment that I have an old ATARI 800, big as a typewritter, and one ATARI Mega ST, both still working fine. With these machines and a cartidge of Atari Basic began my sohn his informatics carrier. Today he is Master programmer, CCNT, and several titles more I don't remember...
In fact today I still playing with my three 7 to 13 y.o. Grandchildrens those old games like Karateka, Barbarian Fighter, Xenon and others, hand to hand, one joystich each...
And, want you now? Despite his own Core 4 or I3 or I7 machines both 3 are glad to play those old games, also with his Oldfather!

Do you use it for crunching?


BOINC WIKI
ID: 1169926 · Report as offensive
Richard
Avatar

Send message
Joined: 10 Jul 99
Posts: 19
Credit: 17,341,684
RAC: 0
Argentina
Message 1170111 - Posted: 11 Nov 2011, 15:43:14 UTC - in response to Message 1169926.  

quote]
Do you use it for crunching?[/quote]

No,no... I use them only to play with my Grandchildren.

I think if that were possible, crunching a single WU would take several weeks, not to say with Astropulse...
ID: 1170111 · Report as offensive
Profile rebest Project Donor
Volunteer tester
Avatar

Send message
Joined: 16 Apr 00
Posts: 1296
Credit: 45,357,093
RAC: 0
United States
Message 1170596 - Posted: 12 Nov 2011, 21:20:03 UTC - in response to Message 1169675.  

...short of using a PCXT to crunch with, Yes I know what one of those is capable of, I've been around that long.

You should upgrade that slow Intel chip with a NEC V20!


LOL! My office was once stocked with IBM PCs. I swapped out the 8088s with V20s. The staff was amazed how I made their machines so fast.

Damn, I'm getting old.

Join the PACK!
ID: 1170596 · Report as offensive
Clarke J Lipscomb

Send message
Joined: 20 Nov 05
Posts: 6
Credit: 4,930,829
RAC: 0
United States
Message 1171338 - Posted: 16 Nov 2011, 6:32:29 UTC - in response to Message 1169564.  

How can 1970,1980 technology still remain at the front end of modern technogy? 7474's 7476's went by the wayside as the 8-bit bus was developed.
ID: 1171338 · Report as offensive
Profile donaldjj

Send message
Joined: 30 Nov 00
Posts: 19
Credit: 1,429,202
RAC: 0
United States
Message 1171375 - Posted: 16 Nov 2011, 13:53:49 UTC - in response to Message 1171339.  

I had a mitts 680B but never had a monitor or keyboard on it to play games with it as only had 512 BYTES of memory in it. 24 switches and leds for output. It did work for about 20 years though as a house alarm.
ID: 1171375 · Report as offensive
Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar

Send message
Joined: 20 Dec 05
Posts: 3187
Credit: 57,163,290
RAC: 0
United States
Message 1171389 - Posted: 16 Nov 2011, 15:51:23 UTC

Getting back to SETI technical matters, I'm having trouble requesting new tasks, both on production and Beta, getting: "11/16/2011 7:46:34 AM SETI@home Beta Test Scheduler request failed: Failure when receiving data from the peer" almost every time I try: occasionally a Beta request will get through...


.

Hello, from Albany, CA!...
ID: 1171389 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51477
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1171406 - Posted: 16 Nov 2011, 16:15:23 UTC - in response to Message 1171389.  

Getting back to SETI technical matters, I'm having trouble requesting new tasks, both on production and Beta, getting: "11/16/2011 7:46:34 AM SETI@home Beta Test Scheduler request failed: Failure when receiving data from the peer" almost every time I try: occasionally a Beta request will get through...


Check the Cricket graphs, my friend.
Nobody is connecting very well right now.
And once the problem is fixed the bandwidth will rapidly become saturated and connecting will still be difficult.

But right now something is fundamentally broken, and you are in the same boat with the rest of us, it's not on your end.
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 1171406 · Report as offensive
Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar

Send message
Joined: 20 Dec 05
Posts: 3187
Credit: 57,163,290
RAC: 0
United States
Message 1171448 - Posted: 16 Nov 2011, 18:37:54 UTC - in response to Message 1171406.  

Yes, but I thought the staff should be made aware...
.

Hello, from Albany, CA!...
ID: 1171448 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 30922
Credit: 53,134,872
RAC: 32
United States
Message 1171500 - Posted: 16 Nov 2011, 21:47:12 UTC - in response to Message 1171448.  

Yes, but I thought the staff should be made aware...

If there is any connectivity, e-mails to mobile phones are sent.

ID: 1171500 · Report as offensive
1 · 2 · Next

Message boards : Technical News : Codex Alimentarius (Nov 09 2011)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.