BOINC Wish List


log in

Advanced search

Questions and Answers : Wish list : BOINC Wish List

Previous · 1 · 2 · 3 · 4 · 5 · Next
Author Message
Profile Eduardo Bicudo Dreyfuss
Send message
Joined: 20 Jun 05
Posts: 58
Credit: 1,375,653
RAC: 18
Brazil
Message 982645 - Posted: 23 Mar 2010, 2:39:42 UTC - in response to Message 982511.

Maybe size is not the appropriate word, but I meant the required processing time. I use to receive tasks that vary from 1 to 22h processing time.
____________

John McLeod VII
Volunteer developer
Volunteer tester
Avatar
Send message
Joined: 15 Jul 99
Posts: 24093
Credit: 517,832
RAC: 152
United States
Message 982729 - Posted: 23 Mar 2010, 10:58:38 UTC - in response to Message 982645.

Maybe size is not the appropriate word, but I meant the required processing time. I use to receive tasks that vary from 1 to 22h processing time.

There are two things that determine the run length of a SETI or AP task. The first is the angle range, and the ones that are going to take longer on a CPU get a longer deadline to match. Note that there is a known problem with GPUs on some angle ranges. The other thing that determines the run length of a task is the noise in the signal. If it is too noisy, the task quits early. This cannot be determined before the task is completed.
____________


BOINC WIKI

Profile Ageless
Avatar
Send message
Joined: 9 Jun 99
Posts: 12259
Credit: 2,552,534
RAC: 773
Netherlands
Message 982749 - Posted: 23 Mar 2010, 12:38:52 UTC - in response to Message 982729.

Added to that, the project doesn't know the run-time of the tasks. That not only varies per task, but also per piece of hardware (CPU model, CPU brand, GPU model, GPU brand). They will only know after the result has been returned.

If Seti knew up front about the run-time per task, it would mean they ran all that work already, and then the question becomes what the use is that we run it? :-)
____________
Jord

Fighting for the correct use of the apostrophe, together with Weird Al Yankovic

Alex Filatov
Send message
Joined: 8 Feb 10
Posts: 90
Credit: 148,574
RAC: 0
Russia
Message 1003007 - Posted: 11 Jun 2010, 15:31:17 UTC

How do BOINC handle unproper computer power off\force program closing?
I noticed that if power goes down or i need to reset the comp, many of resumed results will be with a error later. It this connected somehow?

Profile Ageless
Avatar
Send message
Joined: 9 Jun 99
Posts: 12259
Credit: 2,552,534
RAC: 773
Netherlands
Message 1003055 - Posted: 11 Jun 2010, 16:26:04 UTC - in response to Message 1003007.

Yes. If BOINC is writing to disc at the time that the computer reboots or loses its power, then that data can get corrupted. Therefore, always try to exit BOINC before you restart the computer. In the case of a power loss, there's not much you can do about it, other than to invest in an uninterruptable power supply (UPS).
____________
Jord

Fighting for the correct use of the apostrophe, together with Weird Al Yankovic

Alex Filatov
Send message
Joined: 8 Feb 10
Posts: 90
Credit: 148,574
RAC: 0
Russia
Message 1003174 - Posted: 11 Jun 2010, 19:48:43 UTC

Ok, understood. Is some kind of transaction mechanism put into "todo" list to avoid such a things? It's very annoying when computer works days to finally get an a error result.

UPS is a good thing, but i cannot afford it right now.

Profile Ageless
Avatar
Send message
Joined: 9 Jun 99
Posts: 12259
Credit: 2,552,534
RAC: 773
Netherlands
Message 1003286 - Posted: 11 Jun 2010, 23:18:58 UTC - in response to Message 1003174.
Last modified: 11 Jun 2010, 23:19:19 UTC

Is some kind of transaction mechanism put into "todo" list to avoid such a things?

No, not as far as I know as it's very difficult (neigh on impossible) to do. Especially on Windows Vista and Windows 7 which have quick shutdowns. So really, teach yourself to exit BOINC completely before shutting down. And as for the power loss, unless you can predict when they're going to happen... ;-)
____________
Jord

Fighting for the correct use of the apostrophe, together with Weird Al Yankovic

Alex Filatov
Send message
Joined: 8 Feb 10
Posts: 90
Credit: 148,574
RAC: 0
Russia
Message 1003595 - Posted: 12 Jun 2010, 12:56:39 UTC
Last modified: 12 Jun 2010, 13:01:40 UTC

Why impossible, every true relation database have such a mechanism. Need just to take engine under GPL or something like license and use to store temporary results. In case of unproper shutdown it just rollbacks to last properly saved savepoint and continue from there.

Profile BilBg
Volunteer tester
Avatar
Send message
Joined: 27 May 07
Posts: 2570
Credit: 5,873,202
RAC: 2,573
Bulgaria
Message 1003780 - Posted: 13 Jun 2010, 20:13:06 UTC - in response to Message 1003595.
Last modified: 13 Jun 2010, 20:34:54 UTC

Why impossible, every true relation database have such a mechanism. Need just to take engine under GPL or something like license and use to store temporary results. In case of unproper shutdown it just rollbacks to last properly saved savepoint and continue from there.


This is done long ago and it is working exactly this way -
by default every minute the state is saved. (I like to set this value to 5 minutes (300 seconds))

But if the power loss happens in the exact same second when the state files are saved they may get corrupted.

You obviously want 2 (or more) checkpoints
SETI app stores one - state.sah
BOINC stores two - client_state_prev.xml and client_state.xml

(and client_state_next.xml during state saving, then renames the files:
Creates client_state_next.xml
Deletes client_state_prev.xml
Renames client_state.xml --> client_state_prev.xml
Renames client_state_next.xml --> client_state.xml
)

(I don't know is client_state_prev.xml used automatically by BOINC if it finds that client_state.xml is corrupted
or the user have to find out the corruption himself and rename the files?)

P.S.
Every program here is under GPL license.


____________



- ALF - "Find out what you don't do well ..... then don't do it!" :)

John McLeod VII
Volunteer developer
Volunteer tester
Avatar
Send message
Joined: 15 Jul 99
Posts: 24093
Credit: 517,832
RAC: 152
United States
Message 1003866 - Posted: 14 Jun 2010, 1:24:11 UTC - in response to Message 1003780.

Why impossible, every true relation database have such a mechanism. Need just to take engine under GPL or something like license and use to store temporary results. In case of unproper shutdown it just rollbacks to last properly saved savepoint and continue from there.


This is done long ago and it is working exactly this way -
by default every minute the state is saved. (I like to set this value to 5 minutes (300 seconds))

But if the power loss happens in the exact same second when the state files are saved they may get corrupted.

You obviously want 2 (or more) checkpoints
SETI app stores one - state.sah
BOINC stores two - client_state_prev.xml and client_state.xml

(and client_state_next.xml during state saving, then renames the files:
Creates client_state_next.xml
Deletes client_state_prev.xml
Renames client_state.xml --> client_state_prev.xml
Renames client_state_next.xml --> client_state.xml
)

(I don't know is client_state_prev.xml used automatically by BOINC if it finds that client_state.xml is corrupted
or the user have to find out the corruption himself and rename the files?)

P.S.
Every program here is under GPL license.


BOINC is supposed to read client_state_prev.xml if client_state.xml is missing or corrupt. HOWEVER, the "if corrupt" part never worked correctly which is why the rename - which is an atomic action on the part of the File system in Windows (and probably other operating systems as well).
____________


BOINC WIKI

Profile Ageless
Avatar
Send message
Joined: 9 Jun 99
Posts: 12259
Credit: 2,552,534
RAC: 773
Netherlands
Message 1003951 - Posted: 14 Jun 2010, 8:11:09 UTC - in response to Message 1003866.

BOINC is supposed to read client_state_prev.xml if client_state.xml is missing or corrupt. HOWEVER, the "if corrupt" part never worked correctly which is why the rename - which is an atomic action on the part of the File system in Windows (and probably other operating systems as well).

Which is why the rename is what, John? Oooh, the suspense is killing me... ;-)

BOINC checks whether or not client_state.xml is corrupt by reading whether it has some expected tags at the start an end of the file. So when a corruption is in the middle, it won't read it. And herein lies a problem, for when BOINC finds that there's nothing wrong with the client_state.xml file, it will immediately make a backup of it to client_state_prev.xml even when the whole middle of the client_state.xml file is corrupt or missing.


____________
Jord

Fighting for the correct use of the apostrophe, together with Weird Al Yankovic

Profile Ageless
Avatar
Send message
Joined: 9 Jun 99
Posts: 12259
Credit: 2,552,534
RAC: 773
Netherlands
Message 1003952 - Posted: 14 Jun 2010, 8:17:34 UTC - in response to Message 1003595.

Why impossible, every true relation database have such a mechanism. Need just to take engine under GPL or something like license and use to store temporary results. In case of unproper shutdown it just rollbacks to last properly saved savepoint and continue from there.

The science applications checkpoint. A checkpoint is a little state point saved to disk. It contains information on where you were in the task at the time of saving.

BOINC has a preference for the minimum write to disk time, called "Tasks checkpoint to disk at most every X seconds". The science application doesn't need to follow this preference when you set a value that's lower than the application's own internal checkpointing time. Then it will follow its own scheme.

So if you set this to 30 seconds and the science application checkpoints once every 65 seconds, it won't follow your 30 seconds value. However, if you set to checkpoint once every 3600 seconds, BOINC will follow that. The problem here however is that when something happens, that you will lose the checkpoints of the past 3600 seconds.
____________
Jord

Fighting for the correct use of the apostrophe, together with Weird Al Yankovic

Profile Pawl
Send message
Joined: 16 May 99
Posts: 2
Credit: 142,248
RAC: 18
Canada
Message 1004065 - Posted: 14 Jun 2010, 16:08:03 UTC

I wish the servers would stop crashing!

Profile Gundolf Jahn
Send message
Joined: 19 Sep 00
Posts: 3184
Credit: 356,858
RAC: 17
Germany
Message 1004068 - Posted: 14 Jun 2010, 16:11:23 UTC - in response to Message 1004065.

I wish the servers would stop crashing!

That's a wish regarding the SETI (or other) project, not BOINC. ;-)

Gruß,
Gundolf

John McLeod VII
Volunteer developer
Volunteer tester
Avatar
Send message
Joined: 15 Jul 99
Posts: 24093
Credit: 517,832
RAC: 152
United States
Message 1004264 - Posted: 15 Jun 2010, 0:05:06 UTC - in response to Message 1003952.

Why impossible, every true relation database have such a mechanism. Need just to take engine under GPL or something like license and use to store temporary results. In case of unproper shutdown it just rollbacks to last properly saved savepoint and continue from there.

The science applications checkpoint. A checkpoint is a little state point saved to disk. It contains information on where you were in the task at the time of saving.

BOINC has a preference for the minimum write to disk time, called "Tasks checkpoint to disk at most every X seconds". The science application doesn't need to follow this preference when you set a value that's lower than the application's own internal checkpointing time. Then it will follow its own scheme.

So if you set this to 30 seconds and the science application checkpoints once every 65 seconds, it won't follow your 30 seconds value. However, if you set to checkpoint once every 3600 seconds, BOINC will follow that. The problem here however is that when something happens, that you will lose the checkpoints of the past 3600 seconds.

The checkpoint at most every X seconds is a request to the application. There are some applications that checkpoint whenever they reach the time to checkpoint whether or not the checkpoint at most eve X seconds is exceeded (CPDN tasks of all sorts do this - but they only check point every few minutes anyway). This setting was put in place because some applications (SETI was the example) were checkpointing every 2 to 3 seconds on some high end systems. BOINC cannot actually enforce the minimum time between checkpoints, it can only suggest the interval to the application.
____________


BOINC WIKI

Alex Filatov
Send message
Joined: 8 Feb 10
Posts: 90
Credit: 148,574
RAC: 0
Russia
Message 1004489 - Posted: 16 Jun 2010, 2:04:19 UTC - in response to Message 1003780.
Last modified: 16 Jun 2010, 2:07:12 UTC


This is done long ago and it is working exactly this way -
by default every minute the state is saved. (I like to set this value to 5 minutes (300 seconds))

But if the power loss happens in the exact same second when the state files are saved they may get corrupted.

You obviously want 2 (or more) checkpoints
SETI app stores one - state.sah
BOINC stores two - client_state_prev.xml and client_state.xml

(and client_state_next.xml during state saving, then renames the files:
Creates client_state_next.xml
Deletes client_state_prev.xml
Renames client_state.xml --> client_state_prev.xml
Renames client_state_next.xml --> client_state.xml
)



Not even close 'exactly'. It's not a DBMS system at all.
DBMS, rougly, appends data to a one file in a binary format:
<data>
<checksum>
<new data entry>
<checksum>
...
<new data entry>
<checksum>
So, if at any of this stage some error occurs, even power off, it will be like

<data>
<checksum>
<new data entry>
<checksum>
...
corrupted data entry>

And it just cuts to prevous checksum.

LarryO
Send message
Joined: 23 Jul 02
Posts: 14
Credit: 511,130
RAC: 0
United States
Message 1026803 - Posted: 20 Aug 2010, 20:19:34 UTC

When I joined up with SETI the object was to use my machines cycles to process SETI data. Now it seems I spend 3 or 4 days a week with no sets to process. Will we ever get back to having data to process all the time? Back in the day I could cache 15 or 20 sets to process and I was relatively immune to problems in Berkley. Will we ever get back to that?

LarryO
____________

John McLeod VII
Volunteer developer
Volunteer tester
Avatar
Send message
Joined: 15 Jul 99
Posts: 24093
Credit: 517,832
RAC: 152
United States
Message 1026859 - Posted: 20 Aug 2010, 22:22:52 UTC - in response to Message 1026803.

When I joined up with SETI the object was to use my machines cycles to process SETI data. Now it seems I spend 3 or 4 days a week with no sets to process. Will we ever get back to having data to process all the time? Back in the day I could cache 15 or 20 sets to process and I was relatively immune to problems in Berkley. Will we ever get back to that?

LarryO

We already have the second. Set "maintain enough extra work for X days" to 4.
____________


BOINC WIKI

LarryO
Send message
Joined: 23 Jul 02
Posts: 14
Credit: 511,130
RAC: 0
United States
Message 1027653 - Posted: 23 Aug 2010, 6:47:49 UTC - in response to Message 1026803.

I was set to 10, but changed to 4, and am set to 0(always connected). Doesn't seem to make any difference. I still only have 2 sets, and they are both working.

LarryO
____________

Profile Ageless
Avatar
Send message
Joined: 9 Jun 99
Posts: 12259
Credit: 2,552,534
RAC: 773
Netherlands
Message 1027654 - Posted: 23 Aug 2010, 7:00:16 UTC - in response to Message 1027653.

Where did you set them? On the web site?
Have you ever looked in on your local preferences and exited that window with the OK button? OK saves those preferences into a file in your BOINC Data directory, after which BOINC will use those preferences, overriding the web preferences.

To check this, please do the following:
Open BOINC Manager->Advanced view->Advanced->Preferences->Click the Clear button.
That will revert you back to using the web preferences.
____________
Jord

Fighting for the correct use of the apostrophe, together with Weird Al Yankovic

Previous · 1 · 2 · 3 · 4 · 5 · Next

Questions and Answers : Wish list : BOINC Wish List

Copyright © 2014 University of California