The Server Issues / Outages Thread - Panic Mode On! (118)

Message boards : Number crunching : The Server Issues / Outages Thread - Panic Mode On! (118)
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 48 · 49 · 50 · 51 · 52 · 53 · 54 . . . 94 · Next

AuthorMessage
Kiska
Volunteer tester

Send message
Joined: 31 Mar 12
Posts: 302
Credit: 3,067,762
RAC: 0
Australia
Message 2029056 - Posted: 24 Jan 2020, 22:21:27 UTC - in response to Message 2029030.  
Last modified: 24 Jan 2020, 22:24:58 UTC

If you can use rrd here it is:
https://transfer.kiska.pw/n2FN6/setiathomev8_creation-creation-g.rrd
That rrd file doesn't contain the 'pure' original values but instead averages of more than one value so it is (nearly) impossible to determine the original values to the last decimal position :(


It does contain what I get off the SSP xml.

Like I've ran:
root@vultr:~# rrdtool lastupdate /var/lib/munin/Munin-Node/Munin-Node-results_setiathomev8_creation-creation-g.rrd | sed ':a;N;$!ba;s/.*: \([0-9]*\)/\1/g'
6.6856


Which is consistent with the value seen in SSP at 22:10 UTC

Here is my configuration for munin
    print "multigraph results_setiathomev8_creation\n";
    print "graph_title setiathome sahv8 Creation Rate\n";
    print "graph_order creation\n";
    print "graph_args --base 1000\n";
    print "graph_vlabel Creation rate per second\n";
    print "update_rate 240\n";
    print "graph_category setiathome\n";
    print "graph_info This graph shows result statistics for setiathome sahv8\n";
    print "creation.info This shows the creation rate for setiathomev8\n";
    print "creation.label creation\n";
    print "creation.type GAUGE\n";
ID: 2029056 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14690
Credit: 200,643,578
RAC: 874
United Kingdom
Message 2029057 - Posted: 24 Jan 2020, 22:26:25 UTC - in response to Message 2029023.  

Also i dont know if this matters, but since the problem started,
i did remove my app_info.xml
Put it back, you'll need it. app_info isn't provided by the S@H website; it was created by the Lunatics installer you presumably ran. You'll need it to get the right apps running, or leave it off and the website will give you "stock" apps.
Best to wait a day or two. There is a new installer almost ready to launch, but I'm waiting until things settle down a bit to finalise testing.
ID: 2029057 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 2029058 - Posted: 24 Jan 2020, 22:32:47 UTC - in response to Message 2029043.  

Unlikely as getting work has still been near impossible much of the time, even when there was some work to get.
And I doubt that things will improve while all these blc35 noise bombs are being spewed out. :-(

Cheers.


+1
ID: 2029058 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 2029059 - Posted: 24 Jan 2020, 22:33:13 UTC - in response to Message 2029041.  

Looks like the splitters have been enabled again. I got some work on a host. Looks like 44 tasks.


. . Nothing here :(

Stephen

:(
ID: 2029059 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 2029061 - Posted: 24 Jan 2020, 22:36:11 UTC - in response to Message 2029052.  

If you can use rrd here it is:
https://transfer.kiska.pw/n2FN6/setiathomev8_creation-creation-g.rrd
I couldn't get exact match because of the averaged noise in the data but the window size that matched the most samples was 299 seconds. Multiples of 299 matched a little bit more but so little that it was just by chance. 149.5 (half of 299) matched about half the number of samples compared to 299 and quarter of 299 about quarter the number of samples. These
are the expected results if 299 is the true value.


. . Try 303 ...

Stephen

. .
ID: 2029061 · Report as offensive
Ville Saari
Avatar

Send message
Joined: 30 Nov 00
Posts: 1158
Credit: 49,177,052
RAC: 82,530
Finland
Message 2029062 - Posted: 24 Jan 2020, 22:37:05 UTC - in response to Message 2029056.  

Which is consistent with the value seen in SSP at 22:10 UTC
Thats a single sample. About half of all the unique sample values in the data I downloaded fitted in the four decimal format. The rest were a lot longer. Some of them obvious averages of a few samples with decimals after 4th all being 3s etc.

How often and at what times are you scraping the data? The rrd has 5 minute time resolution so if you sample more frequenty, it'll probably average the samples. Or if your samples happen at different times than the rrd's regularily spaced 5 minute slots, then it might interpolate between two of your samples to get the value at the point of time it wants.
ID: 2029062 · Report as offensive
Kiska
Volunteer tester

Send message
Joined: 31 Mar 12
Posts: 302
Credit: 3,067,762
RAC: 0
Australia
Message 2029063 - Posted: 24 Jan 2020, 22:42:58 UTC - in response to Message 2029062.  
Last modified: 24 Jan 2020, 22:49:50 UTC

Which is consistent with the value seen in SSP at 22:10 UTC
Thats a single sample. About half of all the unique sample values in the data I downloaded fitted in the four decimal format. The rest were a lot longer. Some of them obvious averages of a few samples with decimals after 4th all being 3s etc.

How often and at what times are you scraping the data? The rrd has 5 minute time resolution so if you sample more frequenty, it'll probably average the samples. Or if your samples happen at different times than the rrd's regularily spaced 5 minute slots, then it might interpolate between two of your samples to get the value at the point of time it wants.


Its sampled once per 5 minutes which matches up with the 5 minute time resolution :D

And at what time... well it runs when it runs :D

EDIT: But it typically runs at :(x)0 and :(x)5
I don't monitor the server every second of my life, so that is probably a generalisation of when it runs
ID: 2029063 · Report as offensive
Ville Saari
Avatar

Send message
Joined: 30 Nov 00
Posts: 1158
Credit: 49,177,052
RAC: 82,530
Finland
Message 2029064 - Posted: 24 Jan 2020, 22:46:59 UTC - in response to Message 2029063.  
Last modified: 24 Jan 2020, 22:49:34 UTC

And at what time... well it runs when it runs :D
And there is your problem. From rrdtool manpage:

RRDtool lets you update the log file at
any time you want. It will automatically interpolate the value
of the data-source (DS) at the latest official time-slot
(interval) and write this interpolated value to the log.

Half of the values were correct because you update at twice the frequency the ssp updates, so you always get two identical samples and interpolating between identical values gives the original value.
ID: 2029064 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 2029066 - Posted: 24 Jan 2020, 23:01:53 UTC

I've been able to get a few tasks on the daily driver, but the majority are the BLC35 "noise bombs"

Thankfully, half a dozen are AP tasks and at least those will run for a while. Even have 3 cpu AP tasks good for a couple of hours.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 2029066 · Report as offensive
Profile petri33
Volunteer tester

Send message
Joined: 6 Jun 02
Posts: 1668
Credit: 623,086,772
RAC: 156
Finland
Message 2029073 - Posted: 24 Jan 2020, 23:52:18 UTC - in response to Message 2028638.  

It could be my "Seti Toaster"

To win a SETI Toaster you need to crunch at least 1 Billion credits. LOL


. . I don't think my current toaster will last that long :(

Stephen

:)


I hope you have had time to collect them/those for a bad toaster day. It is a short resource.
To overcome Heisenbergs:
"You can't always get what you want / but if you try sometimes you just might find / you get what you need." -- Rolling Stones
ID: 2029073 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5126
Credit: 276,046,078
RAC: 462
Message 2029082 - Posted: 25 Jan 2020, 1:20:03 UTC

Fri 24 Jan 2020 07:18:54 PM CST | SETI@home | [sched_op] Server version 709
Fri 24 Jan 2020 07:18:54 PM CST | SETI@home | Project has no tasks available
Fri 24 Jan 2020 07:18:54 PM CST | SETI@home | Project requested delay of 303 seconds

A proud member of the OFA (Old Farts Association).
ID: 2029082 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 2029085 - Posted: 25 Jan 2020, 1:26:06 UTC

The splitters have been turned off the great majority of the day and no work available has been the result. Whatever they did to reorganize the database has given them more issues to deal with other than just reducing the results out in the field or the size of the database.

Unless someone continues to work on the project this weekend, I fear the project won't get going anytime soon until Monday or Tuesday.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 2029085 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 31347
Credit: 53,134,872
RAC: 32
United States
Message 2029086 - Posted: 25 Jan 2020, 1:38:33 UTC - in response to Message 2029085.  

The splitters have been turned off the great majority of the day and no work available has been the result. Whatever they did to reorganize the database has given them more issues to deal with other than just reducing the results out in the field or the size of the database.

Unless someone continues to work on the project this weekend, I fear the project won't get going anytime soon until Monday or Tuesday.

AP splitters are running and one MB splitter is running as of 5-10 ago.
ID: 2029086 · Report as offensive
Dave Stegner
Volunteer tester
Avatar

Send message
Joined: 20 Oct 04
Posts: 540
Credit: 65,583,328
RAC: 27
United States
Message 2029087 - Posted: 25 Jan 2020, 1:43:55 UTC - in response to Message 2029005.  

Unless someone has an idea, I guess I will reset the project and lose the 150 completed work units.

I am still unable to report.

Since the project came back up:

I have 2 machines
1 finally recovered and is getting a trickle of work.
The other

1/24/2020 9:37:26 AM | SETI@home | Fetching scheduler list
1/24/2020 9:37:39 AM | | Project communication failed: attempting access to reference site
1/24/2020 9:37:40 AM | | Internet access OK - project servers may be temporarily down.

Have rebooted, etc.

Cannot even get it to report.

Any ideas ?

Dave

Dave

ID: 2029087 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 2029090 - Posted: 25 Jan 2020, 1:48:23 UTC - in response to Message 2029087.  

Unless someone has an idea, I guess I will reset the project and lose the 150 completed work units.

I am still unable to report.

Since the project came back up:

I have 2 machines
1 finally recovered and is getting a trickle of work.
The other

1/24/2020 9:37:26 AM | SETI@home | Fetching scheduler list
1/24/2020 9:37:39 AM | | Project communication failed: attempting access to reference site
1/24/2020 9:37:40 AM | | Internet access OK - project servers may be temporarily down.

Have rebooted, etc.

Cannot even get it to report.

Any ideas ?

Dave

Set No New Tasks in the Manager to report work.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 2029090 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 2029091 - Posted: 25 Jan 2020, 1:49:54 UTC - in response to Message 2029086.  

AP splitters are running and one MB splitter is running as of 5-10 ago.

I see the AP splitters running. No sign of any MB splitter running though.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 2029091 · Report as offensive
Dave Stegner
Volunteer tester
Avatar

Send message
Joined: 20 Oct 04
Posts: 540
Credit: 65,583,328
RAC: 27
United States
Message 2029093 - Posted: 25 Jan 2020, 1:57:35 UTC - in response to Message 2029090.  

Unless someone has an idea, I guess I will reset the project and lose the 150 completed work units.

I am still unable to report.

Since the project came back up:

I have 2 machines
1 finally recovered and is getting a trickle of work.
The other

1/24/2020 9:37:26 AM | SETI@home | Fetching scheduler list
1/24/2020 9:37:39 AM | | Project communication failed: attempting access to reference site
1/24/2020 9:37:40 AM | | Internet access OK - project servers may be temporarily down.

Have rebooted, etc.

Cannot even get it to report.

Any ideas ?

Dave

Set No New Tasks in the Manager to report work.


Tried that hours ago, no luck.
Dave

ID: 2029093 · Report as offensive
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 2029094 - Posted: 25 Jan 2020, 1:58:58 UTC - in response to Message 2029087.  

Unless someone has an idea, I guess I will reset the project and lose the 150 completed work units.

I am still unable to report....
Sometimes I have to limit the number of tasks reported as well as setting NNT, but, I usually limit it to 240. Try limiting it to 50 and see if it helps.
Place it in the cc_config.xml file,
<max_tasks_reported>50</max_tasks_reported>
ID: 2029094 · Report as offensive
Profile bloodrain
Volunteer tester
Avatar

Send message
Joined: 8 Dec 08
Posts: 231
Credit: 28,112,547
RAC: 1
Antarctica
Message 2029095 - Posted: 25 Jan 2020, 2:10:18 UTC
Last modified: 25 Jan 2020, 2:16:59 UTC

with all 5 pc i got. maybe i got 5 wu...... so atm for it not request over 1k worth of wu. i switch to prime grid.
k a update to a odd bug. if you get a astro pulse now it will run on both gpus at once. but only show up as running on one.
ID: 2029095 · Report as offensive
Dave Stegner
Volunteer tester
Avatar

Send message
Joined: 20 Oct 04
Posts: 540
Credit: 65,583,328
RAC: 27
United States
Message 2029098 - Posted: 25 Jan 2020, 2:21:47 UTC - in response to Message 2029094.  

Unless someone has an idea, I guess I will reset the project and lose the 150 completed work units.

I am still unable to report....
Sometimes I have to limit the number of tasks reported as well as setting NNT, but, I usually limit it to 240. Try limiting it to 50 and see if it helps.
Place it in the cc_config.xml file,
<max_tasks_reported>50</max_tasks_reported>


Set as indicated. Still no luck.

Within seconds of requesting update, project goes to 24 Hour back-off.
Dave

ID: 2029098 · Report as offensive
Previous · 1 . . . 48 · 49 · 50 · 51 · 52 · 53 · 54 . . . 94 · Next

Message boards : Number crunching : The Server Issues / Outages Thread - Panic Mode On! (118)


 
©2025 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.