Panic Mode On (105) Server Problems?

Message boards : Number crunching : Panic Mode On (105) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 24 · 25 · 26 · 27 · 28 · 29 · 30 . . . 34 · Next

AuthorMessage
Profile Zalster Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 5517
Credit: 528,817,460
RAC: 242
United States
Message 1862248 - Posted: 18 Apr 2017, 19:34:56 UTC - in response to Message 1862246.  

Yup..
ID: 1862248 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 34862
Credit: 261,360,520
RAC: 489
Australia
Message 1862311 - Posted: 19 Apr 2017, 7:30:00 UTC

Well if that was an outrage then there should be more like them. though I did run out of GPU work again on my main 2 rigs.

Cheers.
ID: 1862311 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1862596 - Posted: 21 Apr 2017, 3:28:49 UTC
Last modified: 21 Apr 2017, 3:34:50 UTC

Currently have been in a Arecibo VLAR storm for the past couple of hours. Been unable to maintain my allotment of GPU tasks with the restrictions on sending to Nvidia GPU's. So, my question is why are we restricting Arecibo VLAR being sent to Nvidia GPU's but it is entirely OK to send BLC VLAR to Nvidia GPU's. Same low angle ranges. What's the difference?
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1862596 · Report as offensive
Profile Zalster Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 5517
Credit: 528,817,460
RAC: 242
United States
Message 1862599 - Posted: 21 Apr 2017, 3:36:24 UTC - in response to Message 1862596.  

Currently have been in a Arecibo VLAR storm for the past couple of hours. Been unable to maintain my allotment of GPU tasks with the restrictions on sending to Nvidia GPU's. So, my question is why are we restricting VLAR being sent to Nvidia GPU's but it is entirely OK to send BLC VLAR to Nvidia GPU's. Same low angle ranges. What's the difference?


This is from memory so I may get this wrong. Originally the VLARs of Arecibo origin caused lock ups of the Nvidia way back. So they restricted those away from the GPUs. The VLARs of BLC resemble VLAR but it has something to do with the size of the packet?? (again fuzzy memory) so technically they aren't really VLARS. Since the SoG was designed with this new parameter in mind, the restriction was removed to allow the NVIDIA GPUs to crunch these new work units.
ID: 1862599 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1862606 - Posted: 21 Apr 2017, 3:50:29 UTC

Has anybody tested Arecibo VLAR's on the SoG app at Beta for example? Maybe the SoG app doesn't have issues with the Arecibo VLAR's like it used to have with the CUDA apps. I really would like the restriction removed if possible.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1862606 · Report as offensive
Profile Zalster Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 5517
Credit: 528,817,460
RAC: 242
United States
Message 1862610 - Posted: 21 Apr 2017, 3:55:35 UTC - in response to Message 1862606.  

Raistmer would know. You could try sending a PM and see what he says.
ID: 1862610 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1862612 - Posted: 21 Apr 2017, 4:01:00 UTC - in response to Message 1862610.  

Raistmer would know. You could try sending a PM and see what he says.

Done. Will await any answer.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1862612 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1862616 - Posted: 21 Apr 2017, 4:07:17 UTC

I looked over at Beta in the 8.21 and 8.22 threads but have no idea what kind of tasks they were benching since all the links to the tasks are expired. Is there anyway to find out what type they were?
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1862616 · Report as offensive
Profile Zalster Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 5517
Credit: 528,817,460
RAC: 242
United States
Message 1862617 - Posted: 21 Apr 2017, 4:13:47 UTC - in response to Message 1862616.  

Last time I check on any Beta work, they were trying to tailor the SoG for low end GPUs. I didn't do much of those work units since I gave away my last 750 some time back. There was a thought to release uncompressed work units and we tested those. No Problem but they decided to continue with current sized work units.
ID: 1862617 · Report as offensive
Profile Zalster Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 5517
Credit: 528,817,460
RAC: 242
United States
Message 1862622 - Posted: 21 Apr 2017, 4:51:10 UTC - in response to Message 1862616.  

I looked over at Beta in the 8.21 and 8.22 threads but have no idea what kind of tasks they were benching since all the links to the tasks are expired. Is there anyway to find out what type they were?


Ok, been so long that it took me a bit to figure out what they were talking about, lol... 8.19 was the original SoG released to Main. Commandlines were not used widely yet. ( I was using some but hadn't shared yet what I was using)

8.21 and 8.22 were the revised apps with testing of different commandlines for the apps with slightly lower gpus (ie 950 in this case). With each revision they were able to adapt the app for how much Memory was on each GPU. Most of it was testing effects of the -sbs, -tt, -high prefer, period_iterations_number etc..

I already knew what the affect were on my 980Tis (those posted results are buried somewhere in the Seti@home v8 beta to begin on Tuesday thread) but they needed to see how it worked on other cards. They also wanted to see effect with GUI .

Zalster
ID: 1862622 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1862641 - Posted: 21 Apr 2017, 6:35:40 UTC

Has anybody gotten a BLC task in the last 3 or 4 hours? I haven't across all three machines. Are the BLC splitters actually doing anything? I'm down to under a 100 GPU tasks on Numbskull. Down about 30 or so on another machine. Lots of

SETI@home 4/20/2017 11:29:12 PM Requesting new tasks for CPU and NVIDIA GPU
SETI@home 4/20/2017 11:29:21 PM Scheduler request completed: got 0 new tasks
SETI@home 4/20/2017 11:29:21 PM Project has no tasks available

across all three crunchers. I've done the toggle bit in Preferences to no effect. I don't think this is a case of the old bugaboo. I just think that there is nothing but Arecibo VLAR coming out of the splitters.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1862641 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13746
Credit: 208,696,464
RAC: 304
Australia
Message 1862644 - Posted: 21 Apr 2017, 7:04:46 UTC
Last modified: 21 Apr 2017, 7:07:32 UTC

Just had a look at my systems.
The slower system is all Arecibo. The faster one has picked up 9 GBT tasks, and 6 of those are resends.
So i'd say (regardless of what the Server Status page might show) is that next to 0 GTB work is being split.

EDIT- and since installing the AP application, my v8 cache remains full.
It would be nice if they could sort out the Application preference work allocation issue, till then this kludge appears to be working.
Grant
Darwin NT
ID: 1862644 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13746
Credit: 208,696,464
RAC: 304
Australia
Message 1862645 - Posted: 21 Apr 2017, 7:09:59 UTC - in response to Message 1862596.  

... why are we restricting Arecibo VLAR being sent to Nvidia GPU's but it is entirely OK to send BLC VLAR to Nvidia GPU's. Same low angle ranges. What's the difference?

Arecibo VLARs can bring a system to it's knees, GBT VLARs take forever to crunch, but don't make systems non responsive.
Grant
Darwin NT
ID: 1862645 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1862647 - Posted: 21 Apr 2017, 7:21:11 UTC - in response to Message 1862645.  

... why are we restricting Arecibo VLAR being sent to Nvidia GPU's but it is entirely OK to send BLC VLAR to Nvidia GPU's. Same low angle ranges. What's the difference?

Arecibo VLARs can bring a system to it's knees, GBT VLARs take forever to crunch, but don't make systems non responsive.


I'm actually looking for GBT VLAR's to crunch. I want them for the Ryzen CPU since they crunch the fastest. I actively reschedule the BLC's that get assigned to the GPU's over to the CPU and take whatever normal Arecibo tasks that get assigned to the CPU and move them to the GPU's. That way each processor gets to work on the most efficient type of task.

I can do the BLC VLAR's on the Ryzen in under an hour compared to the Arecibo CPU task which takes 1-1/4 to 1-1/2 hours. The BLC task on the GPU take 12-15 minutes two up while the Arecibo normal AR tasks get done in 4-8 minutes two up.

But without any BLC tasks on my system, there is nothing to reschedule. I'm down to 80 GPU tasks on Numbskull now. Should be 200.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1862647 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1862648 - Posted: 21 Apr 2017, 7:22:10 UTC - in response to Message 1862645.  

... why are we restricting Arecibo VLAR being sent to Nvidia GPU's but it is entirely OK to send BLC VLAR to Nvidia GPU's. Same low angle ranges. What's the difference?

Arecibo VLARs can bring a system to it's knees, GBT VLARs take forever to crunch, but don't make systems non responsive.


That was the old case with the CUDA apps. But is that the case still with the SoG app?
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1862648 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13746
Credit: 208,696,464
RAC: 304
Australia
Message 1862651 - Posted: 21 Apr 2017, 7:29:17 UTC - in response to Message 1862648.  
Last modified: 21 Apr 2017, 7:29:58 UTC

Arecibo VLARs can bring a system to it's knees, GBT VLARs take forever to crunch, but don't make systems non responsive.

That was the old case with the CUDA apps. But is that the case still with the SoG app?

When on Beta I crunched some Arecibo VLARs on my GTX 1070s, but I can't remember what application was running at the time.
The system was very sluggish to respond, key strokes dropped.
I suspect with older GPU hardware, it would have been brought to it's knees.
Grant
Darwin NT
ID: 1862651 · Report as offensive
Bruce
Volunteer tester

Send message
Joined: 15 Mar 02
Posts: 123
Credit: 124,955,234
RAC: 11
United States
Message 1862657 - Posted: 21 Apr 2017, 7:47:28 UTC

Hi Keith,

You still having trouble getting work?
As I write this, I am down about 150 GPU work units in my cache and the server is telling me - This computer has reached a limit on tasks in progress.
This has been happening since the outage on Tuesday.

Anybody else having the same problem?
Bruce
ID: 1862657 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1862659 - Posted: 21 Apr 2017, 7:57:04 UTC - in response to Message 1862657.  

Yes, still having issues getting work. Down to under 70 tasks now on Numbskull. I couldn't pinpoint the exact time that this problem became very noticeable. It may have been after the outage. I got tasks in small dribs and drabs, not the usual 20 tasks in one shot that I get after an outage once the request queue gets reduced as all the crunchers slowly get refilled. It is has become a case of attrition where there seems to be nothing but Arecibo VLARs coming out of the splitters and they don't get assigned to Nvidia GPU's. CPU's have been at full cache level since the outage since there is no restriction of type.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1862659 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13746
Credit: 208,696,464
RAC: 304
Australia
Message 1862665 - Posted: 21 Apr 2017, 8:04:14 UTC - in response to Message 1862657.  

Anybody else having the same problem?

It has been an issue since late December.
Changing your Application preferences (under Seti @Home preferences) often allows you to get more work.
Changing "Run only the Selected application Astropulse ..." and "If no work for selected applications is available, accept work from other applications? ..." from whatever it is, save it, then on the next Scheduler request the Manager will get those new settings, and on the following request for work it should start flowing. If not, wait a couple more requests and if no joy, change the Application settings back again, save them and after the next Scheduler request work should then start flowing.

However there is bugger all GBT work available at the moment, and a huge proportion of the Arecibo work is VLARs, which aren't sent to the GPU. So even if everything is working OK, getting GPU work at the moment is very much hit or miss (mostly miss).
Grant
Darwin NT
ID: 1862665 · Report as offensive
Bruce
Volunteer tester

Send message
Joined: 15 Mar 02
Posts: 123
Credit: 124,955,234
RAC: 11
United States
Message 1862667 - Posted: 21 Apr 2017, 8:23:41 UTC

Hi Guys,

Thanks Grant, but I was one of the people affected when whoever it was broke that switch on the account page.
I have been using a shortcut that somebody posted awhile back. You do a manual update just after Boinc does the automatic update, the server tells you that it is to soon and you need to wait. The next connection usually starts sending work.

This limit on tasks seems to be something new and different.
Sure wish they would figure out what is wrong.
Bruce
ID: 1862667 · Report as offensive
Previous · 1 . . . 24 · 25 · 26 · 27 · 28 · 29 · 30 . . . 34 · Next

Message boards : Number crunching : Panic Mode On (105) Server Problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.