Message boards :
Number crunching :
Aborted: run time limit exceeded -> gpu task
Message board moderation
Author | Message |
---|---|
Tom M Send message Joined: 28 Nov 02 Posts: 5126 Credit: 276,046,078 RAC: 462 |
I am running Lubuntu 18.04 (I think) and I had a very busy/exhausting week. Since I thought I had everything set on a "set and forget" I didn't check on the system https://setiathome.berkeley.edu/show_host_detail.php?hostid=8684146 during that time. I came home and collapsed all week. The other gotcha is an unreliable LAN cable that I have just replaced. That cable was in place for the last two times it locked up. I have a temporary fix in place till I get another longish LAN cable. I am guessing that the Boinc/Seti locked up caused the above error message because I didn't catch it in time. Is this right or do I need to bark up another troubleshooting path? Tom A proud member of the OFA (Old Farts Association). |
rob smith Send message Joined: 7 Mar 03 Posts: 22504 Credit: 416,307,556 RAC: 380 |
Well, maybe, but there are other issues with that computer as it has more recently dumped something over 400 "Error 11" tasks. So it most definitely needs more investigation to get to the bottom of that issue. (The last time I had a dump of abandoned tasks was when I cold re-started one of my computers which has a flakey BIOS back-up battery, which is another job on my Tuit list) None of you aborted tasks that I looked at show either a valid "estimated run time", or an elapsed time so there is obviously something very wrong with that one. Bob Smith Member of Seti PIPPS (Pluto is a Planet Protest Society) Somewhere in the (un)known Universe? |
Keith Myers Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873 |
That is an application error code and the codes are listed here: https://boinc.mundayweb.com/wiki/index.php?title=Project_application_errors Exit code -11 is a floating point rounding error. Likely due to overheating or overclocking. https://boinc.mundayweb.com/wiki/index.php?title=Exit_code_-11 Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) |
Tom M Send message Joined: 28 Nov 02 Posts: 5126 Credit: 276,046,078 RAC: 462 |
Well, maybe, but there are other issues with that computer as it has more recently dumped something over 400 "Error 11" tasks. Thank you both for your expertise. I was scratching my head. After resetting "everything" to bios defaults I have applied "4.0 Presets", "Die-B, 3200Mhz (fast) from Stilt?, and can't remember if I turned up the phase control/current delivery or not. I did not switch the LLC at all. And I did a manual cpu voltage override of 1.31. When I just looked it it, it was running a bunch cooler (how big is a bunch.... well hold out your hand and let me see how much I can get on there)... Psensor is reporting 69C and the max it has reported is 71C. It was running hotter than that. The cpu watch command still reporting 4.0 to just under that. As many as 4 threads are hitting 4Ghz at the same time. Again thank you..... Heat is the computers enemy.... (and strikes again...). Tom A proud member of the OFA (Old Farts Association). |
Keith Myers Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873 |
After resetting "everything" to bios defaults I have applied "4.0 Presets", "Die-B, 3200Mhz (fast) from Stilt?, and can't remember if I turned up the phase control/current delivery or not. I did not switch the LLC at all. And I did a manual cpu voltage override of 1.31 What does psensor show for cpu voltage at load? How much does it sag from the manual 1.31V BIOS setting. How many cpu tasks are running? I have LLC on Auto and it sags down to 1.33V from the Auto BIOS setting of 1.35V. That is with 8 cpu tasks and 3 gpu tasks running. CPU is set to 4050Mhz manually. Temps are normally around 68-73 °C. under load for my 2700X. That 1.31V might not be enough for 4Ghz. Depends on how much it sags under load. If the cpu loading is minimal, it probably is OK. I would feel better with a voltage closer to 1.35V which is the default for 4Ghz. But that would raise temps and I don't know your cooling solution. I have AIO or custom loop cooling on all my cpus. Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) |
Tom M Send message Joined: 28 Nov 02 Posts: 5126 Credit: 276,046,078 RAC: 462 |
After resetting "everything" to bios defaults I have applied "4.0 Presets", "Die-B, 3200Mhz (fast) from Stilt?, and can't remember if I turned up the phase control/current delivery or not. I did not switch the LLC at all. And I did a manual cpu voltage override of 1.31 Those are great questions. The cooling solution I am using is the Biggest two fan cpu Air Cooler from Notura(sp) and a 40mm fan under the cpu socket. The MB is installed on a modest mining rack. I have 6 gpus running on riser cards but nothing else cooling them. The rack is on the floor under a table where another system may grow. Here is a specific answer to the data question. Thank you for helping get it setup so I can answer these kinds of questions. tom@EJS-GIFT:~$ sensors asuswmisensors-isa-0000 Adapter: ISA adapter CPU Core Voltage: +1.30 V CPU SOC Voltage: +1.04 V DRAM Voltage: +1.40 V VDDP Voltage: +0.63 V 1.8V PLL Voltage: +1.96 V +12V Voltage: +11.83 V +5V Voltage: +4.99 V 3VSB Voltage: +3.36 V VBAT Voltage: +3.23 V AVCC3 Voltage: +3.38 V SB 1.05V Voltage: +1.07 V CPU Core Voltage: +1.30 V CPU SOC Voltage: +1.04 V DRAM Voltage: +1.42 V CPU Fan: 1424 RPM Chassis Fan 1: 4720 RPM Chassis Fan 2: 0 RPM Chassis Fan 3: 0 RPM HAMP Fan: 0 RPM Water Pump: 0 RPM CPU OPT: 0 RPM Water Flow: 0 RPM AIO Pump: 1448 RPM CPU Temperature: +71.0°C CPU Socket Temperature: +47.0°C Motherboard Temperature: +32.0°C Chipset Temperature: +48.0°C Tsensor 1 Temperature: +216.0°C CPU VRM Temperature: +50.0°C Water In: +216.0°C Water Out: +216.0°C CPU VRM Output Current: +80.00 A asus-isa-0000 Adapter: ISA adapter cpu_fan: 0 RPM k10temp-pci-00c3 Adapter: PCI adapter temp1: +71.2°C (high = +70.0°C) tom@EJS-GIFT:~$ ^C Given my low temps and the fact that I am running about as hard as it is possible 90% of available threads are processing something. And it hasn't crashed yet. I am loathed to touch it. It had crashed twice (app locked up OS) previously. I assume the "AIO pump" is because I am plugged into the wrong fan socket. Tom A proud member of the OFA (Old Farts Association). |
Keith Myers Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873 |
Everything that sensors reports looks perfectly normal to me. If I had those sensor readings I would be loathe to change them. The header labelled AIO pump is just a normal fan header only with extra temperature and PWM controls on it. It normally just puts out constant +12V for its output unless you set it up for PWM control and set temperature/PWM percentage thresholds in the BIOS. No problem running a simple fan from it. If you run sensors with the -u parameter, you can see what the native name in the BIOS is for all the sensor outputs. All I'm saying is that if for some reason the system crashes after a few days, the cpu voltage may not be sufficient. I would throw some LLC2-3 on that 1.31V in the BIOS and see if the system stays up longer. I have my systems running for as long as I ignore them. Sometimes over a month of system uptime before I come visit and see that there are pending updates that I allow to go through. Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) |
Tom M Send message Joined: 28 Nov 02 Posts: 5126 Credit: 276,046,078 RAC: 462 |
I will update next week. It seems to be perfectly happy. And I am currently too busy to be hand holding anyway. Tom A proud member of the OFA (Old Farts Association). |
Tom M Send message Joined: 28 Nov 02 Posts: 5126 Credit: 276,046,078 RAC: 462 |
It does seem to have the screen/keyboard become un-responsive. That was with cpu voltage at 1.30~?/auto. So I am going to reset the LLC to 3. Tom A proud member of the OFA (Old Farts Association). |
Jimbocous Send message Joined: 1 Apr 13 Posts: 1856 Credit: 268,616,081 RAC: 1,349 |
I would echo others in that my Linux box was throwing those errors on GPUs from time to time, as you may recall me mentioning in the other thread. Since I found and corrected a problem with a power splitter feeding a 6-pin on one 980, I have not had a recurrence since 9 Sep. Ymmv ... |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.