Posts by ausymark

1) Message boards : Number crunching : CL file build failure in Ubuntu Linux 17.10 using opensource AMD drivers for a RX480 (Message 1907528)
Posted 16 Dec 2017 by Profile ausymark
Post:
Isn't this part of AMD's plan to support the HSA memory management that is being put into the latest 4.14 kernel?


I dont think its directly related as they still need the OpenCL driver to work with seti first, and it isnt.

Its an issue that surfaced with NV cards a couple years ago as well. So Im thinking its an API type issue.
2) Message boards : Number crunching : Gridcoin...? (Message 1907516)
Posted 16 Dec 2017 by Profile ausymark
Post:

Regarding MB's on GPU, it has been available from a time long before AP. So you should be able to find it.
And AP's errors out on RXxx0's under windows too (they run fine on the HD7770). There is a bug in the code somewhere (either the driver or the app). For a short time I had Raistmer looking into it, but he seems to be busy with other things.


Mine wasnt even getting to that point, was failing on the start of the initialisation. We will see what happens in the future though ;)
3) Message boards : Number crunching : Gridcoin...? (Message 1907387)
Posted 16 Dec 2017 by Profile ausymark
Post:
Its not that Grant, its the simple fact that I am getting an error when AP tries to start crunching on my GPU. As per this thread:

https://setiathome.berkeley.edu/forum_thread.php?id=82285

What about MB?


Anything cpu based will work. So far its only used AP on thd gpu. What app does seti use for multibeam under Linux and i will have a look later today as im not on my crunching machine atm.

Cheers
4) Message boards : Number crunching : Gridcoin...? (Message 1907361)
Posted 16 Dec 2017 by Profile ausymark
Post:
Unfortunately I am still waiting for the ability to crunch seti WU on my GPU once again. Hopefully that time will come soon. :)

Your resource share determines what does & doesn't get processed.


Its not that Grant, its the simple fact that I am getting an error when AP tries to start crunching on my GPU. As per this thread:

https://setiathome.berkeley.edu/forum_thread.php?id=82285

A bug report has been filed so we will see what gets fixed in the underlying Linux OpenCL subsystem.

Aint living on the bleeding edge fun? :P

Cheers
5) Message boards : Number crunching : Gridcoin...? (Message 1907341)
Posted 16 Dec 2017 by Profile ausymark
Post:
I see that your system is pretty much identical to mine.

So if I can get 20c a day, I would be rather satisfied. I was hoping to do 10.

I have an extra HD7770 crunching also, so perhaps I can get to 21 :)


Also depends if you are running 24/7. But as you have two RX 480's, and assuming you aren't just running seti, you should be fine. I have 1 x Einstein@home running on my GPU and its credit score is far greater than other projects - it is the main generator of gridcoins on my rig.

Unfortunately I am still waiting for the ability to crunch seti WU on my GPU once again. Hopefully that time will come soon. :)

Happy crunching!
6) Message boards : Number crunching : Gridcoin...? (Message 1907312)
Posted 15 Dec 2017 by Profile ausymark
Post:
I joined the gridcoin system last week and a pool as well.

Little baby steps. Its not raking in the millions (more like 20c a day lol) but at least you get 'paid' for work you are already doing - i.e. boinc projects, and hence reduces/eliminates the electricity usage that would be needed if one actually just crunched crypto currencies. Besides who knows how much these particular crypto currency will be worth in a few years.

Experimentation is fun aint it? :P
7) Message boards : Number crunching : Task Deadline Discussion (Message 1906127)
Posted 10 Dec 2017 by Profile ausymark
Post:
Which is why I added my 2c worth, running on the assumption that if the underlying dispatch system works better then improved deadlines would be a likely outcome ;)

The deadlines are set by the project, and Jeff's initial numbers show that even as things are, there's no need for such long deadlines.


Hi Grant

There maybe extreme cases where a long deadline is needed, I am thinking something like a laptop that has both its buffers set as high as possible, and which only connects to the net every few weeks.

In this day and age that would be pretty rare, but it wasn't rare 15 years ago, so the long deadlines could just be a throwback from earlier times to make allowances for this type of situation.

Cheers
8) Message boards : Number crunching : Task Deadline Discussion (Message 1906119)
Posted 10 Dec 2017 by Profile ausymark
Post:


This is off topic from deadlines, but is also a major cause of bloating.



Which is why I added my 2c worth, running on the assumption that if the underlying dispatch system works better then improved deadlines would be a likely outcome ;)

Cheers
9) Message boards : Number crunching : Task Deadline Discussion (Message 1906113)
Posted 10 Dec 2017 by Profile ausymark
Post:
ausymark wrote:
I'm going to suggest what I think might be a simple system that may help matters.
...
...
So does any of the above have merit and is it simple enough to be easily implemented?

Cheers

Mark
Yes, it definitely has merit, but I don't think any changes relating to quota management are likely to be easily implemented.

The existing system attempts to do some of what you suggest. If you look at the "Applications details" for a given host (for instance, yours would be here), you'll already find fields for "Average turnaround time" and "Max tasks per day", for each application that can be sent to that host.

In general, I suspect your proposal would probably be more restrictive on new hosts, but perhaps overly generous for existing ones. For new hosts, the initial value for "Max tasks per day" is 33. I think that number is much too high as it is, notwithstanding the fact that the value in that field is somewhat misleading. As I recall, the project applies a multiplier to that value to calculate the true maximum, and my recollection is that the multiplier is 8. (If anyone can point to someplace that would verify or refute that number, please do.)

That initial setting is then incremented by 1 for every task that is completed and validated, so it can get to be quite a huge number for reliable hosts. Theoretically, the number is decreased for each time a task is marked as an Error or Invalid (the latter being somewhat questionable). It can never drop lower than 1, but with the multiplier, and with the scheduler having multiple applications to choose from for hosts running stock), that number is kind of meaningless.

Even for hosts with huge "Max tasks" numbers, there are already two limiting factors in play. One is the maximum of 100 CPU tasks per host and 100 tasks per GPU. The other is the "Store at least nn days of work" and "Store up to an additional nn days of work" combination, controllable through user/host Preference settings. Each of these has a maximum value of 10 days which, even in extreme cases, should limit a host to no more work than it can process in 20 days. If that value is less than the maximum allowed by the other limits, that's where it should be capped. Otherwise, the other limits should come into play and, in the case of the most productive hosts, should be far less than 20 days. So, given all that, why a host with a greater than 20-day average turnaround is able to download more than 1 or 2 tasks at a time is beyond me. :^)


Thanks for that Jeff

Knowing the above it might be a 'simple' matter of tweaking the existing system a little. I am thinking the 33 Max Tasks figure and the multiply by 8 figure are there to create a buffer in the first case and compensate for a few days extra storage - to buffer any initial user requirement to download 'xxx' days of extra work. These figures were probably created for when accessing seti was via dial-up modem so that users could disconnect and crunch away (I used to be one of those users ;) ).

By the sounds of it the 100 tasks per CPU and 100 tasks per GPU are there for two reasons:
1) the Max Tasks field isn't being calculated properly and
2) just in case there is a runaway of invalids for whatever reason

The Average Turnaround Time figure is obviously calculated over several validated work units. This figure should increase if more WU's are sent to the host, and decrease with fewer - assuming the computer is operating consistently

So the question really becomes, "how many WU should be allowed to be dispatched in total taking into account the users buffer options?".

BOINC knows the performance of the host so the initial WU dispatch should be whatever that host is calculated to be able process in a 24 hour period multiplied by how many extra hours/days the user is requesting for their buffer. This should be the initial "Max Tasks" number/limit. This also allows us to calculate an initial Average Turnaround Time. Most likely this initial distribution of WU's will be more than the host will process as most aren't running 24/7.

If this is the case then the Max Tasks total should be reduced until the Average Turn Around time is close to the original calculated Average Turn Around time. This must also allow for X days of buffer. So if a 3 day buffer then Average Turnaround Time might be 4 days + 3days of buffer which equals 7 days Average Turn Around Time.

So, for example, if the Average Turnaround Time is 30% longer than expected, then reduce the number of WU by 30%. Though this reduction may have to be done in small amounts over time, perhaps 2% per day, that way it will take 15 days to slowly correct itself. The reverse would occur if Average Turnaround Time was smaller than expected. (As I write that, and knowing how slowly seti reacts to changes, makes me wonder if this isnt how its currently done lol).


This however doesn't account for failed/invalid WU's that are processed very fast. My gut instinct is that any result that is at > 85% smaller variance to the Average Turnaround Time then that WU should be excluded from the calculations. If Host hardware has changed then BOINC should notify seti and the initial startup routine would be initiated again.

Deadline dates with the above should be Average Turnaround Time + Number of Buffer Days + x days that the database people think are fair, I am guessing somewhere between 7 and 10 days, or posibly a multiplication of Average Turnaround Time + Number of Buffer Days.

Anyway, Im just thinking aloud. These thoughts may be not be new, IDK, just adding some ideas into the mix.

Cheers

Mark
10) Message boards : Number crunching : Task Deadline Discussion (Message 1906079)
Posted 10 Dec 2017 by Profile ausymark
Post:
3) No matter how fast a host is its real performance is how many WU it uploads per day

Make that Validates.
Some host upload a lot of results, but they don't validate.

>>> Agreed ;)

Firstly there should be a 30 day average of WU processed per day. This allows for changes in operation and/or new hardware being introduced onto the host. To be calculated once per day. Note: Only the average needs to be stored, not an array of daily WU.s

The Average turnaround time for each application actually gives a good indication of how long it takes to return work.


>>> the issue with using WU average turn around time is that it is something that happens after potentially lots of unneeded WU's are allocated to a host, especially true for slow hosts. I dont know how/if seti uses turnaround time atm, if it does how is it used to allocate buffers and determine WU deadlines?

Cheers :)
11) Message boards : Number crunching : Task Deadline Discussion (Message 1906049)
Posted 10 Dec 2017 by Profile ausymark
Post:
I'm going to suggest what I think might be a simple system that may help matters.

Caveat: I have no idea how seti currently allocates WU's so please excuse any ignorance on my part.

Firstly some assumptions:

1) How fast a host is only indication of its potential processing ability
2) That host may only be on an hour or two per day, or may be running other BOINC tasks
3) No matter how fast a host is its real performance is how many WU it uploads per day
4) Below I talk about an "Issue total" which is the total number of outstanding WU that should be issued to a host. i.e. a host should not be processing more WU's than defined by the Issue Total.
5) In the below proposal I will use some 'constants' that can be tweeked to suit seti better - these 'constants' are given so that the example can be made. I will mark these constants in BOLD to highlight them

There are two situations that I am addressing with this, in the hope that it will improve the other issues mentioned throughout this thread.

Situation One: Existing Host

Firstly there should be a 30 day average of WU processed per day. This allows for changes in operation and/or new hardware being introduced onto the host. To be calculated once per day. Note: Only the average needs to be stored, not an array of daily WU.s

Secondly the Issue Total of WU issued to a host should:

* For a host processing at least 1 WU per day: not be greater than three times the 30 day daily average. This allows both a buffer for outages and hardware improvements.
The deadline for these tasks should be something like ten days . Due to the daily average being used this is equivalent to the host taking 10 times longer to process work units.
e.g. if the 30 day daily average is 40, then total number of WU issued to a host would be 40 x 3 = 120 WU.

* For a host processing less than 1 WU per day: issue a total of no more than 3 WU with a 60 day deadline.

Situation Two: New Host

The problem here is that there is no 30 day average to draw throughput data from. To fix this I propose a simple solution.

Firstly set the 30 day average at a figure of 3 and send the first 3 WU. Give a deadline of say 30 days .

For the first 48 hours of processing for every returned WU issue 3 more WU's - this will build up an initial buffer.

Once the 48 hours is over use that WU throughput as the new 30 day daily average figure (i.e total WU in 48 hours divided by 2 to give average daily figure). The host is now ready to go onto the main WU distribution system above.

If no WU are returned in the first 48 hour period set 30 day average figure to 2. This allows the host to still ramp up WU throughput if it was off to a slow start, it also reduces the number of WU allocated to slow hosts.

If the 30 day average ever drops below 0.33, then only issue 2 WU, and the Issue Total should be set to no less than 2 with a 60 day deadline.

So does any of the above have merit and is it simple enough to be easily implemented?

Cheers

Mark
12) Message boards : Number crunching : CL file build failure in Ubuntu Linux 17.10 using opensource AMD drivers for a RX480 (Message 1904881)
Posted 4 Dec 2017 by Profile ausymark
Post:
Is that available in beta form for testing?
13) Message boards : Number crunching : CL file build failure in Ubuntu Linux 17.10 using opensource AMD drivers for a RX480 (Message 1904222)
Posted 1 Dec 2017 by Profile ausymark
Post:
Hi Team

As many of you may know AMD is open sourcing much of its proprietary graphics driver code and merging it within the linux base code. (Their proprietary drivers are stuck way back in the Ubuntu 16.04 code base - 18 months ago).

I have Einstein at home successfully running in this configuration on the RX 480. However Seti at home is throwing a "CL file build failure" error.

I thought this may have rectified itself after the recent mainlining of much of AMD drivers code into linux but the problem still exists.

Anyone got any ideas?

Cheers

Mark
14) Message boards : Number crunching : Is anyone got AMD RX480 on Ubuntu 16.04 working for SETI or other BOINC GPU projects? (Message 1860472)
Posted 9 Apr 2017 by Profile ausymark
Post:
Try the new AMD driver, is now working for me under Ubuntu 16.10.

http://support.amd.com/en-us/kb-articles/Pages/AMDGPU-PRO-Driver-for-Linux-Release-Notes.aspx

Cheers
15) Message boards : Number crunching : BoincTasks under Linux (Message 1826669)
Posted 25 Oct 2016 by Profile ausymark
Post:
well that is indeed confusing. Didnt know such a thing existed, thought was talking about standard boinc.

sorry for the confusion
16) Message boards : Number crunching : BoincTasks under Linux (Message 1826050)
Posted 22 Oct 2016 by Profile ausymark
Post:
Linux has its own native boinc client.

Running windows boinc under wine is just adding unneeded complexity and overhead....

So I would advise downloading the native boinc client (if you are running Ubuntu linux Boinc is part of the application software centre - makes it easy to install).

SO I would advise to install the linux version first then work out other issues after that point.

Cheers

Mark
17) Message boards : Number crunching : GPU FLOPS: Theory vs Reality (Message 1822895)
Posted 9 Oct 2016 by Profile ausymark
Post:
When I installed my Sapphire Radeon RX 480 I did a fresh install, even fresh home directory. I reinstalled all apps then restored core data. One of the things I noticed was that my boot times were halved, compared to my original nVidia setup, which was about 4 years old, that included about 7 OS upgrades/updates.

So I think there was fair bit of 'garbage' from those years of messing around, including several versions of the nVidia drivers etc.

So in a nutshell, assuming you didnt do it, I would advise doing a fresh install, home directory included, to fully rule out conflicts with old setups.

Apologies if you did do the above as I dont mean to seem condescending, just trying to help.

Cheers

Mark
18) Message boards : Number crunching : GPU FLOPS: Theory vs Reality (Message 1822703)
Posted 8 Oct 2016 by Profile ausymark
Post:
Just installed the 16.10 Beta with the 14.8 kernal, seems as though it is miss reporting the compute units and max speed as per the 16.04 Ubuntu version so it is likely something in the AMD firmware blob. I will wait until they release an updated video driver and see if that fixes things.

The system still works, is just performing under what it can do. No biggie, I am patient ;)
19) Message boards : Number crunching : GPU FLOPS: Theory vs Reality (Message 1822368)
Posted 7 Oct 2016 by Profile ausymark
Post:
I am running the latest AMD driver for Ubuntu 16.04 from AMD's site: http://support.amd.com/en-us/kb-articles/Pages/AMD-Radeon-GPU-PRO-Linux-Beta-Driver%E2%80%93Release-Notes.aspx

Being a beta/alpha tester has never worried me in the past.

In fact I helped to test some of the early Lunatics builds back in the day....

As well as beta apps and OS's....

So, no biggy. Will let you know how I go ;)
20) Message boards : Number crunching : Last few % taking forever to compute under lunix (Message 1822367)
Posted 7 Oct 2016 by Profile ausymark
Post:
That behavior can be normal and mainly depends on the work unit being processed.

Just let it run ;)


Next 20


 
©2018 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.