It seems that the phantom tinkerer has been tinkering again. The BOINC website (housed in the same rack cabinets as the SETI servers) has been intermittently 'down for maintenance' since about midnight, Berkeley time - affecting only the database of messages, not downloads, wiki pages, etc.

And now it's been posted about, it's Ok again.

---

Grant
Darwin NT

And the "Results received in last hour " line is blank.

As are the result turnround times, as a result of the number formatting failures in the warning messages.

Not a good sign, on the server status page

Database/file status

Warning: number_format() expects parameter 1 to be double, string given in /disks/carolyn/b/home/boincadm/projects/sah/html/seti_boinc_html/sah_status.php on line 604 Warning: number_format() expects parameter 1 to be double, string given in /disks/carolyn/b/home/boincadm/projects/sah/html/seti_boinc_html/sah_status.php on line 606 Warning: number_format() expects parameter 1 to be double, string given in /disks/carolyn/b/home/boincadm/projects/sah/html/seti_boinc_html/sah_status.php on line 608

And the "Results received in last hour " line is blank.

---

Grant
Darwin NT

So how do I get boinc to use openssl/1.1.1 instead? I remember getting a Windows client to use a newer version of curl before by doing something, but I don't remember what I did to get it to do that.

I think I remember how I did it with Windows.. in c:\program files\boinc (where the BOINC software itself is installed), there are DLLs for those libraries.

But in linux.. the core software is in the data directory where all the xml files are and it relies on system variables and references for the libraries.

So.. I need to try to figure out why it is falling-back to 0.9.8g and fix that reference, I guess.

---

Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)

but... this line from the startup of boinc:

2019-09-17 04:15:58 Libraries: libcurl/7.18.0 OpenSSL/0.9.8g zlib/1.2.11 c-ares/1.5.1

So how do I get boinc to use openssl/1.1.1 instead? I remember getting a Windows client to use a newer version of curl before by doing something, but I don't remember what I did to get it to do that.

So there's the core of the problem. Boinc uses 0.9.8g where it should use something newer. Now all we need is a good solution!

---

He runs Boinc version 6.10.58, that may just be too old...?

I'm sure I've advised users of older BOINCs to update certificate bundles, and it's worked with the new files. This one is different.

Yes, I've looked around, and it seems the standard fix for the ASN1_item_verify error is to update openssl to version 0.9.8o.
Have you tried that, Cosmic?

[Edit: I checked on one of my Mint hosts, and 'openssl version' says 1.1.1]

root@taurus:~# openssl version
OpenSSL 1.1.1  11 Sep 2018
root@taurus:~#

but... this line from the startup of boinc:

2019-09-17 04:15:58 Libraries: libcurl/7.18.0 OpenSSL/0.9.8g zlib/1.2.11 c-ares/1.5.1

So how do I get boinc to use openssl/1.1.1 instead? I remember getting a Windows client to use a newer version of curl before by doing something, but I don't remember what I did to get it to do that.


also, for reference..

2019-09-17 04:15:58 Starting BOINC client version 6.10.58 for x86_64-pc-linux-gnu
2019-09-17 04:15:58 Data directory: /var/lib/boinc-client
2019-09-17 04:15:58 OS: Linux: 4.15.0-62-generic

---

Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)

He runs Boinc version 6.10.58, that may just be too old...?

I'm sure I've advised users of older BOINCs to update certificate bundles, and it's worked with the new files. This one is different.

Yes, I've looked around, and it seems the standard fix for the ASN1_item_verify error is to update openssl to version 0.9.8o.
Have you tried that, Cosmic?

[Edit: I checked on one of my Mint hosts, and 'openssl version' says 1.1.1]

---

He runs Boinc version 6.10.58, that may just be too old...?

I'm sure I've advised users of older BOINCs to update certificate bundles, and it's worked with the new files. This one is different.

The individual certificates themselves are, of course, encrypted. It rather sounds as if Cosmic_Ocean's problem doesn't lie with the certificates themselves, but with the tool needed to decrypt them.

He runs Boinc version 6.10.58, that may just be too old...?

---

Well I copied the text from https://github.com/BOINC/boinc/blob/master/curl/ca-bundle.crt and saved it as ca-bundle.crt and dropped it into the data directory on the linux machine. Same error:

[http_debug] [ID#1] Info: error:0D0C50A1:asn1 encoding routines:ASN1_item_verify:unknown message digest algorithm

What else can I do/try?

I can't help with a magic script, but I can try to help narrow down the problem.

ca-bundle.crt is actually - I was surprised to find out - a plain text file. You can open it in any text editor and read the description, and the names of each included certificate. It starts:

## Certificate data from Mozilla as of: Fri Jan 26 21:30:21 2018 GMT
##
## This is a bundle of X.509 certificates of public Certificate Authorities
## (CA). These were automatically extracted from Mozilla's root certificates
## file (certdata.txt). This file can be found in the mozilla source tree:
## https://hg.mozilla.org/releases/mozilla-release/raw-file/default/security/nss/lib/ckfw/builtins/certdata.txt
##
## It contains the certificates in PEM format and therefore
## can be directly used with curl / libcurl / php_curl, or with
## an Apache+mod_ssl webserver for SSL client authentication.
## Just configure this file as the SSLCACertificateFile.
##
## Conversion done with mk-ca-bundle.pl version 1.27.
## SHA256: a3ac15b98179dd2f3c5de076d10b1d53048754372f7207c2f327510cdd78fbd8

The individual certificates themselves are, of course, encrypted. It rather sounds as if Cosmic_Ocean's problem doesn't lie with the certificates themselves, but with the tool needed to decrypt them.

BOINC grasped that particular nettle in January 2018, and updated the certificate bundle ('ca-bundle.crt') sent to Windows users - and, I think, Android users too. It was a developer from WCG who took the lead on #2326, so I don't think we're expecting any equivalent of the millenium bug in 2019.

But that doesn't solve the problem of bringing Linux system-level certificates up to 2019 standards.

Well I copied the text from https://github.com/BOINC/boinc/blob/master/curl/ca-bundle.crt and saved it as ca-bundle.crt and dropped it into the data directory on the linux machine. Same error:

[http_debug] [ID#1] Info: error:0D0C50A1:asn1 encoding routines:ASN1_item_verify:unknown message digest algorithm

What else can I do/try?

---

Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)

I don't know if it means anything, but in my googling, I found a boinc forum post from 2009 where someone was asking about this very thing for WCG, and one of the devs replied and said that the cert WCG relies-on is good until "2019".

Maybe we've reached the point in "2019" for that particular cert, perhaps? I know we're not WCG, but it seems to line up.

BOINC grasped that particular nettle in January 2018, and updated the certificate bundle ('ca-bundle.crt') sent to Windows users - and, I think, Android users too. It was a developer from WCG who took the lead on #2326, so I don't think we're expecting any equivalent of the millenium bug in 2019.

But that doesn't solve the problem of bringing Linux system-level certificates up to 2019 standards.

So now what?

I had a problem very similar to this on my Mint hosts. That problem went away when I installed the libnsspem library.

sudo apt-get install libnsspem

As I recall it, the http_debug output in my case complained that it couldn't find libnsspem.so, so it may not have been exactly the same as your problem...

Tried that just now, as well. No change.

---

Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)

So now what?

I had a problem very similar to this on my Mint hosts. That problem went away when I installed the libnsspem library.

sudo apt-get install libnsspem

As I recall it, the http_debug output in my case complained that it couldn't find libnsspem.so, so it may not have been exactly the same as your problem...

---

Cosmic Ocean...read this thread in the recent past...someone else just had this problem

https://manpages.ubuntu.com/manpages/bionic/man8/update-ca-certificates.8.html

root@taurus:~# update-ca-certificates -v
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
root@taurus:~#

Don't know what that did, if anything. I see the file size and modified date of /etc/ssl/certs/ca-certificates.crt changed, and I restarted boinc-client, and it still gives the same error: unknown message digest algorithm.

I don't know if it means anything, but in my googling, I found a boinc forum post from 2009 where someone was asking about this very thing for WCG, and one of the devs replied and said that the cert WCG relies-on is good until "2019".

Maybe we've reached the point in "2019" for that particular cert, perhaps? I know we're not WCG, but it seems to line up.

here's that particular thread: https://boinc.berkeley.edu/dev/forum_thread.php?id=4133

---

Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)

Cosmic Ocean...read this thread in the recent past...someone else just had this problem

https://manpages.ubuntu.com/manpages/bionic/man8/update-ca-certificates.8.html

---

Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)

Cosmic Ocean...read this thread in the recent past...someone else just had this problem

Yeah, I see that now. Looks like some older clients are starting to act up with expiring certificates.

I went and grabbed the current latest one from: http://curl.haxx.se/ and overwrote the old one that was in /etc/ssl/certs, but I'm still getting a/the same problem.

2019-09-17 02:47:49 SETI@home update requested by user
2019-09-17 02:47:54 SETI@home [sched_op_debug] Fetching master file
2019-09-17 02:47:54 [http_debug] HTTP_OP::init_get(): http://setiathome.berkeley.edu/
2019-09-17 02:47:54 SETI@home Fetching scheduler list
2019-09-17 02:47:54 [http_debug] [ID#1] Info: About to connect() to setiathome.berkeley.edu port 80 (#0)
2019-09-17 02:47:54 [http_debug] [ID#1] Info: Trying 208.68.240.110...
2019-09-17 02:47:54 [http_debug] [ID#1] Info: Connected to setiathome.berkeley.edu (208.68.240.110) port 80 (#0)
2019-09-17 02:47:54 [http_debug] [ID#1] Sent header to server: GET / HTTP/1.1

2019-09-17 02:47:54 [http_debug] [ID#1] Info: Connected to setiathome.berkeley.edu (208.68.240.110) port 443 (#0)
2019-09-17 02:47:54 [http_debug] [ID#1] Info: successfully set certificate verify locations:
2019-09-17 02:47:54 [http_debug] [ID#1] Info: CAfile: ca-bundle.crt
2019-09-17 02:47:54 [http_debug] [ID#1] Info: CApath: none
2019-09-17 02:47:54 [http_debug] [ID#1] Info: SSLv2, Client hello (1):
2019-09-17 02:47:54 [http_debug] [ID#1] Info: SSLv3, TLS handshake, Server hello (2):
2019-09-17 02:47:54 [http_debug] [ID#1] Info: SSLv3, TLS handshake, CERT (11):
2019-09-17 02:47:54 [http_debug] [ID#1] Info: SSLv3, TLS alert, Server hello (2):
2019-09-17 02:47:54 [http_debug] [ID#1] Info: error:0D0C50A1:asn1 encoding routines:ASN1_item_verify:unknown message digest algorithm
2019-09-17 02:47:54 [http_debug] HTTP error: SSL connect error

In the boinc data directory, ca-bundle.crt a symbolic link to /etc/ssl/certs/ca-certificates.crt. I changed the link and the CAfile listed in the output there changed to a non-existent path for where it would be if curl was being used.. so it DOES rely on it being in the data dir.

And as a test, I downloaded the latest BOINC for x64 linux and pulled ca-bundle.crt out of that, and it still does the same thing.

So now what?

---

Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)

Cosmic Ocean...read this thread in the recent past...someone else just had this problem

I think it's a SSL certificate problem.


That's what an 'update now' looks like in wireshark.

That's the only thing I can think of at this point is that it's a bad certificate. It looks like the server IS being contacted and responding, so it's not a network problem.

---

Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)