SSL errors whilst upgrading to 0.7.18.0

evoLucidity · August 1, 2022, 6:09pm

Hi all.

Spent most of the day on this part of the upgrade. Would appreciate some help, please!

Running on Ubuntu 22.04, installed ubuntu-flavoured rvm v1.29.12 from GitHub - rvm/ubuntu_rvm: Ubuntu package for RVM as well as the packaged openssl v1.0.1i from the same place, and ruby v2.7.2p137 installed okay.

Done ‘git checkout Gemfile.lock’ in diaspora folder.

steve@pod:~/diaspora$ git checkout Gemfile.lock
Updated 1 path from the index

Done ‘git pull’.

Done ‘script/configure_bundler’

steve@pod:~/diaspora$ script/configure_bundler
Configuring Bundler for production environment and mysql database.
$ bin/bundle config --local jobs 4
$ bin/bundle config --local with mysql
$ bin/bundle config --local without test:development
$ bin/bundle config --local ignore_messages true
$ bin/bundle config --local path vendor/bundle
$ bin/bundle config --local frozen true
$ bin/bundle config --local disable_shared_gems true
Bundler configured! Please run 'bin/bundle install' now.

Problem with ‘bin/bundle --full-index’ command…

steve@pod:~/diaspora$ bin/bundle --full-index
Fetching source index from https://gems.diasporafoundation.org/

Retrying fetcher due to error (2/4): Bundler::Fetcher::CertificateFailureError Could not verify the SSL certificate for https://gems.diasporafoundation.org/.
There is a chance you are experiencing a man-in-the-middle attack, but most likely your system doesn't have the CA certificates needed for verification. For information about OpenSSL certificates, see http://bit.ly/ruby-ssl. To connect without using SSL, edit your Gemfile sources and change 'https' to 'http'.

Retrying fetcher due to error (3/4): Bundler::Fetcher::CertificateFailureError Could not verify the SSL certificate for https://gems.diasporafoundation.org/.
There is a chance you are experiencing a man-in-the-middle attack, but most likely your system doesn't have the CA certificates needed for verification. For information about OpenSSL certificates, see http://bit.ly/ruby-ssl. To connect without using SSL, edit your Gemfile sources and change 'https' to 'http'.

Retrying fetcher due to error (4/4): Bundler::Fetcher::CertificateFailureError Could not verify the SSL certificate for https://gems.diasporafoundation.org/.
There is a chance you are experiencing a man-in-the-middle attack, but most likely your system doesn't have the CA certificates needed for verification. For information about OpenSSL certificates, see http://bit.ly/ruby-ssl. To connect without using SSL, edit your Gemfile sources and change 'https' to 'http'.

Could not verify the SSL certificate for https://gems.diasporafoundation.org/.
There is a chance you are experiencing a man-in-the-middle attack, but most likely your system doesn't have the CA
certificates needed for verification. For information about OpenSSL certificates, see http://bit.ly/ruby-ssl. To connect
without using SSL, edit your Gemfile sources and change 'https' to 'http'.

Same errors if I try ‘bin/bundle install’.

Can someone please help me get past this step?

I’ve tried:

creating a .gemrc with ‘:ssl_verify_mode: 0’ with only partial success.
changing all occurrences of https to http in Gemfile and Gemfile.lock with only partial success
going to https://gems.diasporafoundation.org and trying the ‘gem sources -a https://gems.diasporafoundation.org/’ command but…

I still end up with SSL verification errors.

Anyone help. Please?

Thanks,
Steve

denschub · August 1, 2022, 6:28pm

Three things:

Can you tell me which IP gems.diasporafoundation.org resolves on your machine? (i.e. run host gems.diasporafoundation.org)?
Is the time and date on your server set correctly (you can check with the date command)?
Can you provide the full output of curl -vI https://gems.diasporafoundation.org?

denschub · August 1, 2022, 6:37pm

Ah, better idea to try first. I noticed you’re still on an old RVM version. Unfortunately, v1.29.12 is more than a year old and there isn’t any new release for whatever reason.

Our upgrade guide asks people to use the latest development-version with rvm get master. I don’t know if you can simply run that command to upgrade your package-provided RVM, but if not, you probably should get rid of that package and install RVM the old-fashioned way, using their master branch and not a stable release.

With an upgraded RVM, you’ll get Ruby 2.7.6, and that hopefully already resolves that issue. You kinda have to do that anyway, because ruby 2.7.2 is also really old, and there have been a couple of security hotfixes to ruby in the meantime.

evoLucidity · August 1, 2022, 6:38pm

Hi, Dennis.

steve@pod:~/diaspora$ host gems.diasporafoundation.org
gems.diasporafoundation.org is an alias for diasporafoundation.org.
diasporafoundation.org has address 159.69.231.144
diasporafoundation.org has IPv6 address 2a01:4f8:e0:1f48:d811:38ff:fece:ac46
diasporafoundation.org mail is handled by 10 alpha.0b101010.email.

Yes, this is correct.

steve@pod:~/diaspora$ date
Mon Aug  1 19:37:20 BST 2022

steve@pod:~/diaspora$ curl -vI https://gems.diasporafoundation.org
*   Trying 159.69.231.144:443...
* Connected to gems.diasporafoundation.org (159.69.231.144) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=diasporafoundation.org
*  start date: Jun  2 13:08:35 2022 GMT
*  expire date: Aug 31 13:08:34 2022 GMT
*  subjectAltName: host "gems.diasporafoundation.org" matched cert's "*.diasporafoundation.org"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* Using HTTP2, server supports multiplexing
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* Using Stream ID: 1 (easy handle 0x56186199a550)
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
> HEAD / HTTP/2
> Host: gems.diasporafoundation.org
> user-agent: curl/7.81.0
> accept: */*
>
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
< HTTP/2 200
HTTP/2 200
< server: nginx/1.21.6
server: nginx/1.21.6
< date: Mon, 01 Aug 2022 18:37:58 GMT
date: Mon, 01 Aug 2022 18:37:58 GMT
< content-type: text/html;charset=utf-8
content-type: text/html;charset=utf-8
< content-length: 38365
content-length: 38365
< x-powered-by: geminabox 1.2.0
x-powered-by: geminabox 1.2.0
< x-xss-protection: 1; mode=block
x-xss-protection: 1; mode=block
< x-content-type-options: nosniff
x-content-type-options: nosniff
< x-frame-options: SAMEORIGIN
x-frame-options: SAMEORIGIN
< set-cookie: rack.session=4cf14ffa2c0b2d4aceb2cb3d25c6f8ffdecbd55b3f6427066be1111fe85b192f; path=/; expires=Mon, 01 Aug 2022 18:54:38 GMT; HttpOnly
set-cookie: rack.session=4cf14ffa2c0b2d4aceb2cb3d25c6f8ffdecbd55b3f6427066be1111fe85b192f; path=/; expires=Mon, 01 Aug 2022 18:54:38 GMT; HttpOnly

<
* Connection #0 to host gems.diasporafoundation.org left intact

Thanks,
Steve

evoLucidity · August 1, 2022, 6:39pm

Just spotted your other comment. I’ll do that now.

Steve

denschub · August 1, 2022, 6:39pm

Yeah, your outputs look alright. It most likely is just the outdated RVN and by extension the outdated Ruby. Try that, and if it still fails, let us know.

evoLucidity · August 1, 2022, 7:09pm

Okay, thanks Dennis.

rvm installed as per instructions (hangs head in shame)

So now I’m hitting the openssl errors bit:

+ printf %b 'installing gem /home/steve/.rvm/gem-cache/gem-wrappers-1.4.0.gem --local --no-document\n'
+ command gem install /home/steve/.rvm/gem-cache/gem-wrappers-1.4.0.gem --local --no-document
ERROR:  Loading command: install (LoadError)
        cannot load such file -- openssl
ERROR:  While executing gem ... (NoMethodError)
    undefined method `invoke_with_build_args' for nil:NilClass
+ return 1

Should I install v1.0.1i as before and then ‘rvm install 2.7 --with-ssl-dir=’ like before?

denschub · August 1, 2022, 7:17pm

Ah yeah. Now we found the cause. I already forgot that Ubuntu 22.04 ships OpenSSL 3 by default, which is… an interesting choice that breaks a lot of things.

Yeah, installing OpenSSL v1 and manually providing that should help. @flaburgan ran into this issue recently as well, maybe he can offer help if things go wrong.

evoLucidity · August 1, 2022, 7:22pm

Yes, thanks. Ruby 2.7.6 cleanly installed using openSSL 1.0.1i now.

However, I’m back to the SSL verification errors now.

steve@pod:~/diaspora$ rvm list
=* ruby-2.7.6 [ x86_64 ]

# => - current
# =* - current && default
#  * - default

steve@pod:~/diaspora$ bin/bundle --full-index
Fetching source index from https://gems.diasporafoundation.org/

Retrying fetcher due to error (2/4): Bundler::Fetcher::CertificateFailureError Could not verify the SSL certificate for https://gems.diasporafoundation.org/.
There is a chance you are experiencing a man-in-the-middle attack, but most likely your system doesn't have the CA certificates needed for verification. For information about OpenSSL certificates, see http://bit.ly/ruby-ssl. To connect without using SSL, edit your Gemfile sources and change 'https' to 'http'.

Retrying fetcher due to error (3/4): Bundler::Fetcher::CertificateFailureError Could not verify the SSL certificate for https://gems.diasporafoundation.org/.
There is a chance you are experiencing a man-in-the-middle attack, but most likely your system doesn't have the CA certificates needed for verification. For information about OpenSSL certificates, see http://bit.ly/ruby-ssl. To connect without using SSL, edit your Gemfile sources and change 'https' to 'http'.

Retrying fetcher due to error (4/4): Bundler::Fetcher::CertificateFailureError Could not verify the SSL certificate for https://gems.diasporafoundation.org/.
There is a chance you are experiencing a man-in-the-middle attack, but most likely your system doesn't have the CA certificates needed for verification. For information about OpenSSL certificates, see http://bit.ly/ruby-ssl. To connect without using SSL, edit your Gemfile sources and change 'https' to 'http'.

Could not verify the SSL certificate for https://gems.diasporafoundation.org/.
There is a chance you are experiencing a man-in-the-middle attack, but most likely your system doesn't have the CA
certificates needed for verification. For information about OpenSSL certificates, see http://bit.ly/ruby-ssl. To connect
without using SSL, edit your Gemfile sources and change 'https' to 'http'.

denschub · August 1, 2022, 7:24pm

Meh. Let’s wait if @flaburgan has something to say, he recently did set up diaspora* on Ubuntu 22, maybe he knows what’s up.

Unfortunately, while we could get Ruby to install the Gems via http, that wouldn’t get you too far, as your pod’s federation would still be completely broken.

evoLucidity · August 1, 2022, 7:26pm

Thanks for the help so far, Dennis. At least I’m using the correct software now.
I’ll check back in a bit.

flaburgan · August 1, 2022, 7:30pm

Hello, I only installed it for development purpose so I didn’t have that second problem. Keep us in touch

denschub · August 1, 2022, 7:37pm

@evoLucidity is there any chance you can try it with openssl1.1 instead of 1.0? this issue in the ubuntu_rvm repo suggests that might work

evoLucidity · August 1, 2022, 8:00pm

I followed the instructions in the thread AND downgraded to OpenSSL1.1.

steve@pod:~/diaspora$ openssl version
OpenSSL 1.1.1l  24 Aug 2021
steve@pod:~/diaspora$ bin/bundle --full-index
Fetching source index from https://gems.diasporafoundation.org/

Retrying fetcher due to error (2/4): Bundler::Fetcher::CertificateFailureError Could not verify the SSL certificate for https://gems.diasporafoundation.org/.
There is a chance you are experiencing a man-in-the-middle attack, but most likely your system doesn't have the CA certificates needed for verification. For information about OpenSSL certificates, see http://bit.ly/ruby-ssl. To connect without using SSL, edit your Gemfile sources and change 'https' to 'http'.

Retrying fetcher due to error (3/4): Bundler::Fetcher::CertificateFailureError Could not verify the SSL certificate for https://gems.diasporafoundation.org/.
There is a chance you are experiencing a man-in-the-middle attack, but most likely your system doesn't have the CA certificates needed for verification. For information about OpenSSL certificates, see http://bit.ly/ruby-ssl. To connect without using SSL, edit your Gemfile sources and change 'https' to 'http'.

Retrying fetcher due to error (4/4): Bundler::Fetcher::CertificateFailureError Could not verify the SSL certificate for https://gems.diasporafoundation.org/.
There is a chance you are experiencing a man-in-the-middle attack, but most likely your system doesn't have the CA certificates needed for verification. For information about OpenSSL certificates, see http://bit.ly/ruby-ssl. To connect without using SSL, edit your Gemfile sources and change 'https' to 'http'.

Could not verify the SSL certificate for https://gems.diasporafoundation.org/.
There is a chance you are experiencing a man-in-the-middle attack, but most likely your system doesn't have the CA
certificates needed for verification. For information about OpenSSL certificates, see http://bit.ly/ruby-ssl. To connect
without using SSL, edit your Gemfile sources and change 'https' to 'http'.

denschub · August 1, 2022, 8:15pm

Did you see the note about symlinking the /etc/ssl/certs directory onto the openssl source dir?

denschub · August 1, 2022, 8:21pm

Alternatively, it looks like you can also tell rvm to maintain an openssl version: Cannot build rubies 2.6.x, 2.7.x and 3.0.x on Ubuntu 22.04 because they relies on openssl1.1 and Ubuntu now provides openssl3 · Issue #5209 · rvm/rvm · GitHub

evoLucidity · August 1, 2022, 8:39pm

I did, thanks. Still same result.

supertux88 · August 1, 2022, 8:48pm

@tclaus posted this a while ago about ubuntu 22.04 and openssl 3 … that’s probably all I can contribute to this issue

evoLucidity · August 1, 2022, 9:10pm

Thanks, but I have ruby 2.7.6 installed using OpenSSL v1.0.1.

To bring things up to date, I removed Ruby 2.7.6 and the rvm-packaged OpenSSL 1.0.1a via rvm

My OpenSSL version is now:

steve@pod:~/diaspora$ openssl version
OpenSSL 1.1.1l  24 Aug 2021

I tried installing Ruby 2.7.6 using this ‘native’ OpenSSL version but it doesn’t work.

steve@pod:~/diaspora$ rvm install 2.7.6
Searching for binary rubies, this might take some time.
Found remote file https://rubies.travis-ci.org/ubuntu/22.04/x86_64/ruby-2.7.6.tar.bz2
Checking requirements for ubuntu.
Requirements installation successful.
ruby-2.7.6 - #configure
ruby-2.7.6 - #download
ruby-2.7.6 - #validate archive
ruby-2.7.6 - #extract
ruby-2.7.6 - #validate binary
ruby-2.7.6 - #setup
ruby-2.7.6 - #gemset created /home/steve/.rvm/gems/ruby-2.7.6@global
ruby-2.7.6 - #importing gemset /home/steve/.rvm/gemsets/global.gems...............there was an error installing gem gem-wrappers
................there was an error installing gem rubygems-bundler
................there was an error installing gem rake
................there was an error installing gem rvm
................there was an error installing gem bundler
..
ruby-2.7.6 - #generating global wrappers...............
Error running 'run_gem_wrappers regenerate',
please read /home/steve/.rvm/log/1659387561_ruby-2.7.6/gemset.wrappers.global.log
ruby-2.7.6 - #gemset created /home/steve/.rvm/gems/ruby-2.7.6
ruby-2.7.6 - #importing gemsetfile /home/steve/.rvm/gemsets/default.gems evaluated to empty gem list
ruby-2.7.6 - #generating default wrappers...............
Error running 'run_gem_wrappers regenerate',
please read /home/steve/.rvm/log/1659387563_ruby-2.7.6/gemset.wrappers.default.log

Relevant section from install log:

+ __rvm_log_command_simple command gem install /home/steve/.rvm/gem-cache/gem-wrappers-1.4.0.gem --local --no-document
+ __rvm_log_command_debug
++ __rvm_date '+%Y-%m-%d %H:%M:%S'
++ date '+%Y-%m-%d %H:%M:%S'
+ printf %b '[2022-08-01 21:59:23] command\n'
+ is_a_function command
+ typeset -f command
+ return 1
+ printf %b 'current path: /home/steve/diaspora\n'
+ env
+ __rvm_grep -E '^GEM_HOME=|^GEM_PATH=|^PATH='
+ GREP_OPTIONS=
+ command grep -E '^GEM_HOME=|^GEM_PATH=|^PATH='
+ printf %b 'command(6): command gem install /home/steve/.rvm/gem-cache/gem-wrappers-1.4.0.gem --local --no-document\n'
+ rvm_log 'installing gem /home/steve/.rvm/gem-cache/gem-wrappers-1.4.0.gem --local --no-document'
+ [[ '' == 1 ]]
+ printf %b 'installing gem /home/steve/.rvm/gem-cache/gem-wrappers-1.4.0.gem --local --no-document\n'
+ command gem install /home/steve/.rvm/gem-cache/gem-wrappers-1.4.0.gem --local --no-document
ERROR:  Loading command: install (LoadError)
        cannot load such file -- openssl
ERROR:  While executing gem ... (NoMethodError)
    undefined method `invoke_with_build_args' for nil:NilClass
+ return 1
+ return 1

It only works if I install Ruby with the --with-openssl-directory after rvm pkg install openssl.

I’ve spent a full day on this now and I’ve had enough. If I preserve my diaspora and /var/lib/mysql folders (I installed with MySQL), can I start again on an earlier version of Ubuntu and just copy them over without any issues?

supertux88 · August 1, 2022, 9:19pm

You can downgrade diaspora without any problems … but if you updated mysql to ubuntu 22.04 and started that, that might have upgraded stuff in /var/lib/mysql which then doesn’t work anymore if you want to start the old mysql with that folder … A dump and restore is probably the safer option (unless you didn’t start mysql after updating, or if you still have a copy from before the upgrade. But I don’t use mysql and I don’t know which mysql you used before and if that would be compatible or not.