I think it won’t be, and that’s the wanted behaviour. But we can discuss that.
Actually, the text is from me but I wasn’t sure about it and @supertux88 answer in Backup and restore – account migration - #116 by supertux88 that most of it wasn’t accurate. So don’t spend too much time re-wording it right now, maybe we should simply start from scratch again.
I just had a thought, with the closing of Framasphere (in exactly one month) a lot of users will do the export / import at the same time. But telling to other pod that a user migrated isn’t instantaneous. What will happen if two users sharing with each other are migrating at the same time? With messages about migration crossing each other…
Oh, yes. I noticed also that download profile uses the same database field than upload profile…
Your investigation is indeed very useful. Could you create Issues for each of them?
So we can decide what blocks the next release and what is a plus for later.
I’ll try! to work out which are the useful ones – and create issues for them.
@flaburgan, are you able to create new labels in Github? If so, one for migration would be really useful to hold all of these issues and PRs together. (One for gems or dependencies would also be useful, unless that’s covered by upstream).
One further idea: how about the following addition to the first landing page after the sign-up page – /getting_started?
OK, just a rough mock-up, colour and text are draft only, but a button here could be useful.
Click that button and ‘getting started’ hints are disabled, and you are taken straight to the /user/edit settings page (and scrolled down to the migration section if possible).
We’d need to make it clear that this is not an option for new users to choose so that they don’t miss out on getting started hints and get confused by the migration section in the setting page, but is this a good approach? @tclaus @flaburgan?
Hey @tclaus so I have time to work on diaspora this evening and tomorrow morning probably.
I would like to be able to do the first real migrations using the rake task as it would be easier to monitor what is happening this way. Do you think that you could improve it include the photo import? That would be great.
1 Like
I thought about it some time ago but unfortunately no I don’t have the rights to create them.
About Backup and restore – account migration - #179 by goob well it still is prominent but probably better than what we proposed. I still don’t know how to present it, or if that’s really needed… I think I will focus on the UI in the settings at the moment.
Currently I am out of office. May be back later this day.
I do not know exactly what the task does if uploading a profile. (In terms of setting password or migrating personal profile data)
There is currently not task to import photos.
So, I tried locally the import of my very old profile from diasp.org. The validation went well, but the actual migration failed, is that just because I’m running on localhost @supertux88?
Actually if I launch it again it works, so I don’t know if the initial cert errors were only about some old pods…
I actually hit an interesting case:
Do you really want to execute the archive import? Note: this is irreversible! [y/N]: y
[2021-09-08T21:19:53] DEBUG PID-17442 TID-62280 ActiveRecord::Base: User Load (0.8ms) SELECT "users".* FROM "users" WHERE "users"."username" = $1 LIMIT $2 [["username", "teest"], ["LIMIT", 1]]
[2021-09-08T21:19:53] DEBUG PID-17442 TID-62280 ActiveRecord::Base: User Exists (0.4ms) SELECT 1 AS one FROM "users" WHERE "users"."email" = $1 LIMIT $2 [["email", "flaburgan@XXXX"], ["LIMIT", 1]]
[2021-09-08T21:19:53] DEBUG PID-17442 TID-62280 ActiveRecord::Base: User Exists (0.2ms) SELECT 1 AS one FROM "users" WHERE "users"."username" = $1 LIMIT $2 [["username", "teest"], ["LIMIT", 1]]
[2021-09-08T21:19:53] DEBUG PID-17442 TID-62280 ActiveRecord::Base: User Exists (0.2ms) SELECT 1 AS one FROM "users" WHERE (id != NULL AND email = NULL) LIMIT $1 [["LIMIT", 1]]
[2021-09-08T21:19:53] DEBUG PID-17442 TID-62280 ActiveRecord::Base: Person Exists (0.3ms) SELECT 1 AS one FROM "people" WHERE "people"."diaspora_handle" = $1 LIMIT $2 [["diaspora_handle", "teest@localhost:3000"], ["LIMIT", 1]]
[2021-09-08T21:19:53] DEBUG PID-17442 TID-62280 ActiveRecord::Base: (0.1ms) BEGIN
[2021-09-08T21:19:53] DEBUG PID-17442 TID-62280 ActiveRecord::Base: User Exists (0.3ms) SELECT 1 AS one FROM "users" WHERE "users"."email" = $1 LIMIT $2 [["email", "flaburgan@XXXX"], ["LIMIT", 1]]
[2021-09-08T21:19:53] DEBUG PID-17442 TID-62280 ActiveRecord::Base: User Exists (0.2ms) SELECT 1 AS one FROM "users" WHERE "users"."username" = $1 LIMIT $2 [["username", "teest"], ["LIMIT", 1]]
[2021-09-08T21:19:53] DEBUG PID-17442 TID-62280 ActiveRecord::Base: User Exists (0.2ms) SELECT 1 AS one FROM "users" WHERE (id != NULL AND email = NULL) LIMIT $1 [["LIMIT", 1]]
[2021-09-08T21:19:53] DEBUG PID-17442 TID-62280 ActiveRecord::Base: Person Exists (0.2ms) SELECT 1 AS one FROM "people" WHERE "people"."diaspora_handle" = $1 LIMIT $2 [["diaspora_handle", "teest@localhost:3000"], ["LIMIT", 1]]
[2021-09-08T21:19:53] DEBUG PID-17442 TID-62280 ActiveRecord::Base: (0.2ms) ROLLBACK
rake aborted!
ActiveRecord::RecordInvalid: Validation failed: Email is already taken., Person can't be blank
/var/lib/gems/2.7.0/gems/activerecord-5.2.6/lib/active_record/validations.rb:80:in `raise_validation_error'
/var/lib/gems/2.7.0/gems/activerecord-5.2.6/lib/active_record/validations.rb:52:in `save!'
/var/lib/gems/2.7.0/gems/activerecord-5.2.6/lib/active_record/transactions.rb:315:in `block in save!'
/var/lib/gems/2.7.0/gems/activerecord-5.2.6/lib/active_record/transactions.rb:387:in `block in with_transaction_returning_status'
/var/lib/gems/2.7.0/gems/activerecord-5.2.6/lib/active_record/connection_adapters/abstract/database_statements.rb:267:in `block in transaction'
/var/lib/gems/2.7.0/gems/activerecord-5.2.6/lib/active_record/connection_adapters/abstract/transaction.rb:239:in `block in within_new_transaction'
/var/lib/gems/2.7.0/gems/activerecord-5.2.6/lib/active_record/connection_adapters/abstract/transaction.rb:236:in `synchronize'
/var/lib/gems/2.7.0/gems/activerecord-5.2.6/lib/active_record/connection_adapters/abstract/transaction.rb:236:in `within_new_transaction'
/var/lib/gems/2.7.0/gems/activerecord-5.2.6/lib/active_record/connection_adapters/abstract/database_statements.rb:267:in `transaction'
/var/lib/gems/2.7.0/gems/activerecord-5.2.6/lib/active_record/transactions.rb:212:in `transaction'
/var/lib/gems/2.7.0/gems/activerecord-5.2.6/lib/active_record/transactions.rb:385:in `with_transaction_returning_status'
/var/lib/gems/2.7.0/gems/activerecord-5.2.6/lib/active_record/transactions.rb:315:in `save!'
/var/lib/gems/2.7.0/gems/activerecord-5.2.6/lib/active_record/suppressor.rb:48:in `save!'
/home/fla/diaspora/diaspora/lib/archive_importer.rb:40:in `create_user'
/home/fla/diaspora/diaspora/app/services/migration_service.rb:51:in `find_or_create_user'
/home/fla/diaspora/diaspora/app/services/migration_service.rb:22:in `perform!'
/home/fla/diaspora/diaspora/lib/tasks/accounts.rake:22:in `block (2 levels) in <top (required)>'
Tasks: TOP => accounts:migration
(See full trace by running task with --trace)
So if another user that the one who is imported as the same e-mail than in the archive, the import fails. Is that what we want? I could imagine a valid case where a user has two accounts in a pod, wants to merge a third one in one account, but the email in the archive is the same than the email of the second account. I know, few chances that this would happen but ?
1 Like
I also have some errors like this in the logs:
[2021-09-08T23:51:53] ERROR PID-31334 TID-62280 DiasporaFederation::Federation::Receiver: failed to receive public message: DiasporaFederation::Salmon::InvalidEnvelope: DiasporaFederation::Salmon::InvalidEnvelope
But I guess those messages are just ignored.
The old code tries to create a user every time, but needs to do a “find_or_create”.
Then, not all attributes are set correctly.
Can you try this patch?
Index: app/models/user.rb
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/app/models/user.rb b/app/models/user.rb
--- a/app/models/user.rb (revision d88529ccd10adbfb201423b0183f6c8e6f3bfa92)
+++ b/app/models/user.rb (date 1631138799981)
@@ -404,12 +404,20 @@
end
###Helpers############
- def self.build(opts = {})
+ def self.build(opts={})
u = User.new(opts.except(:person, :id))
u.setup(opts)
u
end
+ def self.find_or_build(opts={})
+ user = User.find_or_create_by(username: opts[:username], email: opts[:email])
+ user.password ||= opts[:password]
+ user.password_confirmation ||= opts[:password]
+ user.setup(opts)
+ user
+ end
+
def setup(opts)
self.username = opts[:username]
self.email = opts[:email]
@@ -417,6 +425,10 @@
self.language ||= I18n.locale.to_s
self.color_theme = opts[:color_theme]
self.color_theme ||= AppConfig.settings.default_color_theme
+ self.strip_exif &&= opts[:strip_exif]
+ self.show_community_spotlight_in_stream = opts[:show_community_spotlight_in_stream]
+ self.auto_follow_back ||= opts[:auto_follow_back]
+ self.getting_started = false
self.valid?
errors = self.errors
errors.delete :person
Index: spec/lib/archive_importer_spec.rb
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/spec/lib/archive_importer_spec.rb b/spec/lib/archive_importer_spec.rb
--- a/spec/lib/archive_importer_spec.rb (revision d88529ccd10adbfb201423b0183f6c8e6f3bfa92)
+++ b/spec/lib/archive_importer_spec.rb (date 1631137969039)
@@ -147,7 +147,6 @@
expect {
archive_importer.create_user(username: "new_name", password: "123456")
}.to change(User, :count).by(1)
-
expect(archive_importer.user.email).to eq("user@example.com")
expect(archive_importer.user.strip_exif).to eq(false)
expect(archive_importer.user.show_community_spotlight_in_stream).to eq(false)
Index: lib/archive_importer.rb
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/lib/archive_importer.rb b/lib/archive_importer.rb
--- a/lib/archive_importer.rb (revision d88529ccd10adbfb201423b0183f6c8e6f3bfa92)
+++ b/lib/archive_importer.rb (date 1631136648126)
@@ -36,7 +36,7 @@
profile_attributes: profile_attributes
}
)
- self.user = User.build(data)
+ self.user = User.find_or_build(data)
user.save!
end
What do you mean by “old code”? I m in your upload_migration_files branch and I do have a find_or_create (migration_service.rb line 49, called line 22)
Maybe we should have those discussions on github, I’ll post my findings there now.
This is just a method name -
It calls
- archive_importer.create_user(username: new_user_name, password: SecureRandom.hex)
and in Archive_user:39 a ‘build’ method, where a user is created every time.
One question that has just occurred to me:
Can the migration feature as currently constituted cope with a second import from the same account? I can imagine this happening in the following scenario.
- Pod A goes down and is down for a while. I decide to open a new account on Pod B, because it looks as though Pod A is dead.
- I open an account on Pod B and import a back-up taken a month before Pod A went down, because that’s better than nothing.
- Later, Pod A goes online once again. Great! I can now migrate my account with all the up-to-date information to Pod B.
When I do this, will it:
a) fail with an error;
b) duplicate the content that was part of my earlier import;
c) recognise a previously imported account from the key and refuse to proceed with the migration;
d) recognise previously imported content and correctly skip that content?
1 Like
I would say it still works but not sure.
I’m leaving this evening for the whole week-end, I’ll be back only on Sunday evening. I started to post messages to explain users of Framasphere what is going on and some of them are willing to be beta-testers of the migration. I will do that with the rake task once we feel we’re ready, maybe beginning of next week? @tclaus could you add the photo import to the rake task?
@supertux88 @jhass @comradesenya I don’t want to put pressure on you, but if you have time to review part of Thorsten work, it would be really, really cool. Framasphère is closing in less than a month, so time is running out.
I don’t know this.
A Close message is send out to your old pod. This signings are repeated several times for some days (weeks?) So if your old pod comes back, chances are high that it will receive the close-message.
Maybe Supertux88 knows more?