Here’s a simple import button without backend:
https://raw.githubusercontent.com/programizer/screenshots/master/importUI1.png
https://github.com/programizer/diaspora/tree/import_profile_gui_2
Rails console script gist for partial import:
In principle, yes. But Senya’s approach omits restoring after server death, so far. That’s why, I’d view backup and restore with regard to server deaths as a separate issue. See here:
https://wiki.diasporafoundation.org/User:Senya/Account_Backup_And_Restore#Description
Duplicate discussions, now closed:
Export data / Import data
Importing and exporting contacts through pods
Personal Data Import/Export
Import profile functionality - user request
Moving/migrating between different pods
Basic idea to move users between pods on D*
I know this is a very old discussion, but there seem to be still no real progress in account migration or even backup for a user so at least contacts are not lost when, for some reason, anyone decides to move to another pod.
A few weeks ago I saw this post that makes it possible for a ordinary user to at least have the contacts saved:
(permalink on my pod, replace the podname if for some reason you can not reach it)
diaspora* already allows you to backup all your data, downloading a zip. And in the develop branch, you can already import everything except images in a new pod.
To summarize the current situation:
- Federation part is done (see documentation, work mainly done in #54 and #89)
- Backend part is done for the data (this doesn’t include images) (original issue #908, most of the work done in #7660). The result of this is a rake task which can be used to import an account from an archive.
- There is no UI which allow the users to import their account, this has to be done. Some questions there, do we want to include the possibility to import an account in the registration process? In the settings once the account is created (the backend is able to merge account)? In both?
- Nothing is dealing with images yet. They can be exported, but not imported. This also raises several questions because URLs to images can be hardcoded, so there is an investigation to do on that point before deciding what to actually implement.
As, in my opinion, this feature is the last main blocker for the release of the next diaspora* version, I really want to see it implemented. 90% of the work is done, let’s get some motivation back and do it 
I just had a call with @tclaus and he agrees to join the effort. @supertux88 and @comradesenya, you’re the one who worked on the backend, could you give us some pointers? Or even better, do you want to team up and get that done? Maybe remote-hackathon style, like the good old days?
2 Likes
This is very heartening development. It would be wonderful to finally see that incredible work done by @comradesenya and @supertux88 brought to completion.
@weex was also interested in joining this effort, I think from their comment here.
Thanks @goob for the mention. Got a chance to review the links in @flaburgan’s post above and learned a lot. I think I can help with documentation and testing.
There are some good questions about a UI and image support but if this is holding up a release, as an agile developer and potential podmin I tend to prioritize release over perfection. Podmins can certainly skip this version if they’re not obtaining value.
1 Like
We’re talking about a major release here, so we want to deliver the complete feature. While we don’t hold minor releases which are (were, activity really went down) pushed every 6 weeks with whatever is ready, major release are (were, once again) usually done yearly and can be delayed if the feature isn’t ready. That is the case here.
1 Like
Totally understand about the major release factor and project norms. For the UI task is there a github issue where those questions would be hashed out or is this the place?
General discussion can happen here I think. Once we start to have some agreement on the direction to go, I’ll start coding and then the conversation will move to github
During registration makes the most sense to import an account from the user perspective. One might be reluctant to create a whole new account with all the mental load of deciding what to call it, what info to put in, will it conflict with what is about to be imported. OTOH, if import is expensive then you open up potential for denial of service so there needs to be some constraint on import attempts.
If it’s offered after registration, then does the imported info simply overwrite everything on the user’s profile, or at least what’s included in the export?
If URLs are hardcoded, I could see some regexes being able to update them, as long as the URLs have enough info to know which image in the export to link. So it would be good to do it on a best-effort basis and simply leave URLs intact if no reliable update can be done.
Is there a link to the test export that is mentioned in 7660 with like 1k posts, 1k comments or is there a better export to use for test?
I have several huge real accounts myself, it’s easy to launch the import in a dev environment to see how it reacts. We did it several times during the hackathon in April 2019 and actually found some bugs that way. Some of the fixes (either #8010 or #8017 still has to be merged btw). To test again the backend still is a good idea, and I’m sure many users will volunteer to give me their export archive to test it. I can start with my accounts are report what I’m finding.
You can also do that on your side with your own account on the dev env (using the docker image if you don’t want to install it, it’s available as script/diaspora-dev). I’m not 100% sure but I think that, run from the local env, the migration doesn’t have effect on your actual production account. Maybe switch off your internet connection when doing so, just to be sure ^^
But actually, all that part (merging a migrated account into a created one) is already done and integrated, a good UI is only what’s missing. Even if done during the registration process, it will start by creating a new account on the new pod anyway.
Okay, so I started thinking about this. We want the users to understand:
- To migrate, they need to first export the original account from its old pod, then import it in the other one
- Two archives are concerned by this download from old → upload into new: the one with the data and the one with the photos
- What exactly will be migrated (profile data, settings, aspects, contacts, posts, comments, private messages…) and what that means for both the new account (how are data merged? imported one overwrite the existing one?), for the contacts (will they receive a notification that the new account started sharing, or will that be quiet?) and the comments made (will the author change transparently?). This of course depends of if the old pod still is up or not. All that has to be clear for the user.
- That the imported account will be deleted from the old pod
- That the migration cannot be undone
It’s been a long time since I looked at the migration, and I don’t remember all the choices made there. I need to dig into the code to be able to answer those questions. Here, I’m just sharing the flow the user should come through, not the actual content.
So, two possible entry points, from the registration and from the settings. It’s already late so I won’t talk about registration right now, but from the settings, I thought about something like this:
- The “Export data” section now has a legend explaining that this is the first step.
- The “Import” section explains what’s possible and how to do it. It already reminds the account which will receives the import, and that’s there is no way back. At this step, I kept the things simple and the user only has one button to click
This is a draft, don’t look at the beauty of the design at the moment, it is just a basis for discussion
When the import button is clicked, a modal opens with 3 sections.
- The first one explains more in details what’s going to happens and the consequences (content here is an example, as said, I need to dig in the code to actually know myself what is really happening)
- The second section allows the user to upload the two archives. Once done, a check is made to verify the integrity of the archives. If the syntax is okay…
- … only after the check, the third section appears, reminding the user which account is going to be imported and into which one, and that it cannot be undone.
I don’t know if the how the backend allows that integrity check separately than the import itself at the moment, there will be tweaks for sure, but that intermediary steps seems important to me. It would also be nice to use it to ping the old pod, see if it is still online, and adapt the list of consequences depending on that. To be seen.
I throw all this out of my head without thinking too much about it, now I need to sleep, and probably tomorrow I’ll see it differently, but that allows us to start thinking about it 
As long as the pod isn’t reachable from other pods under the domain it is configured to, the import doesn’t have any effects on the imported account, so if your dev-pod is just http://localhost:3000 none of the pods can reach that to validate the signature. But don’t test migrations on any pods which are able to federate with other pods (public reachable domain).
At the moment, only import of the data archive is supported. Nothing with photo import is done yet, it’s not even decided how this should work exactly on the backend, and how exactly all this should work on a federation level, how should other pods get the updated URLs (resend all the photo entities to all the pod, which would be a ton of messages … or maybe create a new message with where all the URLs can be packed into a single message as a bulk).
So if you want to include photo-import into the migration, a lot more still needs to be done than just the UI.
Nobody will receive notifications about migrations, contacts just switch to the new diaspora ID.
Yes
The migration doesn’t depend on the old pod still being up, the archive contains everything needed to migrate. But since the data-archive doesn’t contain the photos, in it’s current state photos wont work anymore if the old pod is down, since they still point to the old URL (if the photo migration also is added, then nothing at all will depend on the old pod … but as said, nothing here is done yet).
Then from your screenshot “Import another account”:
No data is overwritten. If the user doesn’t exist, it will be created from the data in the export, if it already exists, nothing will change.
Settings like theme and visibility aren’t even in the export (a few settings are, but for them the same as above applies, they are only set if the user didn’t exist yet, if the user already exists, nothing is overwritten).
The only thing that will be deleted are user/profile attributes (name, bio, birthday … and also private stuff like email and settings), as if the account was closed … the posts and all other linked data will be migrated to the new account/profile. (OK, that’s something that obviously only works if the old pod is still up, if it’s down, it can’t delete anything … what I meant above was that none of the import/migration parts depend on the old pod).
No, as said, migration doesn’t depend on old pod and none of the contacts will receive a notification, independent of if the old pod still exists or not.
Now back to below the screenshot
It does, and the rake-task is already using it.
One last thing:
Besides the photo-import not done yet, I think there are also more things that are still a bit glitchy … for example I think conversations/private-messages are broken after the migration (at least in some cases, I don’t remember all the details, but @jhass had some problems there), and I don’t know how to fix this best … but that needs to be looked into too, and maybe a warning added in the import/migration description.
See also Contact status not properly preserved after account migration · Issue #8106 · diaspora/diaspora · GitHub and Optimize archive import with lots of dead (offline pods) contacts · Issue #8019 · diaspora/diaspora · GitHub. IIRC my conversations weren’t imported at all.
Thanks to both of you. As those are more backend problems maybe @tclaus you want to have a look?
The code of the mockups I posted above is available on that branch btw as I read that you want to start to work on that feature. As said, I can easily handle the UI part, if you’re fine splitting the work.
1 Like
OK, nice. Will you do not only mockups, but also a ‘real’ UI?
Then I can go into jhass’ mentioned issues. 8106 and 8019.
(The post I made this morning was more or less symbolic)
1 Like
I’m confused about the mockups. As far as I remember account migration is a secondary option to account creation, it does not replace or update an existing account.