Backup and restore – account migration

Proposal: Include polls & poll participations to users archive?

See the comment and further discussion.



  • Yes: 9
  • Abstain: 0
  • No: 0
  • Block: 0

Note: This proposal was imported from Loomio. Vote details, some comments and metadata were not imported. Click here to view the proposal with all details on Loomio.

But there is a nuance: I don’t know if it is a design decision or just random, but now it’s impossible to check who voted for which answer. If we export the archive there will be each vote in the archive with the ID of the voter.

I’m surprised to hear that this info is federated by the polls feature. I don’t think there’s any need for anyone except the poll creator to be able to see who voted for which option. I would vote for closing this down so that only poll question, poll answers and number of votes for each answer were federated. But this should be shut down within the polls feature code rather than in the export code. That way exports will be lighter!

@goob: the author of the poll-participation is federated to validate the signature and also to prevent duplicate votes by the same person. And if only the poll-creator sees who voted and sends only numbers to everybody else, he would be able to fake the result. Maybe we should simply make the poll-participants public visible to everybody (like votes here on loomio). But this needs its own discussion and doesn’t belong to the export. :slight_smile:

Also the proposal here is only about including the poll and answers to the poll-creators export. The poll (and answers) is not included to other persons export.

@supertux88, thanks for your reply. I couldn’t (and still can’t) find any indication that this vote relates only to migration by a poll creator, but if so, I think it is important to migrate all data necessary for a poll to work, so that a poll (if still open) can still be work.

It would be good to refine the polls feature with things such as the ability to close polls (and to create them to be open for a limited time only), and to look at ways to reduce the federation of user data when interacting with a poll, if this is appropriate an useful. All this would make the migration of an account including polls more streamlined, but this work doesn’t belong to the migration feature. These refinements would be best added to, and discussed within, the Enhancements for polls feature issue in Github.

Thanks again.

Does backup and restore concept include Pod Migration?

Here’s a simple import button without backend:

Rails console script gist for partial import:

In principle, yes. But Senya’s approach omits restoring after server death, so far. That’s why, I’d view backup and restore with regard to server deaths as a separate issue. See here:

Duplicate discussions, now closed:

Export data / Import data
Importing and exporting contacts through pods
Personal Data Import/Export
Import profile functionality - user request
Moving/migrating between different pods
Basic idea to move users between pods on D*

I know this is a very old discussion, but there seem to be still no real progress in account migration or even backup for a user so at least contacts are not lost when, for some reason, anyone decides to move to another pod.
A few weeks ago I saw this post that makes it possible for a ordinary user to at least have the contacts saved:
(permalink on my pod, replace the podname if for some reason you can not reach it)

diaspora* already allows you to backup all your data, downloading a zip. And in the develop branch, you can already import everything except images in a new pod.

To summarize the current situation:

  • Federation part is done (see documentation, work mainly done in #54 and #89)
  • Backend part is done for the data (this doesn’t include images) (original issue #908, most of the work done in #7660). The result of this is a rake task which can be used to import an account from an archive.
  • There is no UI which allow the users to import their account, this has to be done. Some questions there, do we want to include the possibility to import an account in the registration process? In the settings once the account is created (the backend is able to merge account)? In both?
  • Nothing is dealing with images yet. They can be exported, but not imported. This also raises several questions because URLs to images can be hardcoded, so there is an investigation to do on that point before deciding what to actually implement.

As, in my opinion, this feature is the last main blocker for the release of the next diaspora* version, I really want to see it implemented. 90% of the work is done, let’s get some motivation back and do it :smiley:

I just had a call with @tclaus and he agrees to join the effort. @supertux88 and @comradesenya, you’re the one who worked on the backend, could you give us some pointers? Or even better, do you want to team up and get that done? Maybe remote-hackathon style, like the good old days?


This is very heartening development. It would be wonderful to finally see that incredible work done by @comradesenya and @supertux88 brought to completion.

@weex was also interested in joining this effort, I think from their comment here.

Thanks @goob for the mention. Got a chance to review the links in @flaburgan’s post above and learned a lot. I think I can help with documentation and testing.

There are some good questions about a UI and image support but if this is holding up a release, as an agile developer and potential podmin I tend to prioritize release over perfection. Podmins can certainly skip this version if they’re not obtaining value.

1 Like

We’re talking about a major release here, so we want to deliver the complete feature. While we don’t hold minor releases which are (were, activity really went down) pushed every 6 weeks with whatever is ready, major release are (were, once again) usually done yearly and can be delayed if the feature isn’t ready. That is the case here.

1 Like

Totally understand about the major release factor and project norms. For the UI task is there a github issue where those questions would be hashed out or is this the place?

General discussion can happen here I think. Once we start to have some agreement on the direction to go, I’ll start coding and then the conversation will move to github

During registration makes the most sense to import an account from the user perspective. One might be reluctant to create a whole new account with all the mental load of deciding what to call it, what info to put in, will it conflict with what is about to be imported. OTOH, if import is expensive then you open up potential for denial of service so there needs to be some constraint on import attempts.

If it’s offered after registration, then does the imported info simply overwrite everything on the user’s profile, or at least what’s included in the export?

If URLs are hardcoded, I could see some regexes being able to update them, as long as the URLs have enough info to know which image in the export to link. So it would be good to do it on a best-effort basis and simply leave URLs intact if no reliable update can be done.

Is there a link to the test export that is mentioned in 7660 with like 1k posts, 1k comments or is there a better export to use for test?

I have several huge real accounts myself, it’s easy to launch the import in a dev environment to see how it reacts. We did it several times during the hackathon in April 2019 and actually found some bugs that way. Some of the fixes (either #8010 or #8017 still has to be merged btw). To test again the backend still is a good idea, and I’m sure many users will volunteer to give me their export archive to test it. I can start with my accounts are report what I’m finding.

You can also do that on your side with your own account on the dev env (using the docker image if you don’t want to install it, it’s available as script/diaspora-dev). I’m not 100% sure but I think that, run from the local env, the migration doesn’t have effect on your actual production account. Maybe switch off your internet connection when doing so, just to be sure ^^

But actually, all that part (merging a migrated account into a created one) is already done and integrated, a good UI is only what’s missing. Even if done during the registration process, it will start by creating a new account on the new pod anyway.

Okay, so I started thinking about this. We want the users to understand:

  • To migrate, they need to first export the original account from its old pod, then import it in the other one
  • Two archives are concerned by this download from old → upload into new: the one with the data and the one with the photos
  • What exactly will be migrated (profile data, settings, aspects, contacts, posts, comments, private messages…) and what that means for both the new account (how are data merged? imported one overwrite the existing one?), for the contacts (will they receive a notification that the new account started sharing, or will that be quiet?) and the comments made (will the author change transparently?). This of course depends of if the old pod still is up or not. All that has to be clear for the user.
  • That the imported account will be deleted from the old pod
  • That the migration cannot be undone

It’s been a long time since I looked at the migration, and I don’t remember all the choices made there. I need to dig into the code to be able to answer those questions. Here, I’m just sharing the flow the user should come through, not the actual content.

So, two possible entry points, from the registration and from the settings. It’s already late so I won’t talk about registration right now, but from the settings, I thought about something like this:

  • The “Export data” section now has a legend explaining that this is the first step.
  • The “Import” section explains what’s possible and how to do it. It already reminds the account which will receives the import, and that’s there is no way back. At this step, I kept the things simple and the user only has one button to click

This is a draft, don’t look at the beauty of the design at the moment, it is just a basis for discussion

When the import button is clicked, a modal opens with 3 sections.

  • The first one explains more in details what’s going to happens and the consequences (content here is an example, as said, I need to dig in the code to actually know myself what is really happening)
  • The second section allows the user to upload the two archives. Once done, a check is made to verify the integrity of the archives. If the syntax is okay…
  • … only after the check, the third section appears, reminding the user which account is going to be imported and into which one, and that it cannot be undone.

I don’t know if the how the backend allows that integrity check separately than the import itself at the moment, there will be tweaks for sure, but that intermediary steps seems important to me. It would also be nice to use it to ping the old pod, see if it is still online, and adapt the list of consequences depending on that. To be seen.

I throw all this out of my head without thinking too much about it, now I need to sleep, and probably tomorrow I’ll see it differently, but that allows us to start thinking about it :wink: