Personal Data Import/Export

[Edited by Fla]
This discussion is about account migration between pods.
Current github issue

Summary: We all agree this is an important feature, but it’s hard to deal with considering privacy and security issues.

Note: This discussion was imported from Loomio. Click here to view the original discussion.

Proposal: Abandon the feature?

We could always abandon this feature. If we do, it means clearing that advert text on diasporaproject, letting people know in the GH Issue that it’s not gonna happen in the near future, and closing it out.

I don’t necessarily want to do this, but I figured this is the best time to bring up abandoning an issue, as no work has been done yet.

Outcome: We’re not abandoning this feature. :slight_smile:


  • Yes: 0
  • Abstain: 1
  • No: 4
  • Block: 2

Note: This proposal was imported from Loomio. Vote details, some comments and metadata were not imported. Click here to view the proposal with all details on Loomio.

I don’t feel that the concept should be abandoned. Rather; we need to figure out how to make it work securely based on what’s out there now.

Interestingly enough, both Friendica Red’s Zot2 implementation and Tent are focusing on this idea of a user being able to literally “pack up and move” from one node on a network to another. Maybe they have some ideas on how it could be done; better yet, maybe we could talk to these different projects about possibly allowing users to move not just between Diaspora hosts, but onto Friendica and Tent hosts as well.

We need an existing implementation of this concept to look to for inspiration on how best to do this. I don’t think just a basic “upload your data that you downloaded from the other pod” is exactly going to cut it in the long run.

Just digging out what I thought about it like a year ago

@seantilleycommunitymanager why wouldn’t a quick-n-dirty solution like that work? i thought we could just export some sort of big XML feed with all the text of every post along with a folder of images that were posted by the user, collect it in a .ZIP file and be ready to go. would that not be a good solution?

@jonneha i know it would blow the package up a bit but i think photos are crucial to the migration. it would just be a pain in the ass for users to have to separately download each image.

the way Facebook does it, you can actually pick whether you want photos in the archive. i think we should echo this feature. they also queue the job and email you when its done, since the archive files can get pretty large. I’d be all about doing it this way.

Can this kind of shit even be done on Heroku? We can’t alter the filesystem, so where will we put the .ZIP archive?

Hm maybe we can get access to a tmpfs on Heroku but in general it’s readonly.

@tomscott The issue I have with the quick 'n dirty approach is that it could open up a potential security problem. It’s not necessarily a technical problem, so much as that it’s a problem of social engineering.

If all the process requires is a download/upload of data, what’s stopping someone from finding a way to download someone else’s data, upload it to a new pod, and pretend to be that person?

@seantilleycommunitymanager in terms of pod-to-pod migrating, could we have the pods actually do the transfer and migration? the user would never actually touch the .ZIP archive (though they could optionally download it), rather the pods would facilitate all of the archive creation, unpacking and loading of data.

@tomscott That’s exactly what I was thinking of. There could be an option in Settings to move to a different pod, where you could type in the URL of your Diaspora pod and authenticate with it like it’s an app.

This would push your posts, contacts, Aspects, and bio to the new pod, and your old account would simply become a redirect to your current one.

Of course, that’s all easier said than done, but from an end user’s standpoint, it’d be very easy to migrate by just typing in a URL and clicking a button.

I think this is a key feature for Diaspora, as a decentralized network.
Speaking about a quick approach, the possibility of abuse that Sean Tilley voiced - how possible is it?

really glad to hear this is not being abandoned, I think its a really important feature. As far as downloading a zip of your data goes, I think that’s fine for people to have a copy of their stuff, but not as a way of transferring data between pods.

@seantilleycommunitymanager it may be difficult, but i believe this is the first step to providing 3rd-party applications access to DIASPORA as well as providing us with a much-requested feature. we can use the pod-to-pod authentication and transmission knowledge we gain here to build future things that deal with 3rd-party apps.

but that’s all out of the scope of this proposal. i’ll write up a decision and let’s get to work.

Proposal: Export pod data to an archive

The first step to this whole project is exporting a person’s entire persisted history to an archive. Without this key ingredient, there’s no point in coding pod-to-pod communications. In order to maintain the philosophy that you should not only be in control of, but own the data you post to DIASPORA, it will be an option to download the archive from your pod. Pods will only retain archives for 4 hours, after which they will be purged.

Outcome: N/A


  • Yes: 2
  • Abstain: 3
  • No: 0
  • Block: 0

Note: This proposal was imported from Loomio. Vote details, some comments and metadata were not imported. Click here to view the proposal with all details on Loomio.

@louigiverona depends on how we make it. we can’t just let someone POST to /users and have us set up a secure keypair and password for them. Since we’re not running a centralized network, it would be possible for me to take Sean’s data and upload it to a different pod, and therefore pretend to be him on DIASPORA. Although and would be slightly different, it would be difficult for the common person to distinguish and may lead to some clever social engineering/phishing scams. We don’t want to facilitate that in any way, so we need to make sure this whole thing is done securely.

Export of data is only of use if you can import it to your new pod. Otherwise you may as well just close one account and open a new one.

@jonneha, @goob and @mataloutreach – lol i think i got the idea of “decisions” here incorrect. i should have put the whole proposal up there.

But before I do, does anyone know if we can write files to the HD on Heroku? If not, we need to make this an optional feature so Heroku pods don’t crash when this happens.

We already can export some data? Why would that not be needed - even Facebook allows you to do that :slight_smile:

All we need is an import. I don’t see the security problem since you’re not supposed to give your data archive to anyone else. Own fault if you upload it to the internet and someone uploads your data on another pod.

The security concern is that we need a way for another pod to authenticate as the existing user, otherwise one could simply pretend to be someone else to the network. We just need the private key of the user on the new pod and generate and establish a new keypair on/from the new pod, so that the old pod can make no further posts in the name of the user.

I say let’s just do a simple upload for now and reuse the actual data processing logic there when the automatic wizard thingy is done. This way there is no risk - any more than there is now. I could create and pretend to be him at any time :slight_smile:

For data processing for example contacts in aspects would be nice to be auto-created - imho this could be first step.