I’d actually like to see an export of all data we wish to retain during the move, the reason is that it would make the move feature a backup feature with no basically additional effort, allowing a move even if the original pod went down. With that idea a few more semantics can change, see my old writeup: https://github.com/MrZYX/old_diaspora_wiki/blob/master/Seed-migration-proposal.md
Totally agree, I’m only focusing on the move right now, I wan’t to shut down one of my pods as cleanly as possible and move the 1 user on it to my newer pod.
I’m trying to keep it simple first, move the user, let them rebuild their stuff. If someone feels like focusing on export/import (regardless of pod) that would take care of both too.
I’ve looked a the existing export, no ones maintaining it. People are adding relations to the user/person models and none of them are getting put in the export.
To me the export is a solved problem, it’s just a dump and reload of data, someone just needs to take ownership of it. People have manually moved before, I (yes i now its not for everyone) can sit there and poke my data back into the DB, not a big deal. The big issue was how to move the actual user/person relationship from one to another and notify everyone it happened, I believe what I proposed solves this.
Sure and I wouldn’t mind a first version not migrating the data, but I think we should decide whether we want to support migration when the original pod is down and you just have such a download laying around. We can totally strip the data stuff from my old proposal and have that, if we don’t want to support that use case yours is fine as it is.
Another issue both our proposals don’t accommodate:
It’s quite frequent that a pod receives a comment or a post (through a reshare) from a third pod and fetches the authors profile there. The third pod doesn’t keep track of that and there’s no contact from or to a user on that third pod. So the home pod of the fetched user doesn’t know that he should send a move notice to the third pod. If then after a move someone from that pod want to contact that user, that’ll fail.
Perfect, that last thing is the kind of issue I was looking to locate. I knew it could happen, but I couldn’t work out exactly how. Will work on something to try solve that.
Totally love this idea - I think having this solved would bring a lot of confidence in to the project, since many people seem to feel it was one of the original promises in the kickstarter (umm it was I guess :)).
+1 for implementing this if you are up for working on it @ruxton No solution needs to be perfect at the beginning - that is impossible, the best way is to make something and iterate over it.
As for the export, it would be nice that these two would complement each other. After all, both ideas can use the same contents for the profile ("(and maybe serialized profile)" -part). Please make it JSON, not XML?
One idea I was going to write about was some kind of automatic backup of user profile/contacts/posts (without picture uploads) toanother pod. This pod could be chosen randomly from participating pods (like “allow_backups” setting) and each user would automatically have their data backed up on a random pod. The data would be encrypted so that it could only be opened with their private key, which would given to download for the user on registration (with instructions to store it safely - loss of key=loss of backups).
In the event of catastrophe at the users pod, the user has the option of either downloading data or opening an account using the backup profile, on the other pod - which is when they would need to private key.
In this way user data would be backed up somewhere off-site, making it less likely for user to lose everything if the pod goes down. Also the pod which has the backups would not be able to read the backed up data, until user opens it with their key. The only issue I would see is that the backups would have to of course be incremental - which could be an issue as the data would need to be sent over encrypted… Also the whole thing shouldn’t cause too much strain on the pods of course… DB space is hardly an issue for just post/profile data with no relations - and I’m 100% sure lots of pods would gladly participate.
But really, the only way imho to solve the problem of dying pods is to make it somehow not require user action. Requiring one to manually download data “as a precaution” is not really a working model. Maybe when we have the API there could be clients to automate it of course, but then that would be a big strain on servers
But sorry for rambling in (mostly) unrelated things - would totally love to have the possibility for users to migrate pods!
@jasonrobinson In principle, your idea is quite nice, even with all the technical details to be figured out. The only real conceptual problem I see is that the “automatic backup to a random pod” defeats the whole “choose the pod you trust” idea.
My point is - default experience IMHO should be as little need to decide as possible. Because most users don’t like clicking through a gazillion things to just get to the application.
Maybe uploading to FTP/OwnCloud/DrobBox/something would be an option for automatic backup? Or isn’t it better to put effort into implementing an API instead of automatic backup? Server strain could be reduced by allowing a backup of selected timespan through API.
From what I can see, account migration would have to have a few objectives:
Let’s assume we want to migrate a user’s data from account x on pod A to account y on pod B
The user informs pod A that he’s migrating to y@B. The user initiates the migration from his account y@B.
Pod B requests Pod A send copies of the x@A data to y@B. This includes posts, images, comments, messages, etc. and account connections (e.g., start sharing with the same accounts on y@B as x@A was). Pod A would reject any migration other than to y@B, and uses https certificate validation to ensure it is sending data to the correct pod. Pod B could send a nonce/token to Pod A in its initial message, which pod A sends back with the x@A data for additional security.
Pod A could send out a message to other pods indicating the y@B is the new home for x@A. Pods hosting accounts sharing with x@A could verify the move with pod A and automatically start sharing corresponding accounts with y@B while unsharing x@A. Another alternative would be to send a message to any user sharing with x@A about the move.
After the move is complete, manage interactions with x@A. This could be as simple as a disallowing comments on existing posts/messages with an appropriate message, or as robust as redirecting posts/image requests/etc. to the new pod via 302. The user x@A should not be allowed to create new posts/messages/comments anymore. In fact, deleting the pod A account should be the only modification x@A is allowed to do, or perhaps initiating another migration.
I’ve outlined a fairly complex state machine and will probably require more transitions than what I’ve described (e.g., aborting the migration). And probably a fair amount of code.
I like what everyone has put here so far! My thoughts on how the process could work:
User logs into their home pod
In user settings, user can authenticate to an account on a remote pod using OpenID Connect and a Diaspora Migration API. You could almost think of it as the remote Diaspora pod being treated as an authenticated app.
Posts, images, and contacts get sent securely to remote pod using API.
Existing posts on original pod somehow are turned into references/redirects of some sort that point to the duplicates on the new pod. We may need to implement a proper way to revise posts via the API in order to successfully do this.
After a determined amount of time, original posts are deleted once the new references get federated out to everybody.
It may be basic, meaning fundamental to Diaspora’s concept, but it is in no way basic, meaning simple to create or implement. On the contrary, it’s extremely complex to do properly. That’s why it’s taken so long!
@ruxton confirmed he will not be working on this (on Twitter) - so anyone who wants to, feel free to participate in tuning the idea and/or creating an actual spec for the migration.
It would be great to get this implemented as I think it would reinforce the idea that D is decentralised in potential users minds, and give them more confidence in it. Also it would be a really cool feature.
I’m not sure that allowing the user to back up to another pod is a good idea, mainly from a perception point of view. I think any backups should be dealt with from the podmin end, however I still think allowing a user to download their data is a good thing.
No solution needs to be perfect at the beginning - that is impossible, the best way is to make something and iterate over it.
