You all might already be aware of the fact that after a recent upgrade JD is really a mess .
Mostly agreed issues are
====================
Purge unused old accounts.
Active users - move to smaller pods , federate !
Having said that I personally am very sad at the entire episode of post upgrade maintenance of JD .Yes technically there are issues and maxwell is definitely working hard to fix them but looking at the issues as a end user who just logs into JD as a regular user to communicate with friends , its getting hard and we might be loosing people . By loosing people I don’t mean loosing people to other pods ( which would be awesome ). People who still are stuck to JD are those very early adopters who have gained significant friends and have participated in many discussions and just don’t want to move the account because of the lack of restore option for the backed up data !
What could be done ?
=================
Restore tool as a high priority - So that some big accounts could be moved.
Fedration issues like lack of timeline/stream for small pods could be fixed ( or are they already fixed ? )
very silly thing - but obvious question can’t we have more bigger machines ? like more RAM or CPU ?
Having said that please excuse my ignorance if there are some mistakes in what I said. feel free to correct me.
Note: This discussion was imported from Loomio. Click here to view the original discussion.
Thanks for the link . But is the forum right place ? I felt community discussions should be here. Anyways another reason I was posting was the need for the “Restore” option and also because its affecting the community as a whole not just JD because different pods are showing different things of JD posts.
Community discussion can be anywhere! Loomio is one main channel, particularly used for proposals and decision-making, and discussion leading to these.
This is really an issue concerning the management of an individual pod, rather than anything concerning the diaspora software or project activity, as far as I can tell. Even though it is diaspora’s largest pod (for historical reasons) I don’t see what the project can do about it. All those things, such as adding memory/processor power, are up to Maxwell individually while he is the person running the pod.
What’s the ‘restore’ option you mention? Are you talking about account migration? If so, this is a massive undertaking and very complex to make both easy to use and secure. No amount of saying ‘we need this feature’ is going to make it happen - people who are able to code it need to have the motivation (plus time, money, etc) to do so. It won’t happen before then.
Well, third largest these days, in terms of activity
@maxwellsalzberg found anything interesting in your investigations regarding the pod where the community could help?
As said by @goob , this is a single pod issue which the project can’t do much about. Of course we can improve on the software to handle large (and in this case, old) pods, but to improve we need to know what is going wrong. Since we have no visibility to that side (well, the tags page is slow on all pods I think), it’s difficult to do anything.
Yes its a single pod issue but if I go to a smaller pod will my timeline have everything a large pod has ? How could we suggest normal users to migrate when there is no backup and restore option. Ofcourse backup is implemented. I am talking in terms of normal user and am concerned about the many good old community members in JD
but if I go to a smaller pod will my timeline have everything a large pod has ?
No, though I’m kinda working on something related to public post replication that might help here. More info later…
How could we suggest normal users to migrate when there is no backup and restore option.
Backup is easy, and done in fact, though it fails because it is not automatic. Restore is hard, assuming you mean posts. Profile restore would be easy, and it is covered in this issue, which hopefully someone will implement soon.
An automatic backup system is really needed at some point. I’d love it to happen to other pods or maybe a decentralized network of backup servers - the backups would be encrypted of course. Currently as it is manual it’s kinda useless.
I’ve flagged these because any content importer is going to have to deal with these issues before it can succeed.
One could also import content from a site that exists and is alive if there’s an API for enumerating/fetching it. One can do the fetch bit today, but there’s nothing to enumerate what needs to be fetched and authenticate to see any private conversations (except for scraping the HTML login page and faking it with a server process); until the openid/connect stuff is widely deployed.
Otherwise we’ve got Diaspora import working in redmatrix today so you can at least keep all of your friends and not have to start from scratch. Your friends will have to reshare with you as there is no API for that bit, but we send out all the share notifications from your channel when importing.
Also of concern to an import utility is the sheer quantity of posts (and followups if applicable) to import. While working on my own import/export tools I’ve been leaning towards providing exports/imports of specific date ranges to keep the data size and processing load for any individual member under control. For instance you could provide perhaps a year of posts with the “user dump” and also offer the option to backup/restore any prior year separately. This is critical for redmatrix sites that are on shared hosting, but I’m mentioning it as a way to keep from bogging down large servers (of any importing site/project) as well, by allowing you to batch the imports rather than trying to suck in a long-time member’s complete history all at once and rendering the new pod unusable for a possibly large chunk of time in the process.
but if I go to a smaller pod will my timeline have
everything a large pod has ?
No, though I’m kinda working on something related to
public post replication that might help here. More info
later…
yeah that would be helpful , especially for most of us to spread far and have our own servers.
yeah that would be helpful , especially for most of us to spread far and have our own servers.
I’m afraid my solution might be very heavy on very small pods… but since it would be opt-in I guess that would be ok. I’ll try and write more about it today. Got a PoC solution almost ready.
Check this issue about synchronisation of comments. This definitely is not just the JD issue but it affects the entire community. https://pod.geraspora.de/posts/3820367
Maxwell answered me yesterday. Apparently the sidekiq workers are consuming more RAM than before and dying out - which causes long pauses to happen and thus the queue piles up. He promised to up the ram limits to get rid of the queue. Again, problem caused by joindiaspora being too big (and thus we should help by moving our official project account away).
Last night abt 12 hours ago the queue was approx 87K. It’s now down to 23K queued messages. So I see light at the end of the tunnel for joindiaspora.
Also, @pavithrans the problems you are seeing with comments are exactly due to this pileup of messages, not because the sofware is generally broken.
