Groups are a hugely important feature for many who might think about moving to diaspora. As such, they are a huge priority.
However, there are some technical difficulties - how should we move forward with these?
Note: This discussion was imported from Loomio. Click here to view the original discussion.
The technical difficulties amount to federating messages. By creating a groups feature, we are essentially extending the project to also be a large distributed forum. Similar to newsgroups, I assume, people would subscribe to different groups and receive all updates on those channels. If we want to model groups after Facebook, we’ll need to federate group comments and group messages separately. We can also model groups more like Usenet and simply have 1 mode of transport, leaving the hierarchy of messages up to the view layer and disallowing commenters to alter the subject of their federated group message. Unlike sharing posts, which federate to a whitelisted select few people, group messages would be federating to a much wider group of people. I would raise that as a concern just as a gut reaction, but I’m not sure of the actual impact as I do not run a large pod.
@tomscott - thanks - I’m not sure I fully understand; what does federating messages (as opposed to posts?) involve?
My initial reaction is that group messaging is less important than group posts etc. Or could group messaging be implemented later - client-side (?) could take the list of group members and add it to a private message to: list [I guess this wouldn’t currently work as you have to have mutual sharing to be allowed to pm someone].
In the longer term I would also be in favour of having plugins for groups to be able to have their own wikis or etherpads for collaborative editing…
@nickdowson well, it’s more a matter of volume. if i post in a group that has 1000 subscribers, that message has to get federated out to a maximum of 1000 pods, quickly. that is a different scaling problem than, say, Facebook has, because they can just spin up a bunch of web servers that all connect to the same database cluster. with us, we’re attempting to keep multiple databases in sync to ensure the data is distributed to each necessary outfit. by “group messaging”, i meant group posts, by the way…i’m just using the term “messaging” here to describe how pods communicate. sorry if my language is a bit unnecessarily esoteric. 
anyway, my major concern is how are we going to distribute this data quickly and efficiently across a wide range of pods. in the current federation protocol, we do have some issues wherein a post won’t federate completely across everywhere it should be federating. i don’t believe there’s a fault-tolerance system baked in that ensures posts always make it to their destination. these features seem NECESSARY for group-based interaction, because suppose you make a post in a group and 50/100 of the pods that need to get it, don’t. that discussion either dies, or worse, it gets replied to. the replies federate across the group and are referencing a post that for half of the pods doesn’t even exist. before we implement a group feature, we need to figure out a solid solution to that problem.
So, one of the things groups currently would depend on is federating properly. We also need to think of a basic UX implementation.
Pistos had a groups feature on his fork of Diaspora that I really loved. The problem is that his feature is not compatible with Backbone.JS, and so would probably need to be rewritten, with tests.
What I liked particularly about Pistos’ version of groups was that it was similar to how StatusNet implemented groups. Groups could be mentioned in a post using a bangtag, so !cats@joindiaspora.com would go to a cat group hosted by joindiaspora. Joining a group is kind of like a followed hashtag, except that there’s an admin that can edit the page, or remove entries if they’re offensive/spam/whatever.
The real barrier, I think, is federation support. At some point in the future, we may need to further consider our federation system and think about going with a better-supported protocol, such as OStatus as a whole (Diaspora only uses some of OStatus), or Tent, both of which are exploring solutions to existing problems that we ourselves have seen in Diaspora and other networks.
One of the long-standing problems of our federation system is that it currently needs refactoring. A general consensus is that putting federation into a layer of some sort may be beneficial, and even thinking about touching protocol work hinges on the state of our federation system and workers.
On that last point, @seantilleycommunit , would it be worth doing a call for volunteer developers to form a working party to look into the feasibility of putting federation code into its own layer, and implementing it if feasible? It should be headed up by one of our main current developers, but could include some people new to the project. It seems to be a high priority, but the size of the task seems to be holding up getting it done. I think it might be a good idea to try to form dedicated groups for some of the biggest and most important development tasks such as this and creating an API. What do you think?
That’s really a separate discussion, but I moot the idea here and if you think it’s a good plan, I can start a separate discussion.
Ok, thanks for running through some of the difficulties. @goob that sounds like a good idea - I would think that going with a well-supported protocol should also be high priority; do the federation layer and the protocol support need to be addressed at the same time or from a technical perspective does it matter in what order they are addressed?
@seantilleycommunit - as I understand it Ostatus is not privacy aware? correct me if I’m wrong! I would think that developing/supporting/collaborating on a protocol that goes wider than diaspora is highly important to the success of both diaspora and the social web (perhaps this page is not the place for that particular discussion though)
Great discussion here. I fully agree with Sean, but also want us to keep in mind, that groups in Diaspora* should probably something more than groups in OStatus. Groups in OStatus are like an open channel to which everyone could post messages to and anyone could subscribe that channel just like following a person. But when I think of groups, I’d really want a group to not always be public and give always public information to everyone. The group should especially control, who can join the group (admission-mode) and who can read which of the group’s contents. So to me it seems as if we could take the groups from OStatus, but also need to add a privacy-layer.
Nick Dawson, adding further information to a group like etherpads, wikis or file-repositories isn’t the very big deal after all. Since we rely on the activitystreams-protocol, we can support any activities for any modules in the future. So when Tom Scott says “message”, he most likely means a generic message, that could be anything like a posting, a like or an editing of a wiki-page.
@rasmusfuhse I agree about the importance of privacy options (though as a start I think a blanket option to make all content private/public is fine).
Could you tell me more about the activity streams protocol?
Activitystrea.ms ist the webpage you’re looking for. An activity is a small piece of XML or JSON, that consists of at least subject, verb and object. A casual posting is the triple of “Rasmus”, “post” and “bla bla”. But you could also deliver any activities like for example “Rasmus”, “edit”, “My Wikipage”.
Just bringing this up again while support for groups is hampered by federation, I would propose a halfway house:
We implement groups, but restrict them to only users of a certain pod - and make it clear that this will be the case until future development makes federation possible.
I think this is important as a proof of concept and intent, and will also make diaspora potentially highly useful to a range of organisations and social movements - who could host their own pods and groups for decision making and communicating by members.
This would also be great for growing the diaspora network, as organisations are likely to have the resources to set up and maintain their own pods.
(This discussion is also related to this one here: https://www.loomio.org/discussions/1188)
@nickdowson By forking diaspora/diaspora, you (or someone else) can spike on this right now. There aren’t many roadblocks when getting Diaspora to run off a different Git branch, so by all means…please do some experimentation and tell us what you find!
@nickdowson That’s an interesting thought; at the very least, we could have some basic code in place that could be improved upon later.
I think a good first step, though, is getting @florianstaudacher’s Diaspora federation gem working with Diaspora. That could allow us to abstract away the working federation code in D*, making federated feature development much easier.
Actually, doing so could bring two very good benefits:
Allow for a modular federation abstraction. So, let’s say we fork the existing Diaspora federation gem, and do one for Friendica Red’s Zot2 schema, or we implement the Tent federation API. To some degree, we can make switching out gems easy, so that we can experiment with different protocol standards and see which ones suit our needs best.
With a level of abstraction, all of our existing federated features will make calls to equivalent code in the gem. So, swapping out federation gems would hopefully mean that features would have to only be adjusted very minimally.
I’m not keen on implementing a feature that would only be available to users within a pod and not cross-network. We need to be doing everything we can to enable and encourage inter-pod communication and mobility. Having a feature which only work intra-pod would mean that people who were connected would feel they need to be on the same pod as each other, which would lead to people wanting to migrate towards the huge pods, where all the people are, rather than away from them.
I appreciate your desire to have groups in Diaspora, Nick, but it can only be of use to individuals, and to the network as a whole, if some fundamental things are in place first. Let’s focus attention on getting federation working properly, so that being on a single-user pod gives pretty much the same experience as being on a huge pod, and worry about features like this once the network architecture is ready for them.
@seantilleycommunit do you know where I can find that gem so I can help him? This has been something I’ve wanted to do for a long time and had no idea work was already underway.
Since groups need a special way of federation information to other pods, we couldn’t program the federated groups unless we have put federation into an abstraction layer. It would be no good to make ourselves double work.
But a prototype could be nice. Pistos already did something like this prototype two years ago.