Removing diaspora*s current chat integration

Aloha friends, it’s me again with a fresh new wall of text for you all to consume. This time, I’m writing to you about something that came up in the April hackathon. You’ve read the title, so you know what it’s about.

We spent a significant amount of time during our weekend hackathon reflecting on our past and looking ahead into the future, and even if that sounds super philosophic, we actually came up with some amazing plans. However, as part of such a session, you also have to acknowledge when things did not work out well, and eventually, you have to take appropriate action to fix those things.

One thing that never quite worked out is the attempt of implementing an XMPP-based chat into the interface of diaspora*. The result of discussion among everyone attending the hackathon (which was most of the active, regular contributor team), we decided that we want to remove the chat in its current form.

To bring everyone up to the same point, here’s a quick history recap in regards to the chat.

  • The current attempt was implemented in 2014. At that point, several members of the community voted actively against adding a chat to diaspora*, while others were firmly in favor. We had someone willing to implement it, and who claimed to be prepared to continue maintaining it, so it seemed like a worthwhile experiment.
  • The main idea behind the current implementation was to leverage the power of XMPP because we deemed it suitable for our needs. That made sense, because XMPP is, after all, federated itself.
  • To avoid having to build and maintain a custom web chat thingy, it was decided that we use a ready-to-use web chat applet. JSXC, in this case.
  • Because diaspora* and XMPP are two fundamentally different things, we had to somehow build a bridge. Initially, the attempt was to ship a Ruby-based XMPP server inside diaspora*. Later, this decision got revoked, and instead, a Prosody-based solution was worked out.

During the initial pull request and direct follow-ups, we (as in everyone involved, not just the core team) identified several critical issues in terms of usability and maintainability. One high-level thing to note here, for example, is the fact that the chat applet is just a little interactive blob that gets added to each site, with no persistance whatsoever. This results in a complete disconnect and reconnect for every page load. There are a lot more issues, but let’s not go into detail, because it doesn’t change the situation.

Because of those issues, we decided that we would merge the code, but have it disabled per default, so it’s not released to our mainstream userbase and podmins who want to give it a try could explicitly enable it. We made that move to allow room for improvements, so we eventually could enable it for everyone and have a big chat party with pizza and stuff.

Well, the thing about hindsight, eh? Here is the state as of right now:

  • Almost all of the critical issues we identified in 2014 are still present.
  • The library used for the chat widget has some significant problems, and ideally, we’d use something else, like Converse.js for example.
  • Podmins still need to manually install, set up, configure, and maintain Prosody, with some specialized plugin and special config to make authentication with diaspora* work.
  • People who volunteered to support this project are apparently no longer interested in doing so.

So basically, what we have right now is, in direct language, an unmaintainable, broken mess of code. So in April, we talked about how we can resolve this, and instead of starting to look at the technical side, we went a step back and asked us some fundamental questions:

  • What is the need we’re trying to fulfill?
  • What is the best way to address that need?

This was quite a learning experience for all of us, and we realized the previous chat attempt probably got stuck in ideology, rather than working on user’s needs. The problem we tried to solve is to allow users to exchange messages inside conversations fast and hassle-free, but we did a bad job at working towards that. (Ues, “we” totally includes people who write long comments on GitHub, who could have figured this out earlier)

After our discussion, we came up with the following plan of action ideas:

  • We want to remove the current chat entirely. It never was a released feature, setting it up is complicated, there are lots of issues, and noone is willing to maintain it. There is no point in keeping it around.
  • Instead of supporting future attempts at integrating something else into our application, we should focus on what we can already do very well: our federation layer. Because of that,
  • Instead of starting another attempt at making the XMPP chat happen, let’s first work on making our current conversation UI more… conversation’y in the eventually upcoming UI work. And from there, we can iterate.

We ran some tests, and our current federation layer manages to deliver messages actually quite fast (unless you’re on a slow pod, but that’s a whole other story), usually within a couple of seconds at max. The main reason why people don’t like the current conversations model (we talked to a few folks, especially non-technical audiences) is the high friction in UX/UI. But all the issues we discovered can be resolved, and maybe we should focus on what we have a bit more. So, for our current conversations/direct messages:

  • We should be able to implement real-time UI refreshing in a not too distant future.
  • We can also make the UI feel like a proper conversation, for example by scrolling to the latest message, and
  • Maybe we should get rid of the whole “conversation” thing, and go to a more contacts-based model, where you have a chat with one contact or multiple contacts, compared to our current model where everything is based on a “conversation” where you add contacts to. Although this is not the right time and place to discuss this in more detail, lots of people have great ideas, and we can iterate on those once we’re ready.

So, to end this, as we have consensus within the team (which includes the community team people), our current plan is to go ahead and drop the chat. If you have a strong veto, this is your chance: Please explain why you think that an XMPP-based instant messaging feature inside diaspora* is required, and please also describe your ideas on how to iterate on the current implementation/model, or on good ways to start over. We’d love to hear your feedback.


Quick Q&A

Why can’t you just keep the current chat code in the codebase until a replacement is ready?

We have talked about this a lot, believe me. The simple fact is, however, that the current code is a burden. It adds significant maintenance complexity to certain areas, and it’s a constant support load from and for podmins who want to set it up but run into troubles. Removing it would make it absolutely clear that this is not something we support.

Why do you want to invent your own messaging protocol instead of using XMPP?

We don’t want to do that. We already have UI and federation support for direct messages, and our current ideas evolve around those already existing things. There is no point in having both a “private message” and a “chat” feature, because those are, essentially, the same, so why not make them literally the same?

What happens to the Prosody authentication module?

The Prosody authentication module started as an independent project, and we see no reason why it can’t continue that way. To allow users to use their diaspora* credentials to sign in to the XMPP server, the Prosody module connects directly to the database and accesses a relatively stable scheme. No support is needed inside diaspora* itself for that to work.

What happens to pods who already use the chat?

Pods who currently have the chat enabled will lose the chat UI integrated into diaspora*. As explained above, users can continue using their diaspora* account to sign into the pods XMPP server - they’d just have to use an external client for that.

What happens to the bounties put on some chat issues?

When we close these issues, people who put bounties on the chat will have the option to either get that money back, or to donate the money into our Bountysource pool, allowing us to redistribute that money to other issues.

9 Likes

As a ‘dumb user’ (qua, non-technical) I follow your reasoning for dumping the non-functional parts of the code. As a user, I am ambivalent about the need/desirability of a ‘chat’ feature. I do seem to remember that it did once work, seemingly quite well, on the old Diasp0raca pod ; it was fun sometimes, back then, and also quite useful to be able to turn it off ! Thanks to all involved in the maintenance and development of D*.

1 Like

I realise you’re talking about the experience of using any chat feature in diaspora*; but just for the record, the chat on Diasp0ra.ca was a much earlier chat feature, built by a developer called Pistos on a fork of a much earlier version of the diaspora* code-base. The code-base had changed so much over the next couple of years that it wasn’t possible to reintegrate this code.

1 Like

Hi Goob. Yeah, I’m talking about both. I remember Pistos well and was, I would say, his friend. I remember the whole sorry saga and opened a libertree acc., never pursued it much, though. ‘Ecce Mulk!’

Nevertheless, I would have thought that what could be done then could be done now. As I said, I’m pretty ambivalent about it, what do you think the general user base wants ? I don’t know, but it seems as if there are many other options for chat on switching_social. I haven’t explored them because it’s not something I want right now.

Thanks for your reply ! …Mulk.

I don’t know. That’s what this discussion aims to find out! It’ll be interesting to see what the response is. Like you, I’m ambivalent; I have little use for it myself, but happy for it to be a feature if it doesn’t either take too much time/energy for core developers nor have a negative impact on pod performance.

If you’re a non-coder like me, it can certainly seem that way. Pistos’ feature, as I remember it, only worked on his pod – or, at least, on his fork – so at a far smaller scale and didn’t have to federate. Things do seem to get awfully complicated at larger scales and with federation, and really good solutions for small-scale local situations don’t always work at larger scales.

Me too. Bad days. I hope Pistos is happy and fulfilled, whatever he’s doing now.

1 Like

Thank you @denschub for pointing here, because I didn’t took a look into this discourse for a while.

This is not a strong veto I’ll say: go ahead!

First of all I don’t need a chat feature in my diaspora* because I have already a lot of chat accounts and protocols and clients running (for some reason my ICQ seems not to work ;D) there’s no need of another one (but tbh: there’s also not much harm in it).
So: As a user and an enthusiast I’d have loved to see XMPP in diaspora* because it’s kinda unique selling point to have a chat that actually can interact with other chats on other platforms that have nothing to do with diaspora*. And as far as I know no other social network or social network software has such a ability. (Google had it once I think, they bound G-Talk and G-mail so I guess it was also somehow connected to G-Plus).

To keep this short because I have to go working in a few minutes:
As I said I prefer XMPP over diaspora-Chat because I don’t need a chat but if there is one it should be a cool one (and interacting with other platforms is very cool).
As I understand your FAQ you somehow only remove the XMPP-Client from the code. So the server is independend anyways and there is a module allowing to somehow share the roaster with diaspora* and authenticate over diaspora*.
So this lets me hope that one day there could be an other client an other implementation of the XMPP client-stack. Maybe even some integration into conversations I don’t know.

But as I understand the origin post this is about kicking of a client that doesn’t work well. And I think that’s not too bad.

1 Like

I vote in favour of retaining the XMPP chat integration even if you cant maintain it regularly. As its a useful feature and you never know how it can be used.

  • You can give option for codebase with & without chat code so who wants can use it and/or develop it.
1 Like

Before commenting here, please read the entire opening post, and also the guidelines on this Discourse. This is not a vote, and we absolutely do not do votes anymore since we’ve moved away from Loomio, because votes are harmful. If you have arguments besides “I like it”, then you are very welcome to articulate those here, and we can consider of those arguments are strong enough to not get rid of it.

Maintaining two codebases - one with the chat, and one without - is absolutely not an option. We’re interested in making our stuff more predictable and more maintainable, we’re not interested in duplicating all our efforts.

“You never know how it can be used” is not a great argument to build something. Either we know how it can be used, which means we can spend resources on that feature to make it better for those use cases, and if we don’t know that, everything I wrote in the opening post is valid, and is absolutely not a reason why we should keep something that is cause for pain.

2 Likes

Did I get it right at all,
if a podmin likes to keep the chat up she can still run the modified Prosody and some authentication module. And the users will still be able to use their diaspora-ID with Jabber they will just have to use an external client.
Correct?
What is about controlling the roaster with diaspora*? Because 'til now one adds contacts into a chat-aspect to add them to the XMPP roaster as far as I know. Would that be removed too? So people would have to add their diaspora* contacts using their clients. Or will it still work?

Correct.

That’s another prosody module called mod_diaspora_contacts, which also gets its information via direct database access, not via special APIs or custom functions inside diaspora. That module will not break if we remove the chat integration - but it might break if we do database adjustments, in which case someone needs to update that module, which may or may not happen.

Edit: I’m a goofball, read @supertux88’s reply below.

The roaster integration will break, since it relies on the chat_enabled column which will be removed. But it’s also one of the parts which doesn’t work well, it’s more or less random when new contacts are visible in your roaster (can take hours), people don’t see if something is broken, or if prosody just didn’t update yet. If somebody wants to update that module to work without the chat_enabled column and just add all aspects to prosody, then it will work in the future, but I wouldn’t recommend that. Adding the contacts you want manually works just sooooo much better.

1 Like

What a pity,
but if it’s broken anyways… hum.

Well with direct DB-access one could still take the contacts from a single aspect I’d guess. Maybe the query is much more expensive. But I think it’s possible to read “which aspect is named chat and which contacts of this user are in there?”

Okay, so the removal of the client breaks the whole XMPP-chat-thingy at all. Well not what I hoped but thank’s for the information :slight_smile:

I’m thinking that if there’s that many issues, some of which are fundamental if I’m reading right, I think I’d rather see y’alls limited resources going into the overall UX, which is what most people “see” when they come here. I just don’ think Yet Another Chat should be a priority.

Agree on updating the DM UI/X tho. It’s pretty clunky, and adding features there (DM an aspect, perhaps?) would be much more useful overall.

Hi all,

Thanks for the thoughtful, well-composed dialog. I briefly ran the XMPP chat but found it underutilized and very difficult to support, and there were significant UX issues. I think consolidating efforts to a single, unified solution makes a lot of sense.

:+1::+1:

Heh. I do appreciate your support, but please do note that we removed the chat… a year ago, and it will not be part of the next major release.

My mistake, I should have updated this thread. :slight_smile: