Aloha friends, it’s me again with a fresh new wall of text for you all to consume. This time, I’m writing to you about something that came up in the April hackathon. You’ve read the title, so you know what it’s about.
We spent a significant amount of time during our weekend hackathon reflecting on our past and looking ahead into the future, and even if that sounds super philosophic, we actually came up with some amazing plans. However, as part of such a session, you also have to acknowledge when things did not work out well, and eventually, you have to take appropriate action to fix those things.
One thing that never quite worked out is the attempt of implementing an XMPP-based chat into the interface of diaspora*. The result of discussion among everyone attending the hackathon (which was most of the active, regular contributor team), we decided that we want to remove the chat in its current form.
To bring everyone up to the same point, here’s a quick history recap in regards to the chat.
- The current attempt was implemented in 2014. At that point, several members of the community voted actively against adding a chat to diaspora*, while others were firmly in favor. We had someone willing to implement it, and who claimed to be prepared to continue maintaining it, so it seemed like a worthwhile experiment.
- The main idea behind the current implementation was to leverage the power of XMPP because we deemed it suitable for our needs. That made sense, because XMPP is, after all, federated itself.
- To avoid having to build and maintain a custom web chat thingy, it was decided that we use a ready-to-use web chat applet. JSXC, in this case.
- Because diaspora* and XMPP are two fundamentally different things, we had to somehow build a bridge. Initially, the attempt was to ship a Ruby-based XMPP server inside diaspora*. Later, this decision got revoked, and instead, a Prosody-based solution was worked out.
During the initial pull request and direct follow-ups, we (as in everyone involved, not just the core team) identified several critical issues in terms of usability and maintainability. One high-level thing to note here, for example, is the fact that the chat applet is just a little interactive blob that gets added to each site, with no persistance whatsoever. This results in a complete disconnect and reconnect for every page load. There are a lot more issues, but let’s not go into detail, because it doesn’t change the situation.
Because of those issues, we decided that we would merge the code, but have it disabled per default, so it’s not released to our mainstream userbase and podmins who want to give it a try could explicitly enable it. We made that move to allow room for improvements, so we eventually could enable it for everyone and have a big chat party with pizza and stuff.
Well, the thing about hindsight, eh? Here is the state as of right now:
- Almost all of the critical issues we identified in 2014 are still present.
- The library used for the chat widget has some significant problems, and ideally, we’d use something else, like Converse.js for example.
- Podmins still need to manually install, set up, configure, and maintain Prosody, with some specialized plugin and special config to make authentication with diaspora* work.
- People who volunteered to support this project are apparently no longer interested in doing so.
So basically, what we have right now is, in direct language, an unmaintainable, broken mess of code. So in April, we talked about how we can resolve this, and instead of starting to look at the technical side, we went a step back and asked us some fundamental questions:
- What is the need we’re trying to fulfill?
- What is the best way to address that need?
This was quite a learning experience for all of us, and we realized the previous chat attempt probably got stuck in ideology, rather than working on user’s needs. The problem we tried to solve is to allow users to exchange messages inside conversations fast and hassle-free, but we did a bad job at working towards that. (Ues, “we” totally includes people who write long comments on GitHub, who could have figured this out earlier)
After our discussion, we came up with the following plan of action ideas:
- We want to remove the current chat entirely. It never was a released feature, setting it up is complicated, there are lots of issues, and noone is willing to maintain it. There is no point in keeping it around.
- Instead of supporting future attempts at integrating something else into our application, we should focus on what we can already do very well: our federation layer. Because of that,
- Instead of starting another attempt at making the XMPP chat happen, let’s first work on making our current conversation UI more… conversation’y in the eventually upcoming UI work. And from there, we can iterate.
We ran some tests, and our current federation layer manages to deliver messages actually quite fast (unless you’re on a slow pod, but that’s a whole other story), usually within a couple of seconds at max. The main reason why people don’t like the current conversations model (we talked to a few folks, especially non-technical audiences) is the high friction in UX/UI. But all the issues we discovered can be resolved, and maybe we should focus on what we have a bit more. So, for our current conversations/direct messages:
- We should be able to implement real-time UI refreshing in a not too distant future.
- We can also make the UI feel like a proper conversation, for example by scrolling to the latest message, and
- Maybe we should get rid of the whole “conversation” thing, and go to a more contacts-based model, where you have a chat with one contact or multiple contacts, compared to our current model where everything is based on a “conversation” where you add contacts to. Although this is not the right time and place to discuss this in more detail, lots of people have great ideas, and we can iterate on those once we’re ready.
So, to end this, as we have consensus within the team (which includes the community team people), our current plan is to go ahead and drop the chat. If you have a strong veto, this is your chance: Please explain why you think that an XMPP-based instant messaging feature inside diaspora* is required, and please also describe your ideas on how to iterate on the current implementation/model, or on good ways to start over. We’d love to hear your feedback.
Quick Q&A
Why can’t you just keep the current chat code in the codebase until a replacement is ready?
We have talked about this a lot, believe me. The simple fact is, however, that the current code is a burden. It adds significant maintenance complexity to certain areas, and it’s a constant support load from and for podmins who want to set it up but run into troubles. Removing it would make it absolutely clear that this is not something we support.
Why do you want to invent your own messaging protocol instead of using XMPP?
We don’t want to do that. We already have UI and federation support for direct messages, and our current ideas evolve around those already existing things. There is no point in having both a “private message” and a “chat” feature, because those are, essentially, the same, so why not make them literally the same?
What happens to the Prosody authentication module?
The Prosody authentication module started as an independent project, and we see no reason why it can’t continue that way. To allow users to use their diaspora* credentials to sign in to the XMPP server, the Prosody module connects directly to the database and accesses a relatively stable scheme. No support is needed inside diaspora* itself for that to work.
What happens to pods who already use the chat?
Pods who currently have the chat enabled will lose the chat UI integrated into diaspora*. As explained above, users can continue using their diaspora* account to sign into the pods XMPP server - they’d just have to use an external client for that.
What happens to the bounties put on some chat issues?
When we close these issues, people who put bounties on the chat will have the option to either get that money back, or to donate the money into our Bountysource pool, allowing us to redistribute that money to other issues.
