Changes to the diaspora protocol

@supertux88 Like I already said, I totally forgot about the nodeinfo stuff. When it was short before release I heard of it again (I guess @jasonrobinson pointed me to it) and implemented it in Friendica. By doing this I realised two things that could have been improved to it. One of these things were realised by @jhass very quickly. (I really would have understood it if he just had told that this would be changed in a version 1.1 but not 1.0)

I definitely need persons in the background who constantly point me to things like this so I don’t miss them. (Something like a project manager) For this we need persons like @jasonrobinson. Without him I guess I will miss very much. I simply can’t concentrate too long on a single thing and I can’t overview too much things at the same time.

I’m really sorry that @jhass waited so long for a reply to his proposal and that it came so short before release.

@michaelvogel don’t worry so much, the issue wasn’t that you gave no feedback, the issue was that nobody did. And with this history a proposal like this which demands to wait for feedback from everyone that could possibly be interested for code we never officially declared third party support for and that has design flaws which need to be addressed is just unacceptable.

@jhass I guess we could had agreed to something like “notifying all stakeholders and giving them a fair chance to reply and to listen to their concerns”.

Of course this can’t mean waiting for several months.

Concerning this “not officially declared third party support” we should all accept that the connection between the systems is a fact.

I just had a short look. In my data I found about 240 active Diaspora servers, 112 servers running RedMatrix and Hubzilla and 395 servers running Friendica. (Of course there are much more Diaspora users than Friendica users because Friendica servers are mostly used by only a few people)

All these systems can interact by now.

The number of developers who are working on internals on these systems are low. At Diaspora there are 2 or 3 persons (AFAIK) working at the communication core. In Friendica it’s mostly me and on RedMatrix it’s mostly @mikemacgirvin.

So I guess that in a “case of emergency” it should be no problem to speak with each other. And I guess that in this phase of the communication we all can learn about what could be done, once the protocol is really well defined and there are (hopefully) masses of developers from others systems willingly to implement the protocol.

All these systems can interact by now.

They can interact by now, more or less stable … and I can’t guarantee that this doesn’t break, because the implementation on the diaspora end is buggy and undefined, and I don’t know the implementation at the friendica end. But I try to stay backward compatible as much as I can.

If you want to be informed about every little step, then it is the best, when you follow the federation-gem commits. But if you don’t have that much time, then I will inform you when you really need to change something (because we are removing backward compatibility) and give enough time to change it.

So I guess that in a “case of emergency” it should be no problem to speak with each other.

If something unexpectedly break, then of course we can talk to each other.

I’m happy with some warnings when you know there could be compatibility issues, but also a bit more sensitivity when it comes to fixing issues - waiting a month to revert something that breaks federation in a big way isn’t a suitable outcome. I’m not happy adding a salmon declaration to our XRD to keep it working because we don’t support salmon. (And neither does Diaspora). I’m not happy adding a url alias because we removed lrdd some years ago (and Diaspora never implemented it). Both are a huge support burden for our over-stretched devs, as is Diaspora itself, but our members seem to want Diaspora; even though half of the useful features they insist on having won’t work with Diaspora. I need to figure out how to make post/comment edits and nomadic identity work with Diaspora or we might have to disconnect - we can’t support our core features across that platform and this is a HUGE support burden for us. So I understand that trying to maintain compatibility with “reverse engineered Diaspora” implementations on other platforms is a support burden for you.

If the API was complete we could just use that and be done with it. But in the end we have to respond to our members and fix their problems. Whether you like it or not, we and many of our members are all members of the Diaspora community in some degree, just as you are members of our community. We need to work together to figure out how to lower the support burdens for everybody. Theoretically all you need to do is maintain a consistent protocol for us to be happy and attempt to provide backward compatibility as you move forward. As we’ve discovered that isn’t always as easy as it sounds.

On our side we have to find ways to support stuff that you haven’t even thought of yet and hence don’t have any support for. We have to somehow turn all of these things into something that maps to the Diaspora protocol - or our members complain. It’s daunting. I’ve got no idea how to support magic-auth or nomadic identity on Diaspora and I’ve been struggling with trying to find a way for a few years now. I also don’t know how to make event participation work from your side. Maybe we could parse keywords in comments. We did get forums working across platforms (at least public forums). And polls are easy. I’m mostly waiting for somebody to want those bad enough to do the work, as I’m tired of doing it all.

I sort of question the need to go back now - four years later and fix issues with the XRD hacks Diaspora initially put in place. (I mentioned these to Ilya long ago and he was aware of and trying to figure out what to do about them before his unfortunate demise). XRD has been obsolete for three years and most everybody has moved away from it. I’d rather spend the time converting our Diaspora stuff to rfc7033 so we won’t have to waste resources with 5 or six different URL fetches for every handle lookup. That and scraping HTML webpages for info. Yuck. Adding unimplemented stuff to or requiring unimplemented stuff in the XRD or moving it to an hcard we have to scrape is really missing the point and just making work for everybody. Nobody uses that stuff anymore. Anyway I’m not being confrontational - just trying to provide my viewpoint so perhaps we can all understand each other; even if some of us don’t get along and have deep hatred towards each others’ projects. If I had the deep hatred of Diaspora that some think I do, I wouldn’t have implemented cross platform federation on three different projects. Frustration? Absolutely. Hatred, no.

@supertux88 Following the commits isn’t something I would like to do - especially because I won’t be able to exactly see what has changed.

Is there a documentation of the used endpoints? (Like /.well-known or /p?) I would like to compare them with the equivalent implementation in Friendica to make it more compatible.

BTW: I agree with @mikemacgirvin that the scraping of HTML pages is a strange way to fetch information. I really would suggest something more structured like XML or JSON.

P.S.: I had some suggestions for the protocol. Is Loomio the right place for it?

I disagreed because I don’t agree with “Deliver a plan of changes being done to stakeholders when those are planned or latest when work begins on them” and also I don’t agree with this from the OP “Even if it is "YOLO code”, coming from one developer, that doesn’t mean it doesn’t have to be supported, maintained and more importantly, not broken for ALL stakeholders.“

How would you call that if not communication? So, we agree, you’re not even willing to communicate.

That is not correct: a UI change doesn’t need the OK from everyone.

Is that a joke!?