I support diaspora development since a couple of weeks and I often get lost in the rails / haml / handlebars / backbone /mobile ecosystem.
On implementing new features this combination often feels (for me) quit confusing.
What if… diaspora would make a kind of restart with modern Rails (7) and mostly without backbone and handlebar templating?
Just to get some opinions.
Hello. Yes, obviously, the front-end is aging. We had a discussion on that already, you can read about here: Choosing new frontend framework
The thing is, we don’t even take time to fix bugs or finish the new website for the foundation, so honestly I don’t think that a complete rewrite will happen…
So I agree that current front-end implementation is heavily outdated… That’s was one big demotivator for me during the migration work – it was not clear whether I should build something using the legacy stack if we consider replacing it…
However, it’s true that we don’t have resources to rebuild things.
I’d say a more realistic way is to gradually shift to new technologies. This is what I’ve seen in business as well. Few businesses are ready to invest in rewriting the application from scratch to a different tech stack. But it’s quite fine to replace the old code with new gradually.
So we could just start building new parts with (let’s say) React and rewriting things while keeping everything else in Backbone. There is no issue connecting different stacks, I’ve seen things like that working just fine. Probably not beautiful code-wise, but working.
This is the best pragmatic approach IMO. Much better than just throwing everything away and rewriting. And personally I find it fun to replace things part by part in small steps – this feels quite rewarding – comparing to putting a lot of effort on rebuilding something just to build something that we already have (in terms of user features).
One other alternative could be to develop insporation and make a Web based build of the Flutter app. If we go this way as a team, we could stop developing the current Web front-end at all and just make the Rails app an API backend. I have mixed feeling about this. On one hand we still don’t have much resources to build two clients at the same time, so it kind of makes sense to squeeze as much as possible from one. But on the other hand it’s cool to have some choice, and for me it’s much easier to contribute to the Web app rather than learning Dart and Flutter which I’m totally new to.
So here’s my suggestion of what we can do:
The primary reason why I personally am not a huge fan of the “add a second stack, and rebuild components” approach is that we’d just be re-building what we already have, even thoough we know that there are a lot of papercuts we could fix.
A clean-slate approach would allow us the opportunity to do so, without constantly changing small things over a long period of time, potentially annoying users. However, since there’s a clear lack of resources on this topic, maybe you’re right, and running dual-stack and porting the existing UI would be a first step.
Good to see your name. Take care. 
I wouldn’t have started this discussion if I didn’t have an idea about this.
To be honest, I did not have in mind to change the “backbone” / handlebars system with something like react/vue/angular whatever fancy framework.
What about ‘back to the roots’ and make use of what Rails 7 brings in?
I think about the new turbo / stimulus things which are very powerful.
To get an example of what is possible with little effort, I have made a proposal:
Have a look here:
https://dnext.societas.online/streams/public
If you have already an account on societias.online you can even login. This is made completely with rails 7 and turbo/stimulus. It should only show what can be archived without using extra third party frameworks.
Turbo, and the rest of the Hotwire stack, is a third-party dependency. It’s DHHs companies toy project, and there are no guarantees in the long term. It builds on top of a lot of Chrome-only features, and for other browsers, they just slap a bunch of expensive shims on top. I’m not a fan of anything in that stack, but my opinion may be tainted by the fact that I recently had to spend more than a day of work on figuring out why Turbo caused a race-condition in Firefox and Safari, and DHH and team didn’t even bother investigating.
Rails’ frontend-situation is a hot mess. Sprockets is technically still supported, Webpacker seems to be the current “default”, but they also support rails/importmap-rails which would render Webpacker obsolete for JS (and bundling CSS is rarely ever an issue). Picking anything frontend-related that comes out of the Rails world right now feels like playing poker, and the odds that you end up in an unmaintained stack are against you. And we’re already in a situation where part of our current stack is rotting - the only reason why diaspora still works as is is because the project hosts its own Ruby Gems server, mirroring the rails-assets dependency we use.
If a bunch of time is spent on building something new, I’d much prefer this to be standalone. Not only because that makes the project resistant to whatever the Rails projects decides to do on any given day, but it would also allow the frontend to be used by a backend stack that is not Ruby on Rails - which is an interesting fact all by itself. Copy-pasting React component out of a rails-webpacker based intermediate solution is easy, but turning a Hotwired-based frontend into a standalone thing is anything but.
I think choosing React could lower the bar as low as it can be for people to contribute. It’s basically the most popular JS front-end framework and a lot of people have experience with that one way or another. Sometimes people manage to build React applications without even understanding much of JS.
Using something unpopular lowers the chances that the potential contributors will go through the struggle of learning something they are not familiar with.
However if @tclaus you want to actively work on the new front-end and have a strong objection against using something like React, then definitely your opinion is of more weight that of some potential contributors that we might not have at all.
We could look at this problem a little differently as well… We could start a new front-end application from scratch, in a separate repository based on any stack we want (I would start with React though). And we build the new front-end based on the API calls (the authorization can be a question - I don’t know how to keep the token safely in such use case. local storage?)
And eventually when it’s ready we could just suggest the podmins to install the new frontend alongside with the diaspora Ruby app and configure their web servers to route the application requests to the new front-end app.
@jhass isn’t it true that we can actually make a build of Insporation for Web? That could be a nice alternative to the regular web app
You can but the one time I tried it was a pretty bad experience still. Not even talking about layout not scaling to a big display like that, the ergonomy isn’t very great due to the way Flutter works, rendering everything on its own. You can’t really select text and stuff like that.
Implementation wise we would need to backfill a bunch of native code/find web compatible libraries, namely for appauth (OIDC) and image selection.
That’s pretty sad, I had a hope we could reuse some of this work for a web front-end…
Okay, how about replacing rails-assets with yarn and node_modules ?
I’ve tried doing it for some packages and it seems more or less straightforward.
That would add another build dependency for us – Node.js and yarn. We’d need to update our installation guides and people would have to install Node.js additionally to Ruby in order to build their assets. But on the other hand this is pretty much an industry standard now for front-end development and that’s what everybody else is doing so it shouldn’t be a terrible problem.
And for us it would mean that we wouldn’t need to maintain our rails-assets instance anymore (once everyone upgrades).
Would be a nice first step to upgrade stuff, maybe we could incorporate webpack later, but it’s a separate topic. I think this is valuable by itself.
CC @denschub
Oh, you added the node_modules folder as a source for Sprockets instead of using webpacker or anything. Interesting. I really like that approach!
It’s not more or less “risky” than having to build our own asset gems. The build pipeline for those Gems already just load the package form NPM, and then throws Bower at it. And anything that will get rid of the dependency on our infrastructure is good - I really don’t like the fact that we’re hosting a custom Gemserver right now.
We already let people install nodejs in our installation guides. In theory, some podmins might have discovered this comment and installed therubyracer instead, but that was never supported and the install guides ask to install NodeJS for a long time. I think we’re good on that front.
One thing to note is that we should find a way to run yarn install during/on rails assets:precompile or similar. I don’t think adding another command for podmins to run is a good idea as it will likely be missed. I’m not sure if we can just run something “before” assets:precompile, or if we have to add a custom task like assets:build that does everything, but this shouldn’t be a big issue.
Well, one problem here is that to run yarn install we need to get yarn installed first and npm install -g yarn is another command that could be missed. So we either install yarn during the assets precompilation (doesn’t sound very clean to me), or… we could maybe just use NPM instead of Yarn? Yarn is preferred by many people but I guess in our case NPM can do the job just fine too
Okay, so I’ve just checked the installation guide for Ubuntu.
This is what we do to install nodejs.
sudo apt-get install build-essential git curl gsfonts imagemagick libmagickwand-dev nodejs redis-server libssl-dev libcurl4-openssl-dev libxml2-dev libxslt1-dev libpq-dev
And I’ve realized that we can install Yarn with apt-get too, by adding yarnpkg to the command.
I don’t know if every Distro that we officially support guide for has yarn in the repositories, but at least the most popular (I assume) does.
Mh, I don’t have strong feelings about this. We don’t need to trigger a -g. A npm install -v 1.22.19 yarn for a local installation, and then using npx yarn install to run it. As a bonus, this would allow us to make sure the right yarn version is installed, which also can be… exciting.
But npm is fine as well, of course. Precompiling assets takes so long anyway, I don’t think the performance gain with yarn is relevant. 
I needed to change it a little to make it work without changing or creating any lock files:
npm install yarn@1.22.19 --no-save --no-package-lock
But the approach seems to work.
Looks very reasonable to me.