Choosing new frontend framework

It’s no secret that diaspora frontend needs major changes. As it was discussed earlier the desired way for that is to upgrade the frontend by integrating some modern framework on top of the diaspora API.

Even though the diaspora API is not ready yet, we could actually make some steps towards new frontend based on some new framework. But before we could actually start we should decide which framework it would be.

Today most of the complex applications use some kind of reactive frameworks.

Two popular frameworks I know about are React and Vue.js.

I can’t really argue which is better, but both solve pretty much the same problems.

I have some experience with Vue.js and I would prefer it myself just for that reason. Also I can see that @jaywink uses Vue.js in his socialhome software and adopting Vue.js would allow us to exchange frontend code and solutions.

Vue.js is good in gradual integration. For example there is a vue-backbone compatibility layer which would allow us to use Vue.js along with our old frontend code without full rewrite.

I don’t have any experience with React so I can’t really tell if it worse or better than Vue.js.

So I can’t say which framework fits better for us, but I think we should decide that eventually. Hopefully in this topic we’ll manage to do that.

2 Likes

So, you’ll get a two part answer now. Let’s start with your initial question.

I am a React person, and although I have looked at Vue.js (and by “looked”, I’m talking about several days of research, not just reading the README.md), and I stayed at React. Vue is opinionated, and while that’s nothing bad, it’s just not my taste. :slight_smile:

In detail, I dislike Vues template system, for once. I was really happy when we got rid of template languages in React by either using JSX (which basically mixes plain JS calls with plain HTML code), or by using the createElement functions. I’m liking the latter approach, because it basically allows me to create HTML by not writing HTML, resulting in what is to me pretty code. In addition, it enables (read: forces) you to build generators, maps, … for repeating components. With template languages like the one used in Vue, you often end up doing the classic for loops. That being said, it is true that this makes writing components a bit harder in the first place. However, given that we would end up with a lot of examples to get inspired by, I doubt this is an issue.

Talking about opinionated, let’s talk a bit about data handling. Vue seems pretty opinionated towards Vuex, which is their first-party replacement of Redux, which is a central state management. First, I absolutely do not understand why they even bothered developing Vuex as a fully standalone project. Redux has nothing to do with React, and all their custom integrations could be easily modeled upon Redux. It’s a nice task, but I am not sure if people should reinvent the wheel over and over again. But each their own. Comparing Redux vs. Vuex, Redux is superior in my opinion, mainly for the higher flexibility for extensions, and the tooling around Redux is superb.

At the end of the day, there are some things Vue does nice, and I’d like to completely derail this discussion now and ask another question:

Does it matter?

You made two assumptions I think are not applicable in our case:

  1. Copying/reusing components from other projects is possible.
  2. Gradually shifting/filling/mixing Backbone and Framework X is a good idea and we could/should start gradually embedding another framework.

To 1: out of experience, this is not the case. Even if you find a component, regardless if it’s React or Vue, you need to do a lot of work to make it fit your application. In the end, you mostly end up taking over or reimplementing the logic within your application base, and at that point, it absolutely does not matter if the component you get inspired from is using React or Vue. At the end of the day, both framework are using very similar concepts, and porting between them is not a huge deal. The ability of using components should not impact the choice of Frameworks here.

To 2: I disagree. :slight_smile: There is more to a frontend than the view layer. While it is true that you can use Vue with Backbone, so can you use React with Backbone without any issues. Again, this should not impact our decision either. But I don’t think this is a good idea.

We have a lot of issues in our frontend, and some of them are big enough that I do not have fun working on it, and those are not related to the view layer. We have serious issues in the way our data flows and in the way we store and handle said data. We do XHRs everywhere, we have some global data, we have some data provided via templates, and we have a lot of magic. Adding Vue or React will not make this issue go away, if everything, it will make the entire frontend more complicated.

Now, one could say “well, we’ll just write components, and we can reuse them when we fixed our data handling”, and I do not agree with that. The way components are written is highly dependent on the way you let the data flow in, regardless of how good you design your components. So in the end, you’ll end up with a lot of required changes just to adopt to the new data handling.

I actually have even more concerns against adding another framework, but ultimately, I think that, in our specific case, adding another framework to our current frontend is a bad idea.

As we are now at a point where we could rename this thread to “Engineering the new frontend”, let’s actually talk about data handling. Vue seems pretty opinionated towards Vuex, which would be similar to the famous React/Redux combination. For most applications, I’d totally agree that this is great and happily use it, but I am not sure if it is the right choice for diaspora.

Central state storage and bound components like with Redux/Vuex are complex. Like, really complex. When you consider the amount of points we have to pull data from and push data to, we’ll quickly get to a point where data handling is so complex that only people with experience in React/Redux or Vue/Vuex can contribute more complex tasks. Contributor friendliness, in addition to future maintainability, is something to consider very carefully.

Also, I do not feel like a central state storage with bound components is in a good idea for us in general. If you think about it, we actually do have a lot of standalone components with isolated data sources (think about the notification thingy). And maybe, this is reason enough to really design something that works best for us.

I recently had a chat with some of the folks who work on and with Flight by Twitter, which is going “back” to a global event dispatcher, almost a bit like a PubSub channel for the frontend. I do not like some of the decisions they did (like, depending on a lot of jQuery), but I like the idea of a global event hub. In fact, I like this so much that I started playing around with a custom implementation of a “event based data flow” component that keeps its state in Redux and passes it to React. However, this has some other issues I am still in the process of figuring out, and I am not comfortable with claiming that this is a good solution at all. But if you think about this in the scope of diaspora, it actually would make some sense.

To end this post: these are all just brainstormy ideas, not proposals or worked out plans. But I think we should not keep doing as we’re doing, and as you are suggesting, and not work on embedding another framework into our current frontend. I’d like our frontend to be a 100% API consumer, and a nicely designed one.

Instead, I propose to repeat our “success story” we had with our federation core: Throw it away. Think about it. Think about it some more. Draft specs. Think about the specs. Build something fancy, new and maintainable. I think this is the only way we will end up with something that is good, and ultimately, fun.

I am aware that this is a huge project, and I am aware that technically, we are not there yet. We have no API. Still, this does not stop us. We can plan, we can discuss. We can work out UX concepts. And, we have an API spec, so writing a dummy server that responds with predefined JSON payloads is easy.

Maybe we should have some discussions about this first. That is, if we should target a new, from-scratch, implementation, or a gradually changing one. Because that will impact a lot of decisions further down the road.

1 Like

First of all, I’m very glad to see that discussion happening :slight_smile:

I never worked with React nor Vue so I can’t tell which one should be pick. Most of my developer friends I asked this question answered Vue, saying things like “Something always bothered me with React, Vue feels smoother and more intuitive to use”. However keep in mind that they don’t know the diaspora* codebase, so it doesn’t mean Vue is the solution for us.

As for restarting from scratch vs changing one part at a time, although the first solution looks more fun and nice, I’m afraid we will never see it live. I mean, we have hundreds of small issues open that we don’t have time to fix, where will we find the resources to rewrite everything? That’s such a massive work.

Also, this, from the year 2000.

So was rewriting the entire federation source. Took quite a bit, but you know what, it turned out working, and the result is superior to what we could have achieved with staying at the old codebase. Also, what is the alternative? Replacing all views in Backbone with new stuff? Staying on two solutions forever™? Have you considered the extra amount of work that would be needed for maintaining that? I doubt that the actual work effort is that much higher in the long run.

Funny this blog post is talking about Netscape, when in fact, Mozilla just rewrote large parts of the rendering engine. :stuck_out_tongue: Was it a bad decision? No. Did Netscape/Mozilla go out of business? No. So what. This post is generalizing everything and claiming like re-engineering solutions is never good, completely disregarding the reality. You cannot turn a car into an airplane without starting from scratch. It’s just not going to happen.

Also, this, from the year 2016. You know what, I don’t care what other people are saying. I care about what the people involved in this project are saying, and that’s about it. Decisions have to be made within a given context, and not based by blog posts that ramble on some points because they think they are right.

Would a rewrite be a large task? Yeah. Might it fail? Hell yeah. So what? I’m honestly more concerned by the risk of us not being able to maintain the frontend in an adequate way (which, by the way, is the case right now) then the fact of us potentially delaying or rethinking a frontend reimplementation.

($0.02, and probably the last comment of this kind in this thread)

@denschub

I’m not good enough at frontend development to actually tell why the approach you propose to try might be better than others approaches mentioned. I think I would just trust you with that it is something worth trying.

Do you think there is something I can do now to contribute to this research?

Can you describe some intermediate tasks which could be seen as a step towards our eventual UI approach? For example you mentioned “event based data flow” component based on React/Redux. Maybe it is possible to try implementing some of our UI components logic on top of this approach? And therefore see what issues it brings up.

*Ahem.* https://www.youtube.com/watch?v=M1pbsH08yB4&t=8m09s
; - )

3 Likes

LOL :smiley:

But I agree with @denschub, doing a complete clean rewrite sounds much better than trying to mix things up. It probably takes some time until it’s finished and you see some results, but when it’s finished it’s worth it. When it’s never finished, then that’s sad, but at least we don’t end up with a half mixed frontend (when we would start to mix our current frontend with a new one, but not finish that) where it is even harder to do anything than with the current frontend. And it’s probably less work when you can just start from scratch instead of making sure that you don’t break anything existing by everything you touch. And I don’t regret that I’ve done that with the federation rewrite. It was a lot of work until seeing results, but it was totally worth it!

For the other discussion (which framework to choose), I don’t know both of them, so I can’t say anything about that topic.

3 Likes

Well, actually the federation rewrite was also gradual to some extent. We had some distinct steps and even the huge PR for the new gem integration was split to some commits which were gradual steps as well.

So even if we do accept the new UI paradigm it wouldn’t be “throw everything away at once”. It would be still gradual at least to some extent.

How do you think, guys, maybe I can build the front end for the seed migration feature based on the new UI paradigm? It could be a good playground, since it would be largely isolated from everything else.

Yikes. At the moment, no, not really. I’m only spending some hours per week on that at max and it is not open sourced yet for a reason. But I take this as a motivation to change that.

Well, if we decide to carefully reengineer things, a really good point of start would be to make a list of our current UI components, then go over each and consider UX and UI ideas to come up with a solution on how it could/should be. But since that requires a lot of time, I’d do that only after we decided that we indeed want to “start over”…

That being said, if you really think this is the way to go and want to have a playground to implement against, one could write a little dummy server (possibly using Sinatra or something lightweight) that respons with some API JSON dummies as speced. Not sure if this is a good approach at the current state, though.

Maybe my federation example wasn’t the best. As said before, I do not think it is possible to “gradually rewrite” the frontend without making different design decisions in the first place. And when we decide to rework data handling later on, this would be a huge effort to adopt the “just recently” created components.

If we’re planning on having a new, clean UI that is internally designed well, I think we should not built upon something that is not. :wink:

I don’t think so. At least not until we decided we want to have two frameworks in the frontend. There is no way to have a truly isolated environment in the frontend, since all dependencies are packed together and you cannot use one framework on one page but another framework on another page.


*people.

The only separate steps for the federation rewrite were the webfinger parts. But the last step was really one big PR. Sure there were many commits in there (and I hope that a UI rewrite also has multiple commits and not just a single “New UI :tada: :nail_care:” commit :wink: ). But only after the last commit was functional. The other commits I just deleted old stuff without taking care if something breaks and then add the new federation stuff.

And I think we can save a lot of time when we don’t need to take care that we don’t break the existing UI and just write a completely new one. Maybe we can add part by part to the new UI and already add it as separate beta UI (like the mobile UI is also separate). But I don’t know how easy this is and if it’s worth it (since it would be really incomplete in the beginning).

Does it then makes sense to create new front end features if we want to rewrite everything?

If I write new features now, how do I make it easier to rewrite them later when the time comes?

This is a good discussion here, so I take my chance to contribute…

For the framework decision:

I’m working all day with React and Redux in my daily job, so my preferred way to go would be React.
I did not yet work with Vue.js, but I heard a lot of it.
Like @denschub I love the way React works and how people can use basic JavaScript (e.g. Array.map) to render the components without some templating or learning another framework-specifc syntax (like it was necessary in Angular).
Vue is easier for people coming directly from Angular I heard.

So it seems we are all agreeing on the fact that Angular is dead and we currently have React and Vue in the ring. :slightly_smiling_face:

But things in the web are changing fast these days and since our new UI is still far away, I’m not sure when we should decide for a technology.

For the rewrite discussion:

I also don’t think we should embed any new framework in the current user interface.
It’s not only a difficult task mixing all together, it’s not maintainable. The current UI is not easy and to implement new features, a developer has to know: ERB, Ruby on Rails, JavaScript and BackboneJS. It’s difficult to understand how things are working together and you must know a bit of everything. Adding some React, Vue, Redux or whatever on top of it wouldn’t make things easier.
So in my opinion what we need is a modern web application rewritten from scratch which does nothing than consuming a well designed API.

I don’t know, how it is possible to integrate a beta UI (like with the mobile UI) because it has to be all bundled together somehow and of course we should avoid having multiple UIs for a long time (like with the mobile UI).

Talking about components:

There are lots of nice component collections out there, so we can also think about using some of them instead of write all standalone components from scratch.
Even if you all want to hit me now for posting this link, in my daily business I often work with some of the React components of Microsofts Office UI Fabric React with some good components for a social network like the “Aspect Pickers”. :wink:
There are lots of other well-maintained component libraries out there, so we wouldn’t have to rewrite everything.

1 Like

I thinks it’s totally fine to develop the new UI in a separate repo as a separate project, because when it is based on the API it won’t require to be bundled together with the Rails app.

@denschub

While the Facebook webapp has a similar front-end requirements and similar components as diaspora (streams, notifications, contacts, private conversations, profiles, etc) they make use of Flux and central state storage. I feel your point is right, but how does it work for Facebook then?

Or they don’t use a central state storage and use multiple independent state storages in the same app?

Oh, one can make things happen in a lot of different ways! Certainly, a central trust storage is never a really bad idea, and it makes a lot of sense if everyone is familiar with the concepts and the layout of that central storage.

As for diaspora*, however, we have a lot of different developers and should, at least in parts, aim towards being more contribution friendly. I don’t want to build something new that ends up being incredibly complex, putting contributors in a position where they have to understand that trust storage and its entire data, and also the flow of data, to be able to work with it.

If you, in contrast, consider something like a global event bus, “the experienced, core developers” have to come up with the events and their payloads, but each individual component is able to simply listen to them, and building data flow based on callback functions off those events, for example. That makes the individual components much more isolated from the global environment, and maybe a bit simpler for new people to work on.

I want to get back to the front-end topic with one more question. Time goes by and there’s no news on this one, and I think the community still wants improvements to the front-end of diaspora*. We have talked here about fully rewriting the front-end of diaspora* here, but the work is not in progress, is it? So this doesn’t have to block improvements to the current version of the diaspora front-end.

One of the legacy things we have in our front-end is component dependency management. We’re using rails-assets.org which relies on bower which is kinda deprecated now. Also some of the front-end packages we’re using are built specifically for Rails as Ruby gems. Today NPM is usually used to get packages for front-end.

With the present dependency management if someone wants to import a new JS package to diaspora they have to either create a bower package which doesn’t make too much sense in 2018, or create a Ruby gem manually, which is not a great time investment, or just include the source code of the component directly to diaspora*.

Most of the modern JS libraries are primarily distributed using NPM. Rails now has rails/webpacker which is something like an official support for NPM in Rails.

If we want to keep improving our current front-end, I think that including Rails webpacker support is a nice thing to do. This will make it easier to use some up-to-date packages which are only available as NPM modules. And it will make it easier to share some front-end code which we develop for diaspora as external packages so that these package might be used in some other projects.

As a short example, I’ve been working with @jcbrand on making a headless build of converse.js which can be integrated in other projects, but it is delivered either as a file or as NPM package, so this is where we could use webpacker to easily integrate this package.

So my question is: what’s the diaspora* core team’s opinion on enabling rails/webpacker for diaspora?

UPD: Adopting webpacker will help solving https://github.com/diaspora/diaspora/issues/5194 in a better way.

2 Likes

I actually am not in favor of using rails/webpacker, or anything like it. I actually think we should completely decouple the backend and the UI, and if we do any severe frontend work, this probably (<- my opinion, not a general consent) should be our goal. I think I hinted at that earlier, but never elaborated on it, so here we go.

In my dreams, we have implemented the entire API already. Therefore, we could build our client 100% as an API consumer, except for maybe a little bit different login flow to keep the UX constant. And if that is the case, there is no reason to couple backend and frontend sources together, so they should end up in two separate repositories. This might seem overly complicated in the first place, but there are actually pretty compelling reasons to follow this approach.

  • Separations of concerns always is a huge topic, and by decoupling UI and backend, maintaining parts of the projects can actually become easier. It doesn’t matter what backend changes you perform, as long as it still responds with the API we defined. Also, no matter how your frontend architecture looks like, it’s absolutely fine as long as it can handle the clearly defined API.
  • The backend gets more friendly for Ruby-developers to contribute to, as they are less likely to be forced to touch frontend codes if they just touch the way API payloads are generated.
  • Likewise, frontend developing is much easier and much more contributor-friendly, as they could focus on exactly their frontend world, without the need to touch anything related to Rails. Heck, they even could develop their frontend using a real-world, live production pod to test their implementation, as the API would be considered somewhat stable.
  • Developing new features gets a bit easier for backend folks. Instead of designing controllers and views for the frontend and the API, they just have to create the API. Granted, it might require some more discussion around payloads and response formats, but this still should be a win at the end of the day.
  • Because of this approach, we’d never end up features with non-existent/incomplete API routes. Our official client would consume the API, so implementing a feature without exposing an API would literally be impossible. We’d also immediately be aware of issues and unexpected requirements.
  • Customizing the UI, or even building independent UI projects for different needs (like special accessibility needs, or just fancy UX ideas), would be super easy.

So, in my opinion, we should maybe hold off any frontend work until the API is done (says the one supposed to work on the API for a couple of years, I know), and build something new when the API is done. Is that adventurous and maybe a costly (in terms of work required) idea? Yeah, for sure. But ultimately, I think it’s a good idea, both for our own happiness (as it would make code less dependent on tasks outside one’s individual expertise) and user happiness (as we would finally get the chance to re-imagine the user interface in a way that actually makes sense for the user, without continually having to fight backend restrictions).

If you, or anyone else, have serious interest in starting that work, maybe even based off an API mock server to get things going, please reach out to me. I have both some UI drafts and a general frontend architecture idea, both are unfortunately not in a state where I could share them publicly, but we can get there if needed.

2 Likes

Iam just an outsider, who currently dives into the state of the project. But working with and building frontends is part of my job.

First off, “hold off any frontend work until the API is done” is always a critical point. In a Company it is usually problematic to block feature development for more than half a year (actually more likely 2 years and more for a rebuild)
You can argue this is less of a topic for a OpenSource project with just a hand full of people, so I will try another argumentation-chain.

For OpenSource you have a special situation as you have a very undefined Team with high fluctuations of different skill levels and expectations.
I clearly support putting a majority of work into building a Frontend System, which is just interacting via an API with the Backend. (and I really would like to participate here, but did not found a url in this thread)
But, still allowing continued work on the existing system is important to gain new contributors, who are maybe not so experiences with building new architectures, but are able to modify and extend an existing codebase. Blocking new willing contributors over a timeframe of several months will drive them away longterm and makes it a lot more likely, they will instead bind to another open source project, or even non at all.
By still allowing contributions there, you will only need to invest a minimal ammount of time for reviews. Its even possible to establish from this new contributors a Reviewing group, which can minimize the review work the current core team needs to do. And you dont even need to handle them with the highest priority.

The thing I want to say is: If there are people who want to improve the old approach, see how to make it possible without hindering yourself from working on the new approach.
Be supportive, positive, enabling.

As long as it does not lead to significant problems or more work for the core team in the next few months, there is no good reason to block or deny work on the old approach.

1 Like

In my dreams, we have implemented the entire API already. Therefore, we could build our client 100% as an API consumer, except for maybe a little bit different login flow to keep the UX constant.

Oh good I thought I was the only one who thought that if we had the API finished that this would be the preferred mechanism for interacting with the new UI (desktop and mobile). It’s why I want to make it such a priority with my development time on this project right now.

2 Likes

5 posts were split to a new topic: When will the API be done?