Optional features like chat could also cause problems. Maybe there’s a way of making clear which features are optional, and which features and help texts have been added/changed since the last major version. There’s no need to point out rewording, but sometimes the entire content of a section might change and users should be aware of this.
Yeah, good idea. For issues where it matters, we can simply use the infobox template (like the blue/yellow boxes in the install guides, or the warning template in the wiki) to point that out.
You two are right, though, there are a few potential fallacies when shipping a centralized help, but at the same time, it also is a good thing. If we miss a documentation update for in-app stuff, we have released the version and our opportunity is gone. If we have the documentation centralized, we can update the docs whenever we feel like. Having the docs centralized also makes maintenance for community members easier.
All podmins should update fast anyway, so in my opinion, that’s a tradeoff that works for me personally. But it’s good to keep in mind, maybe we have to revert that decision eventually!
TL;DR of my long post: Overall thumbs up on this idea.
I spent a bunch of time reading over this discussion, looking at the source code, playing with it, writing some example new posts, etc. My off the cuff reactions are all positive with some follow-up questions that I couldn’t determine the direct answers to.
First, Yes absolutely I agree we need a redesign of these things for the exact reasons you’ve stated. There is a wealth of information in there but it’s amazing how hard it is to find sometimes. It was less than a year ago when I started working on the project and the Wiki install guides etc were pretty good for the core install but finding solutions to questions and some other things can be hard. I have bookmarks for some things that I know I spent a bunch of time searching for and could only find by looking at browser history, etc. So kudos for the team addressing that.
Beyond that I had no idea how hard the management of the sites was both from a structural/updating process point of view and a dealing with spam and other content point of view. What you have described is definitely completely unworkable. It’s amazing you guys haven’t done something like this up until this point by dropping to Jekyll or some CMS style system. You hit the nail on the head on limitations of Jekyll (can’t speak to others) and dynamic content. I just got done fighting that with my own blog and getting the Vue code working inside a Jekyll site for that whole “blog discussion timeline” proof of concept usage of the API. That one went so far as only working by embedding in an IFRAME because the Markdown processor was eating my Vue variables alive lol.
In the original post it was stated that some of this may be controversial. I’m not sure what that would be referring to. The only thing potentially controversial about your proposal from my perspective is using a custom system versus an existing one. My first question whenever a suggestion like that is brought up on a project is whether a survey was done of existing technologies. You answered that it has and you’ve identified some really interesting cases for translations and dynamic content we will encounter from the get go that CMS’s don’t/can’t handle. I can definitely see the advantage of your approach and there are some really solid things about it:
- Love editing via Markdown files
- Love managing from Git and pull requests. For people who aren’t comfortable with that path we could have an alternative contributions process similar to what opensource.org uses.
- Dynamic content is very powerful for pages that need that (like the install) and seems to be a clean implementation.
- Much of the content is straight markdown so it “just works” in other platform if ever it was decided to switch years from now. I imported the non-dynamic posts into a Jekyll site and it went in pretty cleanly and my Jekyll blog posts dropped into this site cleanly too. Beyond data portability for static content it also means contributors who are putting up blog posts, simple documents that would only have translation services etc, can edit that in a very familiar way.
- The site translations management system looks interesting but it’s not working on the present version (self-served throws no method error and the hosted one has internal server error) but get the concept form the config files and looks really nice.
Some questions that came up during my review that I couldn’t figure out from just looking at it and/or I thought were worth discussing. I got down in the weeds on some of them which may be too early to worry about. If any fall into that category just say “let’s discuss that later” or something.
- How do we envision search and larger lists/indexes/etc working with this system? Is the way the site is generated such that DDG or another system can index it properly and give us it “automagically” or is there something else we would need to look into to achieve that? Do we have a page we could populate with examples beyond the landing pages were the lists would have dozens or hundreds of items not just a handful? These would be especially necessary for guides, blog posts, FAQs, etc.
- Do we have any support concerns with a home grown approach? The code is pretty standard Ruby and a relatively small code base so it’s pretty straight forward to people familiar with it, which includes Diaspora developers. I felt pretty familiar with it right away when playing around with it tonight. However are there things we should be thinking about that may get in the way later though? Like what would a total reorganization of the site look like if that was done a few years from now? Is it mostly changing configuration files and tweaking code or the other way around?
- How is site layout design handled? For example if there is a page that needs a slightly different styling where is that change made? I see the views/components and views/layouts folders. Is this similar to the _includes and _layouts in Jekyll, respectively? Is cueing off a layout keyword or something what is envisioned?
- What is the pipeline for purely static pages? Are they being rendered and cached on start up or the first time they are accessed and then that is simply rehosted or is it being regenerated on each request? Do we have any scalability concerns related to any of that? Considering people on this forum say it feels much faster I’m assuming the answer is “no” even if there were thousands of articles but I figure it couldn’t hurt to ask.
Overall I’m stoked about what you’ve come up with and addressing the problem with this sort of an approach. According to the commit log you’ve spent 2+ months working on it and thinking about it (probably even longer thinking about it) and that definitely comes across. For some of the questions above (search, site navigation, etc) it may make sense for us to populate this with some stand-in content that looks like years of content to see how the experience feels from all perspectives (readers, creators, designers, administrators, maintainers) and tweak as necessary. I think it feels great from a landing pages perspective in each of the sections and “top view” lists but I can’t get a feel for how I’d find something more buried by time or obscurity, which is a problem we are trying to tackle with this redesign.
Sorry I haven’t managed to engage properly with this proposal yet. For now, I’ll just say that based on earlier conversations with Dennis, this gets a big thumbs-up from me.
One potentially controversial change would be (in my opinion) the switch of the domain. Indeed, we don’t exactly have a foundation structure we’re managing, so diaspora.software better reflects the reality. But “foundation” brings trust, and we probably need that. As we are trustworthy, I don’t feel like a liar when using the diasporafoundation.org domain. That’s the only controversial change I can think of from my point of view.
Looks good to me overall. In general I think that this is the right direction to move in for the diaspora project and this is what I think you’re asking to confirm from us at this point. All potential issues could be solved one way or another. So I find nothing blocking the proposal here.
I like the technical approach you chose. Static pages + minimal ruby backend for providing interactivity. Maybe that could even be turned into some kind of “documentation website framework” at some point in future.
I won’t repeat what others say, but I share some concerns stated above, like for instance what Hank said about search and indexing.
I don’t like the name “guides” for the FAQ/user help. It feels ambiguous to me. The fact that it combines information on different topics like podmin help, developer help and user help makes it hard to invent a good name for it. Podmin and developers will probably expect to find the relevant information under a name like “Documentation” while for user help expected name would be “User help” or “User manual” or “Usage hints” or “For diaspora users” or something. I don’t have a good solution here, but if we don’t have good names for the help section we could get the same problem as we have at the moment: people don’t understand where to look for the information they are interested in. This is probably just my impression though, so if nobody thinks it is an issue I’m okay with that. I just want to make sure that the required information can easily be found at the north star.
Appreciate the feedback.
Using Transifex instead of WTI; dropping the wiki, thus requiring a GitHub account to contribute; losing existing translations, …
I looked at more classical CMS like Wordpress, but also had a look at Drupal, TYPO3, MODX… Basically, in some cases, we simply couldn’t achieve everything we want, or managing the site would be so complex that we wouldn’t gain anything. Even more significant, using those systems would require some kind of backend account, and if you start looking at CMS with file-based backend so we could make the contribution process more streamlined… yeah, not so much. There is a non-zero chance I missed something, but I feel pretty comfortable claiming that building something tailored is likely the best idea.
Ah yeah, that was a fragment from me adding the edit button to the footer while being at a conference. I was aware of that issue, but didn’t find the time to address it. That now is fixed. However, there isn’t much translated content, but at least the “select language” selector will be German.
Everything is discoverable by search robots. Well, everything but the individual steps inside the installation guides, because the selector is based on JS, but the installation index page will be indexed, so that’s a non-issue. There is no hidden content unless we mess up, and even all the FAQs/Guides are listed in a simple list-of-links, for example here, which contains all guides, not just the top n, and this will also list all release announcements.
One thing that we need to do is to make sure to 301 redirect old URLs to the new site, which will make sure we don’t lose the search rank of our old website, but also will make sure people won’t get lost in old bookmarks.
The fun thing about that question is there is only one valid answer: I dunno.
Content itself should be pretty convertable, given it’s literally just a bunch of markdown files. With everything else, well, it depends on what the new site looks like. This is hard to plan, and so impossible to account for. In general, though, adding new subdomains for example should be simply done by adding a new section to the config. Other types of interactive contents could be build by adding a new module to
site_modules and including it, … but that’s all things I can imagine right now. There probably will be another large-scale reinvention in 5 years from now, but that’s nothing bad.
I planned in adding a
layout key to the frontmatter, just like jekyll did. In fact, there is even a
views/layout directory, and
MderbRenderer::mderb takes a
The thing is… I didn’t need it. I managed to build everything we have right now without the need for a second layout. So I didn’t implement it. If we ever need a second layout, we just need to add the layout key to the frontmatter, and implement a way to provide frontmatter defaults like jekyll does, and that’d be it.
Caches, in general, are evil. If you absolutely need them, they are still evil, but a necessary evil. But we don’t need them.
Everything is being rendered on demand, where “demand” is a http request. What this thing does in production is to load all files into redis, but only the files. I parse the frontmatter YML into a serialized object because parsing YML is surprisingly expensive (okay, it’s actually not surprising, but that’s off-topic), so this gets stored as a marshal’ed object, but the site’s contents are stored as unparsed mderb source. There are too many edge-cases we’d have to consider with caching.
As for salability… it doesn’t really matter. Startup times will get slower if we have lots of sites, but that’s only startup. Let’s just quote the redis docs for the command used to lookup pages here:
While the time complexity for this operation is O(N), the constant times are fairly low. For example, Redis running on an entry level laptop can scan a 1 million key database in 40 milliseconds.
We will not get to one million pages. Ever. Even with 100 translations, that would still be 10.000 pages. I don’t see that happening. Parsing markdown/erb has a cost, but I couldn’t reach the limits of that with doing a load test with
ab on my macbook running a single puma thread, so… While response times won’t always be the amazing 4ms that it currently has (that is excluding reverse proxy and load-balancer roundtrips), it won’t be at the 40ms of the current site, or the super fast 310ms on average for a wiki page render.
To me, this is actually less about claiming to be a foundation (which, let’s be honest, nobody cares anyway), but more about readability and type-ability of that domain.
diasporafoundation is actually really read to read and type for people with subpar English knowledge, and although we casually ignore that userbase most of the time, I’d like our official website not to do so.
Quite honestly, me neither. Here’s the thing, though: FAQ works in English, but that abbreviation is impossible to translate. Most projects stick with “FAQ”, but it turned out that that’s actually pretty horrible. The term is very well known within technical circles, but outside of that circle, most people are actually really confused. The traffic on the FAQ wiki pages is, compared to other pages, shockingly low.
With “guides”, I hoped to pick a term that’s both easily translatable, and more meaningful to users. Although it’s not super accurate, it makes clear that behind that link, there is content that explains how something is done. Besides that, it kinda forces us to write the contents in a different way. Instead of just answering hypothetical questions, it puts us into a position where we have to find topics to explain in a guide-like fashion. “How text formatting works”, “How can I discover contents?”, …
Does that change your opinion on that? If not, that’s probably a good sign we should reconsider that.
* you people.
Okay thanks for the clarification. I had a suggestion for the alternate route of contribution a la what opensource.org uses (but that’s their primary) but I have a feeling with the edit-in-browser capabilities on GitHub that will be okay for +80% of contributors.
Understood. If you missed a fringe CMS I’d say it’s better to have something easy and clean like this with all of its features than go to some fringe one whose maintenance would be beholden to god knows who. So I think we are good.
That’s good! Would we tie into that for in-site searches? Good catch on the 301 as well! I ran into that with my own blog migration.
Again overall looks good to me. Even my question about what large collections and histories look like is going to be down to presentation layer changes which should be more than easy to do in this system so that’s an implementation detail not a go/no-go decision thing.
So @denschub you started this discussion with
Looking at the likes and comments here I think we can say that your project is well welcomed by everybody here. So you have the approval you were looking for
Honestly, nobody here had a veto, so I’m not even sure the next planned step “asking the world” is needed. This is our website after all You can make this discussion public, but no need to call for a vote in my opinion. You did an amazing job, now let’s go ahead, fill it with content, polish the design and push to prod! (There are more intermediate steps, obviously, but I’m getting excited )
I don’t know whether the name
guides will produce normal level of traffic. I think we can try that and analyze the traffic level to see whether this works well for people or not. Let’s go with
guides for now then.
Thanks for the feedback, @comradesenya!
Yeah, we generally don’t do votes here on Discourse. ;D But it’s still important that those discussions end up public, so just in case that someone has serious concerns, they have a chance to express those.
Everyone here has expressed their opinion, either in person, in this thread, or before you all knew about this idea in private conversations (like @goob), so I guess our private discussion is done here. Thanks, folks.
I’ll write the reply to post when the topic has been moved, then move the repo, make it public, and move this discussion in a public category.
Hello, World. Here’s a thing we have been working on.
Y’all will see that we’ve had some internal discussions before making this public, so I’d like to ask everyone who got interested after reading the title, to read the whole backstory here. It will also explain why this started of internally.
The URLs shown above are still password protected, but the user credentials are provided as well. This is not done to lock people out, it’s mainly done to avoid search engines indexing the new pages by accident, which would be kinda bad. We (as in the community team and the core team) will also soon come up with a second post, explaining how content migration is going to happen. This plan needs some work, so please bear with us a little bit.
As you will probably see, the diaspora* core team, and our awesome community team, is on a consensus that we want to move forward with this plan. However, this is a community project, so we’d like to invite everyone to make some noise if you have serious concerns.
So let’s start 2 years ago when I created my pod… I managed to follow the instructions even for someone that I would qualify as a low level of linux knowledge. Then I also have the advantage that I know some linux gurus that helped me to correct all my mistakes, because the site was running but it wouldn’t have run for long if I only had myself working on it, that’s for sure. So the instructions are ok but not great.
Concerning the website I think the most important thing to me is the home page and the tutorial section for new users, even though I think the welcome email should contain an offline version of it that could be designed as a static page with the generator you’ve talked about. A possibility to add sections for sites that have optional features like Twitter integration or xmpp chat would be greatly appreciated.
For the new URL I agree that diaspora.software makes much more sense than the current URL.
Talking about /something that is so well hidden that nobody knows about it except the core team, I recently discovered the /public completely by accident, so all those possibilities should be clearly laid out for podmins and users so they’re much more aware of them.
To improve the appeal of the install guide I think showing that the project is up and running just fine would motivate people to invest the time and ressources to run a pod. Ideas I have would include a system where a podmin could have a specific country recommended to him where his pod would make the more sense, same thing for users, even if there is podupti.me website but users don’t want to have to go through a list of hundreds of pods, they just want to use the service so improving that would go a long way towards improved user experience, just like a random chose your pod, so that not everyone joins the same.
I really love this project and will keep supporting it as much as I can, I’m glad to see things moving in other ways than just code improvement.
Coming in late here, but I think this looks very well-thought out. I have been active on the web since the 1990’s and started with open source projects in a big way in 2005, I think, so I’m very familiar with the DocsthatSuck syndrome. The organisation looks great too. May I just add,
This is where many great ideas fall short. No one checks them out for the viewpoint of someone actually needing them. You know what’s a great example? Anyone use YouTube events or Google Calendar? They’ve got millions and staff, yet they’ve obviously never tried to enter an event.
Enough, my comment is already too long.
I want to help. Where do I sign up?
My rule for wine labels: short and memorable
My rule for product names: short and memorable
My rule for domain names…
diaspora.software seems to say what the site is and is as short as possible.
I like a lot about the new site. Especially the install guide. I am a Linux newbie and although I have some IT and engineering background the old guide was a bit… frightening. Also took me two tries to set up a pod. Looking back at it I can’t say it was bad or too inaccurate but on first read the impression was: “wow, this surely is tricky and can go wrong any time”.
About translations. I would like to contribute and help with Russian version. However the linked translation doc says:
If your desired language is not availble yet , please do not propose adding it on Transifex. Instead, open an issue on GitHub, so we can have a discussion about that.
…and currently translation project has just one language added. I suppose it is a bit too early to open Github issues about languages because surely existing translator teams will migrate to the new service?
This is mixing diaspora*s UX and the project sites’s UX a bit here, isn’t it? However, there are a lot of contents on the project site that are hard to discover as well, so your point is still valid. In fact, one of the main motivations for building something new was the fact that some of the current contents are not as good as they could be, but improving them on the current codebase is kinda messy.
I am not too sure where you are heading at. To me, the comment sounds like you are more interested in contributing to diaspora itself, rather than the project site? If that’s the case, look in the #features-and-ideas category, there are a lot of discussions already.
If you were talking about the project site, please hold off until we have some actual contents in there, because right now, it’s really just dummy content without any structure. As I mentioned previously, I can’t make exact statements on when that will be done, but this discussion will get an update, for sure. When we worked out our perspective of how the contents should look like, I would be more than happy to hear other opinions!
Good to hear that people are interested in that. Please note that, at this point in time, the only content that is polished even a tiny bit is the install guides, and even those need a lot of work; and they are not even translatable. All the site’s contents are dummy contents at the moment, so translating anything at this point in time doesn’t make much sense. I will make sure there is a separate “Reaching out to site translators” post once the contents and the documentation for translating are finished!
Actually, no. While we could generate a list of eMail addresses from webtranslateit, signing them up on Transifex would violate multiple privacy laws, and be a kinda thing to do anyway. As mentioned, once the contents are in a state where translating makes sense, we will reach out to everyone and explain how the whole process looks like.
I will help with anything I’m capable of, so both project and site. But my comment about usability is important and I think I can help there, yes, when there is something to look at. Having a bunch of peoiple who already know everything on a topic doesn’t really evaluate how understandable, attractive or memorable a site or text is.
I also would like to help with documentation, maybe by proofreading and giving my evaluation on it? Whatever works, I’m happy to give time to these things. Perhaps it would be wroth pinging me when there’s something like this to do because I don’t hang out much on Discourse.
I only have a small note to the guide section: for first time it’s nice to have a hand-holding version, but the returning visits need the ability to actually pick the page I’m interested in without clicking through 46 steps.
Good thing there is a “Show all instructions on a single page” link below the “install diaspora” button, which you can even bookmark.