Dear friends, it happened again. I wrote a long text I spent some time on a project-related idea and want your feedback on that now. The project is called north-star, and I really like that codename, so I had to share that with y’all.
Let me start with explaining why this is a team-internal proposal: if we decide to go ahead with it, it will contain some quite controversial implications, and it will have a major inpact on our public-facing communication. I worked on this project without the team’s “approval”, so I want to get your opinon first. If we all agree that this is a good idea, I plan on moving this proposal to a public category and ask the world if there are vetos. But I want this proposal to have the chance of getting rejected, if needed, without public outcry. 
What I am proposing ultimately boils down to…
- Drop the current project site.
- Drop the in-application FAQs/Guides.
- Drop the current installation guides.
- Drop the wiki.
- Replace all of the above with something cooler.
Let me explain.
Motivation
I think we are a nice project, and we have some nice people, and even our main product is kinda nice! While we are doing a great job project-wise, I think we are doing a really bad job at doing “public relation”, documentation, and are generally bad at keeping folks informed. Let’s look at what we have right now:
The project website
Our current project website is not horrible, but kinda bad. I’ve heard from users, developers, podmins, and journalists alike that finding information is really hard. Even from people doing translations, I’ve heard multiple times that they translated a string and actually had no idea we had that information on the site, before they discovered it via webtranslateit. What is the target audience for our website? Users? Podmins? Journalists? Is the website and its contents really useful to anyone? I don’t think so, or at least I am having a hard time identifying who it’s for. That’s not good.
If we decide to go ahead and change all the contents, how would that look like? Currently, we would have to change the layout in ERB layouts, then hack multi-line content into the yaml file, then maintain the translations somewhere, … in addition, we have to maintain a database because we have some content in the DB, … it’s quite complex. Back in the day when I started working on the current site, I thought basing it on Ruby on Rails is a good idea, but these days… not so much. People who are doing lots of maintenance on the project site have told me multiple times that even getting the site running on their machine is a pain, and let’s not even talk about how complicated actual editing is. It’s not good.
The in-app help/FAQs
Soooo, we have /help. Hands up, who knew it was there? Well, I can’t see your hands now, but most of our users don’t! Sometimes, people are really surprised when I show them the help texts we ship with diaspora, they are really hard to discover.
Aaaand you guessed it, those texts are not very nicely written, and quite outdated. Also, what happens if someone wants to fix that? They’d have to get diaspora working on their machine, tackle the mix of ERB, a bit of Haml, Rails i18n, … just to fix some texts. That’s also not nice, and also really bad, as bad user documentation is one of the biggest issues we have that seperates us from a wider adoption by non-technical audiences. We also need to do much better here.
The wiki
When we installed the wiki, the main motivation was to enable more people to contribute to our documentation, assuming we eventually end up with super awesome docs on every topic imaginable. That… didn’t work out. Basically all active and current articles are maintained by core staff, or by very active community contributors, and the number of users and irregular contributors editing the wiki is not-actually-zero-but-still-zero.
These days, most of the edits are spam, and even though we enforce ReCaptcha for registrations, there is… a lot of spam. You can’t see it, as changes by new users are not visible, but have to be approved manually, but just to give you a number: on 25th December, there were 11 new users who immediately started spamming on their userpages. It’s bad.
The contents on the wiki are either well maintained articles by core members, or stale articles that have been started by someone years ago, but never received any love. However, most of the contents there are interesting, and should get more attention, but the wiki is maybe a less-than-ideal platform for that.
Also, on a technical base, the wiki is actually surprisingly painful to run. Not only does it require an Apache and a MySQL database (contrary to our project site, which is using nginx and postgres), it also needs a dedicated memcached to be even remotely performant, and it needs quite a lot of babysitting on a daily basis in terms of installing updates, fixing plugins, cache problems …
The installation guides
Those are also part of the wiki, but I wanted to split them off, as these are really important.
Before we had the wiki, we used GitHub wikis to maintain a single-page list of steps podmins have to follow if they want to set up a pod, but that guide was bad. When we had the wiki, we decided to move the install docs there, and some time after that, wonderful people spent amazing amounts of time to make them a bit more interactive, and allow people to select their operating system, their database engine, and make the guides easier to understand.
But still, those guides are not good. Podmins get confronted with a large wall of text, steps are sometimes confusing, the information is not really accurate, sometimes outdated… and, you can see a pattern by now, maintaining those is quite painful. By using variables in templates in macros, we achieved a surprising amount of flexibility, but it’s not what Mediawiki is supposed to do, and it’s not what it’s good at. Not good, and we really need to do better here.
The solution
Okay, I explained to you that our current state is not the best. The good thing is: for once, I have an actual solution! Not only do I have an idea, I actually spent some time developing that idea, into a fully blown proposal, including a live demonstration of code I wrote!
One thing this proposal includes is the deprecation (meaning: only exist as a redirect-hub for old URLs) of diasporafoundation.org. I want this project to use diaspora.software, as it is a much nicer domain. Besides the fact that we, technically, aren’t even a “foundation”, the domain is super long, hard to type, hard to read, and the word “foundation” doesn’t carry much meaning outside of some groups. “diaspora.software” is much more snappy: it’s about diaspora, it’s a piece of software, and besides the project name itself, there isn’t any ambiguity in how it’s spelled. Also, having the gTLD “software” makes it more explicit that we are, indeed, offering a software, and not a service that people can use.
Overall, I want us to have…
-
diaspora.softwareas the new project site -
blog.diaspora.softwareas our blog -
install.diaspora.softwareas the place to find install guides -
guides.diaspora.softwareas the place users, podmins, and developers can find guides that help them getting their frequently asked questions resolved
The core idea™
I like static site generators like Jekyll. If you don’t know what that is: basically a piece of software that turns markdown files into HTML files. Simple webpages are usually a good case for that: you once define an HTML layout, and after that, all you have to do is publish markdown files. Quite nice, easy for editors to work with, easy to have checked in a Git repo, no database needed, no dedicated user-management needed, … it’s quite nice.
However, those don’t work for more complex sites. As soon as you require some kind of interactivity, like handling user input, static site generators don’t work. So, building the project site, and especially the install guides, on something like Jekyll, is really hard. And with requirements like internationalization, it turns into an almost-impossible problem.
But… maintaining contents as markdown files would still be amazing. Imagine if we could just all work on GitHub, send Pull Requests changing markdown files that can be viewed and edited right on GitHub itself, …
I built something.
I deployed that something onto our server, but I put it on some example subdomains and also added password protection, mainly to prevent Google from indexing it. Whenever I link something, you can use username northstar and password kitten to get access. I also pushed the code to GitHub, and you are all added. It’s a private repo, mainly for the same reasons as this proposal being private. But it should give you an idea, I hope. The source code can be accessed at https://github.com/diaspora/north-star
Content editing
Well, it’s markdown. Mostly, that is. I mixed plain markdown documents with some ERB, to allow for more… flexible documents. Imagine the install guides, for example: those would be not very useful as static markdown documents, because we need to inject distribution-specific variables like the package list. The server takes those files, renders them, and serves them to the browser. So we kinda have a static file generator, without there being a static file generator. Check this document for more architectural notes.
Easier contributing
The only dependency of that project, in development, is Ruby. Once you have installed Ruby, you are good to go. No database server needed, which makes local setup for contributing and testing changes much easier.
Actual live demos!
Okay, I teased you a bit, but there you go. Note that this is a technical demo, not a demo of well-written contents. It’s mostly dummy contents, really. I’ll make a proposal for how we can get nicer contents as soon as we decided that this is a good idea. (Yes, I have plans for that.)
The project site
As you can see, it kinda looks like our current site. These are the source files for the contents, should give you an idea of why I think that markdown files are a good idea. Just for reference, this is the code behind the current index page.
Also,
That’s still mostly the same. As you can see, I decided to split the blog content up into release announcements, and other stuff. I want us to post more content to the blog, posts like the “where is my feature” masterpiece, for example, shoudl absolutely be on our blog as well. Each blog post, again, is its own file.
The guides (formally known as FAQ)
These are just like our current in-app help, and the FAQ in the wiki. They are split into sections for users, podmins, and developers, where the user section gets the most visibility. The guides are individual markdown files, can be grouped into subcategories… you get the idea when looking at that site.
Install guides
Noooow, the really cool thing.
Just have a look at that! I attempted to guide podmins through the process, instead of dumping all the contents. You start off by selecting your target environment, and then enter a multi-step guide that shows you what you have to do. And even though I just copy-pasted the texts from the wiki, it’s already much nicer. Imagine how that would look like if we spent some time polishing the texts…! 
The install guides are a great example to show off another neat thing: data files. Have a look at the data file for the installation guides, and most of the time, that’s all you need to touch when adding new distribution versions, fixing something, … Overall, I think this reduces our maintenance effort by a fair bit.
The bad things
So, as you can see, I am quite hyped, and I know other folks are as well. I think it would be a huge improvement to our current state, but as usual, there are downsides as well.
Handling translations
Currently, our project handles translations on WebTranslateIt. There are some limitations with WTI, but one of them is the way WTI handles strings. While you can upload multiple files, it ends up collecting all strings and listing them in an alphabetical list. That works fine if you translate software projects like diaspora, but it falls apart when translating content pages where you have multiple strings on one page, and where context is important.
In my proposal, I wrote automation for Transifex. Transifex is a different service, but a much more commercial one. They support opensource projects, but it’s … a different feel. The largest benefit is how Transifex handles multiple files. Each file is displayed on its own, where the translations are in context, and in the order they appear in the file. That way, all translations are in context. Have a look at this documentation, where I explained the details.
I like Transifex, but it would be another service. And it requires a Transifex account.
The code is still home-brew
I looked at already existing CMS, but none of them really fit our needs of simple content management and flexible layouts, while at the same time being accessible to contributors, and having a content history.
The code is based on Sinatra, but there are 650’ish lines of Ruby that I wrote myself. Most of the code should be easy to understand and maintainable, but it’s still no off-the-shelf solution.
Contributing to the site requires a GitHub account
In reality, I don’t think this is a problem, but while previously, you could contribute to the wiki by creating an account there, you would have to have a GitHub account after this proposal. However, that’s also a benefit, since we would get the whole review-workflow for free. And GitHub’s spam protection.
Outro
So, that’s my proposal. Please let me know what you think. Please think about it, and if you have any concern, questions, or just something you want to say, please do so. I want this to be a well-planned, and well-thought step. If there is anything we can improve (architecture wise, obviously, the contents need lots of work), we should do so. If you’re into Ruby, check out the GitHub repo (especially the _server subdirectory), and let me know if you have any feedback.
There are also a lot more details I want to talk about, but this text is already waaaay too long. SO I hope you understand everything, and can form your opinion. If not, let me know.
Thanks. Merry Christmas.
You can make this discussion public, but no need to call for a vote in my opinion. You did an amazing job, now let’s go ahead, fill it with content, polish the design and push to prod! 
(There are more intermediate steps, obviously, but I’m getting excited 
thing to do anyway. As mentioned, once the contents are in a state where translating makes sense, we will reach out to everyone and explain how the whole process looks like.