A New Documentation Initiative

I recently performed an experiment in which I attempted to emulate the experience of a Drupal evaluator. I highlighted a few glaring pain points and shared my experience and conclusions in a blog post: Stranger in a familiar land: Comparing the novice's first impression of Drupal to other PHP frameworks.

That post sparked a tremendous degree of engagement. Many people commented, wrote responses on their own blogs, and voiced support on twitter. I was pleasantly surprised by the Drupal community's response, which was (nearly) uniform and highly corroborative. The jury is in. Drupal's evaluator experience is fraught.

A few notable long-form responses include:

In this post, I'd like to shift the conversation toward specific solutions. In particular, I'd like to focus on ways that we can improve the evaluator, developer, and site builder experiences through changes to our documentation.

Documentation Initiative Proposal

The 2018 Stack Overflow Developer Survey indicated that 83% of Developers learn via the official documentation. Documentation is very important. To borrow Adam Hoenich's statement:

I think the Drupal community should undertake a first-class, long-term Documentation Initiative. ... it is absolutely worthy of becoming a full-fledged core initiative, with all the resources and support that existing initiatives enjoy.

I'd like to propose a formal Documentation Initiative for community consideration. In drafting this proposal, I've tried to incorporate:

The Proposal

There was a strong documentation effort undertaken not too long ago. It produced the Drupal 8 User Guide, which is far and away the best documentation on Drupal.org.

Unfortunately, Drupal.org's UX doesn't make the Drupal 8 User Guide easy to find or navigate. If you're lucky enough to find the Drupal 8 User Guide, you're unlikely to understand the difference between the Drupal 8 User Guide and competing documentation like the Drupal Community Guide. It makes for a very confusing experience.

I'd like to propose that we elevate the Drupal 8 User Guide to the status of "Official Documentation," and update the UX on Drupal.org to prominently feature the Official Documentation on major site entry points. A Drupal novice should encounter the Official Documentation essentially by default. By contrast, our Drupal Community documentation should be clearly identified as wiki-style and unofficial. We should minimize its visibility.

The team that produced the Drupal 8 User Guide established a set of admirable best practices. They implemented a governance process, version control, and documentation standards. They wrote their docs in markdown format and created a continuous integration process to generate and update screen shots of Drupal interfaces. Bravo.

We should adopt these best practices for all Official Documentation. In fact, we should go further. The Drupal 8 User Guide already has multiple branches corresponding with Drupal core minor and major versions. E.g., 8.5.x, 8.6.x, 9.0.x, etc. Let's integrate these into the Drupal.org UX, much in the way that Symfony and Laravel do. We should be able to intuitively identify and switch the documentation version.

Let's go further and lower the contribution barrier. At present, the Drupal 8 User Guide uses a Drupal Issue Queue. This means contributing by attaching patches and text files to issues. I, for one, would be immediately deterred from contributing by these requirements.

I'm going to be a bit controversial and propose that we use GitHub (or GitLab) to manage this instead. Allowing community members to use pull requests and an in-browser editor to contribute documentation will lower the bar while simultaneously leveraging version control and peer review. That would have massive impact. Drupal.org is apparently planning to migrate to one of these services. Let's not wait for that.

Lastly, I'd like to expand the Official Documentation by creating a new class of documentation that we're lacking: "The Official One Pager." The purpose of this class of documentation would be to provide a one page guide that prescribes the single recommended way to accomplish common Drupal tasks, like installing Drupal locally or using Configuration Management. Initially, I'd suggest starting with the following one page guides.

  • Official Quick Start Guide (download, install, and log into Drupal)
  • Official Configuration Management Guide (export, import, update, deploy)
  • Official Composer Guide (install, update, extend, maintain, troubleshoot)

These are not intended to be exhaustively comprehensive. They are not intended to communicate all the myriad ways you can do things with Drupal. They will simply prescribe a straightforward path for accomplishing a discrete set of tasks. More detailed, nuanced, and comprehensive documentation will still be referenced and made visible.

Lastly, I'd like to propose that we prominently feature high quality, free training videos to help people get started. These should be provided as alternatives to the one page guides. Videos like this already exist. I believe that the Drupal Association could find willing partners to provide these materials in exchange for market visibility.

The picture of success

The documentation initiative would bring our documentation into line with competing frameworks. Let's take a quick look at Symfony as an example of a framework with stellar documentation.

alt text

Finding the official documentation is a breeze. Visiting Symfony.com displays a "Documentation" link as the second primary menu link. One intuitive click gets me to their official documentation.

The first "Getting Started" section begins with "Setup" as the first link. It's easy to find. Upon visiting the "Setup" page, I find that it has all of the elements of this proposal:

  • It provides a single clear path forward that ends with a running web server and a working application (in one concise page).
  • It is version controlled and it clearly indicates the framework version that the documentation refers to.
  • It includes an "edit" button that allows you to contribute on GitHub.
  • It links to free video tutorials.
  • It links to more detailed guides for further reading.

The process of visiting Symfony.com, finding documentation, creating a brand new Symfony project, and visiting that new app in a browser took less than two minutes in practice. That is the bar. Drupal can do this too.

What you can do

Speak Up! To even begin this initiative, we need "buy-in" from the community and the Drupal Association. Assuming you, the reader, belong to one or both of these groups, speak up! Post a comment with your support, suggestions, or constructive criticism. This proposal needs review and modification by experienced community members.

Join a related BOF (Birds of a Feather) at DrupalCon Nashville and lend your voice or your help:

Get involved. If you'd like to contribute, or if you're in a position to facilitate the changes proposed in this post, contact me!

Let's make this a reality.

I fully support this initiative and agree it is critical to Drupal's continued success!

Everything in this post is a great idea, and quickly actionable. A lot of this stuff could be done THIS YEAR. Let's not wait. I'll be in Nashville and would happily participate in a BoF.

These sound like excellent ideas, and as far as the UX is concerned, it may be good to collaboratively iterate on this. I will be at DrupalCon, and also would be willing do participate in a BoF on this.

Nicely done! I was glad to see you refer to the symfony docs. They are a great example of docs done right. Also check out the apollo graphql docs as they are amazing

From my past experience with Drupal I like to share what I was expecting in DO

What should be done : This section should have docs with a MUST video tutorial about installing drupal, configuring permission on diff OS. There will only one solution and single doc will be modified with revision if required. This section should be official driven with moderation.

How to do it : This section to give solution to user for dynamic/custom requirement. For ex: how to make node list from views based on some criteria. This section will be having different solutions from different contributors for same requirement.( replacement to DO forum?) . This could be community driven

Share code: I am still surprised why DO don't have section where user can share their views source code(different galleries,fancy product listings with hover etc) ,Feature bundle, Content type exports,Dummy node content exports, overridden Theme tpls etc. So that user can import & use it instantly.

Showcase in-depth: To motivate newbies towards Drupal with some insights of existing Drupal site's implementations.

Change records: There should be sync between docs and change records.For ex: in all docs some warning should be shown that this code doesn't work anymore. Otherwise newbies trying to use old code may end up flooding "not working" issues.

have a look at community<dot>letsencrypt<dot>org too..

Started to learn drupal about 8 months ago and had (and still have) a lot more trouble learning it than Laravel or Symfony. When you're searching for examples, you start typing whatever you need, and end up with 3 main types of results: UI end-user solutions. Drupal 7 code solutions, Drupal 8 code solutions. It would be nice if this could be better streamlined. Maybe even change the name of the UI or use project names like Android does, so i can search cupcake <insert proper replacement> instead of Drupal 8 and be sure i find drupal 8 programming solutions for my problem.

I had a lot to say and I've published some thoughts on this:

https://blog.acromedia.com/new-documentation

In general, you're very much onto something Matthew. Hopefully we can connect at Drupalcon in a few weeks and discuss practical ways to implement each of your proposals.

Drupal 8 documentation is a mess. Basic example for installing drupal is you can end up here: https://www.drupal.org/docs/8/install or here: https://www.drupal.org/docs/user_guide/en/installation-chapter.html. All of the documentation should be consolidated into 1 place with a searchable interface with facets. Drupal has tools to do just that but we seem to be stuck in some archaic cluster of linked pages that go in circles. The developer docs are even more confusing. You have to drill down into several different pages to even try to figure out what properties are available on form elements till you discover the end result is either incomplete or missing and I usually end up digging through the core module directory that contains the code and hope there are some good comments to go off of. The drupal 7 documentation was much better at least around the form api section. Drupal 8 base field definitions are a major improvement but the documentation doesn't clearly explain it well enough to sell it's advantages. Clearly having the ability to define your schema, form/display structure, and property info all at once is a really nice thing to do for custom entities. It's a huge timesaver compared to drupal 7 but the documentation doesn't really mention the "why" around this. Maybe it does somewhere else... but not in the context of where you would expect to find it.

I think the ultimate problem is the documentation is written from a perspective of the people that wrote and understand the logic but perhaps lack the skills to explain in a little less technical terminology. I'd be more than happy to contribute to this initiative but the problem i have is just having a thorough enough understanding of the correct way to do everything to really document it out. Simple things as developing a custom drupal 8 module and knowing what goes where is challenging. Alot of that coincides with the shift to OOP obviously. Where do i put my custom functions? Do i put them in a class as public or private functions? Do they go in the .module file or somewhere else? Whats best practice? Real world full examples would be helpful as well. Drupal has to have much better organization of code snippets around a particular task related to a particular area of the api... again all searchable from one place. The most frustrating part of the documentation is the unorganized mess of pages in so many places and having to search endlessly outside drupal for examples of how to do something that we "hope" is the right way.

This sounds like a great start!

There are a few things going on within the community and via the Drupal Association that might well help focus this, not least, the creation of a new set of properly researched personas of various roles.

Are you coming to Nashville? We should talk...

I was/am one of the organizers/managers of the User Guide project (which is ongoing, as it needs updates periodically due to changes in Drupal). And before we started the User Guide project, I was part of Drupal Documentation governance for years, and tried/failed to get something like the User Guide project going several times; previous docs team leadership had also failed before I got involved.

The reason that the prior initiatives failed to get started was because the community hated the idea (as it was previously presented): they didn't want to give up the ability for anyone on drupal.org to add/edit documentation pages whenever and wherever they wanted to. So, when we proposed the User Guide, we took a different approach. Instead of "Hey, for documentation quality purposes, we need to limit your ability to create/edit documentation." (which the community hated), we said: "Hey, Drupal needs a User Guide!" (which the community got behind). I believe that this new initiative will have the same problem getting off the ground if it proposes to limit how people can edit and add pages online to the existing Drupal documentation -- and really, that's fair. For instance, a person who maintains a module wants the ability to quickly write up some documentation for it, and we shouldn't stand in their way of doing that.

So... Assuming that this new initiative, like the User Guide, will add new docs rather than getting rid of existing docs, visibility and differentiating between governed/curated documentation and the free-for-all documentation is always going to be a problem. We can promote the curated docs in drupal.org site search, and on drupal.org landing pages, but Google or other search engines (which are used much more than Drupal.org search or navigation to find docs) will always find both types of docs.

I would also add that once the User Guide was done and readable on drupal.org, we *did* try to improve its visibility... I would welcome any efforts to make the User Guide and other docs with greater oversight more visible and the other documentation less visible. But it's a problem that may not be all that easy to solve. Hmmm... Maybe if we put "wiki" or "community" or something like that in the page title and at the top of the page for all the existing regular Drupal docs pages, that would help differentiate the planned, curated, governed docs from the free-for-all? Just a thought.

Thanks for bringing light to the topic! I appreciate the docs team and even what we've been enabled to do more recently to identify "official" docs books for modules we maintain. One problem that I'd add that we need to be cognizant of is documentation search - e.g. restricting search results to a particular guide rather than any docs page is key. (Right now, there's a nice big "Search documentation..." bar once you land at the Drupal 8 user guide, which I actually didn't find hard to locate, but it just dumps you into the general documentation search for any guide on the site. Way too much noise.)

I'd caution against the following idea, though: "propose that we use GitHub (or GitLab) ... Drupal.org is apparently planning to migrate to one of these services. Let's not wait for that." It isn't a sure thing that d.o will be migrating to another platform, and BitBucket was actually the leader over those options (cf. https://www.drupal.org/drupalorg/blog/developer-tools-initiative-part-4-whats-next#tldr). Having used GitLab recently in a client engagement, I feel confident stating it's wholly insufficient for any but the most basic use cases.

However, the bigger issue is that we have a variety of other concerns in evaluating the systems we use (including data ownership, license / rights enforcement, contribution credits, etc.) that we can't just gloss over accommodating. I won't argue the WYSIWYG + PR generation on GitHub isn't neat, but I'd need a lot of convincing to prioritize that feature alone over all of the other concerns.

I am glad someone even proposed this. I have been saying this for years. I have been trying to publish tutorial videos and helping instruct new site builders on Drupal for at least 4 or 5 years now. Since I first starting using Drupal about 6-7 years ago I thought the documentation needed a lot of work. This is something a lot of the site builders who are also contributors to Drupal, have been saying forever.

I know I have not gotten almost any respect in the community for putting out ton of tutorials, podcasts, and writing articles. The problem with that is that new developers and site builders use modules and distros too and need articles, videos, and podcasts talking about how to use different distros, modules, and best practices. Documentation, videos, podcasts, and articles are how we can really explain Drupals usability to those evaluators.

Video, podcast, and articles, are hand in hand with documentation and can be a huge help in helping to push the Drupal agenda, but you can't help push Drupal forward with writing about it, making videos about it, making podcasts about it, evaluators need to hear more about what Drupal can do, because evaluators need to understand that the learning curve isn't that hard, and that it is something really powerful in return.

I may not get a lot of respect in the community for my contributions of videos and podcasts, but I do it anyway, because those people need them, and I hear from them pretty often that they appreciate it. I think it's about time the Drupal community as a whole, starts valuing what people like me and other contributors in the "Documentation" area of Drupal think and feel. It's about time.

I am really glad someone starting trying to take this serious, it's about time an initiative for documentation gets suggested in a real manner. Documentation should never be a second thought in a project as complex as Drupal, the documentation sells Drupal, without it people think Drupal is too hard figure out. We need more people trying to make this whole ecosystem visible and accessible to the evaluators and sitebuilders using Drupal, documentation is how we do that.

Good proposal, I definitely support it.

This is great! Drupal lacks most about (api / best practice) documentation. Sometimes you search for hours to solve a problem and don't know if you solved it best-practice.

I'm looking forward to this :)

Add new comment

Plain text

  • No HTML tags allowed.
  • Lines and paragraphs break automatically.