May 13, 2018
Table of Contents
According to Wikipedia:
Markdown is a lightweight markup language with plain text formatting syntax. It is designed so that it can be converted to HTML and many other formats using a tool by the same name Markdown is often used to format readme files, for writing messages in online discussion forums, and to create rich text using a plain text editor. As the initial description of Markdown contained ambiguities and unanswered questions, many implementations and extensions of Markdown appeared over the years to answer these issues.
In this post I’d like to expand on this and write down some thoughts I have about Markdown. My goal is to shed some light on the exact nature of Markdown which would hopefully explain why Markdown is everywhere through different flavors. I would conclude this post by asking whether Markdown is a suitable option for academic writing.
Origin and philosophy
Scrolling down the Wikipedia page, you’ll learn that Markdown was first released in 2004 (the year of Facebook’s birth) by John Gruber on the original webpage: i.e. https://daringfireball.net/projects/markdown/ where you’’ll find:
- details about the Markdown’s philosophy,
- a description of the original syntax,
- how the conversion is made, i.e. a call to a markdown-to-html converter written in Perl:
- a small web application to try Markdown.
Markdown is intended to be as easy-to-read and easy-to-write as is feasible.
and it really is! Let me select, copy and slightly modify an example from the daringfireball to compare the Markdown text and the equivalent written in HTML, i.e. what the Perl converter renders:
Easy-to-read and easy-to-write indeed! If not before, now you should understand
the play on words: HTML (HyperText Markup Language) is a
markup language and Markdown
is a Lightweight Markup Language (LML)
that reduces the length of the most commonly used tags for content editing
in HTML, so Mark
Less and Sass are two other good examples. They both are dynamic preprocessors style sheet language: the user codes in Less and Sass and those files are compiled into CSS (Cascading Style Sheets). In the case of CSS an obvious benefit from using Less or Sass is the possibility of using variables (does not exist in CSS).
Markdown is a markup language to format plain text not, a programming language but the approach is essentially the same. A better example would be LateX that was build on the top of TeX to ease the usage of its macros, but again this is a bit more complex than the case of Markdown which only converts a very restricted subset of HTLM tags. Note that if you want to use other HTML tags, the converter will keep them as is and you HTML page should be rendered just as you expect. You probably wonder why not expanding the number of tags covered by Markdown, right? Well, this is one of the main reason explaining why there are several flavors of Markdown available!
Despite the substantial information about Markdown that John Gruber provided, he unfortunately did not provide a complete specification, i.e. an unequivocal set of instructions that would allow anyone to write a Markdown converter in any programming languages that would have the same behavior (note that reading the code of the Perl converter does not seem to remove all ambiguities, according to the authors of CommonMark that I’ll introduce below). That led some authors to affirm that Markdown suffered from bad parenting. To me, there is no reason to blame John Gruber for not having written a complete specification. It is however reasonable to consider that this is another partial explanation of the current variability among Markdown flavors. Let me introduce some of them:
- MultiMarkdown by Fletcher Penney:
MultiMarkdown, or MMD, is a tool to help turn minimally marked-up plain text into well formatted documents, including HTML, PDF (by way of LaTeX), OPML, or OpenDocument (specifically, Flat OpenDocument or ‘.fodt’, which can in turn be converted into RTF, Microsoft Word, or virtually any other word-processing format).
MMD is a superset of the Markdown syntax, originally created by John Gruber. It adds multiple syntax features (tables, footnotes, and citations, to name a few), in addition to the various output formats listed above (Markdown only creates HTML). Additionally, it builds in “smart” typography for various languages (proper left- and right-sided quotes, for example).
MultiMarkdown started as a Perl script, which was modified from the original Markdown.pl.
- PHP Markdown Extra by Michel Fortin:
Markdown Extra is an extension to PHP Markdown implementing some features currently not available with the plain Markdown syntax.
- kramdown by Thomas Leitner:
kramdown (sic, not Kramdown or KramDown, just kramdown) is a free MIT-licensed Ruby library for parsing and converting a superset of Markdown. It is completely written in Ruby, supports standard Markdown (with some minor modifications) and various extensions that have been made popular by the PHP Markdown Extra package and Maruku.
Even though all flavors are very alike, they are superset of the original Markdown with flavor-specific variations. Therefore when asking “how can I change the size of an image using Markdown?”, a feature that is not in the original syntax, there may be a flavor-specific way to do it and potentially one answer per flavor. This explains the confusion when somebody asks such a question on StackOverflow. So… when asking a question about Markdown, beware of the flavor!
Markdown would doubtlessly benefit from being clearly defined, i.e. having a minimal set of tags (say the one of the original Markdown) and an accurate description of what a Markdown-to-HTML converter must do (harder that it sounds, think about how to deal with line breaks, nested tags, interference with html tags…). This could be a core Markdown on the top of which extensions (flavors) are built, that would be just great (again it is not the current Markdown landscape)! Fortunately, a bunch of highly capable people have decided to tackle this challenge and have created CommonMark that provides:
a specification for Markdown, version 0.28 currently: http://spec.commonmark.org/
useful links to implementations in various programming languages: https://github.com/commonmark/CommonMark/wiki/List-of-CommonMark-Implementations
a great tutorial: “LEARN MARKDOWN IN 60 SECONDS”.
I really hope this will soon represent a standard. This would be very useful to switch from one converter to another without worrying about its behavior. You might think this is only for advanced users, well it is not! If you use Markdown in different websites, different applications, you are very likely switching from one converter to another on a regular basis! And you probably do so because… Markdown is everywhere 🔥!
It would be a tedious work to provide an exhaustive list of the usages of Markdown on the Internet. I have therefore decided to discuss below a curated set of examples I am familiar with.
In any situation involving the edition of online content, the usage of Markdown brings many advantages (as it was designed for such purpose 😅). It is therefore not surprising that Markdown is supported by tons websites and web applications where people edit content. For instance, you can use Markdown to edit a question or an answer on StackOverflow or to exchange with you colleagues on Gitter.
If you create you own website, you could easily find a way to edit content with Markdown. Indeed, most famous Content Management Systems (CMS) such as Wordpress support Markdown as well as static site generators such as Hugo or Jekyll. Furthermore, when markdown support is not a primary feature, you should easily find a plugin that brings such support. For instance, if you blog with Medium you can use markdown to Medium.
Speaking about plugins Markdown Here (by Adam Pritchard) allows you to write email in Markdown! I really enjoy it, so I’ve decided to make a short screencast of me using it, which is what I do pretty much everyday! Note that I use the plugin for Thunderbird (Thunderbird 52.7.0).
Markdown can also be used to edit the content in HTML presentation frameworks such as reveal.js or remark. For developers, Markdown has become a natural syntax to write README and Wiki files, again easy-to-read and easy-to-write! But also, as a markup language, it is also easy-to-format! That’s how a README file turns into a very helpful and good-looking webpage on Github projects (see the README of this website)!
Last but not least, Markdown is now extensively used to produce dynamic documents including dynamic webpages where text and code are integrated together. For instance, you can format your text with Markdown in Jupyter Notebooks. Similarly, R users now extensively use Rmarkdown to edit reports, create package documentations, presentations and websites. In particular, we have created this blog with Blogdown and use Markdown and R Mardown for our posts. By the way, do you remember what I said previously about switching from one converter to another? Well we do so, as Blogdown allows us to use two different converter.
Markdown for other markup languages?
Once again Markdown is a lightweight markup language that emphasize plain-text readability and was originally designed to be converted to HTML. HTML is one among a long list of markup languages, though one of the most widely used. So now let’s consider a second markup language also designed to format content, say LateX (itself built on the top of TeX 😸!), then why not using Markdown for this second markdown language, it only requires a new converter and this would allow us to use markdown for two different markup languages 🏆:
HTML Markdown LateX
We can easily imagine Markdown been used with as many markup languages as desired. Given that Markdown is rather minimal, Markdown represents a useful subset for all those languages (assuming we are talking about markup language that are concerned at least partially by plain-text readability). What if we could do even more! What if we create an abstract representation of a markup language that would act as an universal language and allow to do a conversion between many markup languages, i.e.:
Markup 1 Universal Markup 2.
In contrast to most existing tools for converting Markdown to HTML, pandoc has a modular design: it consists of a set of readers, which parse text in a given format and produce a native representation of the document, and a set of writers, which convert this native representation into a target format. Thus, adding an input or output format requires only adding a reader or writer.
So now we have this fantastic tools thanks to John MacFarlane (I won’t give much details about Pandoc, which deserves a post it-self, if not more) we can perform the conversion between many markup languages, then Markdown becomes very useful minimal syntax we can use keeping in mind that it will be converted into a richer/heavier markup language. Typically, you can use Markdown to produce pdf files as follows: Markdown LateX and then Latex is converted into a pdf file! Remember that, irrespective of the flavor, Markdown only accounts for a restricted set of tags. So if you want to further format your document you can actually use other tags of the specific markup language you will convert your file into. In this case you can use Pandoc afterwards to convert your file into another markup language if desired.
Is Markdown suitable for academic writing?
Very good question! Truncated answer “No” 😞. Less truncated answer “No, UNLESS you use an adequate flavor of Markdown and an awesome tool such as Pandoc!” 😅. Indeed the original version of Markdown, many essential formatting aspects are not included, among them, the most important for academic writing are:
- support for math editing
- references (crucial)
Given when I wrote above, a solution would be to use Markdown in combination with another markup language such as Latex and most of the document would actually be written in Latex, so would it still be academic writing with Markdown? No!
Fortunately, the needs of academics writing are covered by Pandoc’s Markdown! So,
several ways to create tables
there is a specific Pandoc filter pandoc-citeproc that deal very well with citations.
So, thanks to Pandoc, Markdown it is suitable for academic writing. There are even more advantages such ad using templates and benefit from storing metadata of your manuscript directly in your file fir instance through YAML header (another potential post topic thanks to Clark Evans). Using Markdown also means for the authors to switch from a classical WYSIWYG tool to a Markdown editor. I personally use Atom with the following packages: markdown preview enhanced, autocomplete-bibtex (to handle reference) and document-outline (thanks @tpoi for the suggestion about the latter).
If you looking for more details about this, I highly recommend the tutorial of Software Carpentry on the topic and also the reading of different points of view such as “Why scholars should write in Markdown” by Stuart M. Shieber (more articles listed in the resource section).
A final note about reviews. I personally write my review in Markdown and convert
While I was writing this post I realize that just as Markdown is, resources are everywhere! I though it would be useful to end this posts by providing a personal selection of online material. Before proceeding, I shall mention the existence of several Awesome lists where you will find even more resources. Even better, a list that lists such lists. I recommend you have a look at the list created by mundimark and the one by BubuAnabelas.
Syntax descriptions / Specification
- Original markdown
- PHP Markdown Extra
- Pandoc’s Markdown
- Comon Mark
“A Call for Scholarly Markdown” by Martin Flenner on Plos Blog
“What is Markdown?” by Kirk Strobeck
“Why scholars should write in Markdown” by Stuart M. Shieber on Harvard blog
“Writing Technical Papers with Markdown” by Dheepak Krishnamurthy
“Modern Scientific Authoring” on Software Carpentry website
“Markdown for Writers” by Gene Wilburn
“reStructuredText vs Markdown for documentation” by Victor Zverovich
There are Markdown-specific editors such as Abricotine, Mou or Stackedit. Note that most of modern code editors include specific libraries for highlight syntaxing and preview of documents written in Markdown. For instance, Atom includes a markdown preview as a core package and a quick search reveals that there are 30 packages available (query on May 12th 2018, Atom 1.26.1 on my Debian device):