Author:
José Valim <jose(dot)valim(at)gmail(dot)com>
Status:
Final/27.0 Implemented in OTP version 27.0
Type:
Standards Track
Created:
02-Jun-2021
Post-History:
https://github.com/erlang/otp/pull/7936 https://github.com/erlang/otp/pull/8165 https://github.com/erlang/otp/pull/8127 https://github.com/erlang/otp/pull/8123 https://github.com/erlang/otp/pull/8102 https://github.com/erlang/otp/pull/8089 https://github.com/erlang/otp/pull/8063 https://github.com/erlang/otp/pull/8026 https://github.com/erlang/otp/pull/7999 https://github.com/erlang/otp/pull/7959 https://github.com/erlang/otp/pull/7958

EEP 59: Module attributes for documentation #

Abstract #

This EEP proposes a structured documentation API for Erlang where the documentation is handled as part of the language parser and included directly in the compiled .beam files, as a replacement for EDoc style comments. Python, Elixir, and Clojure are examples of languages that follow this approach of treating documentation as data rather than code comments.

Rationale #

The main limitation in EDoc today is that the documentation is kept as code comments. This requires an explicit tool to parse said code comments, which complicates access to the docs by IDEs, from the shell, etc. There have been recent improvements in this area by making EDoc compile to EEP 48 but it still requires an explicit step.

Furthermore, the “code comments” approach is more complex implementation wise, as it requires parsing the source code alongside code comments, parsing the code comments, and so on. It is also beneficial to have an explicit distinction between documentation and code comments: they have different requirements and different audiences.

This EEP proposes the addition of two module attributes to Erlang: -doc and -moduledoc.

As with EEP 48, this proposal pertains exclusively to API references and their documentation. It doesn’t cover guides, tutorials, and other documentation formats.

New module attributes #

This EEP proposes two new attributes: -doc and -moduledoc. They could be used as follows:

-module(base64).
-moduledoc "
Convenience functions for encoding and decoding from base64.
".

-doc "
Encodes the given binary to base64.
".
-spec encode(binary()) -> binary().
encode(Binary) ->
  % ....

-doc "
Decodes the given binary from base64.
".
-spec decode(binary()) -> {ok, binary()} | error.
encode(Binary) ->
  % ....

The new -moduledoc attribute can be listed anywhere and it will contain the documentation for the given module. The -doc attribute must be listed anywhere before a function, type attribute or callback attribute and it will contain the documentation for the following function, type or callback. For instance, the example below:

-doc "Example".
-spec example() -> ok.
example() -> ok.

is equivalent to:

-spec example() -> ok.
-doc "Example".
example() -> ok.

Listing multiple -doc attributes with string values for the same function should warn or error accordingly, unless the documentation is being set to hidden. For example, this is valid:

-doc "Example".
-doc hidden.
example() -> ok.

But this should warn/error:

-doc "Example".
-doc "Updated example".
example() -> ok.

This as well:

-doc "Example".
example(one) -> 1;
-doc "Updated example".
example(two) -> 2;

Hidden docs #

The module attribute must either be a string OR the atom false. Marking a module as hidden means it won’t be part of the doc. For example, imagine the base64 module above delegates some of its logic to a private base64_impl module:

-module(base64_impl).
-moduledoc false.

Note a module may be hidden but individual functions can still be documented:

-module(base64_impl).
-moduledoc false.

-doc "
Some comments as if it was public.
".
decode64(Binary) ->
  % ...

According to EEP 48, this is intentional. For example, base64_impl should be private for users of the base64 functionality, but a developer working directly on the base64 may still want to access the docs for base64_impl functions directly from their IDE. Each documentation tool should honor hidden accordingly. If no -doc is provided, it defaults to none according to EEP 48.

The -doc attribute accepts the false atom too.

File/include docs #

Some developers prefer to not place the documentation alongside the source code. For such cases, -doc and -moduledoc may also provide a {file, Path}, where Path is a relative path from file containing the doc attribute:

-moduledoc({file, "../doc/src/manual/my_module.asciidoc"}).
-doc({file, "../doc/src/manual/my_module.my_function.asciidoc"}).

The file will be read by the compiler and embedded into the chunk at compilation time.

Callbacks and types #

The -doc attribute can also be used to document types and callbacks.

Private functions #

The -doc attribute can be used for private function as well, so that tools and IDEs can provide docs if the user wants them to. However, the private function should not end up in the EEP 48 doc chunk.

Private types #

In general private types are handled just as private functions, however there are times when types are used for documentation and code-sharing purposes within a module, but the user does not want to export it for general use. An example of this are all the various option types in the ssl module.

Therefore any private type that is referred to by a public function specification or type will also be included in the documentation chunks. Such types will have the exported metadata key set to false.

Metadata #

The new module attributes must also support documentation metadata by passing a map as argument:

-module(beam64).
-moduledoc "
Convenience functions for encoding and decoding from base64.
".
-moduledoc #{
  author => [<<"The Erlang/OTP team">>],
  license => <<"Apache 2 License">>,
  cross_references => [binary]
}.

If the -moduledoc is called multiple times with a map, the maps will be merged. This comes with the added benefit that shared metadata can be moved to a header file:

%% prelude.hrl
-moduledoc #{
  authors => [<<"The Erlang/OTP team">>],
  license => <<"Apache 2 License">>
}.

which we can then include and augment:

-module(beam64).
-include("prelude.hrl").
-moduledoc "
Convenience functions for encoding and decoding from base64.
".
-moduledoc #{cross_references => [binary]}.

A list of built-in attributes is available on EEP 48.

Compilation #

Compiling a module with the -moduledoc or -doc attributes will generate a Docs chunk into its .beam file, making the documentation directly accessible in the shell.

Release tools should also prune the Documentation chunk out of .beam files by default. Note this is already done by beam_lib:strip_release/1 and beam_lib:strip_files/1.

warn_missing_doc #

The compiler will warn if a function, type or callback that is included in the EEP 48 doc chunk does not have a -doc attribute set if the warn_missing_doc flag is passed to the compiler.

The flag can be passed both on the command line as +warn_missing_doc, or as a -compile(warn_missing_doc) attribute the source code.

Documentation format #

One important discussion about documentation is what is the documentation format that the documentation should adopt. Luckily, EEP 48 is agnostic to the format, however one must still be listed.

In order to facilitate multiple documentation formats Erlang/OTP allows the user to place a format key in the -moduledoc metadata which should specify the mime type of the format used, as specified by EEP 48. The default format will be text/markdown.

-moduledoc(#{ format => "text/edoc" }).

Other topics #

Doctests #

A direct consequence of making the documentation more structured and accessible is that Erlang can include doctests, which is the ability to run and validate the examples in your documentation. For example, someone could write this:

-doc """
Encodes the given binary to base64.

1> base64:encode("hello").
<<"aGVsbG8=">>

""".
-spec encode(binary()) -> binary().
encode(Binary) ->
  % ....

And then in your test suite:

doctests(_Config) ->
  ct_doctest:run(base64).

The doctest attribute will access the documentation entries in the base64 Docs chunk, extract all of the examples, and run them. Of course, while there is nothing stopping doctests from being implemented on top of EDoc today, this EEP makes doctests considerably simpler to implement.

Doctests would benefit from a separate EEP, as there are some extra considerations, as doctesting exceptions, unparseable formats, etc, but it is worth mentioning them given their benefits to users and documentation authors.

What about EDoc and erl_docgen? #

If this proposal is to be accepted, what happens with Edoc?

One important aspect of the work proposed by the EEP is to try and unify the documentation tools in the Erlang/OTP ecosystem. Before this, there have been multiple tools, EDoc mostly being used by open source projects, and erl_docgen by the Erlang/OTP project, and other third party solutions.

In the short-term, EDoc will be updated to be able to generate html reports from EEP 48 doc chunks containing text/edoc documentation. The EDoc comment style of specifying documentation will be deprecated, however, for backwards compatibility the support to parse the EDoc comments will remain for a long time.

In the long term, the aim is to switch document rendering engine to use ExDoc, which supports Erlang projects either by running it as an escript or via Rebar3 integration. When using ExDoc, the user can choose to either use EDoc syntax or migrate to Markdown.

Similarly, the Erlang/OTP code will be converted to use ExDoc instead of erl_docgen to generate documentation. All the current XML documentation files will be converted to use Markdown.

Copyright #

This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.