doc-tools: A toolset for Reason / OCaml related documentation generation

Motivation

odoc is an essential building block in the OCaml and Reason ecosystem with a potential to constitute a foundation for precise, interactive, contextual and user-friendly documentation for developers.

In its current form, odoc can be seen as a low-level tool that requires a “driver” to be used. For native development this role is fulfilled by tools like dune and odig, while for frontend development with BuckleScript there is currently no official tooling.

In addition to that, the current features of odoc limit the ability for third-party tools to generate alternative output formats (such as Markdown, LaTeX, etc.), or indices for search engines, without extending the official project. This limitation arises from the fact that odoc operates on an internal .odoc file format that cannot be easily parsed by external tools and is compiler version and odoc version dependent.

This proposal describes work packages that would improve this situation by:

• implementing a JSON-based output format in odoc;

• implementing a command-line tool to generate Markdown files;

• implementing a command-line tool to generate JSON-based search indices;

• defining an integration process for BuckleScript and using that integration to generate documentation for the official BuckleScript modules.

All of the enumerated work would allow odoc to be easily integrated with a wider range of OCaml and ReasonML projects, making the documentation ecosystem more accessible to users. The result of the work packages would be integrated into the reasonml.org website with help from the Reason Association.

Work Packages

The project is split up in following work packages:

WP 200 - JSON Export & Search Index

The ability to export JSON from odoc will enable easy integration with third-party tools and sytems. In particular the JSON output can be used to:

• create a search index for tools like ElasticSearch or Algolia;

• The JSON schema needed for particular search engines, might be different to the one generated by odoc.

• provide a specification of the documentation independent of the compiler and odoc version;

• enable and promote the development of alternative user-interfaces for documentation.

• enable and promote the development of alternative outputs for documentation, such as LaTeX, PDF, etc.

• allow functionality for alternative syntaxes such us Reason to be implemented outside of odoc. For example, the current ReasonML HTML generator could be a separate tool in the future, making odoc a smaller and more portable package.

To achieve this, odoc will be extended (either officially, or as independent implementation) to process .odoc files and generate one or more .json files containing the documentation data models.

Additionally, a command-line tool will be implemented to adapt the emitted JSON files into formats required by a search engine of choice for reasonml.org.

The schema for the JSON output will be defined iteratively, depending on the needs of the user-interface generation and search index requirements.

WP 100 - Markdown Generator

Implement a command line tool that processes module documentation artefacts and produces a file tree with Markdown files. Each generated file would represent a compilation unit (i.e. a module). The resulting files can be directly rendered by Git hosting services like GitHub or GitLab.

This tool can either be implemented directly in odoc (and operate on .odoc files) or it can leverage the proposed JSON output generator. The tool might need to be parametrised by options to generate cross-references that target the rendered version of the markdown files.

In order to overcome some of the limitations of Markdown, like lack of styling and more high-level structure, the produced Markdown could be extended to support MDX. This would enable easy prototyping and iteration for documentation user interfaces.

WP 300 - Documentation for BuckleScript APIs

The current integration between odoc and BuckleScript is not ideal. There is an unofficial integration script, but it is not included by default with BuckleScript, nor does BuckleScript use this tool to generate its documentation.

The purpose of this work package is to define an official integration process for odoc in the BuckleScript compiler and to use that resulting tooling to generate documentation for Belt, Stdlib and other modules in a format suitable for publishing on reasonml.org as seen in the API Section.