Parsing of Dune Files

Parsing dune files is done in two steps:

  • They are parsed as S-expressions using src/dune_sexp/parser.mli;

  • Then they are decoded using src/dune_sexp/decoder.mli. The result of this decoding step is added to an extensible variant using a mechanism in src/dune_lang/stanza.mli.

Instead of writing a parser or using pattern matching, we define decoders, which are abstract values of type 'a Decoder.t (returning a value of type 'a). These decoders are assembled using combinators. For example, we can use simple decoders to write a decoder for a record type. This decoder abstraction is monadic, but the applicative subset is sufficient for most decoders.

As an example, here is how (copy_files) is parsed:

src/dune_rules/stanzas/copy_files.ml
31let long_form =
32  let check = Dune_lang.Syntax.since Stanza.syntax (2, 7) in
33  let+ alias = field_o "alias" (check >>> Dune_lang.Alias.decode)
34  and+ mode = field "mode" ~default:Rule.Mode.Standard (check >>> Rule_mode_decoder.decode)
35  and+ enabled_if = Enabled_if.decode ~allowed_vars:Any ~since:(Some (2, 8)) ()
36  and+ files = field "files" (check >>> String_with_vars.decode)
37  and+ only_sources =
38    field_o
39      "only_sources"
40      (Dune_lang.Syntax.since Stanza.syntax (3, 14) >>> decode_only_sources)
41  and+ syntax_version = Dune_lang.Syntax.get_exn Stanza.syntax in
42  let only_sources = Option.value only_sources ~default:Blang.false_ in
43  { add_line_directive = false
44  ; alias
45  ; mode
46  ; enabled_if
47  ; files
48  ; only_sources
49  ; syntax_version
50  }

The fields are queried individually, and a record is built using all the intermediate results. This will automatically take care of generating “unknown field X,” “duplicate field X,” and similar error messages.

Another interesting thing to note is that the fields are not decoded directly, but use the following pattern:

Syntax.since Stanza.syntax (x, y) >>> decoder

Let’s unpack this: (>>>) will run a unit Decoder.t on the input before passing the input to an actual decoder. The first decoder can be used to implement a check and trigger an error in some cases.

Here, it is used for versioning. For example the (copy_files) stanza started supporting (enabled_if) in version 2.8. Decoding this field is protected by this since call: it means that if the language version in dune-project file is greater than 2.8. In particular, this ensures that the project can not be built with Dune versions older than 2.8.0.

Once decoding succeeds, various stanzas are turned into various types defined in src/dune_rules/stanzas/.