OCaml serializing a (no args) variant as a "string enum" (via Yojson)

Question

Say I am building a record type:

type thing {
  fruit: string;
}

But I want the possible values of fruit to be constrained to a fixed set of strings.

It seems natural to model this in OCaml as a variant, e.g.:

type fruit = APPLE | BANANA | CHERRY

type thing {
  fruit: fruit;
}

Okay so far.

But if I use [@@deriving yojson] on these types then the serialized output will be like:

{ "fruit": ["APPLE"] }

By default Yojson wants to serialize a variant as a tuple of [<name>, <args>...] which... I can see the logic of it, but it is not helpful here.

I want it to serialize as:

{ "fruit": "APPLE" }

Making use of a couple of ppx deriving plugins I managed to build this module to de/serialize as I want:

module Fruit = struct
  type t = APPLE | BANANA | CHERRY [@@deriving enum, variants]

  let names =
    let pairs i (name, _) = (name, (Option.get (of_enum i))) in
    let valist = List.mapi pairs Variants.descriptions in
    List.to_seq valist |> Hashtbl.of_seq
  
  let to_yojson v = `String (Variants.to_name v)

  let of_yojson = function
    | `String s -> Hashtbl.find_opt names s
                   |> Option.to_result ~none:(Printf.sprintf "Invalid value: %s" s)
    | yj -> Error (Printf.sprintf "Invalid value: %s" (Yojson.Safe.to_string yj))
end

Which works fine... but I have some other "string enum" variants I want to treat the same way. I don't want to copy and paste this code every time.

I got as far as this:

module StrEnum (
  V : sig
    type t
    val of_enum : int -> t option
    module Variants : sig
      val descriptions : (string * int) list
      val to_name : t -> string
    end
  end
) = struct  
  type t = V.t

  let names =
    let pairs i (name, _) = (name, (Option.get (V.of_enum i))) in
    let valist = List.mapi pairs V.Variants.descriptions in
    List.to_seq valist |> Hashtbl.of_seq
  
  let to_yojson v = `String (V.Variants.to_name v)

  let of_yojson = function
    | `String s -> Hashtbl.find_opt names s
                  |> Option.to_result ~none:(Printf.sprintf "Invalid StrEnum value: %s" s)
    | yj -> Error (Printf.sprintf "Invalid StrEnum value: %s" (Yojson.Safe.to_string yj))
end

module Fruit = struct
  type t = APPLE | BANANA | CHERRY [@@deriving enum, variants]
end

module FruitEnum = StrEnum (Fruit)

That much seems to type-check, and I can:

utop # Yojson.Safe.to_string (FruitEnum.to_yojson Fruit.APPLE);;
- : string = "\"APPLE\""

utop # FruitEnum.of_yojson (Yojson.Safe.from_string "\"BANANA\"");;
- : (FruitEnum.t, string) result = Ok Fruit.BANANA

...but when I try to:

type thing {
  fruit: FruitEnum.t;
}
[@@deriving yojson]

I get Error: Unbound value FruitEnum.t

It seems to be because I am re-exporting type t = V.t from the variant's module, I don't really understand though. (Or is it because the yojson ppx can't "see" the result of the functor properly?)
How can I fix this?

I would also like to be able to skip defining the variant module separately and just do:

module Fruit = StrEnum (struct
  type t = APPLE | BANANA | CHERRY [@@deriving enum, variants]
end)

...but this gives the error:

Error: This functor has type
       functor
         (V : sig
                type t
                val of_enum : int -> t option
                module Variants :
                  sig
                    val descriptions : (string * int) list
                    val to_name : t -> string
                  end
              end)
         ->
         sig
           type t = V.t
           val names : (string, t) Hashtbl.t
           val to_yojson : t -> [> `String of string ]
           val of_yojson : Yojson.Safe.t -> (t, string) result
         end
       The parameter cannot be eliminated in the result type.
       Please bind the argument to a module identifier.

and I don't understand what is wrong.

Answer 1

Regarding the last error, it's because OCaml requires a 'stable path' to types inside modules so it can refer to them. A stable path is a named path to a type, e.g. Fruit.t.

By contrast, StrEnum(struct type t = ... end).t is not a stable path because the type t is referencing a type t in the module literal which does not have a name.

Long story short, you basically can't skip defining the variant module separately. But it's simple to do it in two steps:

module Fruit = struct
  type t = ...
end

module Fruit = StrEnum(Fruit)

The second definition refers to the first and shadows it. Shadowing is a well-known and often-used technique in OCaml.

Overall, I'm not sure all this PPX machinery is actually justified. You can pretty easily hand-write converter functions, e.g.

let to_yojson = function
  | APPLE -> `String "APPLE"
  | BANANA -> `String "BANANA"
  | CHERRY -> `String "CHERRY"

Answer 2

Well, I was curious to have a go at writing a PPX deriver to perform this transformation.

Here's what I ended up with:

open Ppxlib
module List = ListLabels

let make_methods ~(loc : location) ~(is_poly : bool) (constructors : constructor_declaration list) =
  let (module Ast) = Ast_builder.make loc in
  let v_patt = match is_poly with
    | true -> fun name -> Ast.ppat_variant name None
    | false -> fun name -> Ast.ppat_construct { txt = (Lident name); loc } None
  and v_expr = match is_poly with
    | true -> fun name -> Ast.pexp_variant name None
    | false -> fun name -> Ast.pexp_construct { txt = (Lident name); loc } None
  in
  let (to_cases, of_cases) =
    List.map constructors ~f:(
      fun cd ->
        let name = cd.pcd_name.txt in
        let to_case = {
          pc_lhs = v_patt name;
          pc_guard = None;
          pc_rhs = [%expr `String [%e Ast.estring name] ];
        } in
        let of_case = {
          pc_lhs = Ast.ppat_variant "String" (Some (Ast.pstring name));
          pc_guard = None;
          pc_rhs = [%expr Ok ([%e v_expr name]) ];
        } in
        (to_case, of_case)
    )
    |> List.split
  in
  let of_default_case = {
    pc_lhs = [%pat? yj ];
    pc_guard = None;
    pc_rhs = [%expr Error (Printf.sprintf "Invalid value: %s" (Yojson.Safe.to_string yj)) ];
  } in
  let of_cases = of_cases @ [of_default_case] in
  let to_yojson = [%stri let to_yojson = [%e Ast.pexp_function to_cases]] in
  let of_yojson = [%stri let of_yojson = [%e Ast.pexp_function of_cases] ] in
  [to_yojson; of_yojson]

let type_impl ~(loc : location) (td : type_declaration) =
  match td with
  | {ptype_kind = (Ptype_abstract | Ptype_record _ | Ptype_open); _} ->
    Location.raise_errorf ~loc "Cannot derive yojson_str_enum for non variant types"
  | {ptype_kind = Ptype_variant constructors; _} -> begin
      let invalid_constructors =
        List.filter_map constructors ~f:(
          fun cd -> match cd.pcd_args with
            | (Pcstr_tuple [] | Pcstr_record []) -> None
            | _ -> Some (cd)
        )
      in
      if (List.length invalid_constructors) > 0 then
        Location.raise_errorf ~loc "Cannot derive yojson_str_enum for variant types with constructor args";
      match is_polymorphic_variant td ~sig_:false with
      | `Definitely | `Maybe -> make_methods ~loc ~is_poly:true constructors
      | `Surely_not -> make_methods ~loc ~is_poly:false constructors
    end

let generate_impl ~ctxt (_rec_flag, type_declarations) =
  (* [loc] is "location", not "lines of code" *)
  let loc = Expansion_context.Deriver.derived_item_loc ctxt in
  List.map type_declarations ~f:(type_impl ~loc)
  |> List.concat

let yojson_str_enum =
  Deriving.add
    "yojson_str_enum"
    ~str_type_decl:(Deriving.Generator.V2.make_noarg generate_impl)

to make usable it needs a dune file something like:

(library
  (kind ppx_rewriter)
  (name <lib name>)
  (preprocess (pps ppxlib.metaquot))
  (libraries yojson ppxlib))

After adding <lib name> to the pps in your dune file, usage is like:

module Fruit = struct
  type t = APPLE | BANANA | CHERRY [@@deriving yojson_str_enum]
end

It seems to work fine for my use case. It might be extended per the comment by @Yawar to take args allowing to specify to/from string transform functions for the variant labels. But I was happy just with Fruit.APPLE -> "APPLE" for now. I should also implement the sig_type_decl version.

One part I am a bit uncertain about is this:

      match is_polymorphic_variant td ~sig_:false with
      | `Definitely | `Maybe -> make_methods ~loc ~is_poly:true constructors
      | `Surely_not -> make_methods ~loc ~is_poly:false constructors

I am not very clear when the `Maybe case occurs or how it should most correctly be handled, or if there is a better way of detecting "backtick variants" than using the is_polymorphic_variant method from ppxlib.