Would it be easier to use the macros to generate Aeson instances? I don't know if that's bad or generates unhelpful JSON or whatever as I only have peripheral contact with it, but just wondered if it would make things simpler.

Do you mean the Template Haskell facilities? The current JSON output isn't particularly nice either, but it's very systematic too, which means it's predictable if you're familiar with the ADTs.

Yeah exactly. We already use it in a few places, like Types:

That's probably easier then, yes xD

Alright, did that. Much better. Thanks! 👍

I'm just a little apprehensive about using derived instances here, unless the expectation is that people will only be consuming this JSON in other Haskell programs which are using the compiler as a library, i.e. not using the json itself as an interface. For example, with derived instances, is it likely that we might want to change the internal ADT and not be able to do so without breaking third party programs?

That's a good point, I hadn't really considered that aspect of it.

That's a good point. At least I gained more experience with Aeson 😺

Alright, reversed that, I'll now work on Type and Ann.

All the conversions are now implemented.

@paf31 I switched to MIT. The relicensing of the old commits isn't an issue because I plan to squash everything before merging anyway (commit history is a bit of a mess)

@rightfold Could you please paste an example of the generated JSON here for something small like map (_ * 2) [1, 2, 3]? We should get this merged for the next release.

@paf31 JSON doesn't distinguish between floats and ints so you need the discriminator. Int and Number is probably more descriptive than Left and Right though.

Also I'm not sure about object literals and updates, because turning them into JSON objects discards their evaluation order.

@garyb @andyarvanitis Would you like to review the output format here at all, before this gets merged?

Edit: Here's a little example:

{"Main":{"imports":["Prim","Main"],"builtWith":"0.9.2","exports":["X","test"],"decls":{"test":["Abs","f",["Abs","g",["Abs","v",["Abs","x",["Case",[["Var","f"],["Var","g"],["Var","v"],["Var","x"]],[[[["VarBinder","f1"],["VarBinder","g1"],["ConstructorBinder","Main.Y","Main.X",[]],["VarBinder","x1"]],["Literal",["ObjectLiteral",{"z":["Var","Main.X"],"result":["App",["Var","f1"],["App",["Var","g1"],["Accessor","foo",["Var","x1"]]]]}]]]]]]]]],"X":["Constructor","Y","X",[]]},"foreign":[]}}

and prettified:

{
  "Main": {
    "imports": [
      "Prim",
      "Main"
    ],
    "builtWith": "0.9.2",
    "exports": [
      "X",
      "test"
    ],
    "decls": {
      "test": [
        "Abs",
        "f",
        [
          "Abs",
          "g",
          [
            "Abs",
            "v",
            [
              "Abs",
              "x",
              [
                "Case",
                [
                  [
                    "Var",
                    "f"
                  ],
                  [
                    "Var",
                    "g"
                  ],
                  [
                    "Var",
                    "v"
                  ],
                  [
                    "Var",
                    "x"
                  ]
                ],
                [
                  [
                    [
                      [
                        "VarBinder",
                        "f1"
                      ],
                      [
                        "VarBinder",
                        "g1"
                      ],
                      [
                        "ConstructorBinder",
                        "Main.Y",
                        "Main.X",
                        [

                        ]
                      ],
                      [
                        "VarBinder",
                        "x1"
                      ]
                    ],
                    [
                      "Literal",
                      [
                        "ObjectLiteral",
                        {
                          "z": [
                            "Var",
                            "Main.X"
                          ],
                          "result": [
                            "App",
                            [
                              "Var",
                              "f1"
                            ],
                            [
                              "App",
                              [
                                "Var",
                                "g1"
                              ],
                              [
                                "Accessor",
                                "foo",
                                [
                                  "Var",
                                  "x1"
                                ]
                              ]
                            ]
                          ]
                        }
                      ]
                    ]
                  ]
                ]
              ]
            ]
          ]
        ]
      ],
      "X": [
        "Constructor",
        "Y",
        "X",
        [

        ]
      ]
    },
    "foreign": [

    ]
  }
}

Hmm. It looks like the distinction between top-level let and letrec is gone?

Yes, but we could add it back. I'd probably like to change the syntax a bit though.

It'd be nice if non-recursive things appeared like they do now, and we added a Rec constructor to wrap recursive bindings IMO.

Or:

"decls": {
  "let": {...},
  "letrec": [
    {...},
    {...}
  ]
}

Do we need to distinguish self-recursive declarations from non-recursive ones? If not, I'd rather do something like:

"decls": [
  [ { "foo": ["Abs", "f", ... ] } ]
  [ { "bar": ["Abs", "a", ["Var", "baz"]] }
  , { "baz": ["Abs", "b", ["Var", "bar"]] }
  ]
}

where the top level is a list of binding groups, each of which is a list of function declarations.

@paf31 Sorry, I was on an overseas flight (with no wifi).

I'm still jet lagged right now, but it looks good. I also just looked at the sample posted by @rightfold earlier which included type information -- which is good, since the C++ backend is back to using type information (for optimizations now).

One question, though: are all imported values available? The C++ backend also uses those for some optimations (basically grabbing things from Environment currently).

Do we need to distinguish self-recursive declarations from non-recursive ones? If not, I'd rather do something like:

That looks good to me. It works for data binding groups too.

I don't think it should be the responsibility of the AST format to indicate when a function is self-recursive, since that's not something we even track in the AST, so in doing that we're kinda making the JSON representation a different thing. It should be easy enough to figure out by anything that does need that distinction too.

since that's not something we even track in the AST

I thought we did track self-recursive functions in the core AST, as a Rec constructor with a singleton list of bindings.

Core has Let/LetRec distinction for both top-level declarations and let-expressions

I thought we did track self-recursive functions in the core AST, as a Rec constructor with a singleton list of bindings.

Oh right. I should probably know that since I contributed CoreFn. 😆

I think the question should be: do we need this for the first version, bearing in mind that it might be tricky to implement it in a non-breaking way if we decide to use it later? For that reason, I think we should add it, but I'm happy to spend some time figuring out the syntax.

@andyarvanitis Actually, there are no type annotations in this version. The reason is that I would like to discuss move towards something a bit more like GHC core, where only binders get type annotations. With that approach, the type checker would elaborate directly to functional core, rather than decorating the initial AST.

One question, though: are all imported values available?

The core JSON will include a list of imports, but you would have to pull the definitions of the imports from the appropriate core JSON/externs file. Does that work?

@garyb @andyarvanitis Binding group information has now been added back to the JSON output.

Ok, thanks, I've gone back through the comments and see now what's there for the first version. I probably won't switch the C++ backend over to it just yet, then, but it's looks good for some other things I've been thinking about doing.