Skip to content

Option --dump-coreimp to output imperative-core to JSON#3117

Closed
metaleap wants to merge 37 commits intomasterfrom
unknown repository
Closed

Option --dump-coreimp to output imperative-core to JSON#3117
metaleap wants to merge 37 commits intomasterfrom
unknown repository

Conversation

@metaleap
Copy link
Copy Markdown

@metaleap metaleap commented Oct 16, 2017

New purs compile option --dump-coreimp, exactly like the existing --dump-corefn, to dump to a coreimp.json output file:

  1. the CoreImp AST
  2. plus meta & type information about top-level declarations (exported or not). This in support of any coreimp.json consumer with an "alternative-backend/transpile-to-X/codegen" use-case (predictably most of them), targeting languages less 'dynamic' / more-typed than JS. Where it's highly advisable to e.g. reconstruct type-class/instance relations rather than hackily reinterpret what traces raw CoreImp has left of them, same for ADTs etc. (I do find it highly ergonomic to have this all output into a single .json file with a few top-level fields.)

Requested in #876 and also in #711 in the comments, also previously attempted-but-seemingly-abandoned in #971

(The results can be seen in the various coreimp.json files throughout this directory --- not the most intuitively human-readable representation (that'd be straight JS I guess --- but it's not entirely incomprehensible either, imho on-par with the other *.json files purs already emits, imho) but quite suitable for painless decoding in languages without Aeson.

New modules:

  • src/Language/PureScript/CoreImp/ToJSON.hs --- modeled after the existing src/Language/PureScript/CoreFn/ToJSON.hs.

Existing modules:

Mostly, merely duplications of the existing --dump-corefn option handling in:

  • app/Command/Compile.hs
  • src/Language/PureScript/Make.hs
  • src/Language/PureScript/Options.hs
    (Too small & trivial to generalize into helper funcs for now IMHO --- until a 3rd --dump-xyz request comes along in the future at least.)

src/Language/PureScript/CoreFn/ToJSON.hs modified only to export a small number of its already extant utility functions that overlap with CoreImp/ToJSON needs, where it'll be handy if the latter stays consistent with the former (by importing rather than duplicating them).

JSON format:

has the following top-level fields:

  • equivalent to CoreFn/ToJSON: moduleName, modulePath, exports, imports, builtWith, foreign, comments
  • body: the CoreImp AST itself (not a pretty format but decodes very handily for outside consumers)
  • declEnv: the sub-set of the current Environment that refers directly to top-level declarations in this currently --dumped module (the externs.json-like info about synonyms/constructors/classes/instances etc and offers us type signatures for top-level names)
  • declAnns a sub-set of / the same format as CoreFn/ToJSON's decls but only for top-level definitions (coreimp and corefn naturally diverge from there on down) and only covering annotations/comments/basic-expression-type (like Abs or Var etc)

kcsongor and others added 25 commits September 9, 2017 15:36
* Solve ConsSymbol

This adds and solves a new 'magic' constraint,
`Type.Data.Symbol.ConsSymbol`, with the following functional
dependencies:

class ConsSymbol (head :: Symbol)
                 (tail :: Symbol)
                 (sym :: Symbol) | sym -> head tail, head tail -> sym

* The `head tail -> sym` direction works like `AppendSymbol`, but only
resolves when `head` is a singleton symbol.

* The `sym -> head tail` direction deconstructs the symbol into its
first character and the remaining string.

* Style fixes
* Add proxies

* Add failing proxy test cases

* Make proxy types of higher precedence

* Proxy is no longer reserved

* Move "proxy" to reservedTypeNames

* s/proxy/@/g
* Add applicative do notation

* Test the order of arguments to the result of an applicative do-block

* Implement let statements in applicative do-blocks

* Correctly handle PositionedDoNotationElement in applicative do-notation

* Prefer map over pure when desugaring applicative do-notation

* Delete AdoLetNotSupported variant
* Add instance chain groups

* Make OverlappingInstances an error
* Add inlining for runEffFnN/mkEffFnN

* Magic-do case to improve EffFn usage

From natefaubion #3026 (comment)

* Apply magicDo until fixed point

* Rearrange magic do optimizations

* Stop run(Eff)FnX optimisation firing from local definitions

* Add EffFn optimisation test
* fixes cabal warning about an unspecified Paths module

* Bump to GHC 8.2 stack nightly

* trigger rebuild

* fix more shadowing warnings

* retrigger build

* try again
* Fix and regression tests for #2197

* Updated Prim module documentation

* Added test using Record type

* Added some peace-of-mind tests
Previously, the error message made no mention of a solution to the
error. This update adds possible solutions to the error message in
the form of the remaining covering sets. In other words, the remaining
variables that need to be determined in order to satisfy the
typechecker.
* Update Overlapping instances error message.

Fixes #3071.

* Remove explanation for Overlapping instances error.

* Add to contributors.
Remove old video link
Add note about opening issues
* CoreFn.FromJSON: parse json representation of CoreFn

* CoreFn.Module: ModuleT

Polymorphic ModuleT type: when parsing foreign declarations we don't have
access to their type, and thus we need a `ModuleT ()` type.  The
original `Module` type becomes an alias for `ModuleT Type`.

* Language.PureScript.CoreFn.DCE module

* CoreFn: include `SourceSpan`s in json representation

* CoreFn: add Meta to json representation

It is required when runnings `moduleToJs`.

* Update TestCoreFn

* Change json encoding of CoreFn Bind

* Remove `purs dce` command.

* Remove CoreFN.DCE module

* clean

* remove white space in TestCoreFn

* Object CoreFn representation.

* CoreFn moduleToJSON: new representation

- change the json representation of CoreFn
- store module name on the module object
- `buildMakeAction`: output single module json object, rather an object
with a single key (module name) pointing to module json object.

* CoreFn.FromJSON: update `moduleFromJson`

- read the new json CoreFn representation
- extend tests so that all the CoreFn structures are tested.

* Add Module Comments to json CoreFn representation

- `moduleToJSON`: include module comments inside `comments` field.
- `moduleFromJSON`
- add tests

* CoreFn json repr: modulePath to the top level

* Applied @paf31 & @kritzcreek review
Remove the type variable.
* Add valid location list to orphan instance error

Previously, the orphan instance simply said to consider a newtype. Now,
it lists all the places in the project that would be considered valid
locations for the instance as well. This hopefully closes #3063.

* Updates from review

Firstly, there's a little change to the error output in the case of
there being no place for the instance except Prim. Previously, it just
printed an empty list, which didn't make much sense, gramatically. Now,
however, a different error is displayed to point out that there's no
reasonable home for the instance.

As well as this, the `typeModule` function in `TypeChecker.hs` can now
deal with proxy types. Previously, it couldn't, and the tests were
passing really due to an accident of laziness. Now, the tests pass, and
all is well in the world.

* Tidy the orphan instance error further.

Thinking about it, you'll only ever see, at most, two things in the list
of possible locations. So, we can intercalate the list with `or` and get
a much friendlier-looking error.

* Adjust wording on error feedback

Previously, the error mentioned that the instance "must" be declared in
one of a set of places, which @goodacre.liam (rightly) noted would imply
that there aren't other, equally valid, solutions. The wording has been
altered to address this.
@metaleap
Copy link
Copy Markdown
Author

metaleap commented Oct 18, 2017

A quick note on the general motivation here and on why I'm also dumping a relevant sub-set of the Environment together with CoreImp: I think --dump-coreimp can be a cool alternative option for writing alternative backends (alternative to a backend being a soon-outdated fork of purs that users need to keep around as parallel-to-purs executables). With --dump-coreimp, anyone can hack or script their desired PureScript(CoreImp)-to-XYZ transformations without having to navigate this sizable Haskell code-base and the slowish GHC/Stack/Cabal build process, without having to keep their fork in sync with the main repo, etc. They can also keep it updated much easier along with regular purs releases.

Given that (unless you're transpiling to similarly "dynamic" languages as JS, such as perhaps Python or Erlang) for many candidate targets that'd spring to mind for me (Go right now, potentially C#/Java/the likes) you'd want the Environment sub-set for the whole module, not just the exports already in externs.json. Type classes, instances, data types / ctors, type aliases, and "names" (func arg/ret types). This allows for generating much more idiomatic, correct, performant code. Such languages of course typically would allow us to pass around "dynamic"/"any-typed" (boxed) values with constant casts/type-asserts, but it's not desirable at all when it can be avoided. The philosophy of PureScript to generate readable idiomatic "as-if-human-written" code, I'd also want to afford any --dump-coreimp consumers too.

And that's why --dump-coreimp as offered here dumps quite a (useful) bit more than just-CoreImp-per-se.

@paf31
Copy link
Copy Markdown
Contributor

paf31 commented Oct 18, 2017

I don't have time to fully review this right now. Thanks for opening the PR, but I want to note that we need to discuss this before we land on a design. Imperative core is not finalized yet, so there is some design work which needs to go into the internal representation. And we will probably need to engage in some bikeshedding regarding the JSON format.

@metaleap
Copy link
Copy Markdown
Author

metaleap commented Oct 18, 2017

Sure take your time, just wanted to "plug back in" here and happy to keep updating the CoreImp/ToJSON.hs module in tune with any upcoming coreimp changes landing in master. It's all in flight, that's fine by me.

As regards to "JSON bikeshedding", that's to be expected but I'm about to clean up and "lean up" various parts of the output further as well.

@metaleap
Copy link
Copy Markdown
Author

metaleap commented Oct 24, 2017

@paf31 & @ all, a question re:

Imperative core is not finalized yet, so there is some design work which needs to go into the internal representation

I can tell already that coreimp is a far cry from what it could be: fairly decoupled from JS. I'd ponder that stuff like "use strict";, module.exports, ints with |0 etc. would do well to go into the coreimp-to-JS transformation (it's trivial for a coreimp consumer to ditch manually of course, but the beginnings of a long increasingly-messier neverending series of transformations of what currently amounts to vanilla JS), for example.. I'd even go so far as to say, it'd be desirable for CoreImp to preverse PS type info on values/args/signatures/constraint-type-vars, and only for the JS pass to erase them. (Now of course CoreImp uses JS type semantics as in prim-types/classes/objects/arrays. But it's all coming from some PS type semantics that can be carried along expressions as corefn-to-coreimp happens.)

But that's all fluffy ponderings.. Where's the "CoreImp design work" happening right now, who's involved and cracking this? Is there a thread? Is it all happening in the mailing list, or in some buried github-issue threads I should join? At the least I might like to follow along to pick up everyone's thinking of where this should go. I can also add the perspective of first-hand experience wrestling with the current state of coreimp on-the-ground =)

@natefaubion
Copy link
Copy Markdown
Contributor

This has not seen activity in some time. Should this be closed?

@metaleap
Copy link
Copy Markdown
Author

Fine by me, it's been a year since this the last reaction saying:

Imperative core is not finalized yet, so there is some design work which needs to go into the internal representation

and I've already long moved on from the effort I envisioned back then. =)

@garyb
Copy link
Copy Markdown
Member

garyb commented Oct 23, 2018

Thanks for the time you spent on it anyway!

@garyb garyb closed this Oct 23, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.