Linking back to upstream rluba#35
This also potentially supersedes #1

@Stuart-Mouse I think I roughly understand the gist of the trick you are pulling off here. It's pretty cool actually! I have only one question though. If the implementation of the original Hash_Table changes, does it break the Any_Hash_Table? And if so how easily is it detectable at compile time? If Jon changes Hash_Table one day would we get some easy to follow compilation errors and fix things up?

Also,

Currently only supporting string as the key type.

I don't think any other key type needs any support. Afaik, JSON objects only support string keys anyway. :D

@rexim Unfortunately yeah, I think certain changes to the hash table would cause the Any_Hash_Table to break. I'm not sure how avoidable that is though, since there's no way around the fact that in order to do this in a runtime-generic way you have to entirely reimplement the basic procedures. I could add some more asserts to help prevent basic things like if the struct parameters get reordered or if the internal layout changes at all, but if the actual logic changes for how hashing occurs, then things could just silently break. (From the maintainer's side this is probably not too much of an issue since any changes to Hash Table should be reflected in the changelog for the language, but of course its not ideal if users of Jaison just get an inexplicable break one day because they didn't pull the latest version.)

This is sort of the reason I think it may be better to offer a generalized intercept so that users can poke in their own special handling for things like the Hash_Table. I think if users are knowingly taking the extra dependency then at least they'll be aware that it's a potential point of failure. I have some similar functionality in my own parser that I could try replicating here. If you think that's a better idea I can throw together a little proposal in a different PR and we put a pin in the Hash Table thing until then.

its not ideal if users of Jaison just get an inexplicable break one day because they didn't pull the latest version

Yeah, I feel like subtle undetected changes may send the users down a very annoying rabbit hole...

My idea was to make Jaison operate on Table(string, [N]u8) where N is determined at compile time from the type infos that should also be available at compile time. So basically make Jaison parsers polymorph at different sizes of types. I feel like this would be an ideal solution. I'll try to implement it when I get some spare time. Thank you for submitting this PR!

Hey guys! A simpler solution might be to change table_add to work similar to array_reserve in modules/Basic/Array.jai:

array_reserve :: inline (array: *[..] $T, desired_items: s64) {
    array_reserve(cast(*Resizable_Array) array, desired_items, size_of(T));
}

// This overload of array_reserve takes a non-polymorphic *Resizable_Array,
// which will match any array type (since only the size of the data _pointed at_
// by the array changes with T). We downcast to *Resizable_Array because
// when a leaf like this has many more polymorphs than necessary (due to the size_of(T)
// in vanilla array_reserve() being a constant), it causes everyone up the chain to
// have more polymorphs than necessary, because they need to call different leaves!
array_reserve :: inline (array: *Resizable_Array, desired_items: s64, element_size: s64) {
    …
}

And then one could use the non-polymorphic version in jaison.

For that we would have to modify modules/Hash_Table, of course. I happen to know a guy…

What do you think?

Actually, I have just looked at the implementation of table_add and doing a generic version would be a minor PITA (and also a bit slower since we would replace a bunch of constants with dynamic arguments). Sadge.

@rexim Unfortunately yeah, I think certain changes to the hash table would cause the Any_Hash_Table to break. I'm not sure how avoidable that is though, since there's no way around the fact that in order to do this in a runtime-generic way you have to entirely reimplement the basic procedures. I could add some more asserts to help prevent basic things like if the struct parameters get reordered or if the internal layout changes at all, but if the actual logic changes for how hashing occurs, then things could just silently break. (From the maintainer's side this is probably not too much of an issue since any changes to Hash Table should be reflected in the changelog for the language, but of course its not ideal if users of Jaison just get an inexplicable break one day because they didn't pull the latest version.)

This is sort of the reason I think it may be better to offer a generalized intercept so that users can poke in their own special handling for things like the Hash_Table. I think if users are knowingly taking the extra dependency then at least they'll be aware that it's a potential point of failure. I have some similar functionality in my own parser that I could try replicating here. If you think that's a better idea I can throw together a little proposal in a different PR and we put a pin in the Hash Table thing until then.

This is a good analysis. I (a heavy user of Jaison in prod) definitely don't want a core infra piece to be very fragile like this.

I like the interceptor idea. Unless we get better language support to do this, this approach is both simple and widely used in other languages. It's also what we did for jai-postgres (another great core infra piece by Raphael!) for handling custom types like the ones coming from Postgres extensions.