Probably could have done a better job explaining how this works.

The basic idea is, a schema is registered as a hidden "__proto" object on the Kind. It is in the format of an entity proto, without any actual values. When an entity is compared against a kind, the entity is converted to an entity proto, and matched up against the "ghost protocol".

a schema is registered as a hidden "__proto" object on the Kind

Can you explain this further? I don't quite understand why this needs to be done and what the purpose of __proto is. I've never actually used this property or pattern in the past so I don't have a clue what its purpose is. Googling for "__proto" just returns results on "__proto__" and that "The __proto__ property is deprecated and should not be used." which sounds sketchy.

I'll start this post off the way I started my last one... Probably could have done a better job explaining how this works! There's actually nothing special about __proto, I just used the double underscore to mark it pseudo-private. If it's more clear, I could re-structure the fieldMeta object to look more like:

{
  name: {
    metadata: {
      type: String,
      indexed: false
    },
    protocol: {
      string_value: null
    }
  },
  age: {
    metadata: {
      type: datastore.int
    },
    protocol: {
      integer_value: null
    }
  }
}

It's currently:

{
  name: {
    type: String,
    indexed: false,
    __proto: {
      string_value: null
    }
  },
  age: {
    type: datastore.int,
    indexed: false,
    __proto: {
      integer_value: null
    }
  }
}

The basic idea is to store this "ghost protocol" (it sounds too cool not to call it that) to compare against the later completed protocol, whereby a "protocol" means: an entity in the format the JSON api expects. There are many conversion methods in entity.js, which handle taking something like: "Hello!" and converting it to the datastore format, { "string_value": "Hello!" }. I wanted to take the metadata schema format, such as:

var schema = {
  name: {
    type: String,
    indexed: false
  },
  tags: {
    type: String,
    multi: true
  },
  favArticles: {
    type: 'Key',
    multi: true
  },
  contact: {
    type: {
      telephone: { type: String },
      email: { type: String }
    }
  }
};

...and turn it into an empty (ghost) "protocol".

The whole process looks like this:

1. User registers a Kind:

   dataset.registerKind('Person', {
     name: {
       type: 'String'
     }
   });

   Internally, we flesh out this object to the ghost protocol.

2. User wants to save a new Person entity, but first wants to validate:

   dataset.validate('Person', {
     name: 'Stephen'
   });

   Internally, we convert the object passed in to an entity protocol, then compare it against the ghost protocol.

Hope this helps. Feel free to let me know if you (or anyone else, of course) have any ideas for simplifying the validation. 👍

Updated to box in the metadata and protocol, as shown in #157 (comment)

@rakyll friendly ping. I know this was going to be M2, but if you have any thoughts on this now, I'm happy to work on them so this makes the cut. Plus, since the change is rather large, the longer we wait, the more likely I'm stuck in rebase sad land. If there's anything I can do a better job of explaining, or you have any questions, let me know. If it makes it, yay! If not, at least we got to remove an outstanding PR :)

Could we keep it open for a while? Datastore has plans for an indexing/schema API. We should cover their needs before we publicize a client supports a schema.