ChatScript/HTMLDOCUMENTATION/ChatScript-Json.html at master · Schurke/ChatScript

509 lines (498 loc) · 47.8 KB
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
  <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
  <meta http-equiv="Content-Style-Type" content="text/css" />
  <meta name="generator" content="pandoc" />
  <title></title>
  <style type="text/css">code{white-space: pre;}</style>
  <style type="text/css">
div.sourceCode { overflow-x: auto; }
table.sourceCode, tr.sourceCode, td.lineNumbers, td.sourceCode {
  margin: 0; padding: 0; vertical-align: baseline; border: none; }
table.sourceCode { width: 100%; line-height: 100%; }
td.lineNumbers { text-align: right; padding-right: 4px; padding-left: 4px; color: #aaaaaa; border-right: 1px solid #aaaaaa; }
td.sourceCode { padding-left: 5px; }
code > span.kw { color: #007020; font-weight: bold; } /* Keyword */
code > span.dt { color: #902000; } /* DataType */
code > span.dv { color: #40a070; } /* DecVal */
code > span.bn { color: #40a070; } /* BaseN */
code > span.fl { color: #40a070; } /* Float */
code > span.ch { color: #4070a0; } /* Char */
code > span.st { color: #4070a0; } /* String */
code > span.co { color: #60a0b0; font-style: italic; } /* Comment */
code > span.ot { color: #007020; } /* Other */
code > span.al { color: #ff0000; font-weight: bold; } /* Alert */
code > span.fu { color: #06287e; } /* Function */
code > span.er { color: #ff0000; font-weight: bold; } /* Error */
code > span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warning */
code > span.cn { color: #880000; } /* Constant */
code > span.sc { color: #4070a0; } /* SpecialChar */
code > span.vs { color: #4070a0; } /* VerbatimString */
code > span.ss { color: #bb6688; } /* SpecialString */
code > span.im { } /* Import */
code > span.va { color: #19177c; } /* Variable */
code > span.cf { color: #007020; font-weight: bold; } /* ControlFlow */
code > span.op { color: #666666; } /* Operator */
code > span.bu { } /* BuiltIn */
code > span.ex { } /* Extension */
code > span.pp { color: #bc7a00; } /* Preprocessor */
code > span.at { color: #7d9029; } /* Attribute */
code > span.do { color: #ba2121; font-style: italic; } /* Documentation */
code > span.an { color: #60a0b0; font-weight: bold; font-style: italic; } /* Annotation */
code > span.cv { color: #60a0b0; font-weight: bold; font-style: italic; } /* CommentVar */
code > span.in { color: #60a0b0; font-weight: bold; font-style: italic; } /* Information */
<h1 id="chatscript-json-manual">ChatScript JSON Manual</h1>
<p>© Bruce Wilcox, mailto:gowilcox@gmail.com www.brilligunderstanding.com <br>Revision 3/29/2020 cs10.1</p>
<h1 id="real-world-json">Real World JSON</h1>
<p>JSON (JavaScript Object Notation) is an open standard format using human-readable text to transmit data objects over the web. It is a common standard largely replacing XML which is too wordy and hard to read. JSON has two datatypes that represent collections of values, the array and the object. A JSON array is a list of JSON entities separated by commas and placed within square brackets <code>[]</code>, e.g.,</p>
<pre><code>[ A, 2, [ help, life], [] ]</code></pre>
<p>Indices of an array start at 0, so the above has as values:</p>
<pre><code>[0] = A 
[2] = an array of 2 values 
[3] = an empty array</code></pre>
<p>Note that arrays can hold values of different types (since really everything internally is a text type). The JSON types are array, object, number, string (enclosed in doublequotes), and primitives (text without doublequotes that cannot contain any whitespace). Array values are ordered and always retain that order. A JSON object is a list of key-value pairs separated by commas and placed within curly braces <code>{}</code>, e.g.,</p>
<div class="sourceCode"><pre class="sourceCode json"><code class="sourceCode json"><span class="fu">{</span> 
    <span class="dt">&quot;key1&quot;</span><span class="fu">:</span> <span class="dv">1</span><span class="fu">,</span> 
    <span class="dt">&quot;bob&quot;</span><span class="fu">:</span> <span class="st">&quot;help&quot;</span><span class="fu">,</span> 
    <span class="dt">&quot;1&quot;</span><span class="fu">:</span> <span class="dv">7</span><span class="fu">,</span> 
    <span class="dt">&quot;array&quot;</span><span class="fu">:</span> <span class="ot">[</span><span class="dv">1</span><span class="ot">,</span> <span class="dv">2</span><span class="ot">,</span> <span class="dv">3</span><span class="ot">]</span><span class="fu">,</span>
    <span class="dt">&quot;object12&quot;</span><span class="fu">:</span> <span class="fu">{}</span> 
<span class="fu">}</span></code></pre></div>
<p>Each key must be encased in quotes and joined to an ending colon. Whitespace separates the colon from the value. Again types can be mixed in the values. <code>{}</code> is the empty object. Key-value pairs have no guaranteed order and may shuffle around if you manipulate the structure. You can nest arrays and objects inside each other.</p>
<h1 id="chatscript-json">ChatScript &amp; JSON</h1>
<p>JSON is an excellent language to represent more complex ChatScript facts as well as interact with the web. ChatScript can convert back and forth between the JSON text string passed over the web and ChatScript facts that represent the structure internally. If you tried to create facts using ^createfact(), you would find making the data shown below extremely difficult and non-obvious. Though it can be done. But as JSON, it is easy to create facts to represent the structure and to access pieces of it.</p>
<div class="sourceCode"><pre class="sourceCode json"><code class="sourceCode json"><span class="fu">{</span>
    <span class="dt">&quot;firstName&quot;</span><span class="fu">:</span> <span class="st">&quot;John&quot;</span><span class="fu">,</span>
    <span class="dt">&quot;lastName&quot;</span><span class="fu">:</span> <span class="st">&quot;Smith&quot;</span><span class="fu">,</span>
    <span class="dt">&quot;age&quot;</span><span class="fu">:</span> <span class="dv">25</span><span class="fu">,</span>
    <span class="dt">&quot;address&quot;</span><span class="fu">:</span> <span class="fu">{</span>
        <span class="dt">&quot;streetAddress&quot;</span><span class="fu">:</span> <span class="st">&quot;21 2nd Street&quot;</span><span class="fu">,</span>
        <span class="dt">&quot;city&quot;</span><span class="fu">:</span> <span class="st">&quot;New York&quot;</span><span class="fu">,</span>
        <span class="dt">&quot;state&quot;</span><span class="fu">:</span> <span class="st">&quot;NY&quot;</span><span class="fu">,</span>
        <span class="dt">&quot;postalCode&quot;</span><span class="fu">:</span> <span class="st">&quot;10021-3100&quot;</span>
    <span class="fu">},</span>
    <span class="dt">&quot;phoneNumbers&quot;</span><span class="fu">:</span> <span class="ot">[</span>
        <span class="fu">{</span>
        <span class="dt">&quot;type&quot;</span><span class="fu">:</span> <span class="st">&quot;home&quot;</span><span class="fu">,</span>
        <span class="dt">&quot;number&quot;</span><span class="fu">:</span> <span class="st">&quot;212 555-1234&quot;</span>
        <span class="fu">}</span><span class="ot">,</span>
        <span class="fu">{</span>
        <span class="dt">&quot;type&quot;</span><span class="fu">:</span> <span class="st">&quot;office&quot;</span><span class="fu">,</span>
        <span class="dt">&quot;number&quot;</span><span class="fu">:</span> <span class="st">&quot;646 555-4567&quot;</span>
        <span class="fu">}</span>
    <span class="ot">]</span><span class="fu">,</span>
    <span class="dt">&quot;children&quot;</span><span class="fu">:</span> <span class="ot">[]</span><span class="fu">,</span>
    <span class="dt">&quot;spouse&quot;</span><span class="fu">:</span> <span class="kw">null</span>
<span class="fu">}</span></code></pre></div>
<p>JSON facts are triples where the subject is always the name of a JSON object or array. For objects, the verb of the fact is the name of a field and the object is its value. Such a structure might be composed of these facts:</p>
<pre><code>(jo-3 name Robert)
(jo-3 age 10)
(jo-3 gender male)</code></pre>
<p>This would correspond to a JSON string that looks like this:</p>
<pre><code>{ &quot;name&quot;: &quot;Robert&quot;, &quot;age&quot;: 10, &quot;gender&quot;: &quot;male&quot;}</code></pre>
<p>Actually, the &quot;age 10&quot; fact in ChatScript could represent either what was shown above or:</p>
<pre><code>{ &quot;name&quot;: &quot;Robert&quot;, &quot;age&quot;: &quot;10&quot;, &quot;gender&quot;: &quot;male&quot;}</code></pre>
<p>Because JSON has distinct data types as values (primitives, text strings, etc) and CS represents everything as text strings, JSON facts have flag bits that describe how to treat the object value of the fact. This is automatically handled by CS and normally you have no reason to pay any attention to it.</p>
<p>For arrays, the verb of the fact is the numeric index in the array and the object is the value. Such a structure might be composed of these facts:</p>
<pre><code>(ja-5 0 Robert)
(ja-5 1 Terry)
(ja-5 2 Paula)</code></pre>
<p>This would correspond to a JSON string that looks like this:</p>
<pre><code>[ &quot;Robert&quot;, &quot;Terry&quot;, &quot;Paula&quot;]</code></pre>
<p>Since JSON data structures are implemented as facts, they can use the ^query abilities just as ordinary facts can. Whereas in JSON you can only access data going top down, with CS you can equally access data at any level directly querying on some field name.</p>
<p>Note that JSON has no mechanism for sharing JSON subtrees. Hence anytime you create a JSON fact structure in CS, the facts will all be unique. You do this like this:</p>
<pre><code>    $_tmp = ^createfact($$jsonobject newfield $$jsondata (FACTTRANSIENT JSON_OBJECT_FACT JSON_OBJECT_VALUE) )</code></pre>
<p>If $$jsonobject is a json object name, you add a field called newfield whose value is a JSON object, thus joining the trees. If you serialize (write into text) these two JSON objects they will become completely separate structures when read back in (JSON doesn't share).</p>
<p>Also note that while CS can read in JSON whose value has an empty string like:</p>
<pre><code>    { &quot;input&quot;:&quot;&quot;}</code></pre>
<p>CS cannot represent that itself, since empty strings are treated as null values. So CS cannot export such a structure directly. ## Accessing the web with JSON</p>
<p>You will learn how to create JSON structures below. The other key to a web-enabled CS is ^jsonopen(). To see how easy it is to talk to the web, just look at this code.</p>
<pre><code># The cleanest way to build json data
$_var = ^jsoncreate(transient object)
$_var.fieldname = $myvar 
An alternative limited to max output buffer size (~80K):
$_var = ^jsonparse(^&quot;{ fieldname: $myvar }&quot;) 
And now to send data upstream:
$_url = ^&quot;http://myHost:myIP&quot;
$_userAgent = ^&quot;User-Agent: %bot, ChatScript”
$_header = ^&quot; ~Accept: application/json 
                  $_userAgent
                  ~Content-Type: application/json &quot;
$_response = ^jsonopen(transient POST $_url $_var $_header $_userAgent) </code></pre>
<h2 id="jsonflags---optional-1st-arg-to-some-json-routines">JSONFLAGS - Optional 1st arg to some JSON routines</h2>
<p>Routines that will create facts for JSON will by default create them as transients (they die at end of volley unless you work to save them). You can override this default by saying <code>permanent</code> or <code>transient</code> or <code>boot</code>. This applies to <code>^jsonopen</code>, <code>^jsonparse</code>, <code>^jsoncreate</code>, <code>^jsonobjectinsert</code>, <code>^jsonarrayinsert</code>, <code>^jsoncopy</code>.</p>
<p>Json arrays all start with the name 'ja-' and json objects all start with the name 'jo-'. The next letter indicates the storage media. 't' means transient and the structure dies at the end of the volley. 'b' means boot and the data moves to the boot later at the end of the volley. Anything else (e.g. an immediate digit) means the structure is permanent and is stored in the user's topic file.</p>
<p>You can also add a flag <code>safe</code> (described later) to <code>^jsonparse</code>, <code>^jsonobjectinsert</code>, <code>^jsonarraydelete</code>. You can also add a flag <code>unique</code> to <code>^jsonarrayinsert</code>. You can also add <code>duplicate</code> to ^jsonobjectinsert.</p>
<p>When multiple flags are desired, put them into a simple string, <code>&quot;DUPLICATE PERMANENT&quot;</code>. Case doesn't matter.</p>
<p>When you want to add a reference to a normal factid (as is returned by ^createfact) you can add the flag <code>AUTODELETE</code></p>
<p>You can also assign user flags by listing <code>USER_FLAG1</code>, through <code>USER_FLAG4</code> as a flag as well. The JSON fact will have that flag on it, which you can use in conjunction with <code>^query</code> to limit what matches can be found.</p>
<h3 id="jsonparse-jsonflags-string"><code>^jsonparse</code>( {JSONFLAGS} string )</h3>
<p><code>string</code> is a JSON text string (as might be returned from a website) and this parses into facts. It returns the name of the root node JSON composite. This name will look like this:</p>
<li><code>ja-0</code> – a json array numbered <code>0</code>.</li>
<li><code>jo-0</code> – a json object numbered <code>0</code>.</li>
<p>Note that the above names are true only for PERMANENT JSON objects. Transient ones will be named like <code>ja-t0</code> and <code>jo-t0</code>.</p>
<p>As new JSON composites are created during a volley, the numbers increase to keep them all distinct. JSON composites are all created as transient facts and will die at the end of the volley unless you do something to explictly keep them (typically <code>^jsongather</code> into some factset and then saving that OR using that to remove all their transient flags OR using <code>^delete()</code> to destroy the facts in the set).</p>
<p>If you are keeping JSON across volleys, you should use the optional Json flags argument to make sure numbering never collides (normally the numbers start back at 0 for each new volley).</p>
<p>Real world JSON has stricter requirements on its format than CS does.</p>
<p>While CS will output strict JSON, you can input slack JSON. You do not need to put commas between elements of an array or object. And you do not need to put quotes around a key name. So the following is legal text form:</p>
<pre><code>[a b {bob: 1 sue: 2}]</code></pre>
<p>Formally JSON has specific primitives named true, false, and null, that systems may care about, but to CS they are nothing special. Numbers in JSON can be integer or have a decimal point and an exponent. CS does not currently support exponent notation, so you can only do: <code>1325</code> or <code>566.23</code>.</p>
<p>JSON originally required a JSON string be a composite of some kind. Now it allows any JSON type to be a JSON document. CS sticks to the original requirement, because passing around a single value as JSON is pretty useless.</p>
<p>Once you have a JSON fact structure, you can go the opposite direction and convert it back into a string or display it visually.</p>
<p>CS accepts extended JSON syntax for parsing into a json fact structure. Any place you have an object value, you can refer to a ChatScript user or match variable and that value will be substituted in. E.g.,</p>
<pre><code>^jsonparse(&quot;{ a: $var, b: _0 }&quot;)</code></pre>
<p>Note you should use a regular quoted string and not a function string like <code>^&quot;{ a: $var, b: _0 }&quot;</code>. If you use a function string, substitution will happen before calling ^jsonparse. Which might be a problem if you had something like this:</p>
<pre><code>^jsonparse(&quot;{ a: $var, b: _0aba }&quot;).</code></pre>
<p>where you wanted the value of <code>b</code> to be <code>&quot;_0aba&quot;</code>. Had you used an active string, the _0 would have been replaced with its contents.</p>
<p>Also, you can use json dereference operators to take apart an existing json structure and use values of it in the current one. If $_y points to a json structure, then</p>
<pre><code>^jsonparse(&quot; opensquiggle a: $var, b: $_y.e[2] closesquiggle &quot;)
(saying squiggles because Latex print complains if I use real ones)</code></pre>
<p>would find a json object reference on <code>$_y</code>, get the e field, and get the 3rd array value found there.</p>
<p>An initial argument of <code>safe</code> will locate the correct end of the data for json parsing, allowing you to pass excess data. This is important for passing in json data in OOB information. OOB information comes in <code>[]</code> and Json itself uses <code>[]</code> at times, so it would be impossible to find the correct end of pattern with normal pattern matching. Instead just do:</p>
<pre><code>u: ( \[ _* ) and call ^jsonparse(safe _0)</code></pre>
<p>This will figure out where the OOB marker actually is and only parse within it. You can add NOFAIL before the string argument, to tell it to return null but not fail if a dereference path fails cannot be found.</p>
<pre><code>^jsonparse(transient NOFAIL &quot;{ a: $var, b: _0.e[2] }&quot;)</code></pre>
<p>Note: You cannot send JSON text as normal input from a user because CS tokenization is set to handle human input. So doing something like this as input will not work:</p>
<pre><code>doLogin {&quot;token&quot;: &quot;myid&quot;, &quot;accounts&quot;: [&quot;whatever&quot;]}</code></pre>
<p>Instead you should call a website using <code>^jsonopen</code> which will automatically convert returned JSON data into internal CS data. Or you can pass JSON data on input via OOB notation:</p>
<pre><code>[ {&quot;token&quot;: &quot;myid&quot;, &quot;accounts&quot;: [&quot;whatever&quot;]}]  User message</code></pre>
<p>and then have a pattern to grab the OOB data and call <code>jsonparse</code> with it (using SAFE as a parameter). OOB input is not subject to human tokenization behavior, spellchecking, etc.</p>
<p>Note: There is a limit to how much JSON you can pass as OOB data nominally, because it is considered a single token. You can bypass this limit by asking the tokenizer to directly process OOB data, returning the JSON structure name instead of all the content. Just enable <code>JSON_DIRECT_FROM_OOB</code> on the <code>$cs_token</code> value and if it finds OOB data that is entirely JSON, it will parse it and return something like <code>jo-t1</code> or <code>ja-t1</code> in its place. Eg. <code>[ { &quot;key&quot;: &quot;value} ]</code> will return tokenized as <code>[jo-t1]</code>.</p>
<pre><code>u: GIVEN_JSON(&lt; \[  _* \] ^JsonKind(&#39;_0)) $$json = &#39;_0 </code></pre>
<p>then you can do $$json.field directly.</p>
<p>Note: <code>^jsonparse</code> autoconverts backslash-unnnn into corresponding the utf8 characters.</p>
<h2 id="jsonformatstring"><code>jsonformat</code>(string)</h2>
<p>Because technically JSON requires you put quotes around field names (though various places ignore that requirement) and because CS doesn't, the function takes in a slack json text string and outputs a strict one.</p>
<h2 id="accessing-json-structures">Accessing JSON structures</h2>
<h2 id="direct-access-via-json-variables-myvar.field-and-myvar">Direct access via JSON variables <code>$myvar.field</code> and `$myvar[]</h2>
<p>If a variable holds a JSON object value, you can directly set and get from fields of that object using dotted notation. This can be a fixed static fieldname you give or a user variable value or a match variable value (quoted or unquoted): <code>$myvar.$myfield</code> is legal, as is <code>$myvar._0</code> or <code>$myvar.'_0</code>.</p>
<p>Dotted notation is cleaner and faster than <code>^jsonpath</code> and <code>jsonobjectinsert</code> and for get, has the advantage that it never fails, it only returns null if it can't find the field. If the path does not contain a json object at a level below the top, one will automatically be created if trying to do assignment, and have the same transient/permanent property as the immediately containing object. If the top level variable is not currently an object, assignment will fail. CS will not create an object for you because it doesn't know if it should be transient or permanent.</p>
<pre><code>$x = $$obj.name.value.data.side 
$$obj.name.value.data.side = 7</code></pre>
<p>Similarly you can access JSON arrays using array notation:</p>
<pre><code>$x = $$array[5]
$x = $$array[$_tmp]
$$obj.name[4] += 3
$$obj.$_kind[4] += 3
$$obj._0[4] += 3
$$obj.&#39;_0[4] += 3</code></pre>
<pre><code>$x.foo[] = Bruce</code></pre>
<p>If foo is currently undefined, the system will create a JSON array for you, with permanency that matches the JSON object of $x. You cannot do $x[] and have this happen because at the top level the system does not know what permanency to use. Once there is a JSON array in $x.foo, assignments with <code>foo[]</code> will add elements to the array. You cannot designate the index, it will be the next index in succession.</p>
<p>You can also do</p>
<pre><code>    $_array1 +=  $_array2 # copies contents of array2 into array1
    $_array1 += value     # adds value into array1</code></pre>
<p>Adding an element to an array using these notations will automatically select DUPLICATE entries being legal. If you want the insertion to be unique, use ^jsonarrayinsert(UNIQUE xxx xxx) OR in your botmacro change the default by doing</p>
<pre><code>   $cs_json_array_defaults = #JSON_ARRAY_UNIQUE </code></pre>
<p>The only restriction on arrays is that you cannot add a new array index value without using ^jsonarrayinsert as you are not allowed to create discontiguous indices.</p>
<p><strong>NOTE</strong> JSON is normally a non-recursive structure with no shared pointers. But ChatScript allows you to store references to JSON structures in multiple places of other JSON structures. This has its hazards. It presents no problem when transcribing to text for a website using <code>^jsonwrite</code> . And when you have something like this:</p>
<pre><code>$x = ^jsoncreate(object)
$y = ^jsoncreate(object)
$x.field = $y
$x.field1 = $y
$x.field = null</code></pre>
<p>Assuming that a JSON structure is not available in multiple places, the assignment of null (or any other value) to a field that already has a JSON structure will normally cause the old value structure to be fully deleted, since it's only reference is removed. And the system does check and delete the structure if it is not referred to by some other JSON field. But there are limits. The system has no idea if you have a pointer to it in a variable. Or if it is part of a pathological indirection sequence like this:</p>
<pre><code>$x = ^jsoncreate(object)
$y = ^jsoncreate(object)
$x.field = $y
$y.field = $x
$x.field = null</code></pre>
<p>The two structures point to each other, each only once. So assigning null will kill off both structures.</p>
<p>Assigning <code>null</code> will remove a JSON key entirely. Assigning <code>&quot;&quot;</code> or <code>^&quot;&quot;</code> will set the field to the JSON literal <code>null</code>.</p>
<h4 id="accessing-opposite-ends-of-json-structures">Accessing opposite ends of JSON structures</h4>
<p>For an array, obviously $array[0] returns one end of the array, and you can get the index of The other end by doing</p>
<pre><code>    $_tmp = ^length($array) - 1
    $array[$_tmp]</code></pre>
<p>But a convenient shortcut is</p>
<pre><code>    $array[-1]</code></pre>
<p>For objects, $object[0] returns the name of the most recent field added and $object[-1] returns the name of the first field added.</p>
<h4 id="using-a-word-as-a-field-name-rather-than-as-indirection">using a $word as a field name rather than as indirection</h4>
<p>Just escape the $.</p>
<pre><code>$data.\$varname = hello </code></pre>
<h3 id="jsonpath-string-id"><code>^jsonpath</code>( string id )</h3>
<li><code>string</code> is a description of how to walk JSON.</li>
<li><code>id</code> is the name of the node you want to start at (typically returned from <code>^jsonopen</code> or <code>^jsonparse</code>).</li>
<p>Array values are accessed using typical array notation like <code>[3]</code> and object fields using dotted notation. A simple path access might look like this: <code>[1].id</code> which means take the root object passed as id, e.g., <code>ja-1</code>, get the 2nd index value (arrays are 0-based in JSON).</p>
<p>That value is expected to be an object, so return the value corresponding to the id field of that object. In more complex situations, the value of id might itself be an object or an array, which you could continue indexing like <code>[1].id.firstname</code>.</p>
<p>You can walk an array by using <code>[$_index]</code> and varying the value of <code>$_index</code>.</p>
<p>When you access an array element, you have to quote the text because it consists of multiple tokens to CS which breaks off <code>[</code> and <code>]</code>. If you are just accesing an object field of something, you can quote the string or just type it directly</p>
<pre><code>^jsonpath(.id $object2)
^jsonpath(&quot;.id&quot; $object2)</code></pre>
<p>Of course you don't always have to start at the root. If you know what you want is going to become object 7 eventually, you could directly say .id given <code>jo-7</code> and the system would locate that and get the id field. Likewise if you know that key names are somehow unique, you could query for them directly using</p>
<pre><code>^query(direct_v ? verbkey ?)</code></pre>
<p>Or even if the key is not unique you can restrict matches to facts having the <code>JSON_OBJECT_FACT</code> flag.</p>
<pre><code>^query(directflag_v ? verbkey ? 1 ? ? ? JSON_OBJECT_FACT)</code></pre>
<p>Be aware that when <code>^jsonpath</code> returns the value of an object key, when the value is a simple word, it just returns the word without doublequotes (since CS just stores information as a single word). But if the value contains whitespace, or JSON special characters, that may mess up if you pass it to <code>^JSONFormat</code>. You can get <code>^jsonpath</code> to return dangerous data as a string with double quotes around it if you add a 3rd argument &quot;safe&quot; to the call.</p>
<pre><code>^jsonpath(&quot;.name&quot; $_jsonobject safe)</code></pre>
<h3 id="jsonpath"><code>^jsonpath</code></h3>
<p>Generally speaking the direct access methods are clearer than using this function.</p>
<p>Can also return the actual factid of the match, instead of the object of the fact. This would allow you to see the index of a found array element, or the json object/array name involved. Or you could use ^revisefact to change the specific value of that fact (not creating a new fact). Just add * after your final path, eg</p>
<pre><code>^jsonpath(.name* $$obj)
^jsonpath(.name[4]* $$obj)</code></pre>
<p>Correspondingly, if you are trying to dump all keys and values of a JSON object, you could do a query like this:</p>
<pre><code>@0 = ^query(direct_s $_jsonobject ? ?)
    _0 = ^first(@0all)
    and then you have _1 and _2 as key and value
}</code></pre>
<p>If you need to handle the full range of legal keys in json, you can use text string notation like this <code>^jsonpath(.&quot;st. helen&quot;.data $tmp)</code>.</p>
<p>You may omit the leading . of a path and CS will by default assume it</p>
<pre><code>^jsonpath(&quot;st. helen&quot;.data $tmp)</code></pre>
<h3 id="jsonloopjsonstruct-var1-var2">^jsonloop($jsonstruct $var1 $var2)</h3>
<p>This is like ^loop, but explicitly for json data. It is faster and cleaner than writing a loop. A normal loop might look like:</p>
<pre><code>    $_count = ^length($_jsonarray)
    ^loop($_count)
        $_count -= 1
        $_value =  $_jsonarray[$_count]
        ...
    }</code></pre>
<p>And similarly for a JSON object one might want:</p>
<pre><code>    $_count = ^length($_jsonobject)
    ^loop($_count)
        $_count -= 1
        $_jsonfield = $_jsonobject[$_count]
        $_value =  $_jsonobject.$_jsonfield
        ...
    }</code></pre>
<p>But for large numbers of elements (eg a thousand +) this is ineffient because it has to search for each element each time. The efficient way is:</p>
<pre><code>    ^jsonloop($_jsonobject $_field $_value)
        ...
    }</code></pre>
<p>where the loop runs most recent entry to least recent, putting the field name in <code>$_field</code> and its value in <code>$_value</code>. Similarly for a jsonarray it does the index as <code>$_field</code> and <code>$_value</code> is its value.</p>
<p>You can choose (or annotate) which way the loop runs by adding an optional 3rd argument OLD or NEW.</p>
<pre><code>^jsonloop($_jsonarray $_index $_value OLD) will walk oldest first (i.e. index 0 and increasing).
^jsonloop($_jsonarray $_index $_value NEW) will walk newest first (i.e. index max and decreasing).</code></pre>
<h3 id="length-jsonid"><code>^length</code>( jsonid )</h3>
<p>Returns the number of top-level members in a json array or object.</p>
<h2 id="printing-json-structures">Printing JSON structures</h2>
<h3 id="jsonwrite-name"><code>^jsonwrite</code>( name )</h3>
<p>name is the name from a json fact set (either by <code>^jsonpart</code>, <code>^jsonopen</code>, or some query into such structures). The result is the corresponding JSON string (as a website might emit), without any linefeeds.</p>
<h3 id="jsontree-name-depth"><code>^jsontree</code>( name {depth} )</h3>
<p>name is the value returned by <code>^jsonparse</code> or <code>^jsonopen</code> or some query into such structures. It displays a tree of elements, one per line, where depth is represented as more deeply indented. Objects are marked with <code>{}</code> as they are in JSON. Arrays are marked with <code>[]</code>.</p>
<p>The internal name of the composite is shown immediately next to its opening punctuation. Optional depth number restricts how deep it displays. 0 (default) means all. 1 means just top level.</p>
<h2 id="json-structure-manipulation">JSON structure manipulation</h2>
<p>You can build up a JSON structure without using <code>^jsonparse</code> if you want to build it piece by piece. And you can edit existing structures.</p>
<h3 id="jsoncreate-jsonflags-type"><code>^jsoncreate</code>( {JSONFLAGS} type )</h3>
<p>Type is either <code>array</code> or <code>object</code> and a json composite with no content is created and its name returned. See <code>^jsonarrayinsert</code>, <code>^jsonobjectinsert</code>, and <code>^jsondelete</code> for how to manipulate it. See writeup earlier about optional json flags.</p>
<h3 id="jsonarrayinsert-jsonflags-arrayname-value"><code>^jsonarrayinsert</code>( {JSONFLAGS} arrayname value )</h3>
<p>Given the name of a json array and a value, it adds the value to the end of the array. See writeup earlier about optional json flags. If you use the flag unique then if value already exists in the array, no duplicate will be added.</p>
<h3 id="jsonarraydelete-index-value-arrayname-value"><code>^jsonarraydelete</code>( [<code>INDEX</code>, <code>VALUE</code>] arrayname value )</h3>
<p>This deletes a single entry from a JSON array. It does not damage the thing deleted, just its member in the array.</p>
<li>If the first argument is <code>INDEX</code>, then value is a number which is the array index (0 ... n-1).</li>
<li>If the first argument is <code>VALUE</code>, then value is the value to find and remove as the object of the json fact.</li>
<p>You can delete every matching <code>VALUE</code> entry by adding the optional argument <code>ALL</code>. Like: <code>^jsonarraydelete(&quot;VALUE ALL&quot; $array 4)</code></p>
<p>If there are numbered elements after this one, then those elements immediately renumber downwards so that the array indexing range is contiguous.</p>
<p>If the key has an existing value then if the value is a json object it will be recursively deleted provided its data is not referenced by some other fact (not by any variables). You can suppress this with the <code>SAFE</code> flag. <code>^jsonarraydelete(SAFE $obj $key)</code>.</p>
<h3 id="jsonarraysize-name"><code>^jsonarraysize</code>( name )</h3>
<p>deprecated in favor of ^length</p>
<h3 id="jsontextfactid">`^jsontext(factid)</h3>
<p>^jsontext adds quotes if object of fact is a json text and FAILS if object is not a json fact.</p>
<p>CS represents json text as ordinary unquoted text, but text that looks like float numbers would be subject to possible float truncations or expansion of digits. This preserves it as text.</p>
<h3 id="jsoncopy-name"><code>^jsoncopy</code>( name )</h3>
<p>Given the name of a json structure, makes a duplicate of it. If it is not the name of a json structure, it merely returns what you pass it.</p>
<h3 id="jsonobjectinsert-jsonflags-objectname-key-value"><code>^jsonobjectinsert</code>( {JSONFLAGS} objectname key value )</h3>
<p>inserts the key value pair into the object named. The key does not require quoting. Inserting a json string as value requires a quoted string. Duplicate keys are ignored unless the optional 1st argument <code>DUPLICATE</code> is given. See writeup earlier about optional json flags.</p>
<p>If the key has an existing value and <code>DUPLICATE</code> is not a factor, then if the value is a json object it will be recursively deleted provided its data is not referenced by some other fact (not by any variables). You can suppress this with the <code>SAFE</code> flag. <code>jsonobjectinsert(SAFE $obj $key null)</code>.</p>
<h3 id="jsondelete-factid"><code>^jsondelete</code>( factid )</h3>
<p>deprecated in favor of ^delete</p>
<h3 id="jsongather-fact-set-jsonid-level"><code>^jsongather</code>( {fact-set} jsonid {level})</h3>
<p>takes the facts involved in the json data (as returned by <code>^jsonparse</code> or <code>^jsonopen</code>) and stores them in the named factset.<br />
This allows you to remove their transient flags or save them in the users permanent data file.</p>
<p>You can omit fact-set as an argument if you are using an assignment statement: <code>@1 = ^jsongather(jsonid)</code></p>
<p><code>^Jsongather</code> normally gathers all levels of the data recursively. You can limit how far down it goes by supplying <code>level</code>. Level 0 is all. Level 1 is the top level of data. Etc.</p>
<h3 id="jsonlabel-label"><code>^jsonlabel</code>( label )</h3>
<p>assigns a text sequence to add to jo- and ja- items created thereafter. See System functions manual.</p>
<h3 id="readfile-tab-filepath-fn-and-readfile-line-filepath-fn"><code>^readfile</code>( TAB filepath {'^fn}) and <code>^readfile</code>( LINE filepath '^fn)</h3>
<p>reads a tsv (tab delimited spreadsheet file) and returns a JSON array representing it. The lines are all objects in an array. The line is an object where non-empty fields are given as field indexes. The first field is 0. Empty fields are skipped over and their number omitted.</p>
<p>If an optional 3rd parameter of a function name is given, the code does not create a JSON structure to return. Instead it calls the function with each field of a line being an argument. This is sort of analogous to <code>:document</code> mode in that you can potentially read large amounts of data in a single volley and may need to use <code>^memorymark</code> and <code>^memoryfree</code> to manage the issue.</p>
<p><code>^readfile ( LINE filepath 'function)</code> reads a file and passes each line untouched as the sole argument to the function.</p>
<p>Formerly called ^jsonreadfile (still accepted).</p>
<h3 id="jsonundecodestring-string"><code>^jsonundecodestring</code>( string )</h3>
<p>removes all json escape markers back to normal for possible printout to a user. This translates \n to newline, \r to carriage return, \t to tab, and \&quot; to a simple quote.</p>
<h2 id="web-json">WEB JSON</h2>
<h3 id="jsonopen-jsonflags-kind-url-postdata-header-timeout"><code>^jsonopen</code>( {JSONFLAGS} kind url postdata header {timeout})</h3>
<p>this function queries a website and returns a JSON datastructure as facts. It uses the standard CURL library, so it's arguments and how to use them are generally defined by CURL documentation and the website you intend to access. See writeup earlier about optional json flags.</p>
<table style="width:81%;">
<col width="18%" />
<col width="62%" />
</colgroup>
<tr class="header">
<th align="center">parameter</th>
<th>description</th>
<tr class="odd">
<td align="center"><code>kind</code></td>
<td>is <code>POST</code>, <code>GET</code>, <code>POSTU</code>, <code>GETU</code>, <code>PUT</code>, <code>DELETE</code> corresponding to the usual meanings of Get and Post and url-encoded forms.</td>
<tr class="even">
<td align="center"><code>url</code></td>
<td>is the url to query</td>
<tr class="odd">
<td align="center"><code>postdata</code></td>
<td>is either &quot;&quot; if this is not a post or is the data to send as post or put</td>
<tr class="even">
<td align="center"><code>header</code></td>
<td>is any needed extra request headers or &quot;&quot;. Multiple header entries must be separated by a tilde</td>
<tr class="odd">
<td align="center"><code>timeout</code></td>
<td>optional seconds limitation for connection and then for transfer. else $cs_jsontimeout rules</td>
<p>Note: 'postdata' can be a simple JSON structure name, in which case the system will automatically perform a ^jsonwrite on it and send that text data as the data. Currently limited to 500K in size of the internal buffer.</p>
<p>A sample call might be:</p>
<pre><code>$$url = &quot;https://api.github.com/users/test/repos&quot;
$$user_agent = ^&quot;User-Agent: Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; WOW64; Trident/6.0)&quot;
^jsonopen(GET $$url &quot;&quot; $$user_agent)</code></pre>
<p>where GitHub requires user-agent data. Use <code>http://</code> and <code>https://</code> in front of your urls.</p>
<p>As an example of a complex header value you might create neatly,</p>
<pre><code>$header = ^&quot;Accept: application/json
~Accept-Encoding: identity,*;q=0
~Accept-Language: en-US,en;q=0.5
~Cache-Control: no-cache
~Connection: close
~Host: Chatscript
~User_Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:42.0) &quot;</code></pre>
<p>ChatScript will make each line have a single space separator between line continuations. And JsonOpen will correctly get the header elements that do not include that spacing.</p>
<p>The results are a collection of facts representing the JSON parse tree and the value of the function is the root JSON value. The JSON elements that can be represented are arrays, objects, JSON strings, and primitives (numbers, true, false, null). JSON arrays are named <code>ja-n</code> where <code>n</code> is a unique index. JSON objects are similarly named <code>jo-n</code>.</p>
<p>Unlike JSON, which makes a distinction between primitives and strings, in ChatScript those things are all strings and are not quoted. So a JSON string like this:</p>
<pre><code>[ {&quot;id&quot;: 1 &quot;value&quot;: &quot;hello&quot;} {&quot;id&quot;: 2 &quot;value&quot;: &quot;bye&quot;} ]</code></pre>
<p>returns this value: <code>ja-1</code> and these facts. The facts have flags on them which you can use in queries. You may not have any need to use these flags, so maybe you will just ignore their existence.</p>
<tr class="header">
<th>fact</th>
<th align="left">associated flags</th>
<tr class="odd">
<td>(ja-1 0 jo-1)</td>
<td align="left">#<code>JSON_ARRAY_FACT</code> #<code>JSON_OBJECT_VALUE</code></td>
<tr class="even">
<td>(jo-1 id 1)</td>
<td align="left">#<code>JSON_OBJECT_FACT</code> #<code>JSON_PRIMITIVE_VALUE</code></td>
<tr class="odd">
<td>(jo-1 value hello)</td>
<td align="left">#<code>JSON_OBJECT_FACT</code> #<code>JSON_STRING_VALUE</code></td>
<tr class="even">
<td>(ja-1 1 jo-2)</td>
<td align="left">#<code>JSON_ARRAY_FACT</code> <code>#JSON_OBJECT_VALUE</code></td>
<tr class="odd">
<td>(jo-2 id 2)</td>
<td align="left">#<code>JSON_OBJECT_FACT</code> #<code>JSON_PRIMITIVE_VALUE</code></td>
<tr class="even">
<td>(jo-2 value bye)</td>
<td align="left">#<code>JSON_OBJECT_FACT</code> #<code>JSON_STRING_VALUE</code></td>
<p>Using queries, you could get all values of an array. Or all fields of an object. Or all JSON facts where the field is the id. You could manually write script to walk the entire tree. But more likely you will use <code>^jsonpath</code> to retrieve specific pieces of data you want. For example, you could do a query on the value returned by <code>^jsonopen</code> as subject, to find out how many array elements there are. Then use <code>^jsonpath</code> to walk each array element to retrieve a specific field buried within.</p>
<p>Note - for things like primitive null, null arrays, null strings, null objects, these are represented as &quot;null&quot; and the corresponding fact flag tells you the kind of value it is.</p>
<p>You can also ask CS to show those out visually using <code>^jsontree</code>.</p>
<p>Note that the facts created are all transient and disappear at the end of the volley unless you have forced them to stay via <code>permanent</code>. Forcing them to stay is generally a bad idea because it will congest your user topic data file, slowing it down or exceeding its capacity, and because those facts may then collide with new facts created by a new <code>^jsonopen</code> on a new volley. The array and object ids are cleared at each volley, so you will be reusing the same names on new unrelated facts.</p>
<p>Using the flag values, it is entirely possible to reconstruct the original JSON from the facts (if the root is an array or object because otherwise there are no facts involved), but I can't think of use cases at present where you might want to. You cannot compile CS on LINUX unless you have installed the CURL library. For Amazon machines that means doing this:</p>
<pre><code>sudo yum -y install libcurl libcurl-devel</code></pre>
<p>On some other machines that doesn't install library stuff and maybe you need</p>
<pre><code>sudo apt-get install libcurl3 libcurl3-gnutls libcurl4-openssl-dev</code></pre>
<p>System variables <code>%httpresponse</code> will hold the most recent http return code from calling <code>^jsonopen</code>.</p>
<p>If you call <code>^jsonopen(direct ...)</code> then the result will not be facts, but the text will be directly shipped back as the answer. Be wary of doing this if the result will be large (&gt;30K?) since you will overflow your buffer without being checked.</p>
<p>^jsonopen automatically url-encodes headers and urls</p>
<p>Note: CS blocks until the call is complete. On a production server if you expect that the call can take serious time, you are advised to send the request in OOB to your own additional layer which returns to CS immediately. Then it makes the call and when the call completes, makes a call back to CS with the results in OOB.</p>
<h4 id="jsonopen-and-local-files"><code>JSONOpen and local files</code></h4>
<p>JSONOpen is a wrapper around the <code>Curl</code> library, so you can use as an url a local file reference like <code>file:///c:/myfile.txt</code>. The file, of course, should be a JSON text string.</p>
<h4 id="jsonopen-and-proxy-servers"><code>JSONOpen and proxy servers</code></h4>
<p>If you need JSONOpen to run thru a proxy server, these are the CS variables you need to set up: <code>$cs_proxycredentials</code> should be your login data, e.g. <code>myname:thesecret</code>. <code>$cs_proxyserver</code> is the server address, e.g., <code>http://local.example.com:1080</code>. <code>$cs_proxymethod</code> are bits listing the authorization method to use. They come from the LIBCURL so you should OR together the bits you want. Bit 1 is the most basic choice of name and password. Read- https://curl.haxx.se/libcurl/c/CURLOPT_HTTPAUTH.html</p>
<h2 id="json-out-of-band-output-data">JSON &amp; Out-of-band output data</h2>
<p>Out-of-band data in ChatScript is signaled by the output beginning with data enclosed in <code>[]</code>. Which might be confusing, since JSON uses <code>[]</code> to denote an array. Standalone ChatScript contains a built-in handler for OOB data and if you pass it JSON at the start of output, it will swallow it and not display it (unless you turn on OOB display).</p>
<p>Similarly, std webpage interfaces connecting to ChatScript do likewise. So if you want to see this information, you should put something in the output at the start which is NOT the JSON data. Anything will do. The only time you might actually need the JSON clean at the beginning is from some special purpose application, and in that case you will write your own OOB handler anyway (or not have one).</p>
<h2 id="json-out-of-band-input-data">JSON &amp; Out-of-band input data</h2>
<p>OOB data into ChatScript is similarly signaled by being at the start of input, with data enclosed in [], followed typically by the user's actual input. The ChatScript engine reacts specially to OOB incoming data in that it will be careful to not treat it like ordinary user chat. Tokenization is done uniquely, spell-checking, pos-tagging, parsing, named entity merging etc are all turned off and the data becomes its own sentence (the user's actual input generates more sentences to CS as input). OOB data is then processed by your script in any way you want. So one clever thing you can do is pass in JSON data within the OOB to get temporary facts into your app during a volley. Input might look like this:</p>
<pre><code>[ [ a b { &quot;bob&quot;: 1, &quot;suzy&quot;: 2 } ] ] What is your name?</code></pre>
<p>You can pattern match the oob section of the input as follows:</p>
<pre><code>u: ( \[ _* ) $_tmp = ^jsonparse(&#39;_0)</code></pre>
<p><code>_0</code> will contain an excess right bracket (the end of the oob message), but that won't bother <code>^jsonparse</code>.</p>
<p>Representing JSON in CS facts is more than just a bunch of subject-verb-object facts linked together.</p>
<p>The facts have typing bits on them that describe the structure and arrays have index values that must remain consistent. Therefore you should not create and alter JSON fact structures using ordinary CS fact routines like <code>^createfact</code> and <code>^delete</code>. Instead use the JSON routines provided.</p>
<h2 id="practical-examples">Practical Examples</h2>
<h3 id="objects">Objects</h3>
<p>The write routines <code>jsonwrite</code> and <code>jsontree</code> print out different views of the same data..</p>
<pre><code>u: (-testcase1) $_jsonObject = ^jsoncreate(object)
    ˆjsonobjectinsert( $_jsonObject name “some name” )
    ˆjsonobjectinsert( $_jsonObject phone “some number” )
    ˆjsonwrite ( $_jsonObject ) \n
    ^jsontree ( $_jsonObject )\n</code></pre>
<p>Note in this next example how to escape a json string with ^''. This makes creating json objects from static data very intuitive and clear.</p>
<pre><code>u: (-testcase2) $_tmp = ^jsonparse( ^&#39;{name: &quot;Todd Kuebler&quot;, phone: &quot;555.1212&quot;}&#39; )
    ^jsonwrite( $_tmp ) \n
    ^jsontree( $_tmp ) \n
    name: $_tmp.name, phone: $_tmp.phone</code></pre>
<p>This example shows the . notation access of data inside a json object in chatscript. This is probably the most intuitive way of interacting with the data.</p>
<pre><code>u: (-testcase3) $_tmp = ^jsoncreate(object)
    $_tmp.name = &quot;Todd Kuebler&quot;
    $_tmp.phone = &quot;555-1212&quot;
    ^jsonwrite( $_tmp ) \n
    ^jsontree( $_tmp ) \n
    name: $_tmp.name, phone: $_tmp.phone 
    </code></pre>
<h3 id="arrays-of-objects">Arrays of objects</h3>
<p>In the example below, we add two items into an array of objects and we display the formatted array:</p>
<pre><code>u: ( testcase4 )
    # create a phoneBook as an array of structured items (objects)
    $_phoneBook = ^jsoncreate(array)
    # add first object in the array
    $_item = ^jsoncreate(object)
    # assign values
    $_item.name = &quot;Todd Kuebler&quot;
    $_item.phone = &quot;555-1212&quot;
    ^jsonarrayinsert($_phoneBook $_item)
    # add a second object in the array
    $_item = ^jsoncreate(object)
    # assign values
    $_item.name = &quot;Giorgio Robino&quot;
    $_item.phone = &quot;111-123456789&quot;
    ^jsonarrayinsert($_phoneBook $_item)    
    # display JSON tree
    ^jsontree( $_phoneBook ) \n
    # print formatted items in the phone book
    phone book:\n
    $_i = 0
    $_size = ^length($_phoneBook)
    loop($_size)
      # print out formatted item
      name: $_phoneBook[$_i].name, phone: $_phoneBook[$_i].phone\n
      $_i += 1
    }</code></pre>
Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FilesExpand file tree

ChatScript-Json.html

Latest commit

History

ChatScript-Json.html

File metadata and controls
