refactor/naive bayes with tensors by sudo-ben · Pull Request #142 · machinelearnjs/machinelearnjs

sudo-ben · 2018-11-25T02:55:11Z

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
Rewrite of GaussianNB with Tensorflow support.

The PR also introduces:

Iterator implementation for predict
Tensorflow Linear Algebra refactoring
Updating the doc-processor to support native types such as IterableIteration

JasonShin · 2018-11-25T09:38:28Z

Hey, could you help me fix this issue?

I cloned your repository and ran yarn to install npm modules then yarn test, but I am getting below error

src/lib/naive_bayes/multinomial.ts:84:14 - error TS2349: Cannot invoke an expression whose type lacks a call signature. Type '(<U>(callbackfn: (value: ReadonlyArray<number>, index: number, array: ReadonlyArray<ReadonlyArray...' has no compatible call signatures.

84       return X.map((x):T => this.singlePredict(x));
                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


src/lib/naive_bayes/multinomial.ts:167:30 - error TS2349: Cannot invoke an expression whose type lacks a call signature. Type '{ (callbackfn: (previousValue: number, currentValue: number, currentIndex: number, array: Readonl...' has no compatible call signatures.

167       productReducedRow.push(rowFrequencyCount.reduce((s, c) => s + c, 0));
                                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

^^^ is non issue, I realised I was looking at your fork but you are working on a branch bayes_tensors

JasonShin · 2018-12-01T03:18:52Z

Ok, I resolved the error from doc processor

Using TypeScript 2.7.2 from /home/jason/Desktop/kalimdorjs/node_modules/typedoc/node_modules/typescript/lib
To generate markdown please set option --theme markdown

JSON written to /home/jason/Desktop/kalimdorjs/docs/docs.json

/home/jason/Desktop/kalimdorjs/docs/processor/index.js:274
            if (foundRef.kindString === consts.refKindInterface) {
                         ^

TypeError: Cannot read property 'kindString' of null
    at /home/jason/Desktop/kalimdorjs/docs/processor/index.js:274:26
    at arrayReduce (/home/jason/Desktop/kalimdorjs/node_modules/lodash/lodash.js:683:21)
    at Function.reduce (/home/jason/Desktop/kalimdorjs/node_modules/lodash/lodash.js:9683:14)

You were getting this error due to introducing a TS Native Interface IterableIterator. So far in Kalimdor, we've only been using custom defined Interfaces such as StrNumDict or InterfaceFitModel. The difference between custom and native interface is that the former gives you an ID as a reference, which you can use it to find its definition, on the other hand, the latter does not expose an ID. Here are examples:

Custom Interface

{
					"id": 1130,
					"name": "InterfaceFitModel",
					"kind": 256,
					"kindString": "Interface",
					"flags": {},
					"typeParameter": [
						{
							"id": 1131,
							"name": "T",
							"kind": 131072,
							"kindString": "Type parameter",
							"flags": {}
						}
					],
					"children": [
						{
							"id": 1132

^^ has an ID that we can use to reference its definition within docs.json. However, a native interface such as IterableIteration does not have an ID because it's a native type.

Anyways, thanks @BenjaminMcDonald for helping us improving the documentation processor =) !!

sudo-ben · 2018-12-03T05:33:22Z

Update the branch with added support for constraint in doc types added multinomial naive bayes. This with your above patch fixed the issue with the document generation. yarn test now completes

JasonShin · 2018-12-03T11:44:10Z

I am going to write some TODOs for myself here

TODO:
(/) Removing excessive comments
(/) Add exception tests for MultinomialNB
(?) Add real data tests using Iris for MultinomialNB
(/) modelState can be just class fields
(/) export MultinomialNB in index.ts
(/) Add integration tests
(/) refactor isMatrix usage to tensor_ops.ts APIs
(/) Running yarn doc prints

[ { type: 'reference',
    name: 'IterableIterator',
    typeArguments: [ [Object] ] } ]
[ { type: 'typeParameter',
    name: 'T',
    constraint: { type: 'union', types: [Array] } } ]
[ { type: 'reference',
    name: 'IterableIterator',
    typeArguments: [ [Object] ] } ]

to the screen. Remove the console logs.
(/) Running the MultinomialNB unit tests prints


  console.log node_modules/@tensorflow/tfjs-core/dist/ops/array_ops.js:500
    Tensor
        [[-0.8873032, -0.5306282],
         [-0.6931472, -0.6931472],
         [-0.4700036, -0.9808292]]

  console.log node_modules/@tensorflow/tfjs-core/dist/ops/array_ops.js:500
    Tensor
        [-1.0986123, -1.0986123, -1.0986123]

on the screen

(?) Some methods parameters in the docstrings are missing either description or a default value
(/) Release note

1. Removing duplicate interface types. InterfaceFitModel and InterfaceFitModelAsArray can be consolidated 2. removing modelState, we can just save them as class fields 3. Using Type1DMatrix and Type2DMatrix instead of ReadOnlyArray as type 4. Refactring toJSON and fromJSON accordingly 5. Refactoring fitModel function, mostly typings 6. using tf's sum reduction to get product reduced row

1. toJSON and fromJSON now leverages reshape method 2. fixing TF warnings with typing

JasonShin · 2018-12-09T02:31:56Z

I propose we exclude


  /**
   * @param  {IterableIterator<IterableIterator<number>>} X
   * @returns IterableIterator
   */
  public *predictIterator(
    X: IterableIterator<IterableIterator<number>>
  ): IterableIterator<T> {
    for (const x of X) {
      yield this.singlePredict([...x]);
    }
  }

from this PR. I think it's best to introduce this feature in a separate PR that implements it globally throughout the existing models.

…nferencing

…n utilities

JasonShin

Alll good now

JasonShin · 2018-12-01T02:11:17Z

src/lib/naive_bayes/gaussian.ts

-  public predict(X: Type2DMatrix<number> = []): number[] {
-    validateMatrix2D(X);
-    let clonedX = X;
+  public model(): InterfaceFitModel<T> {


Do we need this when we already have toJSON?

JasonShin · 2018-12-01T02:16:18Z

src/lib/naive_bayes/gaussian.ts

 import { IMlModel, Type1DMatrix, Type2DMatrix } from '../types';
+import math from '../utils/MathExtra';
+
+const { isMatrix } = math.contrib;


We should start using ops/tensor_ops.ts for the Tensor / Matrix validations. It relies on TF.js for validation =)

JasonShin · 2018-12-01T02:20:14Z

src/lib/naive_bayes/gaussian.ts


-    if (this.clone) {
-      clonedX = cloneDeep(X);
+  public *predictIterator(


I am going to do some research on Iterators. I think it's a powerful feature from TS that we haven't been using so far and we should think about supporting it globally in v2 release.

JasonShin · 2018-12-09T11:01:44Z

I think this PR Is fine to be merged now!! GREAT WORK and thanks for your second contributions @BenjaminMcDonald !!!

I will address the documentation issue that I describe in #161 in a separate branch.

naive bayes with tensors

6f842e9

JasonShin changed the title ~~naive bayes with tensors~~ refactor/naive bayes with tensors Nov 26, 2018

Merge remote-tracking branch 'origin/v2' into bayes-tensors

12b2d4a

JasonShin and others added 3 commits December 1, 2018 14:21

fix: doc processor updated to support native TS types

1aa7fa7

Added support for constraint in doc types

5069987

Added multinomial naive bayes machine learning algorithm

e94b478

JasonShin self-assigned this Dec 3, 2018

JasonShin added the feature New feature that does not exist in Kalimdor.js yet label Dec 3, 2018

JasonShin added 8 commits December 7, 2018 18:15

chore: refactor Tensor to tf.Tensor

a15e7d4

chore: adding as tfc.Tensor

1806cc3

refactor: updating tfc. to tf.

79489ca

chore: removing excessive comments

d4be97a

wip: refactoring types

0c54700

Merge remote-tracking branch 'origin/v2' into bayes-tensors

5d6d162

refactor: simplifying typings

1410990

1. toJSON and fromJSON now leverages reshape method 2. fixing TF warnings with typing

JasonShin added 6 commits December 9, 2018 13:42

refactor: simplifying typecheck and validation for predict

9520761

refactor: removing unnecessary try catches

fb72a40

refactor: removing type when using tf.tensor1d, we can rely on type i…

90edece

…nferencing

chore: simplifying validations in predict and fit using the validatio…

37044eb

…n utilities

chore: adding exception tests

0d22ba3

refactor: simplifying fit validations

4e430c4

JasonShin self-requested a review December 9, 2018 03:54

JasonShin added the enhancement New feature or request label Dec 9, 2018

chore: simplifying GaussianNB validation and typings

571340f

JasonShin added 3 commits December 9, 2018 15:10

chore: exposing multinomial through index.ts

933a81d

chore: adding MultinomialNB into require tests

35409ce

chore: simplifying types and updating docstrings

8b51713

JasonShin mentioned this pull request Dec 9, 2018

docs/ The generic T should display its true types on the documentation site #161

Closed

JasonShin added 3 commits December 9, 2018 15:39

refactor: exception tests and improving doc strings

a59ee9d

chore: updating changelog

c117851

chore: implementing alpha value

b133889

JasonShin reviewed Dec 9, 2018

View reviewed changes

JasonShin merged commit 1a578de into v2 Dec 9, 2018

JasonShin deleted the bayes-tensors branch December 10, 2018 09:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor/naive bayes with tensors#142

refactor/naive bayes with tensors#142
JasonShin merged 26 commits intov2from
bayes-tensors

sudo-ben commented Nov 25, 2018 •

edited by JasonShin

Loading

Uh oh!

JasonShin commented Nov 25, 2018 •

edited

Loading

Uh oh!

JasonShin commented Dec 1, 2018

Uh oh!

sudo-ben commented Dec 3, 2018

Uh oh!

JasonShin commented Dec 3, 2018 •

edited

Loading

Uh oh!

JasonShin commented Dec 9, 2018

Uh oh!

JasonShin left a comment

Uh oh!

JasonShin Dec 1, 2018

Uh oh!

JasonShin Dec 1, 2018

Uh oh!

JasonShin Dec 1, 2018

Uh oh!

JasonShin commented Dec 9, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

sudo-ben commented Nov 25, 2018 • edited by JasonShin Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JasonShin commented Nov 25, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JasonShin commented Dec 1, 2018

Uh oh!

sudo-ben commented Dec 3, 2018

Uh oh!

JasonShin commented Dec 3, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JasonShin commented Dec 9, 2018

Uh oh!

JasonShin left a comment

Choose a reason for hiding this comment

Uh oh!

JasonShin Dec 1, 2018

Choose a reason for hiding this comment

Uh oh!

JasonShin Dec 1, 2018

Choose a reason for hiding this comment

Uh oh!

JasonShin Dec 1, 2018

Choose a reason for hiding this comment

Uh oh!

JasonShin commented Dec 9, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

sudo-ben commented Nov 25, 2018 •

edited by JasonShin

Loading

JasonShin commented Nov 25, 2018 •

edited

Loading

JasonShin commented Dec 3, 2018 •

edited

Loading