Skip to content

Conversation

@chemelnucfin
Copy link
Contributor

@chemelnucfin chemelnucfin commented Jan 23, 2018

Fixes #3152.

@lukesneeringer This was your fix, so I believe you should be able to get me an LGTM soon right? :)

@dhermes I would like to ask that you are ok with it since I mostly referred to your test_system code.

@tseaver Do you mind looking it over just so I didn't miss anything?

I consolidated all of our responses and comments and made this pull request.

I backed out the code change and the errors happen.

Separately, I have also investigated that

array_values{
  values{
  }
}

would raise ValueError when put into an entity. It might be desired behavior, but we might need to deal with it at some point. The issue is that len(array_value.values) is 1 and the result for this was not [] as I had expected in the following code:

        result = [_get_value_from_value_pb(value)
                  for value in value_pb.array_value.values]

Anyway, shall we get this merged soon please?

@chemelnucfin chemelnucfin added type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. api: datastore Issues related to the Datastore API. labels Jan 23, 2018
@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Jan 23, 2018
array_pb = array_val_pb.array_value.values
array_val_pb.array_value.CopyFrom(entity_pb2.ArrayValue(values=[]))

unindexed_value_pb1 = array_pb.add()

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

@chemelnucfin
Copy link
Contributor Author

chemelnucfin commented Jan 24, 2018

This is what entity_pb looks like after line 150 in test_helpers.py after my first review change

key {
  partition_id {
    project_id: "PROJECT"
  }
  path {
    kind: "KIND"
    id: 1234
  }
}
properties {
  key: "baz"
  value {
    array_value {
    }
  }
}

@chemelnucfin
Copy link
Contributor Author

chemelnucfin commented Jan 26, 2018

@lukesneeringer @tseaver Are we concerned about something else breaking if we merge this in?

From what I understand of the code, the bug was only a problem if value has [] in it, which is fine.
But the set was checking if whether all of the values had the same exclusion, it was comparing it to length 1, which missed the empty array which is always True.

The other solution would be to check to see if len(set) < 1 instead of != 1 and then make sure we catch the pop KeyError, but that's similar to just checking to see if the array has something in it.

It seems that the only case that the merged code would be different is that value has an [] in it, in which something else would occur. However, since the code only appends to exclude_from_indexes. Every other case of array_value would remain the same, and an empty array would have no excluded indexes to worry about.

Should we do a

try:
  merge
except:
  fix

here or is there still too much risk concern?

@chemelnucfin
Copy link
Contributor Author

chemelnucfin commented Jan 26, 2018

Or maybe Luke's version was better, since maybe the empty list may get excluded also? I originally thought we only needed to check the list, but on second thought, the array_value itself may have been excluded.

@chemelnucfin chemelnucfin force-pushed the datastore-issue-3152 branch 2 times, most recently from 2916603 to a0fe60d Compare January 26, 2018 18:02
@chemelnucfin
Copy link
Contributor Author

Changed to Luke's version

@chemelnucfin
Copy link
Contributor Author

I'll try again later.

@tseaver
Copy link
Contributor

tseaver commented Feb 22, 2018

Superseded by #4915.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: datastore Issues related to the Datastore API. cla: yes This human has signed the Contributor License Agreement. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ValueError: For an array_value, subvalues must either all be indexed or all excluded from indexes.

5 participants