Repository navigation
Backward compatibility for structured properties values has been broken from version 1.8.0 #669
Description
Structured property values are not stored as expected in 2 latest versions.
Models Example:
class FileOutputItem(ndb.Expando):
status = ndb.StringProperty()
url = ndb.StringProperty()
secure = ndb.BooleanProperty(default=False)
class FileOutput(ndb.Expando):
items = ndb.StructuredProperty(FileOutputItem, repeated=True)
class File(ndb.Expando):
_default_indexed = False
date_updated = ndb.DateTimeProperty() # updating on create and explicit update
date_created = ndb.DateTimeProperty(auto_now_add=True)
output = ndb.StructuredProperty(FileOutput)
Description:
When creating File model we are passing property "output" with such value
'output': {
'items': [
{
'status': 'INPROGRESS',
'secure': False,
'url': 'video/19fc1a_e61e07c90b2b43be9d5e3cda89c5653f/144p/mp4/file.mp4',
'fps': '30/1',
'quality': '144p',
'format': 'mp4',
'duration': 21000,
'type': 'video',
'kind': None
},
{
'status': 'INPROGRESS',
'secure': False,
'url': 'video/19fc1a_e61e07c90b2b43be9d5e3cda89c5653f/storyboard/144p/mp4/file.mp4',
'fps': '30/1',
'quality': '144p',
'kind': 'video',
'format': 'mp4',
'duration': 21000,
'type': 'storyboard'
},
{
'status': 'READY',
'secure': False,
'url': 'media/19fc1a_e61e07c90b2b43be9d5e3cda89c5653ff000.jpg',
'format': 'jpg',
'type': 'image',
'kind': None,
'duration': None,
'quality': None,
'fps': None,
},
{
'status': 'READY',
'secure': False,
'url': 'media/19fc1a_e61e07c90b2b43be9d5e3cda89c5653ff001.jpg',
'format': 'jpg',
'type': 'image',
'kind': None,
'duration': None,
'quality': None,
'fps': None,
},
{
'status': 'READY',
'secure': False,
'url': 'media/19fc1a_e61e07c90b2b43be9d5e3cda89c5653ff002.jpg',
'format': 'jpg',
'type': 'image',
'kind': None,
'duration': None,
'quality': None,
'fps': None,
},
{
'status': 'READY',
'secure': False,
'url': 'media/19fc1a_e61e07c90b2b43be9d5e3cda89c5653ff003.jpg',
'format': 'jpg',
'type': 'image',
'kind': None,
'duration': None,
'quality': None,
'fps': None,
}
]
}
In the older ndb versions (include v1.7.3), structured properties stored in datastore as:
output.items.kind
[null,"video",null,null,null,null]
output.items.quality
["144p","144p",null,null,null,null]
output.items.duration
["21000","21000",null,null,null,null]
output.items.format
["mp4","mp4","jpg","jpg","jpg","jpg"]
output.items.fps
["30/1","30/1",null,null,null,null]
output.items.type
["video","storyboard","image","image","image","image"]
In the last 2 versions v1.8.0 and v1.9.0 all null values are missing and properties stored in datastore as
output.items.kind
["video"]
output.items.quality
["144p","144p"]
output.items.duration
["21000","21000"]
output.items.format
["mp4","mp4","jpg","jpg","jpg","jpg"]
output.items.fps
["30/1","30/1"]
output.items.type
["video","storyboard","image","image","image","image"]
There is a problem with model loading in GO runtime (values are missing)
Activity
chrisrossi commentedon Jun 22, 2021
I can see that the interaction of the legacy data format and Expando is not ideal. Since we allow people to do this, I probably do need to dig in and figure out how to make it work properly.
In the meantime, if you can either disuse Expando, or use the non-legacy data format (by passing
legacy_data=FalsetoClient.context()), I think either one of those should fix your immediate problem.fix: fix bug with repeated structured properties with Expando values
fix: fix bug with repeated structured properties with Expando values (#…