Skip to content

Added support for asynchronous publishing on workbook and datasource endpoints#311

Merged
gaoang2148 merged 6 commits intomasterfrom
async-publishing
Jul 5, 2018
Merged

Added support for asynchronous publishing on workbook and datasource endpoints#311
gaoang2148 merged 6 commits intomasterfrom
async-publishing

Conversation

@gaoang2148
Copy link
Copy Markdown
Contributor

No description provided.

@graysonarts
Copy link
Copy Markdown
Contributor

The tests and pycodestyle checks aren't passing. Make sure you run them in both python2 and python3

Copy link
Copy Markdown
Contributor

@graysonarts graysonarts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few small things

parser.add_argument('--filepath', '-f', required=True, help='filepath to the workbook to publish')
parser.add_argument('--logging-level', '-l', choices=['debug', 'info', 'error'], default='error',
help='desired logging level (set to error by default)')
parser.add_argument('--as_job', '-a', help='Publishing asynchronously', action='store_true')
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--as-job would be more in line with the rest of the file

new_workbook = TSC.WorkbookItem(default_project.id)
new_workbook = server.workbooks.publish(new_workbook, args.filepath, overwrite_true)
print("Workbook published. ID: {0}".format(new_workbook.id))
if args.as_job is True:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just if args.as_job:. You don't need to compare it to True explicitly

if mode == self.parent_srv.PublishMode.Overwrite or mode == self.parent_srv.PublishMode.Append:
url += '&{0}=true'.format(mode.lower())

if as_job is True:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

again, no need for is True

logger.info('Published {0} (ID: {1})'.format(filename, new_datasource.id))
return new_datasource

if as_job is True:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

another one :)


if as_job is True:
new_job = JobItem.from_response(server_response.content, self.parent_srv.namespace)[0]
logger.info('Published {0} (JOB_ID: {1}'.format(filename, new_job.id))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe Published async...

@api(version="2.0")
def publish(self, workbook_item, file_path, mode, connection_credentials=None):
@parameter_added_in(as_job='3.0')
def publish(self, workbook_item, file_path, mode, as_job=False, connection_credentials=None):
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For backwards compatibility, you should put as_job as the last argument because if someone has written the code as publish(workbook_item, file_path, 'append', creds), this would break them. You need to do the same thing for datasource.

error = 'Workbooks cannot be appended.'
raise ValueError(error)

if as_job is True:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove is True

new_workbook = WorkbookItem.from_response(server_response.content, self.parent_srv.namespace)[0]
logger.info('Published {0} (ID: {1})'.format(filename, new_workbook.id))
return new_workbook
if as_job is True:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove is True

t8y8
t8y8 previously approved these changes Jul 3, 2018
Copy link
Copy Markdown
Collaborator

@t8y8 t8y8 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔧 then 🚀

I'd love to see a follow-on that actually does job progress tracking, then we can reuse it for AD-Sync groups; but even if not there's a few nits and there's still a test failure in the CI

logger.info('Published {0} (ID: {1})'.format(filename, new_datasource.id))
return new_datasource

if as_job:
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can come as a second PR but it'd be great if we actually tracked the progress of the job, or at least optionally tracked the progress of the job, and printed or reported it somehow.

response_xml = f.read().decode('utf-8')
with requests_mock.mock() as m:
m.post(self.baseurl, text=response_xml)
new_workbook = TSC.WorkbookItem(name='Sample', show_tabs=False,
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Clean this up so it's all shiny-like.

new_workbook = TSC.WorkbookItem(name='Sample', 
                                show_tabs=False,
                                project_id='ee8c6e70-43b6-11e6-af4f-f7b0d8e20760')

sample_workbok = os.path.join(TEST_ASSET_DIR, 'SampleWB.twbx')
publish_mode = self.server.PublishMode.CreateNew

new_job = self.server.workbooks.publish(new_workbook, 
                                        sample_workbook,
                                        publish_mode, 
                                        as_job=True)

Copy link
Copy Markdown
Contributor

@graysonarts graysonarts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@gaoang2148 gaoang2148 merged commit 5e88a41 into master Jul 5, 2018
@gaoang2148 gaoang2148 deleted the async-publishing branch July 5, 2018 16:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants