Add docs about prod ecosystem features #23010

dzhulgakov · 2019-07-18T05:02:00Z

Covering fleet-wide profiling, api logging, etc.

It's my first time writing rst, so suggestions are definitely welcomed.

…ging, etc

soumith · 2019-07-18T06:49:06Z

docs/source/notes/production_ecosystem.rst

@@ -0,0 +1,131 @@
+Integration in production ecosystem


ecosystem -> environment

Integration in production environment
-> Features for large-scale deployments of PyTorch

soumith · 2019-07-18T06:49:48Z

docs/source/notes/production_ecosystem.rst

+It doesn't cover topics of deploying models to production. Check
+:mod:`torch.jit` or one of the corresponding tutorials.
+
+Assumption below is that you either build PyTorch from source in your


"Assumption below is" -> "The note asssumes"

soumith · 2019-07-18T06:50:07Z

docs/source/notes/production_ecosystem.rst

+Assumption below is that you either build PyTorch from source in your
+organization or have an ability to statically link additional code to be loaded
+when PyTorch is used. Therefore, many of the hooks are exposed as C++ APIs that
+can be triggered once in some centralized place, e.g. in static initialization


"in some" -> "in a"

soumith · 2019-07-18T06:50:39Z

docs/source/notes/production_ecosystem.rst

+
+PyTorch comes with :mod:`torch.autograd.profiler` capable of measuring time
+taken by individual operators on demand. One can use the same mechanism to do
+always on measurements for any process running PyTorch. It might be useful for


always on -> "Always ON"

soumith · 2019-07-18T06:51:01Z

docs/source/notes/production_ecosystem.rst

+PyTorch comes with :mod:`torch.autograd.profiler` capable of measuring time
+taken by individual operators on demand. One can use the same mechanism to do
+always on measurements for any process running PyTorch. It might be useful for
+gathering information about PyTorch workload running in a given process or


workload -> workloads

soumith · 2019-07-18T06:52:20Z

docs/source/notes/production_ecosystem.rst

+global sampling rate specified by
+`torch::autograd::profiler::setSamplingProbability`.
+
+Potential example might look like:


Potential example might look like
-> Here's an example

soumith · 2019-07-18T06:53:13Z

docs/source/notes/production_ecosystem.rst

+often convenient to bundle additional information together with the model, for
+example, description of model producer or auxiliary artifacts.
+
+It can be achieved by passing ``_extra_files`` argument to


by passing -> by passing the

soumith · 2019-07-18T08:36:45Z

docs/source/notes/production_ecosystem.rst

@@ -0,0 +1,131 @@
+Integration in production ecosystem
+===================================
+


Add a table of contents here

There's already a table of contents rendered on the right, but sure

soumith · 2019-07-18T08:37:47Z

docs/source/notes/production_ecosystem.rst

+    });
+
+
+API usage logging


move this section above "Attaching metadata to saved TorchScript models" as it flows much better after the operator profiling section

ilia-cher

Checked the part on operator callbacks - LGTM, please also add that pushCallback and setSamplingProbability are not thread safe and should be called during initialization only and should never be called while any operator is running.

ilia-cher · 2019-07-19T21:25:44Z

docs/source/notes/large_scale_deployments.rst

+
+    void onFunctionEnter(const RecordFunction& fn) {
+        std::cerr << "Before function " << fn.name() 
+                  << " with " << fn.inputs().size() << "inputs" << std::endl;


nit: space before inputs

ilia-cher · 2019-07-19T21:27:02Z

docs/source/notes/large_scale_deployments.rst

+
+When running in a broader ecosystem, for example in managed job scheduler, it's
+often useful to track which binaries invoke particular PyTorch APIs. There
+exists simple intrumentation injected at several important API points that


typo: instrumentation

facebook-github-bot

@dzhulgakov is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

facebook-github-bot · 2019-07-24T22:35:08Z

@dzhulgakov merged this pull request in d6dcec3.

apaszke · 2019-07-25T03:07:46Z

docs/source/notes/large_scale_deployments.rst

+often convenient to bundle additional information together with the model, for
+example, description of model producer or auxiliary artifacts.
+
+It can be achieved by passing the ``_extra_files`` argument to


Well we should never document arguments which have an underscore prefix... It's there to signify it's private. This has been put as a temporary measure after a discussion I had with @zdevito, but we never intended this functionality to be publicized and to have a lot of people depend on it...

Sorry, I missed this comment. I think passing metadata through PT files might be useful for higher level workflows built on top of PT. How do you feel about dropping underscore all together and making it official functionality? I don't see too many drawbacks as the interface name->blob is pretty generic and our container is a zip file already.

Of course, one can argue that additional files can be appended with just ZipFile python API. While it's ok for reading, there might be unintended effects for writing as ZipFile doesn't guarantee data block alignment (something our writer does). For reading, suggesting to use ZipFile should be sufficient.

I would not be super happy about it, but it would at least resolve this issue 😕 I guess it's ok...

Add docs about prod ecosystem features: fleet-wide profiling, api log…

6121545

…ging, etc

dzhulgakov requested review from brianjo, ilia-cher and soumith July 18, 2019 05:02

pytorchbot added the module: docs Related to our documentation, both in docs/ and docblocks label Jul 18, 2019

soumith suggested changes Jul 18, 2019

View reviewed changes

soumith's comments

fc67a50

ilia-cher reviewed Jul 19, 2019

View reviewed changes

Ilia's comments

81bcecf

ilia-cher approved these changes Jul 22, 2019

View reviewed changes

facebook-github-bot reviewed Jul 24, 2019

View reviewed changes

facebook-github-bot closed this in d6dcec3 Jul 24, 2019

facebook-github-bot added the merged label Jul 24, 2019

apaszke reviewed Jul 25, 2019

View reviewed changes

mruberry added the Merged label Oct 28, 2020

		@@ -0,0 +1,131 @@
		Integration in production ecosystem
		===================================

Add docs about prod ecosystem features #23010

Add docs about prod ecosystem features #23010

Uh oh!

Conversation

dzhulgakov commented Jul 18, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ilia-cher left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot left a comment

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Jul 24, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants