Skip to content

Improve CPUAllocator OOM message #20618

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
dzhulgakov wants to merge 1 commit into from
Closed

Improve CPUAllocator OOM message #20618

dzhulgakov wants to merge 1 commit into from

Conversation

@dzhulgakov
Copy link
Collaborator

@dzhulgakov dzhulgakov commented May 17, 2019

Spotted while debugging some problem

Before

>>> torch.empty(10**15)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: [enforce fail at CPUAllocator.cpp:56] posix_memalign(&data, gAlignment, nbytes) == 0. 12 vs 0

After

>>> torch.empty(10**15)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: [enforce fail at CPUAllocator.cpp:65] . DefaultCPUAllocator: can't allocate memory: you tried to allocate 4000000000000000 bytes. Error code 12 (Cannot allocate memory)

@pytorchbot pytorchbot added the module: internals Related to internal abstractions in c10 and ATen label May 17, 2019
facebook-github-bot
facebook-github-bot reviewed May 17, 2019
View reviewed changes
Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dzhulgakov has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

fmassa
fmassa reviewed May 17, 2019
View reviewed changes
c10/core/CPUAllocator.cpp
"DefaultCPUAllocator: not enough memory: you tried to allocate %dGB. Buy new RAM!",
nbytes / 1073741824);
"DefaultCPUAllocator: not enough memory: you tried to allocate ",
nbytes,
Copy link
Member

@fmassa fmassa May 17, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't it be simpler for the user to print the amount in MB or GB?

@colesbury
Copy link
Member

colesbury commented May 17, 2019

But how will users know now that they need to Buy new RAM? R.I.P. Samsung, SK Hynix, Micron

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented May 18, 2019

@dzhulgakov merged this pull request in d9dcfac.

@EagleW EagleW mentioned this pull request Jul 7, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@facebook-github-bot facebook-github-bot facebook-github-bot left review comments

@fmassa fmassa fmassa left review comments

@jerryzh168 jerryzh168 jerryzh168 approved these changes

@smessmer smessmer Awaiting requested review from smessmer

@li-roy li-roy Awaiting requested review from li-roy

+1 more reviewer

@soumith soumith soumith approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

Merged module: internals Related to internal abstractions in c10 and ATen

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@dzhulgakov @colesbury @facebook-github-bot @soumith @jerryzh168 @fmassa @pytorchbot @mruberry