Skip to content

[JIMT][SL-566] - calls _onblur when creating p5 obj - #51377

Merged
thomasoniii merged 3 commits into
stagingfrom
jimt/SL-566-onblur-sprites
Apr 18, 2023
Merged

[JIMT][SL-566] - calls _onblur when creating p5 obj#51377
thomasoniii merged 3 commits into
stagingfrom
jimt/SL-566-onblur-sprites

Conversation

@thomasoniii

@thomasoniii thomasoniii commented Apr 17, 2023

Copy link
Copy Markdown
Contributor

Summary

This is a proposed fix, since it'll resolve it, but I'm hoping people more knowledgeable with the system will take a look at it and comment as to whether it's viable or if it points to a deeper issue. It feels like a bandaid.

If you're in sprite lab/p5 lab in the cdc mapping landmarks module, at least, you can make the character fly off the screen.

Repro is just to be holding down an arrow key when the timer resets. When you re-start the app, the character will fly off the edge in whatever direction you had previously pressed. However, if you re-press the same key, then motion will stop.

While investigating, I found additional ways to repro - the simplest one is to put a breakpoint in p5.prototype._onkeydown, and run with the debugger. You don't need to do anything, just let the debugger pause and immediately resume and the issue will reproduce.

The root cause is that inside of p5, it's storing a global object called downKeys keeping track of whatever keys are currently pressed. This is completely decoupled from the p5 object itself.

When a user presses a key, it fires off _onkeydown and sticks the key into that global object. BUT - if the key is still being held down when execution stops, the p5 object is promptly destroyed. That means that when the user finally lets go of the key, the p5 object no longer exists and cannot fire the _onkeyup action that clears out the global object. Hence it sticks around in that global object and re-appears later.

Similar things happen in the debugger - if you hop out via this, you're naturally going to let go of the key. So you resume execution, the _onkeyup never fires to clear it, and your character keeps running forever.

So. How to clear it? Fortunately, p5 provides an _onblur event, which clears out the downKeys object. The comments claim it fires when the user is no longer focused on the p5 element, but that's not applicable here. Regardless, we can sneak in and call it right after our p5 object is created to explicitly wipe out any pre-existing downKeys.

Also - please note that this fix will not fix the debugger case, and I don't think that one's fixable with the way p5 is has this implemented. That's a trap where you can hop out of its expected control flow so the keyup doesn't fire (until the user presses a key again), but presumably most of our users won't be adding a breakpoint there so I think we're good.

And that's it.

Alternatives could include:

  • the original fix I had did the _onblur when the p5 object was destroyed, but I opted to move it to the create case so it resets at the beginning - just in case there are more destroy cases I don't know about.
  • I don't think we can directly clear out this object elsewhere, nor have I found another method to try in p5. But this may point to something else we may want to revisit in drawing code on our side? Again, I'm not familiar and am just speculating.
  • Maybe we should upgrade p5? The newest versions of it seem to have moved the downKeys global into a _downKeys attribute, so are presumably unaffected. That'd also allow multiple canvases on screen at once.
  • other?

Links

Testing story

Deployment strategy

Follow-up work

Privacy

Security

Caching

PR Checklist:

  • Tests provide adequate coverage
  • Privacy and Security impacts have been assessed
  • Code is well-commented
  • New features are translatable or updates will not break translations
  • Relevant documentation has been added or updated
  • User impact is well-understood and desirable
  • Pull Request is labeled appropriately
  • Follow-up work items (including potential tech debt) are tracked and linked

@thomasoniii thomasoniii changed the title jimt/SL-566 - calls _onblur when nuking p5 obj jimt/SL-566 - calls _onblur when creating p5 obj Apr 17, 2023
@thomasoniii
thomasoniii marked this pull request as ready for review April 17, 2023 21:40
@thomasoniii
thomasoniii requested a review from a team April 18, 2023 11:46

@mikeharv mikeharv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change looks very safe and fixes the bug that is impacting students. I looked into _onblur and found it's very simple:

/**
 * The onblur function is called when the user is no longer focused
 * on the p5 element. Because the keyup events will not fire if the user is
 * not focused on the element we must assume all keys currently down have
 * been released.
 */
p5.prototype._onblur = function (e) {
  downKeys = {};
};

It seems very reasonable that we'd want to assume all keys currently down have been released when a program starts, so this feels like a good fit. One could maybe argue that the name of the function isn't a perfect fit for our use case, but considering that a rename would require updating both repos and doing a version bump, that seems out of scope.

I recommend we move forward with this change.

@mikeharv
mikeharv requested a review from a team April 18, 2023 12:53

@bencodeorg bencodeorg left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉 Nice work Jim! Makes me a little nervous to use undocumented function every time we execute (ie, could this regress if we bump p5 versions?), but I guess it would be pretty obvious if it did, and we do use undocumented p5 functions elsewhere, so not a first. I think it's worth adding a comment as to why this is here before merging.

@bencodeorg

Copy link
Copy Markdown
Contributor

Two questions for my understanding:

  • how did you find this _onBlur function? Just grepping for downKeys in the p5 library?
  • how did you trace this to p5, rather than p5.play? We were totally down the rabbit hole that this problem was in p5.play 🤦

@thomasoniii

Copy link
Copy Markdown
Contributor Author

🎉 Nice work Jim! Makes me a little nervous to use undocumented function every time we execute (ie, could this regress if we bump p5 versions?), but I guess it would be pretty obvious if it did, and we do use undocumented p5 functions elsewhere, so not a first. I think it's worth adding a comment as to why this is here before merging.

The current version of p5 still has an equivalent function (though it does something slightly different since the downKeys global has become a _downKeys instance variable), so it wouldn't break. But at the same it would probably no longer be necessary since just destroying the object should destroy the list of pressed keys.

I also added a comment just explaining what it does, and of course the git blame will point back to this PR with the additional info.

@thomasoniii

Copy link
Copy Markdown
Contributor Author

Two questions for my understanding:

  • how did you find this _onBlur function? Just grepping for downKeys in the p5 library?

Pretty much. I was first looking at the _onkeydown function, and I spotted the downKeys object in there. Then I realized that was a separate global separated from the object, so I poked around to see if we'd get lucky and any pre-existing functions emptied that list. Otherwise we would've needed to add in an equivalent function to p5.

  • how did you trace this to p5, rather than p5.play? We were totally down the rabbit hole that this problem was in p5.play 🤦

I'd added a few breakpoints to try and narrow down where the actual keydown event handler was and walked back up the stack that way. The _onkeydown function looked like it was probably the actual event handler (and confirmed by looking at it loop over the list of events). From there, it was just lucky that I saw that the error also reproduced on a breakpoint in that function, so I inspected it further and realized it was the separate global.

@thomasoniii thomasoniii changed the title jimt/SL-566 - calls _onblur when creating p5 obj [JIMT][SL-566] - calls _onblur when creating p5 obj Apr 18, 2023
@thomasoniii
thomasoniii merged commit 54787e2 into staging Apr 18, 2023
@thomasoniii
thomasoniii deleted the jimt/SL-566-onblur-sprites branch April 18, 2023 18:47
new window.p5(
function (p5obj) {
this.p5 = p5obj;
this.p5._onblur(); // This is to explicitly wipe out the downKeys object over in p5.js so no old keys hang around.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this comment should go above the function call, to match the styling in the rest of the file (and is our general practice, too).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants