add BlockMask to batch_to_device #1651
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchtune/1651
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 0dd8ec5 with merge base bf93806 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
facebook-github-bot
added
the
CLA Signed
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1651 +/- ##
==========================================
- Coverage 71.04% 69.09% -1.95%
==========================================
Files 296 297 +1
Lines 15112 15120 +8
==========================================
- Hits 10736 10447 -289
- Misses 4376 4673 +297
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
RdoubleA
approved these changes
Sep 22, 2024
| @@ -142,7 +149,10 @@ def batch_to_device(batch: dict, device: torch.device) -> None: | |||
| batch_to_device(v, device) | |||
| elif isinstance(v, torch.Tensor): | |||
| batch[k] = v.to(device) | |||
| elif _SUPPORTS_FLEX_ATTENTION and isinstance(v, BlockMask): | |||
There was a problem hiding this comment.
Could you just combine this case and the above with or?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Context
What is the purpose of this PR? Is it to
batch_to_device only checks dict and tensors. When input is BlockMask (packed=True), it triggers the RaiseError condition.
Changelog
Test plan
After this PR, the code below runs well
tune run full_finetune_single_device --config llama3_1/8B_full_single_device dataset.packed=True tokenizer.max_seq_len=4096If I remove the check for BlockMask, the error message is improved: