Excruciatingly slow speed when moving a macOS virtual machine to an external SSD

[yemartin]

Describe the issue
This is a follow-up to the closed issue #6187 (it was closed as "won't fix", but that was in relation to implementing a progress bar. I am reopening for the root issue: the excruciatingly slow speed of the "Move" operation).

I am trying to:

• move a fresh macOS VM (18GB), to an external SSD (Samsung T9)
• both source and destination are using APFS (Encrypted) filesystems

As I saw @adamallo's comment on the previous issue, I thought I would just wait, 7 hours isn't that bad...

But I have now waited 60 (sixty) hours, and the transfer is not even 50% done:

Configuration

• UTM Version: Version 4.4.4 (93)
• macOS Version: 14.3.1
• Mac Chip (Intel, M1, ...): Apple M1 Max

Crash log
The app has not crashed.

Debug log
The problem occurs when the virtual machine is not running.

Upload VM
n/a

---

2024-04-20 Update
The Move operation completed successfully after 1 more day, for a total of around 85 hours.

[osy]

Okay I spent too much time on this issue but the tl;dr is that Apple's implementation of copyfile for sparse files is broken because:

1. It uses the FS block size for copying (4096 bytes) instead of the optimal IO transfer size (usually 1MiB).
2. In each iteration of the loop, it calls lseek to find the next hole in the file which implicitly does a flush and prevents system caching.

Since copyfile is open source and since there doesn't seem to be any other way of doing APFS sparse copying, I went ahead and fixed the code and also wrote a Swift wrapper around it: osy/SwiftCopyfile@faac327

Fixing #1 gives a 4x improvement and fixing #2 gives another 10x improvement for a 40x improvement over Apple's original code. Testing this with a 18GB VM, I was able to match Finder's copy speed (6 minutes) to my USB 3.0 HDD.

[yemartin]

For the record and closure, I must report a 10,000x improvement!

My case may be extreme, maybe because I was using the App Store version, slightly older. But anyway, what took 85 hours now takes only 23 seconds. Thank you so much @osy !

https://youtu.be/oTTBqAEwCuA