Skip to content

fastboot: overlap flash writes with downloads - #3997

Open
JianyuWang0623 wants to merge 3 commits into
apache:masterfrom
JianyuWang0623:fastboot-overlap-flash-writes
Open

JianyuWang0623 wants to merge 3 commits into
apache:masterfrom
JianyuWang0623:fastboot-overlap-flash-writes

Conversation

@JianyuWang0623

Copy link
Copy Markdown
Contributor

Summary

Publish getvar's integer variables by reference, raise the default
download buffer to 64 KiB, and overlap flash writes with the next
download on a worker thread.

Impact

  • max-download-size halves (the download buffer is split in two for
    the overlapped writes), and a flash write failure is now reported on
    the response to the next command rather than the one that triggered
    the write.
  • The default SYSTEM_FASTBOOTD_DOWNLOAD_MAX changes from 40960 to
    65536, adding 24 KiB of download buffer on every board that enables
    CONFIG_SYSTEM_FASTBOOTD, so the halved size (32768) stays above what
    older host fastboot (libsparse 1:10.0.0+r36) needs to split an
    ordinary image.

Testing

Internal measurement on the board this was developed on, flashing a
32 MiB image to eMMC over USB high speed in 131 segments of 256 KiB,
same max-download-size before and after:

Sending Writing Total Throughput
before 1.04 s 1.36 s 2.80 s 11.4 MiB/s
after 1.05 s 0.27 s 1.40 s 22.8 MiB/s

Build Host: Ubuntu 22.04.5 LTS x86_64; xtensa-esp32s3-elf-gcc
(crosstool-NG esp-12.2.0_20230208) 12.2.0; Arm GNU Toolchain 13.2.rel1
aarch64-none-elf-gcc 13.2.1; QEMU 6.2.0 (Debian 1:6.2+dfsg-2ubuntu6.31)

Target: lckfb-szpi-esp32s3:fastboot_usb, with CONFIG_ESP32S3_SPIFLASH=y
to expose /dev/mtdblock0; SYSTEM_FASTBOOTD_DOWNLOAD_MAX left at the
Kconfig default

$ fastboot -s 1234 getvar max-download-size
max-download-size: 32768
Finished. Total time: 0.001s

$ fastboot -s 1234 flash mtdblock0 fb_200k.bin
Invalid sparse file format at header magic
Sending sparse 'mtdblock0' 1/8 (28 KB)             OKAY [  0.033s]
Writing 'mtdblock0'                                OKAY [  0.001s]
...
Sending sparse 'mtdblock0' 8/8 (4 KB)              OKAY [  0.037s]
Writing 'mtdblock0'                                OKAY [  0.372s]
Finished. Total time: 3.057s

A download: request larger than max-download-size is rejected with
FAILData too large, and the device keeps responding afterwards.

Target: qemu-armv8a:fastboot (TCP transport, default defconfig)

$ qemu-system-aarch64 -cpu cortex-a53 -nographic -machine virt,virtualization=on,gic-version=3 -global virtio-mmio.force-legacy=false -netdev user,id=u1,hostfwd=tcp:127.0.0.1:15554-:5554 -device virtio-net-device,netdev=u1,bus=virtio-mmio-bus.0 -kernel ./nuttx

nsh> fastbootd &
nsh> mkrd -m 1 -s 512 400

$ fastboot -s tcp:127.0.0.1:15554 getvar max-download-size
max-download-size: 32768
Finished. Total time: 0.002s

$ fastboot -s tcp:127.0.0.1:15554 flash ram1 fb_200k.bin
Invalid sparse file format at header magic
Sending sparse 'ram1' 1/8 (28 KB)                  OKAY [  0.048s]
Writing 'ram1'                                     OKAY [  0.005s]
...
Sending sparse 'ram1' 8/8 (4 KB)                   OKAY [  0.046s]
Writing 'ram1'                                     OKAY [  0.003s]
Finished. Total time: 0.418s

The whole 204800-byte ramdisk, read back over fastboot after the flash,
matches the source image (same sha256).

Integer variables may change value at run time.  Storing the value by
copy freezes it at startup, so getvar keeps returning the old value
after it changes.  Store a pointer instead and read the current value
in getvar; the existing variables keep their behavior.

Assisted-by: OpenCode:mimo-v2.6-pro
Signed-off-by: wangjianyu3 <wangjianyu3@xiaomi.com>
Run flash writes on a worker with the download buffer split in half,
so a write overlaps the next download and max-download-size halves.
A flash response reports the previous segment's result; "fastboot oem
flash" returns the result of the last write.

Flashing a 32 MiB image to eMMC over USB high speed, in 131 segments
of 256 KiB, with the same max-download-size before and after:

          Sending   Writing   Total    Throughput
  before  1.04 s    1.36 s    2.80 s   11.4 MiB/s
  after   1.05 s    0.27 s    1.40 s   22.8 MiB/s

Assisted-by: OpenCode:mimo-v2.6-pro
Signed-off-by: wangjianyu3 <wangjianyu3@xiaomi.com>
The download buffer is split in half to overlap flash writes with
downloads, so max-download-size is half of
SYSTEM_FASTBOOTD_DOWNLOAD_MAX.  With the default of 40960 that is
20480, and older host fastboot (libsparse 1:10.0.0+r36) fails to split
an ordinary image for a max-download-size below roughly 28 KiB,
reporting "Sparse file is too large or invalid".  Raise the default to
65536 so max-download-size is 32768.

Assisted-by: OpenCode:claude-sonnet-5
Signed-off-by: wangjianyu3 <wangjianyu3@xiaomi.com>
@JianyuWang0623
JianyuWang0623 marked this pull request as ready for review November 9, 2026 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants