• src/bench/zmodem/README.m

    From Rob Swindell (on Debian L@VERT to Git commit to main/sbbs/m on Thursday, July 23, 2026 23:03:00
    https://gitlab.synchro.net/main/sbbs/-/commit/5c5ad4d427e47dce536082cc
    Added Files:
    src/bench/zmodem/README.md fixA.patch fixB.patch matrix.sh zbench.py zbench_pty.py zbench_sock.py ztx_buf.c
    Log Message:
    bench/zmodem: add the ZMODEM benchmark harness

    Tooling behind docs/zmodem_comparison.md and the (reverted) sexyz
    throughput work, kept for the eventual redesign tracked in GitLab #1195:

    - zbench_sock.py: the main harness (userspace relay with latency,
    bandwidth-asymmetry, and forward bit-corruption injection; one
    bidirectional socket per endpoint so it drives sexyz, lrzsz, and
    Forsberg sz).
    - zbench_pty.py / zbench.py: pty and two-pipe variants.
    - ztx_buf.c: a SyncTERM-model sender linking the real zmodem.o behind a
    buffered send_byte, to separate the zmodem.c cost from the sexyz.c cost.
    - matrix.sh: block-size / CRC / latency / bandwidth / error sweeps.
    - fixA.patch, fixB.patch: the reverted batching prototypes.

    The README captures the design constraint the first attempt missed: a
    correct batched sender must keep servicing the back-channel while sending
    (the reverted prototypes blocked on a whole-subpacket flush and starved
    ZRPOS handling under errors), and it must be gated on multi-run
    error-recovery, not a single pass.

    Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Rob Swindell (on Debian L@VERT to Git commit to main/sbbs/m on Thursday, July 23, 2026 23:03:00
    https://gitlab.synchro.net/main/sbbs/-/commit/1c8c59b14b74fa47a4e82be9
    Modified Files:
    src/bench/zmodem/README.md
    Log Message:
    bench/zmodem: record the confirmed bottleneck and the batching dead-ends

    Update the README with the results of the throughput-redesign attempt:

    - Confirmed the send bottleneck is the producer's per-byte RingBufWrite
    (~11 MB/s vs ~85 for ztx_buf doing the same escape/CRC without the ring);
    consumer-side coalescing of the output thread was tried and did nothing.
    - Three distinct producer-batching prototypes (Fix A, Fix B, and a third
    that flushes on a threshold inside send_byte with error propagation) each
    reached ~66-88 MB/s and each FAILED the 3x error-recovery gate that the
    per-byte sender passes 3/3.
    - Root cause: batching decouples what zmodem believes it has sent from what
    is actually on the wire; ZMODEM error recovery (ZRPOS -> seek -> resend)
    assumes those track, and the buffered-but-unsent bytes are not reconciled
    on a reposition. A correct fix needs an abort/reposition-aware transport
    (a zmodem.c hook to purge the pending output buffer on ZRPOS/abort), which
    is a coordinated zmodem.c + sexyz.c/term.c change, not a transport tweak.

    No source change ships; the robust per-byte sender is retained. GitLab #1195.

    Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Rob Swindell (on Debian L@VERT to Git commit to main/sbbs/m on Thursday, July 23, 2026 23:03:00
    https://gitlab.synchro.net/main/sbbs/-/commit/904ab5e13a1f307d6f517d9a
    Modified Files:
    src/bench/zmodem/README.md
    Log Message:
    bench/zmodem: record the abort-aware-purge attempt and the recommended path

    The abort-aware transport fix was implemented and tested: a purge_output callback in zmodem.c (called from zmodem_handle_zrpos) that clears txbuf, re-inits the ring, and drops the output thread's linear buffer on every reposition, plus a bounded (2 KB) output-thread read. It fires correctly
    and gets the transfer noticeably further under errors, but STILL fails the multi-run error gate: under heavy corruption the block size collapses, retransmit storms fill the ring, the receiver gives up and the sender
    stalls. That is now five distinct batched-transport variants failing while
    the per-byte sender passes 3/3.

    Record the recommended path: re-architect the sexyz *sender* as a single-threaded, protocol-integrated buffered send (as lrzsz does) instead
    of the two-thread ring + output_thread, so the output buffer is owned by
    the protocol thread and flushed/repositioned synchronously with protocol
    state. That is a rewrite, not a patch. No source change ships; the robust per-byte sender is retained. GitLab #1195.

    Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Rob Swindell (on Debian L@VERT to Git commit to main/sbbs/m on Friday, July 24, 2026 11:06:00
    https://gitlab.synchro.net/main/sbbs/-/commit/8beefc9c2679374e69975f23
    Modified Files:
    src/bench/zmodem/README.md fixC-buffered-producer.patch
    Log Message:
    bench/zmodem: window-full flush helps but does not fix windowed sends

    Adds the zmodem.c half of the buffered-producer fix to the parked patch
    and records why the pair still does not ship.

    zmodem.c's window-full path logs, sets backchannel_wait and continues
    straight back to the back-channel wait without a zmodem_flush(). A
    per-byte producer never noticed, because everything it produced was
    already in the ring; a buffered producer is left holding the ZCRCW that
    was supposed to elicit the ACK. Adding the flush there is correct on its
    own terms -- the sender is about to block for a reply -- and it helps:
    -w32768 goes from a roughly one-second stall per window to completing at
    1.36 MB/s, with no cost to streaming, which still measures 115.80 MB/s.

    It is not enough. The shipped sender does -w32768 at 4.74 MB/s, so 1.36
    is still a 3.5x regression, and the obvious next move makes it worse
    rather than better: raising OutbufSize to 64 K so the SO_SNDBUF bound
    exceeds the 32 K window times out entirely. So "bound at least the
    window size" is not the missing rule, and windowed mode needs its own
    diagnosis rather than another guess.

    One thing that looked like a regression is not: -w8192 fails on the
    shipped sender too, rc=-8 with nothing transferred, so that is
    pre-existing and unrelated.

    Both files are therefore back to HEAD and the combined change is saved as fixC-buffered-producer.patch with the measurements in the README. Also
    note for whoever picks this up: 71 MB/s appeared once in a streaming measurement and was load noise from another build on the box -- re-run
    before believing a throughput drop. Refs #1195.

    Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net
  • From Rob Swindell (on Debian L@VERT to Git commit to main/sbbs/m on Friday, July 24, 2026 23:28:00
    https://gitlab.synchro.net/main/sbbs/-/commit/8411fb88763ab83a4568a383
    Modified Files:
    src/bench/zmodem/README.md
    Removed Files:
    src/bench/zmodem/fixC-buffered-producer.patch
    Log Message:
    bench/zmodem: drop the superseded fixC patch

    The buffered streaming send producer it prototyped shipped in sexyz.c
    3.4, gated on streaming mode so it doesn't regress windowed sends -- the
    exact problem that kept fixC parked. Remove the stale patch and point
    the README row at the shipped code.

    Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

    ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net