Atomic Buffer Swap: Thread-Safe Batching Without Blocking Writers
Holding a lock during network I/O turns your logging layer into a serializer for your application threads. Swap the buffer under the lock; POST outside it.
The problem
A shared in-memory buffer that multiple threads append to, flushed periodically to an HTTP endpoint. The naive implementation holds a lock for the entire flush: acquire, copy, POST, clear, release. This blocks every writer thread for the duration of a network round-trip — which can be hundreds of milliseconds.
The approach
Atomic swap: under the lock, grab the current list and replace it with a new empty one, then release. POST the grabbed list outside the lock.
_LOCK = threading.Lock()
_BUFFER = []
def flush():
with _LOCK:
if not _BUFFER:
return
to_send = _BUFFER
_BUFFER = [] # writers immediately get a fresh list
# lock released — writers unblocked before any network I/O
post_to_api(to_send)The key is that Python name rebinding is atomic at the bytecode level when protected by a Lock. After the with block exits, _BUFFER points to a new empty list and to_send holds a reference to the old one. Writers appending during the POST see the new list; their events go into the next flush batch, not into the one being POSTed.
What I learned
The pattern avoids two failure modes. First, holding the lock during I/O: any writer that tries to append while a slow POST is in flight blocks, which under high concurrency means your logging layer starts serializing your application threads. Second, the naive "copy then clear" without a lock: a writer that appends between the copy and the clear has its event silently dropped from the buffer (it was copied but then the clear removes the reference the copy was made from). The swap makes the handoff a single atomic operation — there is no window where an event can be lost.