fix: prevent worker threads from blocking forever on an empty queue - #57

Problem

process_host() guards its loop with host_queue.qsize() and then performs a blocking host_queue.get():

while host_queue.qsize() and not stop_event.wait(0.01):
    try:
        host = host_queue.get()          # blocking - never raises queue.Empty
        ...
        host_queue.task_done()
    except queue.Empty:                  # unreachable as written
        break

With --threads N, several workers can pass the qsize() check while only one item remains. One wins the get(); the others block in it forever.

Every queued item still gets its task_done(), so work_queue.join() in loop_hosts() returns, main() runs all the way through, and the results are written — the scan looks completely successful. But the leaked workers are non-daemon threads, so threading._shutdown() joins them forever at interpreter exit and the process never terminates.

This is easy to miss precisely because the output is complete. The log just ends normally and then hangs:

analyze_hosts version 1.16.0 starting
...
JSON results saved to out/SET-results.json
Output saved to out/SET-log.txt

It is very visible under Docker, where analyze_hosts is PID 1: the container stays Up indefinitely, docker run --rm never returns and so never cleans up, and whatever invoked it waits on it. We accumulated six such containers over a month of nightly scans — one of them had ~150 zombie children — before tracking it down.

Reproduction

This is the loop_hosts/process_host shape reduced to just the queue handling:

import queue, threading, time, random

def process_host(host_queue, stop_event):
    while host_queue.qsize() and not stop_event.wait(0.01):
        try:
            host = host_queue.get()
            time.sleep(random.uniform(0, 0.05))   # "scan" the host
            host_queue.task_done()
        except queue.Empty:
            break

def run(targets, nthreads):
    stop_event = threading.Event()
    work_queue = queue.Queue()
    for t in range(targets):
        work_queue.put(t)
    threads = [threading.Thread(target=process_host, args=(work_queue, stop_event))
               for _ in range(min(nthreads, work_queue.qsize()))]
    for t in threads:
        t.start()
    while work_queue.qsize() and not stop_event.wait(1):
        time.sleep(0.0001)
    work_queue.join()          # returns: all work really is done
    time.sleep(0.3)
    return sum(1 for t in threads if t.is_alive())

for i in range(30):
    stuck = run(targets=25, nthreads=10)
    if stuck:
        print(f"run {i:2d}: join() returned, but {stuck} worker(s) still blocked in get()")

Run it under timeout 120 python3 repro.py.

• Before: 29/30 runs leaked 1–6 threads, and the script itself hung at exit (timeout had to kill it — exit code 124).
• After the one-line change: 0/30 leaked, exit code 0.

The fix

Read the queue with block=False, which makes the existing except queue.Empty: break reachable and lets a worker that loses the race exit cleanly.

The two sibling loops in this same file already do exactly that — remove_from_queue() uses finished_queue.get(block=False) and process_output() uses output_queue.get(block=False). process_host() looks like the one that was missed.

Breaking out is the correct behaviour here: a worker only sees queue.Empty when the queue is genuinely empty, and every target is enqueued before the workers start, so an empty queue means there is no more work.

Unrelated, and happy to open a separate issue rather than bundle it: the current gofwd/analyze_hosts image prints [-] Please install required modules ...: No module named 'pkg_resources' at start-up and silently disables --framework. python-Wappalyzer imports pkg_resources, which ships with setuptools — and since Python 3.12 venv no longer seeds setuptools, while setuptools ≥ 82 has removed pkg_resources outright, so adding setuptools back does not help. Just say the word if that would be useful.