Many checks without self-inflicted outages

Step 01 of 06

Learn the concept

Go makes it cheap to start goroutines, which is how monitors accidentally attack themselves. Ten thousand targets with no limit means ten thousand chances to hold sockets, DNS work and memory at once. Concurrency needs a speedometer and a brake.

The target list can be long because the worker count is not. Results flow back to one collector, so workers do not fight over a shared output slice or interleave JSON writes.

Step 01

The ideas this is made of

Cheap goroutines still spend real resources

A goroutine starts with a small stack and the Go scheduler parks it while network I/O waits. That is perfect for probes. It does not make DNS queries, sockets, TLS handshakes or target capacity infinite. A design that starts one goroutine per target has a worst case equal to the configuration size. Production code states the limit.

Channels transfer ownership instead of sharing memory

A jobs channel feeds targets to workers. A results channel returns complete Result values to a collector. Workers do not append to the same slice, so there is no shared write to race. Closing the jobs channel says no more work is coming; closing results says collection is complete. That signal is as important as the values.

A worker pool is an operations contract

With workerCount = 3, at most three checks hold sockets and pressure targets, no matter how many URLs are queued. Increase the number and you change capacity deliberately. That makes file descriptors, ephemeral ports and downstream load discussable. go func per target is easy. A bounded pool is explainable during an incident.

WaitGroup accounting must happen before launch

Call wg.Add(1) before starting each worker, then defer wg.Done() at the top of the goroutine. Calling Add inside the goroutine races with Wait; forgetting Done leaves the collector waiting forever after an early return. The pattern is small because the bug it prevents is boring and vicious.

Cancellation must spread as far as concurrency did

When the parent context is cancelled, workers should stop taking jobs and in-flight checks should return promptly. That requires selecting on ctx.Done() in orchestration and passing the same context into DNS, TCP, TLS and HTTP. Otherwise the main goroutine exits politely while the sockets keep living their private lives.

A small worker pool without shared writes

package main

import (
	"fmt"
	"sync"
)

func main() {
	jobs := make(chan int)
	results := make(chan int)

	var wg sync.WaitGroup
	for worker := 0; worker < 3; worker++ {
		wg.Add(1)
		go func() {
			defer wg.Done()
			for job := range jobs {
				results <- job * job
			}
		}()
	}

	go func() {
		for _, job := range []int{1, 2, 3, 4, 5} {
			jobs <- job
		}
		close(jobs)
		wg.Wait()
		close(results)
	}()

	for result := range results {
		fmt.Println(result)
	}
}

Workers receive jobs and send values; the collector is the only reader of results. A goroutine waits for every worker before closing the results channel, which lets the final range loop end cleanly.

Unbounded fan-out versus a pool

Design	Maximum in flight	Failure mode
One goroutine per target	Number of targets	FD and port exhaustion
Bounded worker pool	Worker count	Queue waits instead
Shared result slice	Unclear ownership	Data race risk
Result channel	Single collector	Serialised output

What these are called on the job

Goroutine — A lightweight concurrent execution unit managed by Go.
Channel — A typed conduit for values and close signals.
Worker pool — A fixed set of goroutines consuming a queue.
Data race — Concurrent unsynchronised access where at least one goroutine writes.
Backpressure — Intentional slowing when workers or queues are full.