Technology & Theory – λ ∧ ¬λ

Batching tasks with xargs

This post has been sitting as a forgotten draft since WC Lisbon, over a year and a half ago. Posting on this site after such a long dry spell feels unavoidably uncomfortable, but at this point any post will have to do.

A reminder that xargs is often all you need when splitting large operations into batches. For example, to delete all pending comments on a WordPress site in batches of five hundred:

wp comment list --status=hold --format=ids \
  | xargs -n500 wp comment delete

Although no solution is perfect, xargs often offers the best tradeoff between ease of implementation, overall speed of operation, and safeguard against service disruption.

xargs is almost always preferable to manual shell looping:

# Worse: looping with `for`
for id in $(wp comment list --status=hold --format=ids)
do
    # Will run once for each comment
    wp comment delete "$id"
done

# Worse: looping with `read`
wp comment list --status=hold --format=ids \
    | while read id
      do
          # Will run once for each comment
          wp comment delete "$id"
      done

Many implementations of xargs support parallel operations with the simple -P switch, if your problem can take advantage of parallelisation:

# - Fetch files from a list,
# - 1 line at a time,
# - up to 5 concurrent curl processes
cat file-urls | xargs -L1 -P5 curl -O

(For more advanced uses, see GNU Parallel.)

Awkward solutions to Advent of Code challenges

In day 6 of the 2019 Advent of Code edition, we are presented with a system of objects orbiting in space. All objects orbit the center of mass of the system (modeled as object COM), either directly or by orbiting a larger object.

COM)B  # B orbits COM
B)C    # C orbits B
C)D    # D orbits C
...

These chains of orbits can be solved as a directed graph. It just so happens that the old Unix toolkit has tools to operate on directed graphs. They aren’t designed for the use that we will make of them, but that’s what makes this fun.

Jobs and pipes

I just watched one of Gary Bernhardt’s excellent screencasts, Tar, Fork and the Tar Pipe. It’s a succinct and powerful overview of key Unix concepts and I highly recommend watching it. If you’re new to Unix, it can be an eye-opening, refreshing look at computing. If you’re seasoned, Bernhardt’s fine execution is still a delight to watch.

While on that topic, I thought I could share some toy code that I’ve had lying around since last year. I will take the time to explain the process, hoping that in the end the result will, it too, seem refreshingly simple.

Superposition & Indetermination

Or: The List Monad

Last year, I introduced monads under a more “intuitive” light—focus was placed on the semantics of monads rather than formal definitions, turning to the Maybe monad as a first contact. The following assumes the reading of said article.

Make the Web

I recently stumbled upon The Lost Art of the Makefile. It was a timely find, in that I too had been circling back to make and the article expresses some of my opinions. It’s a tool that is not only still relevant, but, I argue, more enjoyable and apt for many scenarios than the modern tools that get chosen by default or vice. Continue reading “Make the Web”