Paul's Programming Notes

apt-get install --no-install-recommends

I reach for --no-install-recommends in a lot of Dockerfiles now. Debian and Ubuntu packages have hard dependencies plus a set of “recommended” packages, and apt-get install pulls in the recommends by default. That is fine on a desktop, but it is bloat you rarely want in an image.

RUN apt-get update && apt-get install -y --no-install-recommends \
    curl \
 && rm -rf /var/lib/apt/lists/*

apt install takes the same flag, but stick with apt-get in a Dockerfile. In a script apt prints WARNING: apt does not have a stable CLI interface. Use with caution in scripts., and its man page recommends apt-get for scripts because it keeps backward compatibility.

Ubuntu measured a 60% image size reduction from this flag on their own images.

To make it the default for every install in an image instead of repeating the flag, set it in an apt config:

RUN echo 'APT::Install-Recommends "false";' > /etc/apt/apt.conf.d/99norecommends

The one gotcha: once in a while a recommended package is actually one you need, and something breaks in a non-obvious way without it. When that happens, install it explicitly by name alongside the rest rather than dropping the flag.

WLED

Today I learned about the amazing WLED project: https://github.com/Aircoookie/WLED

All you need to do is flash their binary to an ESP32 and it will bring up a WIFI access point that allows you to control the LEDs through an app or web page.

A wooden sign outlined with a red LED strip, wired to an ESP32 board and a power adapter on a desk

Microservice Communication

I’ve been learning more about async communication between microservices and came across this article that talks about some of the trade-offs between sync and async communication: https://dzone.com/articles/patterns-for-microservices-sync-vs-async

Summary

Sync

Pros

  • “synchronous calls are simpler to grasp, debug and implement”

Cons

  • “A temporary burst at one component can flood other services with requests.”
  • Risk of Cascading Failures (domino effect of failures if one service experiences an otuage)
  • tighter coupling (requires endpoint versioning in the owning service)

Async

Pros

  • “removes the need to wait for a response thereby decoupling the execution of two or more services”
  • “deals better with sporadic bursts of traffic”

Cons

  • “Asynchronous systems tend to be significantly more complex than synchronous ones.”
  • “consumers need to adapt to work with an asynchronous system”
  • “the message bus the Achilles heel of the system as it remains a central point of failure”
  • Eventual Consistency (potentially out of date data)

Python - Mypy

My new job has me working on a larger codebase than previous jobs, and it’s also my first time using mypy.

I’m starting to understand why Guido’s work on mypy had a lot to do with Dropbox’s Python 3 migration. Dropbox wrote an article with details on how they used mypy on their “4 million lines of Python”. With that much code, I can understand why they needed to treat code like “cattle, not pets”.

Python 3 disallows some comparisons with None types, makes huge changes to strings/bytes, and has many other changes involving types. When your codebase is massive, you have to reach for automated tooling to consistently find and prevent those bugs. With the same tooling you can also prevent entire categories of other bugs from reaching production. The type hints can also be helpful documentation. For those reasons and more, I think implementing type hints on a large codebase like Dropbox’s will definitely be worth it in the long run.

It has me wondering if I should have been taking the extra time to use type hints and checking (or a statically typed language) this whole time. Was going without type hints one less distraction? Or will be the price be paid in maintenance difficulties and bug fixes later?

At this point, not much of the Python ecosystem has type hints (not even Python 2 compatible comment-style type hints). I’m starting to think it would be a good use of time to work on changing that.

Ruby - Jekyll

I migrated this blog from blogger/blogspot to a static site generated with Jekyll and hosted with Netlify.

This blog post describes the process: http://joshualande.com/jekyll-github-pages-poole

Initially, I started using Python’s Pelican, but pelican-import (pelican’s tool for migrating from blogger) doesn’t work as well as jekyll-import. It turned comments into posts and threw exceptions while processing draft posts without content. Also, the first docs that show up on google for Pelican aren’t the latest docs. This causing issues when I followed the old docs that said to use Python 2.7, but Pelican only supports Python 3.6+ now.

Arduino - Oven Thermometer

Parts:

  • HiLetgo DC 3-5V MAX6675 Module + K Type Thermocouple Temperature Sensor
  • UCTRONICS 0.96 Inch OLED Module 12864 128x64 Yellow Blue SSD1306 Driver I2C
  • ESP32

Code: