Docker

We’re doing a DevRomagna meetup on container security. It will happen on June 26 at 7pm, it will be in Italian, and the speaker will be the one and the only Ugo Lattanzi. Details and signup here.

Today, I learned the hard way that the default port for ASP.NET Core 8 container images has been updated from port 80 to 8080, quite a remarkable breaking change. We upgraded our web application from .NET 7 and let the CI pipeline do its work. Finally, we checked the application in the browser to ensure everything was okay, but unfortunately, we got a 502 Bad Gateway error. The Nginx logs revealed that the app was rejecting connections, which was unexpected because we didn’t make any changes there. Further investigation showed that the web app listened on port 8080 while Nginx was reverse-proxied to 80. So that was the problem. But why did the port change? ...

Another day, another unexpected problem. Launching a .NET 8 app from a docker container, I got this error: Failed to create CoreCLR, HRESULT: 0x80070008. I was puzzled as the same container ran smoothly in our test environment but not in production. I ruled out resource problems (memory or disk full, maybe?) but then compared the Docker Engine versions we run in test and production. Both were old (20. xx when 25 is available), but interestingly, the production version was older. I searched online and found a two years-old ticket hinting at some problems between .NET 8 and older Docker versions. ...

I added a new tool to my amateurish DevOps toolbox. Developed in the open by Tom Williams, the Docker Event Monitor is a “tiny container that monitors the local Docker event system in real-time and sends notifications to various integrations for event types that match the configuration. For example, you can trigger an alert when a container is stopped, killed, runs out of memory or health status change.” At its core sits a simple python script that monitors the docker.sock file for noticeable changes. The code is straightforward and looks safe to me. It only took a few minutes to set DEM up so that our alerts channel on Slack gets notified of any health status changes. Some handy options are included; my favorite is silence to set a time window during which alerts are not fired. It avoids unnecessary spam when routine maintenance goes off on your stack. ...

We want our test and production stacks to be automatically updated every time something new is pushed to the test or release branch. CI builds the docker image on successful test runs, then stores it in our private registry. But how do you automatically pull and deploy those updated images? I looked into the Watchtower project, which is interesting. You add Watchtower to the stack, and it will diligently check for new versions of the images used by the containers in the stack, pulling, building and deploying as needed while the stack is up and running. In my experiments, however, I had little luck in making it talk with our private registry. Also, I’m not too fond of polluting my stack with foreign containers. I want my docker stack to be simple, tidy, clean, and single-tasked. ...