Posts

Simon Wilson, who’s recently been my go-to person for all AI-related stuff, has an excellent 2023 AI round-up on his website. 2023 was the breakthrough year for Large Language Models (LLMs). I think it’s OK to call these AI—they’re the latest and (currently) most “interesting development in the academic field of Artificial Intelligence that dates back to the 1950s. Here’s my attempt to round up the highlights in one place! The links contained within the post are also valuable. You may know Simon’s website if you are interested in LLMs and AI. If you don’t, I suggest you start following him, preferably via his RSS feed like real hackers do. ...

Yesterday evening, we watched The Vast of the Night, and what a pleasant surprise it was. One night, in a small New Mexico town, a girl who works at a local radio station and an older reporter boy listen to a recording of some strange noises. Through the radio and its listeners, throughout a single night, they uncover a series of sighting stories that, from clue to clue and radio testimony to radio testimony, bring them close to uncovering something big. ...

I read 24 books for a total of 7070 pages in 2023. That’s seven more books than last year, which is quite an outstanding result considering the seemingly unstoppable decline in book reading I have suffered in recent years. Most have been fiction books, and that’s something new and influential with the final result, as I tend to read non-fiction more slowly. The bad news is that I did not review most of the books I read this year, and that sucks. The last review was in August, a catch-up review of several books clearly showing I was in trouble. ...

Yesterday was a beautiful, sunny, cold winter Sunday. I felt like going out and enjoying nature, so I took a solo hike in the Foreste Casentinesi National Park, about an hour’s drive from home. This one marks my very first technology-assisted hiking adventure. It may seem weird for someone who’s been hiking for so long, is a notorious geek and is a professional computer programmer to have never used technology before. A trail map and sometimes a compass were all I was used to, and deliberately. I wanted to avoid technology in this aspect of my life. I welcomed the orientation challenges and superbly looked down at the crowds of phone-smartwatch-compulsive hikers I met on the route. Alas, 2023 was the year I surrendered my motorcycling habits to intercom systems and GPS navigation, and that spoiled me. In a couple of situations yesterday, the app spared me some trouble by warning me of the wrong direction I was going. Nothing major. I would’ve realized the error and backtracked, but being these the shortest and coldest days of the year, I appreciated not risking getting caught by dusk. Also, I found that I can turn on voice-assisted navigation as I do on my motorcycle, and that’s nice (albeit surreal - walking alone in the wilderness, miles from anybody, with a voice coming out of nowhere and whispering when I should take turns on the trail): it avoids looking at the cellphone all the time so I can stay focused on the experience even more, I suspect, than before when the don’t-get-lost alarm bell was constantly ringing in the back of my head. ...

For those who don’t know me, I’m a demographer. I study population. And my first love in fantasy was, of course, Middle Earth. How many people live in Middle Earth? Being a demographer, I was mainly interested in the data side of things. Tolkien is frustratingly vague about population. He almost never gives us estimates of settlement sizes, and many of the larger metropolises of Middle Earth (like Pelargir) never actually appear on the page. Sizable armies make frequent appearances, yet because his adventurers almost exclusively traverse the wilds of Middle Earth, we rarely see where those soldiers are coming from. ...

The Guardian’s The Winterkeeper: A Lifetime Spent Protecting Yellowstone National Park is a beautiful short documentary I truly enjoyed watching. A little research on Steven Fuller, the protagonist, allowed me to dig out some promising reading material.

I always struggle a bit with I’m asked about the “hallucination problem” in LLMs. Because, in some sense, hallucination is all LLMs do. They are dream machines. We direct their dreams with prompts. The prompts start the dream, and based on the LLM’s hazy recollection of its training documents, most of the time the result goes someplace useful. It’s only when the dreams go into deemed factually incorrect territory that we label it a “hallucination”. It looks like a bug, but it’s just the LLM doing what it always does. ...

Andrej Karpathy has a very well-done Intro to Large Language Models video on YouTube. As a founding member and research scientist at OpenAI and with a two-year hiatus working on Tesla Autopilot, Karpathy is an authority in the field. He is also good at explaining hard things. As a Kahneman reader, I appreciated the Thinking Fast and Slow analogy proposed at about half-length in the video: “System 1” (fast automatic thinking, rapid decisions) is where we’re now; “System 2” (rational, slow thinking, complex decisions) is LLMs next goal. Also, I suspect Karpathy’s intriguing idea of LLMs as the center of a new “operating system style” is not too far off from what will emerge soon. The final segment on AI security and known attack vectors (jailbreaking, prompt injection, data poisoning) is also super interesting. ...

In a few days, on Monday, December 4, at 10 a.m., I will present at the .NET Conference Italia 2023, which will be held at the Microsoft House in Milan. My session is titled “C#12 What’s new and interesting” and is part of a rich agenda focused on the recent release of .NET 8. I presented on the same subject at a DevRomagna meetup a few months ago when C# 12 was in preview and still had several moving parts. Now that the final release is out, it will be interesting to return to the topic to consolidate my knowledge. I suspect that of the many reasons I challenge myself to speak publicly, enriching my knowledge is prevalent. As the adage goes, for learning a subject, nothing beats explaining it in front of a knowledgeable audience. ...

Today, I (re)learned how to serialize an XML to a UTF-8 string. Like all the other times I did this, I got backstabbed by StringWriter, which only supports UTF-16. A simple code snippet like this: await using var sw = new StringWriter(); await using var w = XmlWriter.Create(sw, new() { Async = true }); ... await w.FlushAsync(); return sw.ToString(); Will emit this output: <?xml version="1.0" encoding="utf-16"?><... There’s nothing inherently wrong with UTF-16, but XML is usually UTF-8, so one must do something about it. StringWriter exposes an Encoding property, but it is read-only for unknown reasons. One might think that given that the XmlWriter allows setting its own Encoding value, something like this would work: ...