Posts

During the Christmas/New Year break, I achieved my goal of updating my website with a new theme. I loved Casper, the previous one I ran for a very long time, but it has not been updated in years and looks abandoned. I wanted new features like fuzzy search, archive and tags pages, cover images, table of contents, title anchors, and more. Also, the old theme kept me anchored to an ancient Hugo version, something I felt uncomfortable with. There are still a few loose parts, but I’m confident I made the right call by switching to PaperMod; it offers all the needed features (and plenty more), and it’s frequently updated, which allows me to open tickets and offer improvements confidently. ...

Bring back self-hosted blogs, reinstall a feed reader, make your feed icon prominent on your blog. Blogs + Atom/RSS is the best decentralized social media system we’ve ever had! And yes I am saying that as co-author of ActivityPub: self hosted blogs is the best decentralized social networking we’ve had. – Christine Lemmer-Webber

Simon Wilson, who’s recently been my go-to person for all AI-related stuff, has an excellent 2023 AI round-up on his website. 2023 was the breakthrough year for Large Language Models (LLMs). I think it’s OK to call these AI—they’re the latest and (currently) most “interesting development in the academic field of Artificial Intelligence that dates back to the 1950s. Here’s my attempt to round up the highlights in one place! The links contained within the post are also valuable. You may know Simon’s website if you are interested in LLMs and AI. If you don’t, I suggest you start following him, preferably via his RSS feed like real hackers do. ...

Yesterday evening, we watched The Vast of the Night, and what a pleasant surprise it was. One night, in a small New Mexico town, a girl who works at a local radio station and an older reporter boy listen to a recording of some strange noises. Through the radio and its listeners, throughout a single night, they uncover a series of sighting stories that, from clue to clue and radio testimony to radio testimony, bring them close to uncovering something big. ...

I read 24 books for a total of 7070 pages in 2023. That’s seven more books than last year, which is quite an outstanding result considering the seemingly unstoppable decline in book reading I have suffered in recent years. Most have been fiction books, and that’s something new and influential with the final result, as I tend to read non-fiction more slowly. The bad news is that I did not review most of the books I read this year, and that sucks. The last review was in August, a catch-up review of several books clearly showing I was in trouble. ...

Yesterday was a beautiful, sunny, cold winter Sunday. I felt like going out and enjoying nature, so I took a solo hike in the Foreste Casentinesi National Park, about an hour’s drive from home. This one marks my very first technology-assisted hiking adventure. It may seem weird for someone who’s been hiking for so long, is a notorious geek and is a professional computer programmer to have never used technology before. A trail map and sometimes a compass were all I was used to, and deliberately. I wanted to avoid technology in this aspect of my life. I welcomed the orientation challenges and superbly looked down at the crowds of phone-smartwatch-compulsive hikers I met on the route. Alas, 2023 was the year I surrendered my motorcycling habits to intercom systems and GPS navigation, and that spoiled me. In a couple of situations yesterday, the app spared me some trouble by warning me of the wrong direction I was going. Nothing major. I would’ve realized the error and backtracked, but being these the shortest and coldest days of the year, I appreciated not risking getting caught by dusk. Also, I found that I can turn on voice-assisted navigation as I do on my motorcycle, and that’s nice (albeit surreal - walking alone in the wilderness, miles from anybody, with a voice coming out of nowhere and whispering when I should take turns on the trail): it avoids looking at the cellphone all the time so I can stay focused on the experience even more, I suspect, than before when the don’t-get-lost alarm bell was constantly ringing in the back of my head. ...

For those who don’t know me, I’m a demographer. I study population. And my first love in fantasy was, of course, Middle Earth. How many people live in Middle Earth? Being a demographer, I was mainly interested in the data side of things. Tolkien is frustratingly vague about population. He almost never gives us estimates of settlement sizes, and many of the larger metropolises of Middle Earth (like Pelargir) never actually appear on the page. Sizable armies make frequent appearances, yet because his adventurers almost exclusively traverse the wilds of Middle Earth, we rarely see where those soldiers are coming from. ...

The Guardian’s The Winterkeeper: A Lifetime Spent Protecting Yellowstone National Park is a beautiful short documentary I truly enjoyed watching. A little research on Steven Fuller, the protagonist, allowed me to dig out some promising reading material.

I always struggle a bit with I’m asked about the “hallucination problem” in LLMs. Because, in some sense, hallucination is all LLMs do. They are dream machines. We direct their dreams with prompts. The prompts start the dream, and based on the LLM’s hazy recollection of its training documents, most of the time the result goes someplace useful. It’s only when the dreams go into deemed factually incorrect territory that we label it a “hallucination”. It looks like a bug, but it’s just the LLM doing what it always does. ...

Andrej Karpathy has a very well-done Intro to Large Language Models video on YouTube. As a founding member and research scientist at OpenAI and with a two-year hiatus working on Tesla Autopilot, Karpathy is an authority in the field. He is also good at explaining hard things. As a Kahneman reader, I appreciated the Thinking Fast and Slow analogy proposed at about half-length in the video: “System 1” (fast automatic thinking, rapid decisions) is where we’re now; “System 2” (rational, slow thinking, complex decisions) is LLMs next goal. Also, I suspect Karpathy’s intriguing idea of LLMs as the center of a new “operating system style” is not too far off from what will emerge soon. The final segment on AI security and known attack vectors (jailbreaking, prompt injection, data poisoning) is also super interesting. ...