What’s unsettling is that the whole argument seems to make a whole lot of sense.

We are on a clear path towards a world of division. One should think that if a [LLM] model is too dangerous for everyone, then it is too dangerous for Americans too. Instead the US is treating these models like weapons that need to be controlled. It is not just about capabilities, it is about racism and nationalism. If you have the wrong passport, you are not to be trusted. This is a very different thing from safety, and Europeans should pay close attention to it.

I think the future probably belongs to the small companies // individuals more than sprawling conglomerates. There is a huge opportunity for people who use AI to remove everything that isn’t judgment from their workload, and apply that judgement to good products and good services. My theory is that one person, with an AI-augmented operational layer plus taste is the company of the future - and I’d bet on that again and again.

A bold statement for sure. Like many others, I’ve been experiencing this AI acceleration firsthand — more tasks, faster pace, less time to think. It can feel overwhelming. But Westenberg’s framing resonates with me: the goal isn’t to do everything faster, it’s to consciously offload the repetitive and the mundane, and reclaim that space for the work that actually matters. I run a tiny company myself, so I can only hope Joan is right.

Senior engineering practitioners from major technology companies gathered for a multi-day retreat to confront the questions that matter most as AI transforms software development. The discussions covered more than twenty topics across breakout sessions, but the most significant insights didn’t emerge from one single session. Instead, they surfaced at various intersections; we found that the same concerns kept appearing in different conversations, framed by different people solving different problems.

This publication synthesizes those cross-cutting themes, organized around the patterns that senior leaders need to understand and act on now. The retreat did not produce a single, unified vision of the future, but instead produced something more useful: a map of the fault lines where current practices are breaking and new ones are forming.

– The Future of Software Engineering

Great read. It also taught me about the Chatham House Rule.

Two things stood out: the PR is in draft, and it includes an AI disclosure:

the fix and the test were created by Claude. I don’t have mongo available or all the necessary python versions for testing, so I’m making this a draft PR so that I can verify all the tests/checks pass before marking it as ready to review.

I’m not at all surprised that Claude was used for the fix (I use it daily myself, to great benefit), but I appreciate the disclosure. A careful review of the code wouldn’t have revealed AI involvement.

The idea of submitting a draft to let the CI run and intervene where needed before marking it as ready for review is also smart — especially since, as the author explains, they can’t run the tests locally.

All in all, an exemplary case of AI use in open source, and welcome news at a time when maintainers are under pressure, often flooded with auto-generated junk PRs (in my own small corner, it has happened to me too). As always, it’s not the tool that’s the problem — it’s who uses it, and how.

Someday years from now we will look back on the era when we were the last generation to code by hand. We’ll laugh and explain to our grandkids how silly it was that we typed out JavaScript syntax with our fingers. But secretly we’ll miss it.

We’ll miss the feeling of holding code in our hands and molding it like clay in the caress of a master sculptor. We’ll miss the sleepless wrangling of some odd bug that eventually relents to the debugger at 2 AM. We’ll miss creating something we feel proud of, something true and right and good. We’ll miss the satisfaction of the artist’s signature at the bottom of the oil painting, the GitHub repo saying “I made this.”

– Nolan Lawson, We mourn our craft

I’ve never felt this much behind as a programmer. The profession is being dramatically refactored as the bits contributed by the programmer are increasingly sparse and between. I have a sense that I could be 10X more powerful if I just properly string together what has become available over the last ~year and a failure to claim the boost feels decidedly like skill issue. There’s a new programmable layer of abstraction to master (in addition to the usual layers below) involving agents, subagents, their prompts, contexts, memory, modes, permissions, tools, plugins, skills, hooks, MCP, LSP, slash commands, workflows, IDE integrations, and a need to build an all-encompassing mental model for strengths and pitfalls of fundamentally stochastic, fallible, unintelligible and changing entities suddenly intermingled with what used to be good old fashioned engineering. Clearly some powerful alien tool was handed around except it comes with no manual and everyone has to figure out how to hold it and operate it, while the resulting magnitude 9 earthquake is rocking the profession. Roll up your sleeves to not fall behind.

– Andrej Karpathy

As AI generated content becomes the norm, I believe that human-generated content and raw thoughts and emotion will become more valuable. In many ways, I’d rather read a raw, unproduced, and scrappy blog post with grammatical mistakes over perfectly generated content that serves little value.

– Armeet Singh Jatyani, Importance of Writing

AI can replace most of programming, but programming isn’t the job. Programming is a task. It’s one of many things you do as part of your work. But if you’re a software engineer, your actual job is more than typing code into an editor. The mistake people make is conflating the task with the role.

It’s like saying calculators replaced accountants. Calculators automated arithmetic, but arithmetic was never the job. The job was understanding financials, advising clients, making judgment calls, etc. The calculator just made accountants faster at the mechanical part. AI is doing something similar for us.

AI Can Write Your Code. It Can’t Do Your Job

The fine guys at ASP Italia just published the video in case someone is interested.

Yeah, it is in Italian. I got a transcript from MacWhisper and then asked Claude to translate and clean it up. It did a pretty good job, so let me know if there’s any interest in an English transcript; I might post it here.

Speaking of English, I’ve started submitting to international events again, without any luck so far. The conference landscape has changed quite a bit, especially since the COVID break. I still get invited to local ones, while internationals have been elusive. I suspect my change in technology stack plays a role. It was relatively easy to speak abroad when I was into Python. With .NET, not so much. But again, I was into Python in the pre-COVID era, so it is hard to tell which factors are at play and what their weight is.

I’m trying to figure out how to use generative AI as a designer without feeling like shit. I am fascinated with what it can do, impressed and repulsed by what it makes, and distrustful of its owners. I am deeply ambivalent about it all. The believers demand devotion, the critics demand abstinence, and to see AI as just another technology is to be a heretic twice over. Today, I’d like to try to open things up a bit.

I’m surprised by how brilliantly Rick Rubin, Brian Eno, Hayao Miyazaki, and Miyazaki’s Spirited Away are packed into this talk.

We had long used an internal API by our accounting software, which was also utilized by thousands of our end users. We decided to make our experience and expertise available to external developers, allowing them to integrate electronic invoicing into their applications quickly via a public API. Thus, the Invoicetronic project was born.

We had made a similar choice in the past with the open-source release of FatturaElettronica.NET, also a library initially created for internal use. Eve and Cerberus for Python started as an internal projects too.

We didn’t limit ourselves to just opening our API to the public. That alone would have been challenging (as any veteran well knows, opening a private product to a broader and more diverse audience is already a real challenge in itself). We seized the opportunity, years after its creation and with millions of invoices having passed through the system in the meantime, to refresh the design, improve performance, and add important features that proved stimulating to implement.

I certainly wasn’t lacking experience with REST APIs, but I still learned a lot, especially while working on the tools accessory to the actual API. An example is the client SDKs. We support all the most common programming languages: JavaScript/TypeScript, Python, PHP, Java, C#, Ruby, and Go. They’re built using OpenAPI Generator, a tool I initially underestimated but which turned out to be excellent, thanks also to the remarkable ecosystem built by the community. Working on the SDKs also provided me with the opportunity to improve my Continuous Integration skills. Beyond the usual test-containerization-staging-deploy cycle, every API update is followed by an automatic release of all SDKs, each with deployment on both GitHub and the reference package manager for the language: PyPI, NuGet, npm, etc. Seemingly obvious tasks (such as inferring the new API version number from CI and then reusing it in the package version of individual SDKs) required considerable brainstorming.

The Invoicetronic project, which gave me the most satisfaction and taught me the most, is invoice-cli. This open source command-line tool enables sending and receiving electronic invoices from the terminal, and is available for Linux, Mac, and Windows. You don’t need to be a developer or study the API in depth to use it, and, not secondarily, it can be conveniently inserted into a script. A trivial usage example would be:

# send a single invoice.
$ invoice send file1.xml

# send multiple invoices, then delete the local copies.
$ invoice send *.xml --delete

# individually send multiple files.
$ invoice send file1.xml file2.xml file3.xml

# receive all unread invoices.
$ invoice receive --unread

From my point of view, though, the interesting thing about invoice is that it’s written in Go, a language I hadn’t yet had the chance to use professionally. A veteran would likely find fault with the implementation details and the chosen approach, but I hope they’ll be merciful, considering I’m a beginner in this space. We could have used C#, which is now our reference language. Still, in the case of the command-line, Go seemed like a better choice, first of all, for the exceptional support in creating native multi-platform binaries, then for performance both cold and hot, and for deployment simplicity. If I had to make another command-line tool, I would return to Go rather than orient myself toward languages I know better and with which I’m more comfortable, like Python and C# (speaking of C#, the recent improvements in AOT and cross-platform support are remarkable - we’re on the right track, come on guys!)

A second stimulating project was the Invoicetronic MCP Server. We were interested in having developers interact with the Invoicetronic API via LLM, and an MCP server serves precisely that purpose. Fortunately, C# has an excellent official library (still in preview but sufficiently mature) that simplifies the work, and implementing and deploying a first experimental version was relatively simple. There is enormous potential in this area, and I want to stay on top of it: objective achieved (I also gave a couple of presentations on the topic).

We released Invoicetronic for our own use, but I hope other developers will adopt it. Throughout many years of activity, our primary client has consistently been the end-user, the company that utilizes our accounting applications. In recent years, we also engaged with developers, but only in the context of our open-source projects. I’d love to serve them professionally as well.

The old timers who built the early web are coding with AI like it’s 1995. Think about it: They gave blockchain the sniff test and walked away. Ignored crypto (and yeah, we’re not rich now). NFTs got a collective eye roll. But AI? Different story. The same folks who hand-coded HTML while listening to dial-up modems sing are now vibe-coding with the kids. Building things. Breaking things. Giddy about it.

We Gen X’ers have seen enough gold rushes to know the real thing. This one’s got all the usual crap—bad actors, inflated claims, VCs throwing money at anything with “AI” in the pitch deck. Gross behavior all around. Normal for a paradigm shift, but still gross.

The people who helped wire up the internet recognize what’s happening. When the folks who’ve been through every tech cycle since gopher start acting like excited newbies again, that tells you something.

– Christina Wodtke (via)

Everyone’s either panicking that AI will replace them or assuming they don’t need to learn anything anymore. Both miss the point entirely. AI amplifies what you already know. If you understand distributed systems, you’ll use AI to build better ones. If you don’t, you’ll use AI to create distributed disasters.

The difference? When that AI-generated code breaks in production — and it will — you need to know why. When it doesn’t scale — and it won’t — you need to understand the bottlenecks. When it creates race conditions, memory leaks, or architectural nightmares, GitHub Copilot won’t save you. Your fundamentals will.

What Doesn’t Change

I have many thoughts right now, and it will take some time for them to settle. A good analysis and explanation of why this move is needed and is a good first step can be found in yesterday’s article by Dries Buytaert’s, The Web’s Broken Deal with AI Companies, which I recommend everyone read.

For 25 years, we built the Open Web on an implicit agreement: search engines could index our content because they sent users back to our websites. That model helped sustain blogs, news sites, and even open source projects. AI companies broke that model. They train on our work and answer questions directly in their own interfaces, cutting creators out entirely. Anthropic’s crawler reportedly makes 70,000 website requests for every single visitor it sends back. That is extraction, not exchange.

The ongoing discussion on Hacker News is also worth reading

Side note, I am fascinated by the technical details surrounding the 402 Payment Required response (and Cloudflare’s implementation), which was designed explicitly for micro-payment handling but never gained traction, until now.

But, there’s a bit, and that’s that I’m not programming anymore, or very little now, and I love coding. I love entering “the zone” and solving complex problems, one at a time. It’s always been my superpower. Will I still have fun in the future now that I can delegate most tasks to Claude Code?

In his The Hidden Cost of AI Coding, Matheus Lima expresses the same concern:

Instead of that deep immersion where I’d craft each function, I’m now more like a curator? I describe what I want, evaluate what the AI gives me, tweak the prompts, and iterate. It’s efficient, yes. Revolutionary, even. But something essential feels missing — that state of flow where time vanishes and you’re completely absorbed in creation. If this becomes the dominant workflow across teams, do we risk an industry full of highly productive yet strangely detached developers?

I’m afraid there’s no going back; agentic coding is so helpful that it’s tough to let go, especially in tiny teams like mine. Matheus suggests we preserve some space in our work where the flow can still happen, and that’s probably what I’m going to do: on some projects, maybe the open-source ones I maintain, resist the temptation of a claude /init and keep going like the good old times. One can at least try.

Today I learned about a nice NPM package called MCP Remote, which bridges the gap by allowing MCP clients that only support local servers to connect to remote MCP Servers, even with authentication support. Thanks to this tool, Claude Desktop is now talking to my remote server¹.

The only problem is that the OAuth2 handshake is not initiated at Claude startup or the first 401 response, as I expected, so I had to fall back to my secondary authentication method, the custom header.

I could, of course, retrieve my OAuth2 token out of band and then pass it to MCP Remote, but that’s cumbersome and boring. For now, I’m satisfied with the custom header approach. It’s only a matter of time before Claude Desktop matures enough to fully support Remote MCP servers. Right?

As a broke teenager, the web was this strange wonderland where you could access all kinds of copyrighted material using tools developed by fringe individuals/communities: Napster, Kazaa, Torrents, Usenet, etc. These tools (at least in the beginning) weren’t really made for profit, just to subvert the gatekeepers (and yeah, steal their profits).

Now — in a strange twist of irony — things seem to have flipped:

• 1999: Individuals use digital tools to steal intellectual property from corporations.
• 2025: Corporations use digital tools to steal intellectual property from individuals.

The empire strikes back.

On the topic of Agenting Coding he recently published:

• AI Changes Everything (you should read it)
• GenAI Criticism and Moral Quandaries

Both already reported.

I had intended to record the audio with the idea of transcribing it with MacWhisper and then publishing it here on my site, but I forgot to do so, which is a pity.

The session lasted almost two hours (I had thought it would take less time), during which I deviated somewhat from the script, using slides as a guide that were essentially an adaptation of the notes I had taken during my experiments. I showed the code for the MCP servers I created (stdio and streamable HTTP transports), demonstrated the various ways to link them with LLMs (Claude Desktop, Claude Code, and VS Code), and then shared my thoughts on the entire matter.

I received quite a bit of feedback, both in the room and afterward, when we moved to a pub where we stayed quite late by my standards, discussing AI, LLMs, and their impact on our daily work, which we can certainly define as significant, if not massive. Everyone uses LLMs’ UIs at their job to some extent, but very few are currently into agentic coding (the idea of a future session on that topic is tempting).

One remarkable tool that surfaced during the pub chat is Kotaemon, “an open-source RAG-based tool for chatting with your documents.”.

In the 2030s, intelligence and energy—ideas, and the ability to make ideas happen—are going to become wildly abundant. These two have been the fundamental limiters on human progress for a long time; with abundant intelligence and energy (and good governance), we can theoretically have anything else.

– Sam Altman

Hey, why did you build a CLI tool instead of an IDE?

Yeah, it’s a good question. There are two reasons. We started this at Anthropic, where people use a broad range of IDEs. Some people use VS code. Other people use Zed, Xcode, Vim, or Emacs. And it was just hard to build something that works for everyone. And so the terminal is just the common denominator. The second thing is that at Anthropic, we see firsthand how quickly the model is improving. I think there’s a good chance that by the end of the year, people won’t use IDEs. And so, we want to prepare for this future and avoid over-investing in UI and other layers on top. Given the way the models are progressing, it may not be practical to work on them soon.

Well, that’s one fascinating bold statement right there. It aligns with a vision I’ve had for a while: in a not-too-distant future, people will use LLM UIs (think Claude.ai with plug-and-play MCP support and voice recognition¹ both built in) as their primary interface instead of many different, specialized applications, at least for most use cases.

The initialization of Claude on the repository surprised me; the analysis (reported in CLAUDE.md) is thorough and reveals a good understanding of the project, including both the code and the general functioning, strategies, techniques, technologies and libraries employed.

I asked Claude to generate a test suite from scratch, and it did so rather effortlessly (it went back and forth a few times, and it is fascinating following its reasoning and attempts from a distance). Claude added the tests to the existing project, a big no-no for me. I asked it to go back and create a dedicated test project, and it complied, adding the new project to the solution. It took several iterations to achieve a good result. We now have 134 tests that cover 98% of the project’s business logic. Upon request, Claude also added code coverage by installing the missing tools.

Once satisfied with the test suite and having committed and pushed (via Claude) the work done, I asked Claude to set up a GitHub Action to run the tests on every push, then to update the existing workflows (container build; deployment to test or production; etc.) so they all run tests first.

In half a day of work, during which I mostly did other things, such as helping a colleague while Claude was chugging away, I achieved what would have taken me a week or even just a few days.

I have very little to complain about in terms of code quality, and where I wasn’t satisfied, I informed Claude and the unsatisfying code was quickly and pleasantly refactored. However, some refactoring is better done with Rider because it’s faster, doesn’t miss anything (like Claude tends to do if there are multiple similar refactorings to be done and they are scattered across various files) and doesn’t waste tokens that count toward usage limits (with the Pro subscription, we have access to a lighter tier of Claude Code. You want the Max subscription for unlimited - or higher anyway- ceiling).

One valuable feature is memory. You can have Claude memorize directives. This memory can be local to the repo or global. I instructed Claude to prefer collection expressions and utilize C#’s new targeted-type initialization feature, among other things, which were duly noted. I then asked Claude to refactor in light of the new directives, and it did. New code is now produced according to my preferences, and it will be on all my projects.

A thorough code review is always necessary, of course, but wouldn’t you do that for any code coming into your project?

Color me impressed so far. Copying and pasting from LLM’s web UIs is already a thing of the past. The present is all about agents, and they’re just getting started.

It wasn’t the purpose of his article, but I wish Armin had gone into the details of how, why, and when he delegates tasks to Claude Code.

Update: Armin later a follow-up.

When it rains heavily, streets fill with earthworms for several reasons:

1. Earthworms breathe through their skin, which must remain moist. During heavy rainfall, their underground tunnels flood, forcing them to surface to avoid drowning.

2. The vibrations of raindrops on the soil can be mistaken by earthworms for those produced by predators like moles, causing them to flee to the surface.

3. Rainy periods are ideal for earthworms to move and find new habitats or mates, as they can travel on the moist surface without risking dehydration.

4. On paved surfaces like roads and sidewalks, water tends to accumulate more than in soil, creating conditions that attract earthworms.

Unfortunately, once earthworms reach the asphalt, they often become trapped and unable to return to the soil, becoming easy prey or risking dehydration when the sun returns.

I’ve been using LLMs (ChatGPT and Claude) more and more lately, especially for work. The more I use them, the more I understand how to leverage their capabilities. I would have never thought about sharing my reading experiences with ChatGPT before. Ethan Mollick has it right; we should all put at least 10 hours into LLMs before judging them.

Your goal is simple: spend 10 hours using AI on tasks that actually matter to you. After that, you’ll have a natural sense of how AI fits into your work and life. You’ll develop an intuition for effective prompting, and you’ll better understand AI’s potential. Don’t aim for perfection - just start somewhere and learn as you go.

[…] more and more people across the globe get their news and cues from social media. It has a scary ability to generate anxiety and rage, and it is addictive. Expert after expert I’ve talked to over the years has made the same point — in the new paradigm, engagement equals enragement. This is made worse by the people who run these companies, for whom anticipation of consequences is lacking and whose first instinct is to let it all through the gate, regardless of potential damage or danger. What’s the opposite of the mommy state? Parent-free chaos.

Read her article here.

I appreciate how Swisher was an initial proponent of digital advent, so much so that she co-founded All Things Digital. Yet, that didn’t prevent her from foreseeing the troubled waters ahead.

Hiring engineers is about composing teams. The smallest unit of software ownership is not the individual, it’s the team.

Have you ever been on a team packed exclusively with staff or principal engineers? It is not fun. That is not a high-functioning team. There is only so much high-level architecture and planning work to go around, there are only so many big decisions that need to be made. These engineers spend most of their time doing work that feels boring and repetitive, so they tend to over-engineer solutions and/or cut corners—sometimes at the same time. They compete for the “fun” stuff and find reasons to pick technical fights with each other. They chronically under-document and under-invest in the work that makes systems simple and tractable.

The best teams are ones where no one is bored, because every single person is working on something that challenges them and pushes their boundaries. The only way you can get this is by having a range of skill levels on the team.

Short version: ChatGPT and other similar tools do not directly learn from and memorize everything that you say to them.

Every time you start a new chat conversation, you clear the slate. Each conversation is an entirely new sequence, carried out entirely independently of previous conversations from both yourself and other users. Understanding this is key to working effectively with these models. Every time you hit “new chat” you are effectively wiping the short-term memory of the model, starting again from scratch. This has a number of important consequences.

More here.

However, the most critical aspect of this release is not technical, as Ethan Mollick correctly pinpoints in his timely What Open AI Did post:

Likely the biggest impact of GPT-4o is not technical, but a business decision: soon everyone, whether they are paying or not, will get access to GPT-4o1. I think this is a big deal. When I talk with groups and ask people to raise their hands if they use ChatGPT, almost every hand goes up. When I ask if they used GPT-4, only 5% of hands remain up, at most. GPT-4 is so, so much better than free ChatGPT-3.5, it is like having a PhD student work with you instead of a high school sophomore. But that $20 a month barrier kept many people from understanding how impressive AI can be, and for gaining any benefit from AI. That is no longer true.

He then speculates how this change will impact significant areas such as education, work, and global entrepreneurship.

It’s very fast to build something that’s 90% of a solution. The problem is that the last 10% of building something is usually the hard part which really matters, and with a black box at the center of the product, it feels much more difficult to me to nail that remaining 10%. Closing that gap with gen AI feels much more fickle to me than a normal engineering problem. It could be that I’m unfamiliar with it, but I also wonder if some classes of generative AI based products are just doomed to mediocrity as a result.

– Moxie Marlinspike

I find my feelings about AI are actually pretty similar to my feelings about blockchains: they do a poor job of much of what people try to do with them, they can’t do the things their creators claim they one day might, and many of the things they are well suited to do may not be altogether that beneficial. And while I do think that AI tools are more broadly useful than blockchains, they also come with similarly monstrous costs.

Brilliant.

But the reality is that you can’t build a hundred-billion-dollar industry around a technology that’s kind of useful, mostly in mundane ways, and that boasts perhaps small increases in productivity if and only if the people who use it fully understand its limitations. And you certainly can’t justify the kind of exploitation, extraction, and environmental cost that the industry has been mostly getting away with, in part because people have believed their lofty promises of someday changing the world.

Full article here.

The feeling is having a good, untiring Linux connoisseur (and of all the various ancillary packages) sitting right next to you, always ready to lend a hand. I also think, and I’m probably wrong (but my experience seems to bear this out so far), that because these are all simple questions about stuff well carved in stone, hallucinations are unlikely to occur.

My assistant today was ChatGPT 3.5, which is very old by today’s LLM standards. It is also free to use, which makes it all even more impressive.

“Coding” was never the source of value, and people shouldn’t get overly attached to it. Problem solving is the core skill. The discipline and precision demanded by traditional programming will remain valuable transferable attributes, but they won’t be a barrier to entry.

– John Carmack

I’d be tempted to call bullshit on this one, but it’s coming from Carmack, so hey, let me think about it for a minute (it’s not the first time I’ve caught him endorsing AI).

I don’t think content creators are at risk with Sora, not anytime soon, but, as Marquees repeatedly notes in the video above, just one year ago we thought AI-generated video was a joke.

Let me start with the headline: Gemini Advanced is clearly a GPT-4 class model. The statistics show this, but so does a month of our informal testing. And this is a big deal because OpenAI’s GPT-4 (the paid version of ChatGPT/Microsoft Copilot) has been the dominant AI for well over a year, and no other model has come particularly close. Prior to Gemini, we only had one advanced AI model to look at, and it is hard drawing conclusions with a dataset of one. Now there are two, and we can learn a few things.

Things are getting hotter in the LLM space, and competition is always good.

Full article here.

Have you ever wasted some time watching a youtube video, that got you kind of interested because of the click-baity topic, but in the end turned out to be nothing more BUT click-bait? Or have you ever wanted to just quickly recall what a video that you’ve watched some time ago was about? Just Tell Me has you covered!

You can run it from the website or locally on the command line, with your OpenAPI key and the LLM model of choice. I have not looked at the code, but I understand it leverages the video transcript to do its magic¹. I’ve been using it for a while, and it’s been good.

To ground ourselves, we can start with two quotes that should inform any estimates about the future. The first is Amara’s Law: “We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.” Social change is slower than technological change. We should not expect to see immediate global effects of AI in a major way, no matter how fast its adoption (and it is remarkably fast), yet we certainly will see it sooner than many people think.

It’s an insightful read that pairs well with Simon Wilson’s 2023 round-up. Also, his 30-second fake video of himself telling things he never told is impressive.

2023 was the breakthrough year for Large Language Models (LLMs). I think it’s OK to call these AI—they’re the latest and (currently) most “interesting development in the academic field of Artificial Intelligence that dates back to the 1950s. Here’s my attempt to round up the highlights in one place!

The links contained within the post are also valuable. You may know Simon’s website if you are interested in LLMs and AI. If you don’t, I suggest you start following him, preferably via his RSS feed like real hackers do.

I always struggle a bit with I’m asked about the “hallucination problem” in LLMs. Because, in some sense, hallucination is all LLMs do. They are dream machines.

We direct their dreams with prompts. The prompts start the dream, and based on the LLM’s hazy recollection of its training documents, most of the time the result goes someplace useful.

It’s only when the dreams go into deemed factually incorrect territory that we label it a “hallucination”. It looks like a bug, but it’s just the LLM doing what it always does.

– Andrej Karpathy

As a Kahneman reader, I appreciated the Thinking Fast and Slow analogy proposed at about half-length in the video: “System 1” (fast automatic thinking, rapid decisions) is where we’re now; “System 2” (rational, slow thinking, complex decisions) is LLMs next goal. Also, I suspect Karpathy’s intriguing idea of LLMs as the center of a new “operating system style” is not too far off from what will emerge soon. The final segment on AI security and known attack vectors (jailbreaking, prompt injection, data poisoning) is also super interesting.

On his website, Karpathy also has a promising zero-to-hero video series, “a course on building neural networks from scratch, in code.”

We only publish significant news. To find them we use AI (ChatGPT-4) to read and analyze 1000 top news every day. For each article it estimates magnitude, scale, potential and credibility. Then we combine these estimates to get the final Significance score from 0 to 10. And now the best part: We’ll only send you the news scored 6.5 or higher. Sometimes it’s 5 articles, sometimes 2, sometimes 8. And sometimes — none at all. But one thing is constant — you can be sure that you haven’t missed anything important.

The concept is brilliant and well executed, but I can’t help but feel uncomfortable at the notion of an AI curating news for me. Yet, this is the best use case for LLM/AI I’ve seen until now.

However useful these programs may be in some narrow domains (they can be helpful in computer programming, for example, or in suggesting rhymes for light verse), we know from the science of linguistics and the philosophy of knowledge that they differ profoundly from how humans reason and use language. These differences place significant limitations on what these programs can do, encoding them with ineradicable defects.

More here.

Last month one of our journalists received an interesting email. A researcher had come across mention of a Guardian article, written by the journalist on a specific subject from a few years before. But the piece was proving elusive on our website and in search. Had the headline perhaps been changed since it was launched? Had it been removed intentionally from the website because of a problem we’d identified? Or had we been forced to take it down by the subject of the piece through legal means?

The reporter couldn’t remember writing the specific piece, but the headline certainly sounded like something they would have written. It was a subject they were identified with and had a record of covering. Worried that there may have been some mistake at our end, they asked colleagues to go back through our systems to track it down. Despite the detailed records we keep of all our content, and especially around deletions or legal issues, they could find no trace of its existence.

Why? Because it had never been written.

More here.

Software is just a tool to help accomplish something for people – many programmers never understood that. Keep your eyes on the delivered value, and don’t over-focus on the specifics of the tools.

I have often fallen into the over-focusing trap in my career. The whole thread is well worth reading:

(via)

L’elaborazione collaborativa distribuita è diventata famosa grazie a progetti come SETI@home e Folding@home. Semplificando molto potremmo dire che questo tipo di applicazione prevede che un piccolo programma venga installato e fatto girare su decine di migliaia di computer volontari. La capacita elaborativa del progetto è data dalla somma delle elaborazioni individuali.

La novità di Chess@Home consiste nell’idea di ricorrere a codice JavaScript che gira nel browser, dunque senza alcuna necessità di client dedicati. Appositi widget presenti nelle pagine dei siti aderenti innestano l’elaborazione sul computer del visitatore, potenzialmente decuplicando il numero di nodi che partecipano all’elaborazione (più visitatori accedono alla stessa pagina contemporaneamente).

E’ facile intravedere complicazioni etiche ma occorre tenere conto del fatto che il progetto, messo in piedi in sole 48 ore, è nato soprattutto come esperimento per testare il limite delle tecnologie impiegate (Node.JS, MongoDB, JavaScript). Vi consiglio caldamente l’articolo di uno degli ideatori, Sylvain Zimmer, che descrive genesi e dettagli tecnici del progetto.

Già oggi potete giocare la vostra prima partita a scacchi contro la rete. Il mio cuore di nerd (che in anni ahimè troppo lontani ha fatto girare impunemente il client SETI per mesi e mesi) tifa per voi.

PS: domani al Romagna Camp 2011 terrò una relazione dal titolo “Quattro passi fra le nuvole (e non scordate il paracadute)”. Per saperne di più date una occhiata qui.