Blog on chrislaing.net

High-Code: The Unexpected Path to AI Democratisation

Thu, 01 Aug 2024 15:00:00 +1300

Years ago, I read a blog post called Always Bet on Text that left a lasting impression on me. Not only did it accord with my own experience, but it gave me a good, compact reference frame to think about technology. No recent technology is more suited to this reference frame than Generative AI, for the simple reason that, at its core, modern GenAI is all about text completion.

On the Economics and Ergonomics of LLMs

Tue, 16 Jul 2024 10:00:00 +1300

For all the millions of words that have been devoted to the capabilities of LLMs, the economics and ergonomics of these systems seem to me to be under-discussed. In much the same way that technology changes - from records to CDs to streaming - have been a dominant factor in the evolution of popular music, prosaic matters such as cost structures will have a substantial influence on the development and impact of AI. While it is still early days for LLMs, there are already signs that economic factors are impacting everything from product design and technology choice to adoption and even regulation.

The Decomposition of Large Language Models

Tue, 30 May 2023 09:00:00 +1300

While many AI systems can be very complex, Large Language Models (LLMs) seem to be refreshingly simple to understand. With their ability to generate human-like text, applications like ChatGPT have captured the popular imagination. However, as compelling an experience as chatbots are, the tradeoffs that they make can lead us to misunderstand LLMs as a whole. In this post, we will explore the idea that the functions of LLMs can be thought of as discrete capabilities, and how different composition of these functions leads to a much wider variety of user experiences.

The Large Language Model Iceberg

Sat, 01 Apr 2023 13:30:00 +1300

As the capabilities of Artificial Intelligence (AI) continue to evolve, the term “chatbot” has become synonymous with a new era of digital interactions. What was once a seemingly futuristic concept is now an integral part of our everyday lives. However, it is the Machine Learning models underlying them - Large Language Models (LLMs) - that are the true powers in these waters.

Three Categories of AI

Sun, 25 Jul 2021 19:00:00 +1300

The term “AI” has at least three entirely different meanings in the modern business context, and any particular business leader usually has only one of these meanings in mind when they use the term. Not only is it common for participants in a discussion to have different interpretations of the term “AI”, but most will be unaware that this terminological confusion exists, leaving leaders in both public and private sector organisations poorly equipped to discuss one of the most important technologies of our time.

If it's not mysterious, it's not AI

Thu, 08 Jul 2021 17:30:00 +1300

Businesses are deeply confused about what AI is, and they have every right to be. Differences in terminology have plagued the Data-AI-Software landscape for many years, as anyone who has ever looked for a job as a “Data Scientist” will attest. Attempts to demystify the topic either conjure a series of analogies that are then strained to breaking point, or try to delve into the actual technology behind the hype, leaving everyone more confused than they were before.

I will attempt something different: removing the mystery by embracing the mystery.

The Memo

Sun, 20 Jun 2021 14:00:00 +1300

In 2002, Amazon’s Jeff Bezos issued a memo that has entered tech industry canon. The memo, known as the “API Mandate”, is generally perceived as being a statement about technology at Amazon, and is therefore widely admired by technologists and wholly ignored by executives. This is unfortunate, because it’s no exaggeration to say that the API Mandate completely transformed Amazon as a business and laid the foundation for its success. Better still, unlike many things that global technology titans do, it is something that can be replicated and put to use by almost any business.

In this post, we’ll talk about the memo, and how it created the systems and incentives for radical organisational transformation.

The Business Implications of Combinatorial Optimisation by Deep Learning

Mon, 31 May 2021 19:30:00 +1300

Recently, Deep Learning has revolutionised a multitude of technical fields, most notably Computer Vision and Natural Language Processing. The implications for business are enormous, with the ramifications of these change still filtering through the mostly technology-shy world of business. There are signs that Combinatorial Optimisation might be the next technical field to undergo Revolution-By-Neural-Network. If this comes to fruition, then the impact on business will be seismic.

The Bitter Lesson Conditions

Thu, 27 May 2021 11:30:00 +1300

The hard truth of AI is that methods that exploit deep, hard-won human knowledge about a particular domain are outperformed by methods that cleverly exploit the increasing power of computation. I have a modest proposal to distill this fundamental truth into a set of conditions which, when met, will revolutionise a given field. Lineage I do not claim this insight about the fundamental mechanism of progress in AI as original; I have been heavily influenced by Richard Sutton’s powerful essay The Bitter Lesson.

Odds and Ends #4

Wed, 22 Jan 2020 00:00:00 +0000

I just discovered kops, apparently an excellent way to deploy and manage Kubernetes clusters on the cloud. Since managing a k8s cluster seems to be a full-time job at the best of times, I’m interested to see if this really does make it much easier. I have a very mild case of podcast addiction, and am always looking for something great to add to my collection of subscriptions. Nikita Voloboev has published a very interesting list of podcasts.

Odds and Ends #3

Sun, 02 Jun 2019 00:00:00 +0000

My former colleagues at Blue Yonder (now part of JDA) have introduced Kartothek, software for managing tables stored as parquet files. A great set of documentation on going zero to JupyterHub with Kubernetes. A related piece from Jim Crist on installing JupyterHub on an existing Hadoop cluster. An interesting paper interoducing PATE from a couple of years ago that had passed me by. PATE stands for Private Aggregation of Teacher Ensembles, and is a method for doing semi-supervised transfer learning from private data.

Odds and Ends #2

Fri, 01 Mar 2019 00:00:00 +0000

Naftali Tishby on the Information Theory of Deep Learning (embedded below). I’m very enthusiastic about this kind of work, and have resolved to find and read more of it. a16z’s AI Playbook may be a couple of years old now, but it’s still an important read. Traces is a Python library for unevenly-spaced time series analysis. I haven’t had a problem to really try this out on, but the website looks slick 👌🏻.

Odds and Ends #1

Sat, 23 Feb 2019 00:00:00 +0000

Spektral is a framework for relational representation learning, built in Python and based on the Keras API. The field of Geometric Deep Learning is starting to get some traction. In particular, doing Deep Learning on graphs presents some interesting possibilities. RPC Frameworks: gRPC vs Thrift vs RPyC for python. The state of Python Packaging. Packing in Python is both improving dramatically and harder than it should be.

Keybase: Keys for everyone! Part II

Mon, 20 Feb 2017 00:00:00 +0000

In a previous post, we discussed Keybase, a clever company that is solving a lot of the classic problems that users have had with PGP: managing keys, verifying identities, trusting third parties, and user experience.

Keybase: Keys for everyone! Part I

Sun, 13 Nov 2016 00:00:00 +0000

Encryption is a topic that has fascinated me for years, less because of its mathematical basis than its applications, both existing and potential. By far my favourite project in this realm is Keybase.

A New Hope

Thu, 03 Nov 2016 23:04:26 +0100

Many years ago, I had a blog. Being perfectly honest, it wasn’t a very good blog; it consisted mainly of rants of one kind or another, inane musings, and the occasional photo from my holidays. On the modern web, social media is the default depository for said rants, musings, and photos, and so I had no need of a dedicated space on my website to share them.