Simon Willison’s Weblog

airoboros LMoE. airoboros provides a system for fine-tuning Large Language Models. The latest release adds support for LMoE—LoRA Mixture of Experts. GPT-4 is strongly rumoured to work as a mixture of experts—several (maybe 8?) 220B models each with a different specialty working together to produce the best result. This is the first open source (Apache 2) implementation of that pattern that I’ve seen.

# 24th August 2023, 10:31 pm / generative-ai, opensearch, gpt-4, ai, llms, fine-tuning

How The Post is replacing Mapbox with open source solutions (via) Kevin Schaul describes the Washington Post’s emerging open source GIS stack: OpenMapTiles, Maputnik, PMTiles and Maplibre-gl-js.

# 17th February 2023, 6:45 pm / maps, washington-post, openstreetmap, opensearch

Add OpenSearch to your site in five minutes. OpenSearch is easy. DeWitt demonstrates how you don’t even need a site search engine to implement it if you take advantage of Google’s site: operator.

# 9th February 2007, 12:52 am / google, opensearch, dewitt-clinton