Simon Willison’s Weblog

Subscribe
Atom feed for edge-llms

62 items tagged “edge-llms”

LLMs that can run on consumer hardware like laptops or mobile phones.

2023

Large language models are having their Stable Diffusion moment

Visit Large language models are having their Stable Diffusion moment

The open release of the Stable Diffusion image generation model back in August 2022 was a key moment. I wrote how Stable Diffusion is a really big deal at the time.

[... 1,810 words]

7:15 pm / 11th March 2023 / facebook, ai, gpt-3, openai, generative-ai, chatgpt, whisper, llama, edge-llms, llms

Running LLaMA 7B on a 64GB M2 MacBook Pro with llama.cpp. I got Facebook’s LLaMA 7B to run on my MacBook Pro using llama.cpp (a “port of Facebook’s LLaMA model in C/C++”) by Georgi Gerganov. It works! I’ve been hoping to run a GPT-3 class language model on my own hardware for ages, and now it’s possible to do exactly that. The model itself ends up being just 4GB after applying Georgi’s script to “quantize the model to 4-bits”.

# 11th March 2023, 4:19 am / facebook, gpt-3, generative-ai, ai, llama, edge-llms, llms

Related