Simon Willison’s Weblog

DALL·E: Creating Images from Text (via) “DALL·E is a 12-billion parameter version of GPT-3 trained to generate images from text descriptions, using a dataset of text–image pairs.”. The examples in this paper are astonishing—“an illustration of a baby daikon radish in a tutu walking a dog” generates exactly that.

# 5th January 2021, 8:31 pm / machine-learning, ai, openai, dalle, generative-ai

Tempering Expectations for GPT-3 and OpenAI’s API. Insightful commentary on GPT-3 (which is producing some ridiculously cool demos at the moment thanks to the invite-only OpenAI API) from Max Woolf.

# 18th July 2020, 7:29 pm / machine-learning, max-woolf, gpt-3, ai, openai, generative-ai, llms

gpt2-headlines.ipynb. My earliest experiment with GPT-2, using gpt-2-simple by Max Woolf to generate new New York Times headlines based on a GPT-2 fine-tuned against headlines from different decades of that newspaper.

# 31st January 2020, 2:13 am / llms, generative-ai, ai, max-woolf, gpt-2, openai

Reinforcement Learning with Prediction-Based Rewards (via) Fascinating result: by teaching a reinforcement learning agent that plays video games to optimize for "unfamiliar states" - states where it cannot predict what will happen next - the agent does a much better job of playing some games.

... for the first time exceeds average human performance on Montezuma’s Revenge. RND achieves state-of-the-art performance, periodically finds all 24 rooms and solves the first level without using demonstrations or having access to the underlying state of the game.

# 31st October 2018, 11:51 pm / machine-learning, ai, openai