Using gpt-4o-mini as a reranker. Tip from David Zhang: "using gpt-4-mini as a reranker gives you better results, and now with strict mode it's just as reliable as any other reranker model".
David's code here demonstrates the Vercel AI SDK for TypeScript, and its support for structured data using Zod schemas.
const res = await generateObject({
model: gpt4MiniModel,
prompt: `Given the list of search results, produce an array of scores measuring the liklihood of the search result containing information that would be useful for a report on the following objective: ${objective}\n\nHere are the search results:\n<results>\n${resultsString}\n</results>`,
system: systemMessage(),
schema: z.object({
scores: z
.object({
reason: z
.string()
.describe(
'Think step by step, describe your reasoning for choosing this score.',
),
id: z.string().describe('The id of the search result.'),
score: z
.enum(['low', 'medium', 'high'])
.describe(
'Score of relevancy of the result, should be low, medium, or high.',
),
})
.array()
.describe(
'An array of scores. Make sure to give a score to all ${results.length} results.',
),
}),
});
It's using the trick where you request a reason
key prior to the score, in order to implement chain-of-thought - see also Matt Webb's Braggoscope Prompts.
Recent articles
- Qwen2.5-Coder-32B is an LLM that can code well that runs on my Mac - 12th November 2024
- Visualizing local election results with Datasette, Observable and MapLibre GL - 9th November 2024
- Project: VERDAD - tracking misinformation in radio broadcasts using Gemini 1.5 - 7th November 2024