How Does ChatGPT Work? As Explained By The ChatGPT Team
- Gergely Orosz tl;dr: When you ask ChatGPT a question, several steps happen: (1) Input: We take your text from the text input. (2) Tokenization: We chunk it into tokens. A token roughly maps to a couple of unicode characters. You can think of it as a word. (3) Create embeddings: We turn each token into a vector of numbers. These are called embeddings. (4) Multiply embeddings by model weights: We then multiply these embeddings by hundreds of billions of model weights. (5) Sample a prediction.featured in #508
How We Built Text-to-SQL At Pinterest
tl;dr: “We took the rise in availability of LLMs as an opportunity to explore whether we could assist our data users with this task by developing a Text-to-SQL feature which transforms these analytical questions directly into code.” The authors describe the tools evolution and implementation.featured in #507
Lessons After A Half-Billion GPT Tokens
- Ken Kantzer tl;dr: “I thought I’d share some of the more “surprising” lessons after churning through just north of 500 million tokens, by my estimate.” Lessons include: (1) When it comes to prompts, less is more. (2) You don’t need langchain. You probably don’t even need anything else OpenAI has released in their API in the last year. (3) Improving the latency with streaming API and showing users variable-speed typed words is actually a big UX innovation with ChatGPT.featured in #506
Notes On How To Use LLMs In Your Product
- Will Larson tl;dr: “I’ve been working fairly directly on meaningful applicability of LLMs to existing products for the last year, and wanted to type up some semi-disorganized notes. These notes are in no particular order, with an intended audience of industry folks building products.” Will discusses opportunities re-configuration, combining LLMs with unsophisticated algorithms to retrieve data. And more.featured in #505
Developing Rapidly With Generative AI
- Shannon Phu tl;dr: From the engineering team at Discord: “We break down the process of building with LLMs into a few stages. Starting with product ideation and defining requirements, we first need to figure out what we’re building and how it can benefit users. Next, we develop a prototype of our idea, learn from small-scale experiments, and repeat that process until our feature is in a good state. Finally, we fully launch and deploy our product at scale. In this post, we will dive deeper into each stage of this process.”featured in #505
Claude And ChatGPT For Ad-Hoc Sidequests
- Simon Willison tl;dr: The author demonstrates a quick ”sidequest" task where he converted the shapefile of a largest park in NY to a GeoJSON polygon in just 6 minutes. “One of the greatest misconceptions concerning LLMs is that they’re easy to use. They aren’t: getting great results requires a great deal of experience and hard-fought intuition, combined with deep domain knowledge of the problem you are applying them to.”featured in #501
Meta's New LLM-Based Test Generator Is A Sneak Peek To The Future Of Development
- Leonardo Creed tl;dr: “Meta claims that this “this is the first paper to report on LLM-generated code that has been developed independent of human intervention (other than final review sign off), and landed into large scale industrial production systems with guaranteed assurances for improvement over the existing code base.” Furthermore, there are solid principles that developers can take away in order to use AI effectively themselves.”featured in #492
Engineering Practices For LLM Application Development
- David Tan Jesse Wang tl;dr: “LLM engineering involves much more than just prompt design or prompt engineering. In this article, we share a set of engineering practices that helped us deliver a prototype LLM application rapidly and reliably in a recent project. We'll share techniques for automated testing and adversarial testing of LLM applications, refactoring, as well as considerations for architecting LLM applications and responsible AI.”featured in #489
The Pain Points Of Building A Copilot
- Austin Henley tl;dr: What are the pain points, and what are the opportunities for tools. ”We conducted semi-structured interviews with 26 developers from a variety of companies that are working on copilots. We analyzed their responses to identify themes. Then we conducted two focus group sessions with tool builders that involved reviewing our interview findings and brainstorming possible solutions.” Austin shares the results here.featured in #486
featured in #478