/LLM

Did GitHub Copilot Really Increase My Productivity?

- Yuxuan Shui tl;dr: “I had free access to GitHub Copilot for about a year, I used it, got used to it, and slowly started to take it for granted, until one day it was taken away. I had to re-adapt to a life without Copilot, but it also gave me a chance to look back at how I used Copilot, and reflect - had Copilot actually been helpful to me?”

featured in #513


What Can LLMs Never Do?

- Rohit Krishnan tl;dr: “Over the past few weeks I have been obsessed by trying to figure out the failure modes of LLMs. This started off as an exploration of what I found. It is admittedly a little wonky but I think it is interesting. The failures of AI can teach us a lot more about what it can do than the successes.”

featured in #511


How Does ChatGPT Work? As Explained By The ChatGPT Team

- Gergely Orosz tl;dr: When you ask ChatGPT a question, several steps happen: (1) Input: We take your text from the text input. (2) Tokenization: We chunk it into tokens. A token roughly maps to a couple of unicode characters. You can think of it as a word. (3) Create embeddings: We turn each token into a vector of numbers. These are called embeddings. (4) Multiply embeddings by model weights: We then multiply these embeddings by hundreds of billions of model weights. (5) Sample a prediction. 

featured in #508


How We Built Text-to-SQL At Pinterest

tl;dr: “We took the rise in availability of LLMs as an opportunity to explore whether we could assist our data users with this task by developing a Text-to-SQL feature which transforms these analytical questions directly into code.” The authors describe the tools evolution and implementation. 

featured in #507


Lessons After A Half-Billion GPT Tokens

- Ken Kantzer tl;dr: “I thought I’d share some of the more “surprising” lessons after churning through just north of 500 million tokens, by my estimate.” Lessons include: (1) When it comes to prompts, less is more. (2) You don’t need langchain. You probably don’t even need anything else OpenAI has released in their API in the last year. (3) Improving the latency with streaming API and showing users variable-speed typed words is actually a big UX innovation with ChatGPT.

featured in #506


Notes On How To Use LLMs In Your Product

- Will Larson tl;dr: “I’ve been working fairly directly on meaningful applicability of LLMs to existing products for the last year, and wanted to type up some semi-disorganized notes. These notes are in no particular order, with an intended audience of industry folks building products.” Will discusses opportunities re-configuration, combining LLMs with unsophisticated algorithms to retrieve data. And more.

featured in #505


Developing Rapidly With Generative AI

- Shannon Phu tl;dr: From the engineering team at Discord: “We break down the process of building with LLMs into a few stages. Starting with product ideation and defining requirements, we first need to figure out what we’re building and how it can benefit users. Next, we develop a prototype of our idea, learn from small-scale experiments, and repeat that process until our feature is in a good state. Finally, we fully launch and deploy our product at scale. In this post, we will dive deeper into each stage of this process.”

featured in #505


Claude And ChatGPT For Ad-Hoc Sidequests

- Simon Willison tl;dr: The author demonstrates a quick ”sidequest" task where he converted the shapefile of a largest park in NY to a GeoJSON polygon in just 6 minutes. “One of the greatest misconceptions concerning LLMs is that they’re easy to use. They aren’t: getting great results requires a great deal of experience and hard-fought intuition, combined with deep domain knowledge of the problem you are applying them to.”

featured in #501


Meta's New LLM-Based Test Generator Is A Sneak Peek To The Future Of Development

- Leonardo Creed tl;dr: “Meta claims that this “this is the first paper to report on LLM-generated code that has been developed independent of human intervention (other than final review sign off), and landed into large scale industrial production systems with guaranteed assurances for improvement over the existing code base.” Furthermore, there are solid principles that developers can take away in order to use AI effectively themselves.” 

featured in #492


Engineering Practices For LLM Application Development

- David Tan Jesse Wang tl;dr: “LLM engineering involves much more than just prompt design or prompt engineering. In this article, we share a set of engineering practices that helped us deliver a prototype LLM application rapidly and reliably in a recent project. We'll share techniques for automated testing and adversarial testing of LLM applications, refactoring, as well as considerations for architecting LLM applications and responsible AI.”

featured in #489