/SQL

GPT In 500 Lines Of SQL

tl;dr: "Before a text can be fed to a neural network, it needs to be converted into a list of numbers. GPT2 uses a variation of the algorithm called Byte pair encoding to do precisely that. Its tokenizer uses a dictionary of 50257 code points - in AI parlance, 'tokens' - that correspond to different byte sequences in UTF-8, plus the 'end of text' as a separate token. This dictionary was built by statistical analysis performed like this: Start with a simple encoding of 256 tokens: one token per byte. Perform the collapse 50000 times over."

featured in #478

SQL As API

- Valentin Willscher tl;dr: "I know what you are thinking: Exposing an API that accepts SQL is crazy. It's a terrible idea. Especially if the API is exposed on the internet. Doing that is insecure and will lead to SQL injection attacks, it is a nightmare to maintain and it will lock the backend implementation into a specific technology (some ANSI SQL database). But is that really true? Time to re-evaluate!"

featured in #476

The Case Of A Curious SQL Query

- Justin Jaffray tl;dr: Justin provides a deep dive into SQL's foundational aspects, highlighting the importance of a formalized approach to query behavior. Using the example of "predicate pushdown," Justin presents a SQL query that behaves differently across databases like DuckDB, SQLite, and CockroachDB. "I think it's a fun little mind bender that gives you some insight into the internals of these databases query engines without having to actually look at any code."

featured in #460

How We Built A Streaming SQL Engine

- Maor Kern tl;dr: “So you probably wake up every morning asking yourself three of life’s most pertinent questions- how do I build a streaming SQL engine, what even is a streaming SQL engine, and can our Lord drop tables owned by another user. I too found myself asking these questions, sometimes even dreaming about them- often in the form of various SQL operators pointing and laughing at my incompetence as I beg them to answer me.” A streaming SQL engine keeps queries’ results up to date without ever having to recalculate them and Maor discusses topic like data ingestion, query optimization, and stream processing.

featured in #458

SQL Join Flavors

- Anton Zhiyanov tl;dr: Anthon provides a look at SQL joins, covering types like "Qualified Join," "Natural Join," "Cross Join," "Partitioned Join," and "Lateral Join." He explains that a "Qualified join" is an umbrella term for the most common types of joins: inner, left, right, and full. "A qualified join connects records from two datasets into one, according to the matching criteria you specify," he states. The article also warns against the use of "Natural Joins," stating they are "almost always a bad idea."

featured in #450

Is 20M Of Rows Still A Valid Soft Limit Of MySQL Table In 2023?

- Yisheng Gong tl;dr: “There’s rumor around the internet that we should avoid having > 20M rows in a single MySQL table. Otherwise, the table’s performance will be downgraded, you will find SQL query much slower than usual when it’s above the soft limit. These judgements were made on HDD many years ago. I’m wondering if it’s still true for MySQL on SSD in 2023, and if true, why is that?”

featured in #417

Online Gradient Descent Written In SQL

- Max Halford tl;dr: Max implements a ML algorithm within a relational database, using SQL. Some databases allow doing inference with an already trained model. Training the model in the database would remove altogether the need for a separate inference / training service. Max attempts to do this with the Online Gradient Descent algorithm.

featured in #398

SQL Should Be Your Default Choice For Data Engineering Pipelines

- Robin Linacre tl;dr: "SQL should be the first option considered for new data engineering work. It’s robust, fast, future-proof and testable. With a bit of care, it’s clear and readable. A new SQL engine - DuckDB - makes SQL competitive with other high performance dataframe libraries, making SQL a good candidate for data of all sizes."

featured in #387

Writing A Python SQL Engine From Scratch

- Toby Mao tl;dr: "This post will cover why I went through the effort of creating a Python SQL engine and how a simple query goes from a string to actually transforming data." Toby covers tokenizing, parsing, optimizing, planning and executing.

featured in #381

Postgres Auditing In 150 lines Of SQL

- Oliver Rice tl;dr: "Data auditing is a system that tracks changes to tables' contents over time. PostgreSQL has a robust set of features which we can leverage to create a generic auditing solution in 150 lines of SQL."

featured in #303