The archive's meter: you pay for the bytes read

The morning invoice

The morning starts with a call: the station commandant has forwarded you a statement. For yesterday's queries to the Far Archive the meter drew a noticeable chunk of the donor station's energy — and next to every query there is a number.

You look through the list and notice something strange: the most "expensive" query returned five rows. The cheapest returned fifty. The price is clearly not about how many rows came back.

QUERY: I went through the archive's price list while you slept. Bad news: what you pay for here is reading, not the answer. Good news: that means the price is something you can steer.

The on-demand model: bytes, not rows

In its basic mode (on-demand) BigQuery charges for one thing: how many bytes of tables the query read. In Earth money that is about $6.25 per TiB read (the first TiB each month is free). A TiB is a tebibyte, a binary terabyte: 1024 GiB ≈ 1.1 of an "ordinary" TB. Google's price list counts in exactly those, so the course does too. The station's meter works the same way, only the currency is different.

Recall the previous lesson: data lies in columns. So the volume read is counted simply:

scan = the sum of the sizes of the columns read, each one whole

• SELECT name FROM fa_users — one column, name, is read across every row of the table;
• SELECT * FROM fa_users — EVERY column is read;
• the number of rows in the answer does not affect the bill at all.

In Postgres you never thought about the "cost" of a SELECT: your server, your disk, and the difference between a fast query and a slow one is minutes of waiting. In BigQuery every query has a price in money — and it is known BEFORE you run it.

Why LIMIT does not save money

A habit from the row world: "I will add LIMIT 10 — less work for the database". In a row store that is close to the truth: read ten rows from the start, then stop.

A columnar engine works differently: to hand you even a single row it has already read the columns it needs whole — distributed, in large blocks. LIMIT is applied at the very end, to the finished result. By that moment the bill has already been issued.

The same goes for WHERE (as long as the table is not partitioned — that is the chapter "Money and scale"): the filter cuts the number of rows in the answer, but the columns from WHERE and SELECT were still read across the whole table.

Let us do the arithmetic. Imagine the colony's event log has grown to production size: ten columns of roughly equal weight, 5 TiB in total — about 500 GiB per column. Then:

-- reads ~1 TiB (two whole columns): ~$6.25
SELECT user_id, event_name FROM journal;

-- reads EXACTLY the same: ~$6.25
SELECT user_id, event_name FROM journal LIMIT 100;

-- reads all 5 TiB: ~$31
SELECT * FROM journal LIMIT 100;

One SELECT * "just to look" is like leaving the tap open: the water has already run out, however many glasses you pour.

Dry run: the price before the launch. Cache: a repeat is free

The main survival tool is the dry run: a pass over the query without executing it. BigQuery parses the query, looks at the tables' metadata and returns the exact size of the future scan — free and instantly. In the BigQuery console the estimate appears as you type ("This query will process 1.2 TB when run"), in the CLI it is bq query --dry_run, in the API the dryRun flag.

The Postgres habit of EXPLAIN turns into the habit of a dry run here: there you read the plan to save minutes — here you read the bytes to save money.

The second friend is the results cache. Repeat exactly the same query while the data has not changed, and BigQuery hands back the stored result for free (the cache lives about a day). So "run it once more to look" is nothing to fear; what is frightening is "tweak it a little and sweep the whole table again". What exactly breaks the cache (non-deterministic functions such as CURRENT_TIMESTAMP(), for instance) — we will unpick that in the chapter "Money and scale".

Slots: when counting bytes gets old

The "per byte" model has a ceiling: once the queries become very many, a company moves to the capacity model — it buys slots, units of compute power, and pays for them at a fixed rate however many bytes the queries read. Companies switch when the load has become constant, or when knowing the spending ceiling in advance matters more than pricing every single query. That is an administrator's choice, not an analyst's: the billing changes, the syntax does not.

For you the rule is simple: while the bill is counted in bytes, every extra column costs money; if the company runs on slots, a wasteful query still takes shared power away from your colleagues. Cheap habits pay off under both models.