BigQuery
5 articles in the last 30 days.
-
What's new in Managed Service for Apache Spark 4.0
Managed Service for Apache Spark 4.0 is now generally available, with a new runtime on Spark 4.2, Java 25, Python 3.13, and Iceberg 1.12, plus faster startups and VS Code and Claude Code integration.
-
How Vodafone is using the Data Engineering Agent to improve their data pipelines
Vodafone Czech Republic used the Data Engineering Agent to convert 2,000–3,000-line BigQuery stored procedures into modular Dataform pipelines, halving pipeline execution times.
-
Accelerating Polars + BigQuery pipelines with the polars-bigquery package
The new polars-bigquery package, built on Google's Rust libraries, loads BigQuery data into Polars 3x to 4x faster. Its eager read_query and lazy scan_table methods push filters and column selection down to BigQuery.
-
How Virgin Media O2 cut compute costs 30% by giving dbt a memory
Virgin Media O2 describes how dbt v2 and dbt State cut its 1,424-model Nutcracker job from about 1h53 to 1h13. BigQuery slot use also fell from roughly 4,500 to 3,400 per hour.
-
How to put generative AI directly into a dbt pipeline with plain SQL
BigQuery's AI.GENERATE and related functions let you call an LLM from a SELECT, with output_schema for typed JSON. Wrapping them in dbt incremental models with is_incremental() limits how many rows reach the model.