From the archive
Friday, 20 November
Folded from 34 sources that day.
Sorted by quality and by how many sources covered it, the same for everyone.
-
What's new in Managed Service for Apache Spark 4.0
Managed Service for Apache Spark 4.0 is now generally available, with a new runtime on Spark 4.2, Java 25, Python 3.13, and Iceberg 1.12, plus faster startups and VS Code and Claude Code integration.
-
Scaling Black Friday pipelines with Managed Airflow agents
Google shows how agents in the Airflow console diagnose failed tasks and check DAGs for upgrades, while Data Agent Kit writes and deploys DAGs from an IDE, using a clickstream example.
-
QCon San Francisco 2026: The Future of Apache Iceberg
Daniel Weeks, Apache Iceberg co-creator, outlined the proposed V4 of the table format. It redesigns table metadata for better scalability and more frequent writes, plus new data management and schema evolution features.
-
Hebbia sees improvements in financial research with GPT-6 Astra
Hebbia reports that GPT-6 Astra beat the best alternative it tested by 13.6 points on citing required source types, with 99.5% of cited excerpts relevant. It also improved presentation output.