DuckDB Labs recently released DuckLake 1.0, a data lake format that stores table metadata in a SQL database rather than ...

We are looking for a Junior Data Scientist who doesn’t just "do data," but builds intelligent systems. In this role, you won't just be analyzing the past; you will be building the future of customer ...

Definity raises $12M to embed AI agents inside Spark pipelines, catching failures and bad data before they reach the agentic ...

Spark Declarative Pipelines automate flows for batch and streaming data, while Lakeflow Jobs coordinate tasks from SQL queries to machine learning model deployment, supporting streaming tables, ...

A GitHub project now offers an Azure Databricks medallion architecture pipeline built with PySpark, Python, and SQL. It processes e-commerce data through Bronze, Silver, and Gold layers, adding ...

Google Cloud introduced a new AI agent platform, updated data architecture, and eighth-generation TPUs at Next 2026.

Google's Agentic Data Cloud rewires BigQuery, its data catalog and pipeline tooling around autonomous AI agents — not the ...

Zaharia began building Apache Spark as a doctoral student at UC Berkeley in 2009, a faster alternative to Hadoop MapReduce, which had become the default framework for large-scale distributed data ...

Meta Platforms (META) introduced its Muse Spark AI model on Thursday, the first model from its Superintelligence group and one aimed at products. “Muse Spark is the first model in our new Muse series ...

Llama is out; Muse Spark is in. Meta has unveiled its rebooted AI efforts with a new model it calls Muse Spark. Unlike Anthropic’s Claude Mythos Preview announcement yesterday, Muse Spark is actually ...

Databricks is introducing a security information and event management service called Lakewatch. The privately held company sees an opportunity to challenge mature cybersecurity vendors using ...