data governance

Avoiding Entity Key Drift in a Data Lake: Step 2, When Fuzzy Matching Stops Working

Data lakes often suffer from entity key drift, a challenge that normalization alone can’t fully resolve. Our latest post, “Avoiding Entity Key Drift in a Data Lake: Step 2,” details a critical juncture where fuzzy matching proves insufficient for reliable data cleanup. We initially developed a matcher to address this, but real-world testing revealed inherent limitations. This article outlines the resulting architecture, born from setting aside the matcher and charting a new course.

Buried in Meta’s $18B settlement is a legal pass on kids’ data

Meta’s $18 billion settlement with 29 states includes a notable provision: the continued retention of children’s data for training and testing age-detection models. This represents a significant privacy trade-off, allowing Meta to maintain access to data from users under 13. While the settlement aims to resolve privacy concerns, it underscores the complex balancing act between innovation and safeguarding user data. For a deeper dive into responsible AI development, explore our guide on "How to Work with AI Coding Agents."

AWS Introduces Specification Driven Composition for Flexible Data Workflows

AWS has introduced Specification Driven Composition, a progressive approach to data workflow management designed for flexibility and efficiency. This architecture separates intent from processing logic using declarative specifications and reusable capabilities, enabling validation before execution. Early results indicate significant improvements, potentially reducing dataset onboarding from weeks to days while bolstering traceability, versioning, and governance. For a deeper dive into the broader context of AI-powered workflows, explore our article, "Is Agentic AI Just Automation?".

The Data & AI Leadership Questions That Will Define the Next Stage of Enterprise AI

For leaders translating data and AI strategy into tangible enterprise results, the next phase demands focused attention. We’ve identified the critical questions shaping this evolution – inquiries around agent integration, secure model deployment, and the evolving role of AI in development workflows. Explore these pivotal considerations and discover how to navigate the complexities of enterprise AI adoption. For deeper insight into agent-native platforms, see our interview with OpenAI’s Thibault Sottiaux on TechCrunch.

Instinct’s powerful AI assistant is raising privacy and security concerns

Instinct’s AI assistant is generating excitement – and critical questions – among early adopters. While testers praise its power, concerns are surfacing regarding its extensive access, broad terms of service, and ability to act on users' behalf. This raises important privacy and security considerations as AI increasingly integrates into workflows. We’re closely monitoring these developments, and recognize the need for transparency and robust safeguards. For deeper insights into AI security challenges, explore our recent article, "Alabama launches investigation into OpenAI’s hack of Hugging Face."

AI data giant Alation confirms cyberattack

Alation, a leading provider of data search and AI solutions, has confirmed unauthorized access to its systems following an incident on Tuesday. The company is actively investigating the breach and working to secure its environment. This event highlights the evolving cybersecurity landscape and underscores the importance of robust data protection measures. For further insights into related AI infrastructure developments, explore our article on Ramp’s new AI model routing service, Router. We will continue to provide updates as more information becomes available.

OpenAI seeks to one-up Anthropic with new customer privacy protections

The competition for enterprise AI trust is heating up. OpenAI is responding to Anthropic’s privacy focus with new customer data protections, signaling a direct challenge for leadership in secure AI solutions. This move underscores a growing demand for robust data governance as businesses increasingly integrate generative AI. Explore how these evolving protections impact your data strategy, and for a deeper dive into AI content identification, see our related article, "How to Remove Claude Watermarks from Text, Code, and Files.”

I Thought Loading Data Was the Finish Line. It Was the Starting Point.

Many believe data loading marks the end of a project, but it’s often just the beginning. My recent journey building dbt models illuminated the true meaning of "analysis-ready" data—a concept far beyond simply moving data from point A to point B. Discovering this shift transformed my approach to data management, emphasizing the importance of structured, reliable datasets. If you’re exploring the nuances of data transformation, consider "Before Q, K, and V: Reconstructing the Transformer" for a deeper look at foundational architecture.

Building Trustworthy Snowflake AI Agents with Semantic Governance

AI Slop Is Costing You Hours. Here's How To Stop Sending It.

AI-generated data errors – often called "AI slop" – are silently eroding productivity, costing teams countless hours in correction and rework. It’s a common problem, but not an inevitable one. Explore practical strategies to identify and mitigate these errors, reclaiming valuable time and ensuring data integrity. Discover how to refine your AI prompts and validation processes for more reliable outputs. For deeper insights into leveraging AI effectively, see our article, "Top 5 Claude Skills for Writing (Ranked by GitHub Stars)."

The Medallion Data Architecture: An Introduction

Navigating modern data pipelines can feel complex, but the Medallion Data Architecture offers a clear, practical framework. This guide introduces the Bronze, Silver, and Gold layers—a proven approach to structuring data for reliability and analytical readiness. We’ll explore each tier with a working Python and DuckDB example, empowering you to build robust data workflows. For a deeper dive into related challenges in AI agent memory management, see "Asana's AI agents share memory across your company — but not your secrets."

Asana's AI agents share memory across your company — but not your secrets

Enterprise teams are encountering a common challenge: AI agents capable of responding to prompts but lacking memory and consistency. Asana’s Agentic Work Management (AWM) tackles this, leveraging the company's 18-year-old Work Graph—a comprehensive, graph-based database—to create AI teammates that share knowledge and operate alongside human colleagues. AWM also incorporates robust access controls to safeguard confidential data and dynamically routes prompts to optimize performance, demonstrating a future-focused approach to scalable AI integration, as highlighted by early adopters like FedEx and CoreWeave.

Do Legacy Organizations/Government Have More AI Talent Than AI Problems?

Many organizations, particularly legacy institutions and government entities, possess significant AI talent but face a surprising bottleneck: a lack of foundational data maturity. Discussions often leap to advanced AI solutions like RAG and agent frameworks before addressing core issues—data accuracy, governance, and accessibility. Before pursuing autonomous agents, establishing reliable data pipelines and answering fundamental questions about data origins and ownership is critical. As explored in "Stop Graphing Everything," even seemingly advanced techniques benefit from a solid data foundation.

CareCloud begins to notify hundreds of thousands after hackers stole medical records

CareCloud, a leading health tech provider managing extensive patient medical data, has begun notifying hundreds of thousands of individuals regarding a recent data breach. Hackers accessed one of CareCloud’s protected health data stores, compromising sensitive records. This incident underscores the growing importance of robust data security, particularly as AI increasingly interacts with sensitive information. For deeper insights into securing AI agents, explore our recent article, "NTT DATA AIVista and Snowflake: Identity alone won’t secure enterprise AI agents."

Companies are finally seeing AI ROI — and now they know how much more value it can deliver

Companies are finally realizing the substantial ROI of AI, and the SAP Value of AI Report 2026 reveals just how much further that potential extends. Based on a survey of over 2,600 business leaders, the report indicates AI now supports nearly one-third of organizational tasks, with ROI expectations significantly increasing. However, realizing this full potential hinges on strategic data governance—a challenge many organizations are only beginning to address. Explore the full findings and discover how to unlock transformative value with AI.

Target SVP says its real AI moat isn't the models — it's everything built around them

Target SVP Siobhán McFeeney asserts that Target’s competitive advantage in AI isn’t solely reliant on advanced models, but rather the robust infrastructure built around them. The company’s approach prioritizes deliberate agent deployment, ensuring they address high-value problems and “earn” autonomy through demonstrable results. This framework, encompassing architecture, taxonomy, and rigorous observability, enables scalable AI investment and allows Target to strategically leverage models—from frontier to specialized—for optimal cost-benefit. For deeper insight into agent architecture, explore Microsoft’s recent reference architecture for AI agents on AKS.

At VB Transform 2026, Zillow's engineering chief said AI ROI numbers only hold up if you measure before you build

At VB Transform 2026, Zillow's engineering chief, Toby Roberts, underscored a critical lesson for enterprise AI: establish measurement baselines *before* implementation. Zillow’s experience revealed that context, not just raw data, presents the most significant challenge when building AI architecture to support customers navigating complex real estate transactions. Their solution—a persistent context layer—demonstrates the value of owning this layer, alongside partners like Glean, to streamline workflows and optimize costs by leveraging smaller, task-specific models.

Automatically Assign a Category to Uncategorized Rows in Power Query and DAX

Categorized data is foundational for effective reporting and analysis; uncategorized rows hinder grouping and aggregation. When faced with data lacking assigned categories, establishing rules for assignment becomes essential. This post explores a practical solution for automatically assigning categories to uncategorized rows, demonstrated through a facility management project using Power Query and DAX. Discover how this approach unlocks deeper insights from your data. For further exploration of related techniques, see "TabFM Studio" and its application to spreadsheet predictions.

Could Your AI Systems Already Be High-Risk Under the EU AI Act?

Navigating the EU AI Act can feel complex, but understanding its implications is critical for responsible AI deployment. Could your current AI systems already be considered high-risk under the new regulations? Access our on-demand webinar to gain clarity on the latest guidance and define your next steps for AI governance. We'll explore practical strategies to ensure compliance and mitigate potential risks. For a deeper dive into building a robust AI foundation, see our article, "Many Companies Use AI.