One of the biggest challenges in using Large Language Models (LLMs) in real-world applications is keeping them accurate, efficient, and predictable—especially when they’re interfacing with structured systems and business processes.At Dataception, we’ve fo...

AI is supposed to be the breakthrough that transforms modern enterprises—yet for many, it's turning into a frustrating and costly disappointment.Recent commentary from Dan French on the article “C-suite leaders grapple with conflict, silos amid AI adoptio...

In our journey through graph structures, we’ve covered data graphs, which define relationships between entities. Now, we shift gears to execution graphs—the dynamic frameworks that power computational processes.While data graphs help us discover relations...

Data management often feels like a compass spinning without true North—different teams pulling in opposite directions, governance operating in a vacuum, and customer needs frequently overlooked.To address this, I’ve reimagined the Compass of Confusion—a m...

In the world of business process automation, AI-driven decision-making, and real-time data interactions, traditional homogenized data models and centralized graph approaches fall short. Businesses need a way to query, navigate, and interact with processes...

After years of leading AI transformation initiatives, one truth stands out: successful AI adoption isn’t just about technology—it’s about business transformation, culture, and execution.Many organizations sink millions into AI, only to see projects stall ...

Just had a great discussion with Eddie Short on how GenAI is revolutionizing the journey from idea to industrial prototype—with real UX and business functionality in front of the customer faster than ever before.The emergence of Large Language Models (LLM...

Whoooaaa! We’ve just hit a major milestone.For years, the gap between operational (transactional) systems and analytical (decision-making) systems has been a major challenge. But that divide is vanishing—right now!Using our Data Product Pyramid GenAI tool...

What's the Right Approach to Data Products?Which approach delivers the most value when building data products?🔼 Top-Down: Start with the business use case first. Define the problem before doing any data work. Once clear, find and package only the data yo...

Artificial intelligence is on the cusp of a new chapter: Reasoning Language Models (RLMs). These next-gen models don’t just predict patterns or rely on probabilistic outputs like traditional LLMs—they think.RLMs represent a leap forward, incorporating str...

Just as Gutenberg’s press democratized knowledge, ushering in an era of rapid innovation, AI—particularly Generative and Agentic AI—is doing the same for data and technology. It’s rewriting the rules, short-cutting end-to-end software and analytics creati...

A common question we get asked is: How do we ensure predictability and transparency in AI-driven, agentic workflows?Traditional AI agents often operate within opaque, sequential memory buffers and LLM-generated decision paths, making it difficult to track...

🐾 Agent DOG Deploys AI to Transform Risk Management at Guardian InsuranceAt dawn, Agent DOG infiltrated Guardian Insurance’s Risk Department, scanning the landscape of outdated spreadsheets, slow manual reviews, and fragmented data pipelines.💬 "Your ris...

Mission: Transform an Enterprise with AI & Data Object GraphsIn the vast digital landscape of Data and AI, a new intelligence emerged: Agent DOG (Data Object Graph). Built for one mission—to revolutionize business transformation through AI-driven automati...

Building the PackRunner Architecture: The Future of Multi-Agent AI SystemsFollowing recent discussions on AI agents and Data Object Graphs (DOGs), many have asked how to architect these systems to support multi-agent workflows. The answer? PackRunner—an A...

The Real Role of AI: Speeding Up, Not ReplacingAI isn’t about replacing human expertise—it’s about accelerating existing workflows. I recently had a conversation with a data leader who was struggling with the slow pace of good data design between upstream...

From Prompting to Programming: The Future of AI System OptimizationThe era of manually tweaking prompts to get language models (LMs) to work is rapidly fading. As AI systems become more complex and multi-layered, a more structured and systematic approach ...

Your Data Product Platform is About to Be Eclipsed!The era of traditional, tech-heavy, data-focused platforms is rapidly fading. With cloud-based GenAI solutions like ChatGPT, Gemini, Claude, and others advancing at an unprecedented pace, the way organiza...

Finding Hidden Business Value with Data ProductsImagine sitting with a finance leader, combing through operational data to uncover opportunities for cost savings. Last year, I did exactly that. What started as a deep dive into financial data turned into a...

In the latest episode of the Data Product Workshop Podcast, we dive deep into the advanced capabilities of Small Language Models (SLMs) and their application in real-world text mining use cases. From extracting information to building composable workflows...

In today’s fast-paced business world, solving problems effectively requires more than just gut feelings or assumptions—it demands data-driven insights that inform decisions and guide action. Let’s explore how we can use data products to create compelling ...

Beyond Buzzwords: Unpacking the True Potential of Data in BusinessIn the ever-evolving landscape of data management and analytics, the term “data product” has emerged as a cornerstone concept, underpinning countless discussions and debates. Yet, as we del...

For years, experimenting with large language models (LLMs) and high-performance computing (HPC) has been a playground reserved for organizations with massive GPU server farms and deep pockets. But now, a revolutionary shift is underway. The world’s first ...

Marc Andreessen famously said, "Software will eat the world." Today, this notion can be updated to "AI will eat business." The pressing question is: will AI revolutionize business from the inside out or from the outside in?Inside Out: The Role of Leadersh...

In the fast-paced world of finance, timely and accurate credit rating reports are crucial for banks and lending institutions to manage risk and optimize capital allocation. Traditionally, generating these reports has been a manual, time-consuming process ...

As we kick off 2024, it's time to reflect on a year filled with significant strides in AI and data product development. After taking a brief hiatus over the holidays, I've been working on a solution poised to accelerate data products and AI capabilities. ...

The recent surge in the capabilities of Large Language Models (LLMs) has opened up numerous possibilities, one of which is the ability to query internal documents effectively using minimal computational resources. Our article delves into a novel approach ...

In recent years, large language models (LLMs) have transformed natural language processing, making significant strides in industries such as finance, healthcare, and customer service. However, the standard Retrieval Augmented Generation (RAG) framework, d...

At Dataception Ltd, we've developed a comprehensive architecture that leverages Large Language Models (LLMs) and transformer-based AI models to revolutionize the development, design, and operation of data products. This innovative architecture is segmente...

In the world of large language models (LLMs), transformers have been the dominant architecture, thanks to their ability to handle sequential data efficiently. However, despite their strengths, transformers are not without flaws. Enter Retentive Networks (...

IntroductionThe intersection of sustainability and advanced AI has often been seen as a challenging paradox. However, recent advancements have begun to bridge this gap, proving that efficient and sustainable AI development is possible. This week, we highl...

We have just concluded an incredible three-day workshop on large language models (LLMs) and generative AI tailored to a client business. These sessions are designed to cut through the hype and focus on delivering practical value. Here’s a breakdown of wha...

In this blog post we're going to discuss The Value of Data: Driving ROI and Innovation with Data ProductsIntroductionIn today’s data-driven world, the strategic use of data products is essential for organizations aiming to enhance their business value. Th...

We've been having a lot of conversations about data products and the whole shift-left movement. But what exactly does shifting left with data mean, and why is it crucial for the modern data ecosystem?Understanding Shift-Left in DataShifting left with data...

The modern data ecosystem needs strong, proven foundations. For me, this journey started years ago with my first data product-oriented solution. Long before the advent of data mesh and during the rise of Hadoop, I designed and built this solution with a c...

We often receive questions about data products and the operating models organizations need to truly harness their value. One critical aspect to consider (after deciding what types of data products to build) is their life-cycle—how to effectively build and...

IntroductionOrganizations that can deploy business-focused data products using the OODA loop not only win but thrive in a bear market. This blog post expands on the ideas of data transformation as expeditions and building a business-as-usual (BAU) capabil...

IntroductionThe true value of data products lies in their ability to empower all types of analytics developers, including business professionals, to create, iterate, pivot, and retire end-use-case-focused analytics that respond "at the clock speed of the ...

There’s been a lot of buzz around data catalogues, and rightly so. They are essential tools for organizing and managing data assets. However, despite the advancements over the last decade, there's still a gap, especially when it comes to the needs of deve...

In these challenging times, businesses must rethink their data and analytics strategies to ensure they can react swiftly and effectively to market changes. A concept that has resonated deeply with me is the OODA loop, which stands for Observe, Orient, Dec...

With all the talk about economic downturns, I’ve been reflecting on how businesses can navigate these challenging periods. As a business owner, there are three critical questions I would ask if my revenue suddenly dropped:How quickly will I know?For small...

Engaging in vibrant discussions around data products, it's clear that many organizations are still in the fabrication phase. This means they spend about 80% of their time building infrastructure and laying the groundwork, with only 20% dedicated to actual...

I recently came across an insightful article on the realities of working with AI and machine learning (ML). It resonated with my own experiences, particularly the points that "ML is not software engineering" and "the biggest gains are not from choosing th...

Centralised data platforms have been around since the invention of the data warehouse (created last century by Ralph Kimble amongst others) to provide businesses the analytics capability they need. Since their invention, the industry and its associated da...

In my previous blogs in the Data Lake series, I have outlined what a data lake is and how to get started. In this blog I will outline one of the biggest challenges in getting the maximum value of using a Data Lake, being able /to share it across an organi...

Data lakes, as I said in my previous blog, are the latest buzz word in analytics. While I warned about jumping into the world of data lakes without thinking carefully about what you want to achieve, I do believe that these data depositories are the long-t...