Lightdash & dbt metrics

Lightdash was the first (and still only?) BI tool to support dbt metrics.
So how did we get here?

‍

A new source of truth

‍

Our experience was that BI tools were not friendly to modern analysts. We saw team after team ripping their transformations out of Looker and other tools to pick up dbt. We saw analysts getting more technically savvy and gaining independence from engineers.

But the workflow was still broken. You’ve probably read Tristan’s infamous post “How do you decide what to model in dbt vs lookml?”

Time-strapped analysts didn’t want to maintain logic in two places; it doesn’t follow DRY principles and it causes unnecessary work. Folks were crying out for syncing between dbt (now adored) and their BI tool (continually frustrating).

Have you ever had a merge conflict in git? Then you know that keeping stuff in sync is a terrible headache. So we set out to fix the analytics engineering workflow using an age-old (2019) engineering paradigm:

“Don’t sync state, derive it!”

In other words, instead of writing logic in our BI tools AND dbt and keeping those in sync, what if we treated our dbt projects as a source of truth, and derive what we need for BI from there?

The most amazing part of this new world was that dbt already had most of the information we needed. We just needed metric definitions.

‍

A wild BI tool appears…

‍

So dbt2looker was born as an experiment to write metrics in dbt and it clicked!

A week later we were building an MVP for Lightdash, a BI tool that would fit the modern analysts workflow and integrate with the tools they love. The vision for Lightdash remains to create a lightning fast development loop. Making it easy to switch from looking at raw data -> to transforming it -> to serving it up for end users.

‍

Metrics become official

‍

All of a sudden we found ourselves in the middle of peak data hype around a brand new “metrics layer”. It really felt like overnight. Building and defining metrics went from a curiosity to the future of the data stack with Benn Stancil declaring there existed a “Missing piece of the modern data stack”.

It was only a few months later that metrics moved from a wild idea to a first-class citizen in dbt, which must have created the hottest Github issue of all time. Metrics in dbt brought a tonne of benefits like showing metrics in dbt documentation and kicked off a new wave of metrics standardisation.

All this culminated with the official launch of dbt metrics and an all new way to define metrics as code and visualise them with Lightdash. We’re really excited to see dbt’s metrics evolve into an open-standard.

‍

The future of metrics

‍

We believe modern analysts will do more with their raw data to power internal reporting, interactive data visualisations, embedded customer facing analytics. At Lightdash we’re creating tools for analysts to take this new technology and create amazing data experiences with much less effort and more reliability.

Ready to get started?

‍