Dataform

Dataform has changed the way that we process and transform data stored in BigQuery and Redshift warehouses - can't recommend this enough for data teams building out complex SQL-based workflows. I like to think of Dataform as an "IDE for the data warehouse", where teams can leverage best practices from software development (CI/CD, Q/A checks and assertions), ensuring that downstream stakeholders are always consuming data of high quality.

I couldn't recommend Dataform more highly to any data team. Dataform has enabled us to fundamentally change the way we work, giving us the ability to easily schedule and perform quality checks on our data. We can then be sure that our data consumers throughout the business are using the single source of truth we have defined. It is an indispensable part of our stack, sitting between our data warehouse and our self service data tool (we use Looker).