DecisionBox
Autonomous AI discovery on your data — open source
Start new thread

DecisionBox for Databricks - Connect DecisionBox to your Databricks to validate findings

by•
Connect DecisionBox to your Databricks workspace. The agent writes its own SQL, validates every finding against your data, and ships a ranked backlog — no prompting. Read-only, Unity Catalog–scoped. Works with Serverless, Pro, or Classic SQL warehouses. Open source, AGPL v3.

Add a comment

Replies

Best
Maker
📌
Hey PH, Selçuk here from DecisionBox, If you're on Databricks, your lakehouse already has the data. Unity Catalog tells you what's in it. A SQL warehouse will run anything you ask. The hard part is figuring out which questions are worth asking, and getting validated answers without dedicating a team on it for weeks. That's what we built DecisionBox for. Connect it to your workspace, point it at a domain pack, and the agent writes its own SQL, validates every finding against the data, and ships a ranked backlog of insights and recommendations. No prompting, no question-writing. Today we're adding Databricks to the list of supported warehouses, with the same posture your security team already lives with for dashboards and dbt jobs. Read-only, scoped by Unity Catalog. The agent connects with a principal you choose, typically USE CATALOG and USE SCHEMA on what you want exposed, SELECT on the tables you opt in to, and CAN USE on the SQL warehouse. Unity Catalog is the boundary. The agent cannot reach anything it hasn't been granted. Runs on the SQL warehouse you pick. Serverless, Pro, or Classic, whichever size, whatever Auto Stop you've set. Every cost guardrail you've already configured for your dashboards and other workflows applies to the agent the same way. The whole Databricks provider is in the public repo, AGPL v3 — the Unity Catalog reads, the OAuth flow, the SQL the agent writes. Same agent runs against BigQuery, Redshift, Snowflake, Postgres, and MSSQL too. If your stack moves, your DecisionBox install moves with it. Happy to dig into the service principal setup, best practices, or anything else in the comments.

Congrats on the launch! The part where it validates the SQL against real data before surfacing the insight is something I keep having to rebuild. Good luck with the Databricks rollout.

Maker

 Thanks a lot for your feedback! 🙏🏻 Would love to hear more if you have further comments & feedback.

Are you seeing teams self host this inside their Databricks VPC, or is it mainly used as a local dev tool?

Maker

 Great question, and it's the more interesting of the two patterns by far.

Both happen, but they're different stages. Local dev tool is how most people first try it - clone the repo, docker compose up, point it at a warehouse, see what it finds in 20 minutes. That's the evaluation path.

Self-hosted inside the VPC is where teams actually run it. The whole thing is built for that: Helm charts and Terraform modules in the repo, all pods run non-root with read-only filesystems, MongoDB is the only infra dependency. You deploy it next to your other internal services and it talks to Databricks over the network like anything else in your VPC.

The Databricks connection is read-only, so the agent only sees what the access token's grants allow. Nothing leaves your environment - the agent runs in your cluster, the SQL runs on your warehouse, the findings land in your MongoDB. The only outbound call is to whatever LLM provider you've configured, and if you don't want even that, you can point it at a self-hosted model via Ollama or vLLM and keep the whole loop inside your perimeter.

So: local to try, in-VPC to run. Happy to walk through the Helm setup if useful.

Congrats on the launch 🔥

Maker

  Thank you, Yuliia! 🙏 Means a lot!

validation step is what caught my eye here. Text-to-SQL is easy to hallucinate, but actually verifying the findings against the live data before shipping the backlog is a brilliant touch.Good job team 👏