Varan - Instant cross-database SQL joins, run entirely local
by•
Varan is one desktop workspace that fuses DuckDB, MySQL, PostgreSQL, Python and your spreadsheets into a single queryable surface — with cross-source SQL, anomaly detection, lineage and git-style rollback. Now in private beta.
Replies
Best
Maker
📌
So what is special about Varan and what motivated me to build it:
I have seen too much friction in data related fields and tried to solve it by myself. I want to give you a real story that I have faced. I am working for a one of the most respected companies in my country and we had a project going on for almost half a year. I was maintaining the database and backend architecture for the project. So we had several issues about the DB, it was going out of sync very frequently, it was based on an old technology and so on, it was problematic. Therefore, everyday I was getting bunch of data sheets from managers and they were asking me to update the DB according to the sheets, or delete some values, create a report and so on. Naturally, it is not possible for a classic SQL engine to join lets say Excel sheet to MySQL table, and do the operations there. So, it took me several hours everyday just to update bunch of columns according to a data sheet in the DB.
I got so frustrated that I started to build Varan, which allows you to join anything to anything totally locally! You do not depend on any server, and all you need is read access to join 2 or more unrelated sources. Now it allows me to join lets say MySQL users table to Excel sales table in just seconds, and I can update, delete and do any other write query on any of them if I have write access. It literally shrinked hours of work to minutes, and even seconds!
Another issue I have seen was how fragile DBs are to data loss. Forget the where clause, delete the whole table in the DB and pray that you are not getting fired. Even if you have backups, it can be inconsistent, and take you hours to recover.
I thought that it should not be that much of an issue, as everyone can make mistakes, especially juniors or interns. So Varan supports git like system, where every write query is committed and you can go back to any commit at any time in seconds! Lets say you accidentally forgot to select where clause in your SQL editor, deleted the whole table. No worries, open the versions tab, click where you want to rollback, and your original table is back! With this feature you can also see who run what query, all the mutations and so on with 3 views: timeline, git-like strings and DAG. It allows better data governance.
Above were the main features, and additionally Varan also has built in Python and py editor where all your tables are data frames on demand automatially. Also all tables are scanned for mistakes and scored on open automatically, to prevent your mistakes before it ever happens.
Report
Love the idea of querying spreadsheets alongside Postgres without copying data around. One thing I'd want before fully committing is a way to schedule and email query results as snapshots, so non-technical teammates can get the same numbers my Python pipeline sees without logging in.
Report
Finally a tool that lets me query Postgres and a CSV in the same window without exporting anything. The git-style rollback on schema changes is a really smart touch for anyone who breaks prod at 2am.
Report
Finally something that lets me query postgres and a random csv in the same place without copy pasting into a notebook. The git style rollback is a really nice touch for when my one-off queries inevitably break something.
Report
Big fan of the cross-source SQL idea, one thing that would really help is native keyboard shortcuts for switching between query tabs without reaching for the mouse, plus the ability to save snippets per connection so I don't keep rewriting the same joins.
Replies
Love the idea of querying spreadsheets alongside Postgres without copying data around. One thing I'd want before fully committing is a way to schedule and email query results as snapshots, so non-technical teammates can get the same numbers my Python pipeline sees without logging in.
Finally a tool that lets me query Postgres and a CSV in the same window without exporting anything. The git-style rollback on schema changes is a really smart touch for anyone who breaks prod at 2am.
Finally something that lets me query postgres and a random csv in the same place without copy pasting into a notebook. The git style rollback is a really nice touch for when my one-off queries inevitably break something.
Big fan of the cross-source SQL idea, one thing that would really help is native keyboard shortcuts for switching between query tabs without reaching for the mouse, plus the ability to save snippets per connection so I don't keep rewriting the same joins.