I don't need your API, just give me database access
I don't need your API, just give me database access
Extreme database collaboration taps the hidden potential of your people
Everyone in your organization should have access to your databases. This level of extreme database collaboration allows your team to more quickly catch and correct data quality and performance issues while reducing the time necessary to learn the schema and write new queries.
Anyone with experience managing databases gets worried when presented with this idea. How you handle security, performance, reliability, business objective alignment, permissions, audit trails, and more, matters immensely. A wrong step could mean catastrophe. Often, many of these considerations are met by placing APIs in front of your database, as Matt Swanson recently hinted:
Today, APIs are everywhere and that’s a great thing because it allows developers to build resilient products. On the other hand, APIs also contribute to organizational blockades which can reduce data quality and operational efficiency.
For example, have you ever had to deal with the wrong data coming from an API? Or an API endpoint occasionally timing out causing your integration to fail? Well, of course you’ll file a JIRA ticket and explain the situation to the API product manager who will put you in touch with the API developers who you’ll explain the problem to again who’ll connect you with the data team to whom you’ll describe the problem again and eventually a fix will be rolled out. Phew! In a large organization this process can take weeks if not months. You tell yourself: “Not too bad compared to the last incident which took even longer!” but have you ever stopped to imagine what this process could be like if you had direct access to the database / the queries / stored procedures running on that database? 🤔
More 👀 = More Improvements
As the number of people who have access to the content increase, the number of ideas, fixes and improvements contributed also increase.
Can you think back to a time when you were trying to learn something new but didn’t have a mentor or even (I’m dating myself) no YouTube videos to explain how to do it? 😅 I can. It can be done but it’s a grind and happens pretty slowly through a series of knowledge / theory / experiment iterations. Contrast that with something for which you had a mentor or teacher. An instructor’s feedback keeps us on the right path so we learn more in less time.
Now take a moment to pause and appreciate how the world has come together to produce mountains of educational content on sites like Wikipedia, YouTube, or various online education platforms. You can learn a language, skill, profession, technique or simple home repair by taking a few minutes to watch a video. A single person couldn’t create all of this content on their own, it took billions of people collaborating together to create these amazing resources.
As the number of people who have access to the content increases, the number of ideas, fixes and improvements to the content also increases — and databases are no exception.
Case Study: Dune
When I first discovered Dune, I was amazed at how they let tens of thousands of people write arbitrary SQL against their open databases. Their community started coming up with creative queries and conclusions from the data. This was the first time I saw such a large database open to the public without the need to get data through a structured API call. Dune had no idea how customers would use the data so instead of creating arbitrary API methods they gave you SQL access and let you create the queries yourself.
This level of extreme database collaboration tapped into the public’s talents to discover hidden value in the data and establish a strong community still going strong today.
Data collaboration is here to stay
Your organization should share these same database collaboration benefits too. Even small companies often have a dozen databases or more lying around which new team members need to discover, learn, integrate and improve. The more people who can review and understand your data, schemas, indexes, and queries the more you benefit from your collective creative wisdom.
At Sort, we’re building a supercharged database collaboration platform for Postgres and Snowflake. Learn more and sign up to get early access at sort.xyz today.