The right tool for the job: data store

Intro

The possibilities for software development have changed tremendously over the past couple of years. Actually, it’s better to say that the number of options has increased tremendously. With the final breakthrough and continuous expansion of the cloud, the supply of different types for resources like compute and data store is larger than ever before. And while you can even find a decision tree for selecting the right type of compute to use in for instance Microsoft Azure, selecting the right type of data store to use for your application is no simple feat. This article tries to give some background on why this is the case, and to give you a direction for making that choice.

The good old days

Back in the good old days the choice where to put your data was simple: use a relational database. The reason for this was both practical and limiting; in most cases the only type of data store that was available, was a relational database. This was true both in software and in the knowledge of the employees. *

Because these databases used to run on on-premises hardware, the first solution to any kind of performance problems used to be to get more or faster hardware. When bringing your relational database to the cloud, the model is different. You cannot solve performance issues by simply giving it more or faster hardware. Well, you can if you scale up to a more performant tier, but with that come higher (recurring!) costs.

Different data store types

Relational databases aren’t going out of style. Moreover, they’re still immensely popular and you can use them, too. But for the right type of data. To determine what the ‘right type of data’ actually is, you can see documentation like Choose the right data store and Criteria for choosing a data store to give you some pointers.

Next to relational databases, these are the different categories of data store types we can determine:

• Document database (NoSQL)
• Key/Value storage
• Graph database
• Search engine database

Modern business software processes more and more data, from both internal and external sources. This gives us diversity in the characteristics of the data and the way this data needs to be processed. Because of this, it’s often better to use different data store types, depending on the shape of, and need for, each set of data. Systems that use multiple data tore types use what’s called polyglot persistence.

Consequences

Does this mean you have to go back to the drawing board? Do you have to revisit the entire data structure of your application? Fortunately this isn’t the case in most situations; looking critically at what’s currently there can already give you a lot of insight. By searching for types of data that would better fit a different type of data store you can start dissecting and optimizing the data structure of your application.

Be sure to use technical solutions that help you switch out classes that implement connecting to different data store types. You can do so by using Dependency Injection, for instance. This way, you don’t tie in too hard to one specific type of data store.

Conclusion

The choice for the type of data store is a fundamental design choice. The first and most important step is to determine the category of data store that fits best. As soon as you’ve made that choice, search for the best type of data store within that category. Factor in things like costs, available features and the ease of maintenance.

And if, for some reason, you make a bad choice; don’t sweat. One of the nice things about the cloud is that it’s just as simple to tear down a data store as it is to setup a new one. 😁

Want to discuss on the topic? Contact me!

* Disclaimer: of course, some companies had more options than just a relational database. And yes, there were employees that for sure had knowledge of more types of systems than just such a database. But the answer of the vast majority was simple; “Data? That goes into our database.” And for a large part this is still true.

• azure