Evaluating and Selecting Technology
The best of times and the worst of times
There has never been a diversity of new database and data handling systems on the market as there is in 2017. It seems every week I hear from somebody about a new graph and/or document database. This excitement is tempered, however, by a profound conservatism, which in turn derives from the central role of the database in an online system and the difficulty of replacing it later. Thus, we still see people attack graph and document problems with traditional databases such as PostgreSQL since this gives clarity as to reliability, performance, and other "non-functional" requirements such as backup.
A database, and the structures that you construct in the database, can be the most difficult thing to change, so you owe it to yourself, investors, and other stakeholders to get a second opinion from an experienced engineer who has seen a variety of projects through all phases of their lifecycles.
Steps of a Technology Evaluation Project
My experience with hundreds of database applications is that the most of the problems that people have as they build, grow and scale and scale database applications come from a short list of causes. Thus, you and I can work through your requirements on a checklist and from that create a checklist of important unknowns about a proposed solution.
Most companies are looking for something between a least-cost and a least-risk solution to be a platform for their ideas. Others are looking for something special from their database that will be part of their unique selling proposition. Either way, in addition to the generic issues on our checklist, it is also important to determine requirements that are unique and essential to your application.
A go/no-go decision is made at that point, and if it is "go", the next step is prototyping an application based around the database. First, it is a matter of turning some part of your conceptual data model into a concrete data model and based on your product of choice. Most database applications involve, to a varying degree, both the use of a framework and of repetitive "boilerplate" code that follows both certain conventions. It's important that these conventions are easy to write, easy to understand, and easy to maintain, as well as designed to prevent common errors such as script injection and null pointer exceptions. The prototype phase is a great opportunity to flesh out exactly how you'll access the database with a set of unit tests, free from the distractions of implementing a user interface.
A go/no-go decision is also made at the end of the second phase, before possibly moving to a third phase, testing, in which the assumptions made so far are challenged. First, it is necessary to create a generative model of the data which will fill you database, so that the database can be stress tested with any required volume of data. After that, running a heavy workload against the database will reveal the challenges involved in running at production scale and help estimate, ahead of time, what the cost of running your database will be, be it in either the data center or the cloud.
Managing business and social relationships
On top of strictly technical concerns, we can help you make the best of the business and social relationships that come with your database choice. In some cases, you may be dependent on a vendor for support (particularly in the clase of a cloud database product such as Amazon's DynamoDB.) In other cases you'll need to work with the open source community around a free database, or have the option to supplement community support with a contact with a vendor such as DataStax or Pivotal.
A coworker I met early in my IT career taught me that managing relationships with vendors is key to getting what you pay for, thus that is an area where I've built up skills. Working with either established vendors or open source community, my ability to get trust and be taken seriously by both technical and management people can be an ace in the hole when it comes to getting the support you deserve.
Make risk disappear
An Ontology2 engagement to evaluate a database or similar tool begins with an interview process to characterize your proposed application, working through our requirements checklist. This can be done on-site or via telephone or teleconference. The second phase, prototyping, is best done together with your technical stuff, often with sessions of team programming. Execution of the third phase can vary, depending on if you are using an open source or commercial database, or if you plan to run this database on premise or the cloud.
Write to inquiries@ontology2.com or call (607) 539-6254 to get started.