The new buzzword for DevOps is “continuous.” We have continuous integration, continuous delivery, continuous obsolescence and for many developers, continuous consumption of Five Hour Energy. Yes, the world of DevOps has radically changed development as we know it and will continue to do so for the foreseeable future.
For those of you just starting down the DevOps journey, the process can be overwhelming. In this post, I will highlight five points to consider when implementing test data management for DevOps.
User Acceptance Testing
There are many stages of testing required when launching new applications. The speed and agility that comes with DevOps means that you need faster and more efficient test cycles. Historically, the most robust test process comes with User Acceptance Testing (UAT), but these tests occur so late in the cycle that finding bugs and implementing fixes can significantly lengthen development times.
The right strategy is to test more thoroughly earlier or “shift-left” in industry parlance. By performing more robust tests earlier, you can find and fix bugs well before you get to UAT. The key enabler to enhanced UAT is the ability to test on full production datasets with close to production performance earlier in the cycle.
Supporting Very Large Databases
Many customers have huge databases, and development teams are often tasked with creating applications that support these massive data constructs. The challenge is that it is difficult or even impossible to provision test copies of these critical data assets in a timely manner.
As a result, many developers are only given access to a subset of the required databases. While this can be beneficial from an operational and storage usage perspective, it can limit the effectiveness of both development and testing since the teams are using a data set that is not reflective of the actual information. An optimal test data management strategy will allow developers to generate copies of huge production databases needed to achieve development schedules.
Developer Self-Service
In order to deliver on the benefits of agile development, developers need access to database and development resources in a rapid and efficient manner. While consistent processes are critical for information protection and control, these same processes can create delays in provisioning or updating development environments negatively impacting developer productivity.
As part of a self-service infrastructure, users need to think about automating development environment creation and the creation and maintenance of test data. This means that developers should have the ability to easily provision and update copies of all databases required for development activities.
Integration with DevOps tools
There is an incredible ecosystem of solutions available in the market as illustrated in the Periodic Table of DevOps Tools. The key takeaway is that there a ton of both open and closed-source tools to help you in your DevOps journey. These offerings can perform a variety of functions including repository management, automated testing, security, environment orchestration and many others.
DevOps tools can significantly automate and streamline development, but they need to be integrated into a complete solution that simplifies the entire process. These integrations are enabled through Application Programming Interfaces or APIs, and you should carefully inspect the API offered by all solutions to ensure that it offers a comprehensive feature set. Remember that you also need to consider how you can automate the provisioning and presentation of test data.
Broader business requirements
Finally, as you wade in DevOps processed, you should not forget the importance of other related business processes like data protection and disaster recovery. You probably have a mature process to address these with your traditional applications, but what about your development environment? Do you feel equally confident with those protection strategies?
Ideally, you should incorporate data protection into the development process at inception, and choose a solution that provides this technology in an integrated fashion. While this might not always be possible, at a minimum, you should consider the protection implications of whatever offering you select.
In summary, implement DevOps processes can be challenging for organizations of all sizes. Test Data Management is one part of the problem, but its impact can be amplified because of the large volumes of data involved. Considering these five points early in the process of designing your DevOps strategy can significantly streamline the process and avoid future pitfalls. As you are thinking about test data management, I encourage you to check out Actifio’s offerings which can completely automate the test data management process.
If you found this article helpful, please share.
Planning you DevOps Strategy?