Introducing our new utility to get you from YAML to AWS parameter store.
Why
One of the drawbacks with Spring Cloud Configuration Server is that the server needs to be running before applications can be spun up. As we have become more cloud native on AWS we’ve wanted to move to AWS centric configuration systems, but to do that we needed a path from the existing git version control system (VCS) based config server.
So what we were missing was an easy conversion to AWS Parameter Store from Spring Cloud Config.
How
We liked Spring Cloud Config Server for many years, as it provided the following benefits:
- git Version control with encryption-at-rest for application config.
- a single point of control for all applications as we could set global configurations that affected all applications deployed.
- A very simple bootstrap.yml file for startup without having to specify a lot of configuration.
We use Spring Cloud AWS (now awspring.io) libraries in a lot of our applications, and the support for both AWS parameter store and secrets manager are now baked into a spring boot starter.
A quick experiment showed some benefits for going to AWS parameter store based config
- configuration always available without remote hosted config server.
- use of secureString could replace our encryption at rest with config server
- bootstrap is even simpler with just the application name required.
- still supports “global” spring application configuration, which we use a lot with Jackson.
We like having our application config in git, as this gives us a simple code on branch, review and merge process using bitbucket. This was the only drawback with going to AWS PS, but surely could be solved with some code.
We’re in a slow move to serverless, so any chance to remove the need for a low utilisation server gets us a step closer to no clusters.
Result
Our code and how to use it: https://bitbucket.org/limemojito/yaml-to-param-store.
So we are pleased to announce a small Open Source java jar that allows you to convert a single or a directory of yaml spring configuration files to AWS parameter store following the path and naming convention for Spring Cloud AWS. It included support for spring profiles conversion, AWS tagging the parameters and updating changed or new values on repeated runs. The command line tool does NOT delete parameters, though the code has support for removing an application by name including all of its profiles.
We have configured our own build server to checkout the configuration server repo, and run our tool over the yaml files to keep them in sync with parameter store.
Details on usage is available on bitbucket at https://bitbucket.org/limemojito/yaml-to-param-store.
For more information on using parameter store with a boot application, please see the configuration steps using Spring Cloud AWS in your Spring Boot application.