31 Oct 2018 Road to InSpec(t) AEM (OpenCloud)
Adobe Experience Manager (AEM) is an enterprise web content management system that, like many other enterprise applications, is a complex piece of software to set up and configure.
We can’t eliminate this complexity completely, but we can reduce it for many use-cases. AEM OpenCloud is an open source project being led by Shine Solutions that automates the setup of a complete ready-to-use AEM environment in the cloud within 15 minutes.
However, testing and verifying that an AEM installation is working correctly is laborious and time-consuming. Done manually, testing can certainly take longer than the 15 minutes required to actually build the environment in the first place.
Fortunately, automated testing was identified early on in the project as an important part of OpenCloud’s modular design, as is made clear by this diagram created by Cliff Subagio, one of the project founders:
However, it’s one thing to say that testing is important, it’s another thing to actually do it. In this post I’ll talk about why and how we used InSpec to implement automated testing in OpenCloud.
An Example
If you’re still unconvinced as to why a comprehensive and reliable test suite is so important for OpenCloud, consider how we generate AEM machine and container images.
We use Packer to do this (see packer-aem for details), but we still needed to run a bunch of manual acceptance tests afterwards to ensure that the images that are produced are correct and secure. For example: do certain files and directories exist? Do they have the correct permissions? Are certain services enabled, up and running?
Whilst each of these tests seems simple on its own, collectively they add up to a lot of work. Automating them saves us a lot of time and gives us confidence that the AEM installation process is working correctly.
Choosing a testing tool
Having decided we wanted to automate testing, the next question we had to answer was: which test framework did we actually want to use? Our requirements were that it must:
- Be OS independent
- Come with basic operating system tests built-in
- Give us the ability to develop and integrate our own test scenarios
- Be able to make use of AEM OpenCloud libraries
- Have human readable output
- Have a marketplace for easily sharing and consuming test scenarios
After doing some research, we decided to use ServerSpec. ServerSpec provides us with basic acceptance testing, but since ServerSpec doesn’t suit all of our needs we moved forward to InSpec developed by CHEF. Being written on top of ServerSpec and in Ruby, InSpec allows us to
- develop acceptance & compliance tests
- easily integrate our existing ServerSpec tests
- use the existing ruby_aem and ruby_aem_aws AEM OpenCloud libraries
- share and use developed test scenarios on the CHEF Supermarket.
In short, InSpec’s “Compliance as Code” philosophy matched exactly what we need, and let us write test scenarios in a language that is understandable to people. Here’s an example taken from the tests for packer-aem:
describe file("/opt/aem/author") do
it { should exist }
it { should be_directory }
its('mode') { should cmp '00775' }
it { should be_owned_by 'aem-author' }
it { should be_grouped_into 'aem-author' }
end
It should be reasonably straightforward to understand what this is testing. On line 1 we are defining the resource we want to test. Lines 2 through 6 then specify what we are expecting from the resource. Specifically:
- It should exist
- It should be a directory
- Its mode should be 00755
- It should be owned by the user aem-author
- It should be owned by the group ‘aem-author’
When we run this test, InSpec produces excellent human-readable output too:
Profile: tests from author.rb (tests from author.rb) Version: (not specified) Target: local:// File /opt/aem/author ✔ should exist ✔ should be directory ✔ mode should cmp == "00775" ✔ should be owned by "aem-author" ✔ should be grouped into "aem-author" Test Summary: 5 successful, 0 failures, 0 skipped
A check mark for each successful test! Isn’t this easy to understand?
Deciding what to test
Before we were able improve the automation process of AEM OpenCloud by introducing automated testing, we needed to know when and what we want to test. To help ourselves, we categorised the tests into the following main categories:
- Installation (baking) – Packer AEM
- Configuration (provisioning) – AEM AWS Stack Builder
- AEM resources – AEM AWS Stack Provisioner
- AEM OpenCloud (Platform) – AEM Test Suite
Once we determined the main test categories, we then broke them up into components that needed to be tested. For example, for the installation and baking process we knew we had to test:
- Operating system
- AEM Base
- Author
- Publish
- Dispatcher
- Consolidated
- Orchestrator
Now that we knew which components we needed to write test scenarios, we then asked ourselves the question: what would give us confidence that the installation process of that component was successful?
The best place to start was all of the checks and tests that we were doing manually. For example:
- Are certain packages installed ?
- Do certain directories exist ?
- Do the directories have the right permissions ?
- Do certain files exist ?
- Do the files have the right permissions ?
- Is the AEM Author service enabled and running ?
After we had written down all checks & tests, we started automating them. Out first test scenarios were for packer-aem, to check if the operating system and AEM were installed successfully. With those tests in place we were able to verify the machine images.
Next, we implemented checks for aem-aws-stack-builder, which installs the AEM software onto the machine images.
To verify the AEM installation at run-time, we developed the InSpec-Profile inspec-aem, which tests the work done by the aem-aws-stack-provisioner . This checks if particular AEM resources do (or do not) exist, and if certain tasks were successfully executed.
For making sure the AEM OpenCloud platform is working properly, we developed inspec-aem-aws, which includes acceptance, readiness and recovery testing. Since everybody knows security is very important (but sometimes also very complicated to test) we built inspec-aem-security to check the AEM installation for the kinds of security holes described in the AEM Security Checklist, and more security tests are going to be implemented in the near future. Together, inspec-aem-aws & inspec-aem-security together combined to form the aem-test-suite.
Conclusion
With the implementation of the InSpec Test Framework on different layers in AEM OpenCloud, we were able to easily improve the automation process. We developed ~180 different test scenarios, published three InSpec profiles on the CHEF Supermarket and introduced the new Ruby library ruby_aem_aws . There’s much more to be done but we’ve been really happy with how it has gone so far, and are looking forward to continuing with this approach in the future.
No Comments