DEV, QA and PROD – Old Fashioned?

When I first started in IT, there was a tried and true process of Development, Quality Assurance and finally Production. This meant that there were three different environments used to test scripts, applications and processes. In the day of cloud and hosted apps, is this anachronism still valid? Yes, absolutely. The issue now is not relevancy, but more consistency or how well the DEV, QA and PROD environments match each other.

First, let’s briefly describe what each environment should consist of as well as any relevancy for PowerShell.

PROD (Production) – This is our ‘live’ environment. Systems involved here are all end-user applications, security and networking devices, directory services and major applications like databases, email and communications. Anything introduced here should have been tested in Dev and QA. This means any hosted or cloud based apps that may be affected by the changes or additions to the environment. For PowerShell, this means any scripts whether or not they make changes. The script should be pre-vetted.
QA (Quality Assurance) – This environment should closely reflect the production environment as much as possible taking into account budget constraints, personnel dedicated to this and data volume in PROD. However, the closer it can be maintained, the more accurate testing in this environment will be. Too many times the QA environment is left to pasture and not maintained consistently. This produces inaccurate results and possible impacts on the testing process.
DEV (Development) – Now the DEV environment is a bit of a hard egg to crack. While Dev needs to be close to QA, it does not have to be a complete duplicate. Now that may not be true of system which can be tested on a smaller set a resources, objects, etc. Some systems can be easily replicated and tested in DEV in a similar fashion to QA. However larger systems, or more integrated systems, may require a smaller footprint to keep costs down, but development as close to par as possible.

PowerShell – Dev, QA and Prod
PowerShell is the perfect candidate for this type of framework as it does require an environment or two to validate a script and the way it interacts with an environment. Testing in Dev and QA allows for the author to build in error checking, logging, event logging and to validate the cmdlets do what one expects on a variety of objects in an environment. Using DEV can help weed out a lot of the bugs, help provide ways to add error handling and more. Once the script includes all of the needed features and enhancements, it can then move over to QA for polishing.
What I have found is that QA and DEV are also the place to take an existing script – for example an Exchange Prerequisite installer, Exchange Health Check script and Mailbox Move management script – and test out wanted features like additional logging, better error handling (error messages go to a log file in detail) and more. Then these new features can get approval at the end and put into production.

Cloud and DEV/QSA/PROD

But what about the cloud? Yes. Still relevant. For example, most companies with cloud systems (i.e. Exchange Online, Skype and SalesForce) still have components that PowerShell can touch via a remote PS session to the cloud resource or the local resource that may not have been migrated. This having a QA and/or DEV environment to test the scripts is invaluable. With Office 365, creating a second tenant is a matter of budget and then creation. Similar to SalesForce which has a sandbox environment.
We can create scripts that interact with both environments, report on both environments and more. It is recommended to have at least a QA tenant for testing. The environment does not have to 100% reflect the PROD environment in terms of users and licensing, but it should be a good enough sampling as to be a good environment to test with.

Practical Example

Now, what wouldn’t this article be without a relatively practical example. I have a script that I wrote that is used for mailbox moves. The script has many moving parts, but none of them were production ready on day one. Below is a chunk of code as it progresses from DEV to QA to PROD.

Exchange Archive Mailbox Sizes

This code sample will pull the basic information on the sizes of mailbox archives in Exchange or Exchange Online:
[sourcecode language=”powershell”]
Get-Mailbox -Archive -ResultSize Unlimited | Get-MailboxStatistics
[/sourcecode]
Short. Simple. Works perfect in your DEV environment, assuming you have archive mailboxes to test with in DEV. Now we move to QA to development some more requirements as we have more data and items to query in QA. However, we get a few errors while running the cmdlet. A server was down, or maybe a database was dismounted, all for testing. Now we need to build some basic error handing to the cmdlet:
[sourcecode language=”powershell”]
Try {
Get-Mailbox -Archive -ResultSize Unlimited | Get-MailboxStatistics
} Catch {
$Log = 'There was an error running the Get-Mailbox -Archive request. Please investigate'
}
[/sourcecode]
The above code works great in QA and provides the level of logging required for the project. If we were to take this code to PROD, it would work as expected. However, as in my case, after a few trials, reiterations and tweaking of real live data showed, more changes were needed. This means taking the code, putting it back into QA and testing the additional requirements.
[sourcecode language=”powershell”]
Try {
Get-Mailbox -Archive -ResultSize Unlimited | Get-MailboxStatistics | Select-Object DisplayName,TotalItemSize
} Catch {
$Log = 'There was an error running the Get-Mailbox -Archive request. Please investigate'
$Log = "$Date,Error message – $_.Exception.Message"
}
[/sourcecode]
Notice the change to the statistics, we are pulling only two items, instead of everything. Then an additional line was added in the Catch which will provide us with the real onscreen error instead of hiding it. This error message could help troubleshoot the issue if one occurs. Again, after testing in QA, this would then be put in production.

To be honest, this may not be the best example of a DEV, QA and PROD environmental setup, however, there are enterprises that do require this level of testing still. Even a benign Get command needs to be validated in QA before going to PROD.