Mastering ERP Testing: Implementation Phase & Best Practices

This post is the seventh of our ERP implementation series. If you want to learn more about prior ERP implementation phases, look at:

There are few things that erode an accountant’s credibility and career faster than providing inaccurate information or not meeting ERP implementation expectations.

Give your boss inaccurate data once, and while they’ll be upset, you can probably ride it out and move on. Do it a second or third time, and you might as well start looking for the next job, or at a minimum forget about moving to the next level any time soon. Also, after spending thousands of dollars on an ERP to improve processes, controls and reporting, missing the company’s implementation expectations is not exactly a recipe for success either.

Ensuring that the new ERP is processing transactions and reporting accurately, and implementation expectations are being met is managed through user testing.

This post discusses two testing scenarios:

Report and Data Testing- Ensures that all required reports and their data sources are capturing transactions and reporting data accurately.
User Acceptance Testing- Ensures that ERP functionality as described in the system requirements and design phase is operating as expected.

Testing important ERP functionality is critical to the success of an ERP implementation. Short cutting test steps or using a haphazard testing approach without pre-defined testing scripts, expected results or reporting documentation is an accident waiting to happen.

An implementation process flow example appears below. The process flow consists of five phases. Testing is part of the “Training and Testing” phase.

Note that the training and testing phase also includes testing support. Testing support includes things like copying data and setting up additional test databases. These tasks are usually completed by the ERP consultants and are not discussed here.

Best Practices

This post discusses some best practices that can help ensure data accuracy and in meeting functionality expectations. The post also identifies “pitfalls” which if not mitigated, often result in post cutover issues.

Over the course of dozens of ERP implementations, six top testing best practices have emerged:

Visible Executive ownership
Set up a test database
Use the requirements and design documents as your functional testing base
Prepare a set of test transactions and expected results
Review report designs, run test reports and reconcile for accuracy
Leverage testing to increase training efficiency

Visible Executive Ownership

Visible executive ownership is the number one success driver across all ERP implementation phases. Executive support has a direct impact on the testing process. Review test plans with executive team members and be sure to explain the importance of thorough testing.

Lobby to have an adequate amount of time and resources assigned to the testing effort. Be sure that the executive team understands that shortcutting testing (especially accuracy testing) is risky. Gain a consensus of executive support. Make sure they’re aware of testing cost in terms of both resources time and dollars. If warranted, discuss the negative impact of issues such as YE audit adjustments, and inaccurate reporting.

Set up a Test Database

The usual ERP implementation plan includes the setting up of several system databases (usually four, sometimes five):

Configuration
Testing
Training
Production
Development (optional)

Use the testing database to process test transactions. Build this database from the configuration database after the initial design configuration and sample data/history loading has been completed and reconciled.

Using the configuration database as the basis for the test database ensures that the testing can be completed using the agreed upon configuration, as well as vendors, customers and GL accounts which were previously test loaded, making the test process easier.

It’s nearly impossible to complete all of the required implementation tasks in a timely manner using a single, or even two databases. ERP implementations are based on sets of tasks being completed in a structured order. If you don’t have a sufficient number of databases available, implementation task completion becomes compressed, tasks overlap, compete against each other, and data becomes corrupt. This situation has a decidedly negative impact on implementation quality. Databases are easily set up and copied, so don’t skimp here.

Use the Requirements and Design Documents as Your Functional Testing Base

A lot of time and effort has been spent gathering requirements and completing the Fit/Gap analysis. These documents contain a wealth of information that can be used in building the functional test scripts.

Use the requirements document to identify major processes to be tested. Requirements responses should be able to provide some clarity as to process importance. There is no need to test each and every ERP function. Test the more important processes to keep the testing effort within reason.

Review the sample requirements responses and associated test script examples below to understand the relationship between requirements responses and test scripts.

Use the Fit/Gap analysis to review any Gap workarounds or customizations. Testing Gap workarounds and customizations is an important task as each represents a change to an ERP process or function. Use the appropriate documentation to build the test scripts.

Prepare a Set of Test Transactions and Expected Results

Testing success depends on organization and attention to detail. You can’t successfully test an ERP by merely poking around and processing transactions without understanding what the expected results should be and documenting the results. If you use this approach, you’ll never know if your testing is adequate and if the tested functionality is operating correctly.

Prepare a set of test transactions including the expected outcomes to be sure your test efforts will be systematic and can identify issues effectively. Use transactions from your current system as a base. For example, choose a sample of AP vouchers from the current system and use them as test transactions in the new system.

Using current system transactions provides the ability to know what the expected results should be. Also, since the test database is built from the configuration database, vendors, customers and GL accounts will already be set up.

The example below illustrates this concept.

Notice above how a set of steps is used to outline how each transaction should be processed.

Run your testing and reconciliation plans past your auditors to be sure that they are satisfied and will accept the work papers and process. You don’t want to complete the implementation, then get to year end and have to pay for additional audit fees to get a set of audited financials.

Review Report Designs, Run Test Reports and Reconcile for Accuracy

Financial reporting accuracy is paramount. Accounting departments are expected to provide accurate information to their audience allowing them to make informed business decisions.

During the requirements gathering process, the ERP consultant should be gathering a list of day one reporting requirements and report samples. Day one reports are those scheduled to be developed prior to the new ERP cutover.

The new reports need to be tested and reconciled to ensure accuracy. Reporting inaccuracies can usually be traced to two common issues:

Data differences between the current system and the data loaded into the new system
New report design issues

If the data and history load process is completed properly, data differences should be non-existent. All data and history loading should be reconciled for accuracy.

To gain a better understanding of this process see: Data and History Load: How To Ensure Data Quality.

The more common accuracy issue results from errors in the new report’s design. Most report writer tools today use a combination of report row and column settings. Usually, the reporting rows reflect GL account data, headers and totals. The columns usually reflect data sources (GL and Budget) and calendar data (This year/Last year). Errors in this structure will cause the reports to not reconcile with the supporting data.

When testing reports, be sure report structure documentation is available. This task should be completed during requirements gathering and helps to identify errors more quickly. If you can, let the ERP consultant write a couple of the most important reports. They have experience in report building that you may not have.

You don’t need the entire data conversion and history load process to be completed and reconciled before building and reconciling the reporting. A month or two is all that’s usually needed. Run the reports in the new ERP and reconcile them to the existing system’s reports for the same period. These tasks can be completed long before the cutover, so don’t procrastinate.

I’ve seen many instances where a customer assumes they’ll save money by building the reporting themselves. Unless there’s someone on staff with report writer experience, this goal is rarely met. What usually happens instead is that the report development process becomes over extended, then there’s a rush to get everything completed at the end of the implementation. At this point, the ERP consultant gets involved and any assumed cost savings evaporate.

Keep day one reporting needs as simple as possible. ERP systems have a number of “canned” reports available. Use these reports until any post cutover issues are addressed and you’re comfortable using the new ERP. Canned reports can be downloaded into Excel spreadsheets making them easier to use and distribute in the interim.

Leverage Testing to Increase Training Efficiency

Accurate and detailed testing plans can be used to leverage training efficiency. Use the same testing process steps in the training process. If you set up a separate training database, you can use the same transactions. Just be sure that you don’t set up the training database from the testing database.

Typical end-user training should focus more on the completion of assigned processes rather than a detailed knowledge of the entire system.

Power users and system administrators are usually tasked with that detailed knowledge.

Conclusion

Thoroughly testing the ERP for functional accuracy makes the cutover period go much more smoothly. Adequately tested functions result in an end-user experience that meets expectations. There should be no processing surprises, and everything should work “as advertised”.

Reporting and data accuracy are key. There is nothing quite as embarrassing to an Accountant as distributing inaccurate reports. So be sure that all your reports are built correctly and that they reconcile to the source.

Accuracy and Functionality User Testing