Manual vs Automated testing of PLC logic

PLC logic testing and validation is one of the important steps in control system design and development as it significantly reduces logic errors and improves application safety and reliability. However, variations in practice among PLC engineers exist, as some don’t conduct tests and rely on commissioning, while others take it very seriously and use a number of tools to validate PLC logic to increase quality of their work and reduce time spent during commissioning stage.

If you are planning on PLC logic testing upon completion of the development, one of the primary decisions you’ll have to make is if you’re going to do manual testing or use automated testing:

• Manual testing is considered when the tester manually toggles and forces PLC bits and words and/or presses push buttons on partly energised equipment.
• Automated testing is used when the tester creates a simulated environment using tools such as PLC Logic Prover, SIMATIC S7-PLCSIM, Beckhoff TwinCAT 3 PLC, separate PLC with testing logic, etc.

While neither of these options are technically better or worse than the other, the size, budget and timeline of a project will certainly be factors that affect which method will work best in your testing procedure. Understandably, in small and simple projects PLC logic is tested during the commissioning stage. However, there is a simple rule of thumb: if the project has more than 20 IOs, PLC logic testing prior to commissioning is essential.

Below, I’ll outline the best times to use each, and the disadvantages and advantages of automated and manual testing.

Manual testing is essentially self-explanatory — testing of a PLC logic is done manually, by human action. This means that someone uploads a PLC program to an actual equipment (or an emulator) to evaluate numerous components including logic integrity, functionality, and performance by pressing buttons on the equipment, adding temporary “jumper” wires and/or toggling and forcing PLC bits. Sometimes, engineers use the terms “live testing” or “factory acceptance test” with manual testing interchangeably.

Advantages

The method of manual testing is inherently valuable to OEM at the final stage because the tester is able to have the full user experience, making it easier to assess visual components such as panel LEDs and HMI (SCADA) indication. Also the tester is able to perform full commissioning of independent subsystems of the equipment. These subsystems usually don’t require integration with other systems of the application.

Additionally, most QA processes will include some form of manual testing because it’s a part of factory acceptance tests which are often required by clients.

Disadvantages

One key limitation of manual testing is that it’s not the best method for PLC logic that has frequent repetition. An example of this is alarm logic for switch-type protection devices. Additionally, manual testing would take an exponential amount of time for logic that needs to be redone frequently after client’s reviews. If the tests aren’t repeated, the changes could break logic integrity which has been manually tested earlier. Especially if changes are done after the commissioning (see the graph below).

Furthermore, manual testing with simulated logic in the production PLC program requires additional attention. The created simulation logic will pose high safety risks, if left enabled during the production.

Automated testing uses the assistance of tools and software to perform test cases by repeating predefined actions. These predefined actions usually simulate activity on Analog and Digital inputs and specify expected behaviour. You can perform test automation by writing test logic in a separate PLC or using PLC logic testing tools (PLC Logic Prover, SIMATIC S7-PLCSIM, Beckhoff TwinCAT 3 PLC, etc.). All automated testing, however, requires some manual testing at first to create an initial testing case.

Advantages

The biggest draw of automated testing is that it’s dramatically faster than manual testing. This is because after simulating PLC inputs, the automated testing tool checks expected behaviour and instantly moves to the next testing action. In some cases, automation allows multiple tests to be run in parallel.

Another key advantage is the timing of testing. With automated means, testing can be started on the early development stage and automated test cases can be created as soon as the project is started. The first run of a test case can be done after a day of PLC logic development. This is beneficial as this also encourages the PLC engineer to develop logic which is easier to test.

Additionally, automated testing is preferred for large projects that have a number of variations during the development. Specifically, the engineers will be able to retest logic after it has been modified due to new requirements. While manual testing is still applicable in this case, it is not advised as it would be very time-consuming.

Disadvantages

The downfall of automated testing is that, unlike manual testing, there is little human observation possible. While automated testing is more efficient in many ways, it runs too fast for checks on HMI and SCADA during the PLC logic test. SCADA and HMI indications will be flickering with high speed and human observers would not be able to accept the visual testing. Despite this, alarms and event logs will still be generated and tested.

For small projects, automated testing is relatively expensive because it requires additional resources. However, if your project calls for automated testing, such as high safety requirements, this investment will most likely pay for itself and be more cost-effective over time.

Whether you’re deciding to pursue a manual or automated testing strategy, there are a number of ways to make the process faster, more efficient, and more accurate for your QA team.

The key benefits of automated testing are action logs (test results) and exports of the test cases. The action logs can be used for reporting purposes. Reports are useful documents to supply to your clients and also a great tool to track the results of previously completed tests. On the other hand, exporting the test cases will help in creating a library of test actions which can be imported in other projects and reused by other engineers in your team. This will effectively reduce time spent on the development of test cases.

Overall, the success of your testing process will depend on your ability to measure which testing method is best for your project. By evaluating the pros and cons of manual testing and automated testing, their best uses, and what you would like to accomplish through testing, you can launch a solid strategy that will continue to improve your PLC logic.