PowerSlim vs Fitnesse vs SpecFlow

PowerSlim Fitnesse (SLIM) SpecFlow
Web Access GreenCheck_16 GreenCheck_16 Delete_16x16
Test In/Out WYSIWYG WYSIWYG Gherkin scripts are used to generate test fixtures. Actual test results you can view by using Visual Studio Test View interface.
Verification Code Verification done by framework itself. You just need to provide the results. Verification done by framework itself. You just need to provide the results. You need to duplicate your specification statements again for underlying testing framework (MSTest, NUnit)
Built in Fixtures GreenCheck_16 Delete_16x16 Delete_16x16
Visual Studio Integration GreenCheck_16
Force HLT GreenCheck_16 Delete_16x16 Delete_16x16

Test In/Out

PASSED

TestInOut

FAILED

TestInOutRed

Verification Code

SpecFlow – check CompareToInstance or CompareToSet examples.

Fitnesse – check method query(). It just returns the results. There are no comparisons/expectations/asserts. Finesse does it for you because of  In/Out (see above).

Built in Fixtures

  •  Built in possibility to call product through public interface
  •  Built in possibility to see results of such a call

PowerSlim already implemented Fitnesse fixtures (Script, X Query, Scenario, Decision ) for you. So you don’t need to write any test classes to make a glue between your test and production (SUT). All you need is to find a way how to setup testing environment or to call your production through a public interface (command line, PowerShell CMDLets, REST, HTTP).   PowerSlim will send the results back to Fitnesse by using SLIM protocol. And the good things is that Fitnesse by himself is responsible for different type of verifications (see Verification Code above).

Example: get-service is all what you need to put in your test to get the idea about the state of a particular service.

Subset Query:Local get-service
ServiceName Status DisplayName
SENS Running GO SENS GO

Visual Studio Integration

I really want to put something for SpecFlow. They did the great job by integrating with Visual Studio 😉

HLT

The requirement to test/call your product through a public interface is still the #1, even if I don’t completely agree with myself back to 2010 https://vlasenko.org/2010/03/17/beyond-acceptance-testing-framework/. It is not an Acceptance Tests if you need to mock part of your product. Probably this one is not a bad unit test (Stop Mocking, Start Testing)

Beyond Acceptance Testing Framework

It looks like there are several acceptance testing frameworks (ATF) available (RobotFramework, Cucumber, Fitnesse on FIT, Fitnesse on SLIMZenTest, Concordion). Not sure if each of them are really ATF, but can say that Concordion is not a great choice. Fitnesse is the best ATF I know. But it’s up to you, which way to go.

We should keep in mind the  High Level Testing (HLT) pattern and make our application ready for HLT. While performing HLT you interact with your application via Web Services, EXEs, “jobs” in the database. Test doubles are  unacceptable in the acceptance tests:

  1. We don’t need to stub “customer” environment.
  2. We don’t want to duplicate production code in the acceptance test
  3. And we don’t want to have tests for nothing

Fitnesse: the results for |check not| for BLANK is confusing me

It is too confusing that the Fitnesse highlight expected [BLANK] in second case.

The right exporting folder

check execute cmd /c dir /B “C:\ExportEmpty” BLANK
check not execute cmd /c dir /B “c:\ExportNotEmpty” [2d1313b9-cfb8-4405-8e5f-91f34b5a1e08 ]

expected [BLANK]

The best way is to say NOT EXPECTED [BLANK]

the script for the page:

|check|execute|cmd /c dir /B “C:\ExportEmpty”|BLANK|
|check not|execute|cmd /c dir /B “c:\ExportNotEmpty”|BLANK|

BTW: Why should I type BLANK for the second scenario? I’ll get the IGNORE result if I left them empty