SpecFlow uses an IDE integration (extension, addon, plugin or whatever it’s called). Instead use TickSpec. NO IDE extension/addon required. Supports F# / C#. Full example listed below

Cucumber, Specflow and Gherkin

Cucumber is the most the most popular library for this, supporting Ruby, Java, JavaScript. For .NET though, Specflow has become the most popular.

Both libraries tend to agree on the same syntax plain text file syntax though; gherkin.

BDD with gherkin recap

A quick re-cap, BDD is the act of defining automated tests by the behaviour of the system (an alternative to TDD). Because BDD focuses on behaviour, tests are typically called

outside-in. Said another way, they sit higher up on the test pyramid and they test much of the application when compared to TDD tests. BDD style tests could be written in source code, however, because they involve writing out what the application should do, this tends to involve other people (those who don’t write code). As a result, gherkin was introduced as a simple plain text language to describe the tests so those without little coding experience could read them.

Gherkin language

The language is very simple and has 3 main keywords, given, when, then. The full spec is here: https://docs.cucumber.io/gherkin/reference/ here is a simple example (taken from the docs):

1: 
2: 
3: 
4: 
5: 
6: 

Feature: Guess the word

#The first example has two steps
Scenario: Maker starts a game
    When the Maker starts a game
    Then the Maker waits for a Breaker to join

SpecFlow makes this hard

Specflow makes all of this possible by having a plugin to the IDE, Visual Studio and community support for Visual Studio For Mac (thanks @jimbobbennett). This is required, as Specflow emits code when the user saves the gherkin file (called the feature file). Sometimes things change in the IDE, and these tools are broken, this is more of an issue on VS4Mac than windows. I also don’t like code gen when it can be avoided - it feels like magic and code should be understood.

TickSpec as the alternative

There is a lesser known library that provides the same functionality called https://github.com/fsprojects/TickSpec. It supports the gherkin language and does not require an IDE plugin. Without the plugin, there is no syntax highlighting, but these style of tests are not for developers. Best of all, with TickSpec, there is not code gen. No magic. Nothing to go stale. With a few code tweaks, this library can be used to build out feature files (plain text gherkin language tests) for a Xamarin app, running on either Mac or Windows.

Creating a Xamarin UI test with TickSpec

• Given an existing app that needs testing
• Add a new UI test project
• Update Xamarin.UITest to the latest
• Add TickSpec NuGet package

All code snippets will be in F# (because it’s an awesome language), C# is supported too though. You can even write your app in C#, and make just this UITest project in F#.

It’s available in the drop-down when you create the project. To bootstrap TickSpec, a bit of glue code is required for the tests to be discovered for each platform. Add the following to your AppInitializer

1: 
2: 
3: 
4: 

module AppInitializer =

    // Sadly we need a variable and null :(. It will be the only one though. 
    let mutable app: IApp = null

If you’re using C#, then create a new file and translate the following code to C#. If you’re us F#, then add the following to the bottom of your AppInitializer.fs file.

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
15: 
16: 
17: 
18: 
19: 
20: 
21: 
22: 
23: 
24: 
25: 
26: 
27: 
28: 
29: 
30: 
31: 
32: 
33: 
34: 
35: 
36: 
37: 
38: 
39: 
40: 
41: 
42: 
43: 
44: 
45: 
46: 
47: 
48: 
49: 
50: 
51: 
52: 
53: 
54: 
55: 
56: 
57: 
58: 
59: 
60: 
61: 
62: 
63: 
64: 
65: 
66: 
67: 
68: 
69: 
70: 
71: 
72: 
73: 
74: 

/// Class containing all BDD tests in current assembly as NUnit unit tests
[<TestFixture>]
type FeatureFixture () =
    /// Test method for all BDD tests in current assembly as NUnit unit tests
    [<Test>]
    [<TestCaseSource("Scenarios")>]
    member __.Bdd (scenario:Scenario) =
        if scenario.Tags |> Seq.exists ((=) "ignore") then
            raise (new IgnoreException("Ignored: " + scenario.ToString()))
        try
            let platform = 
                match scenario.Tags |> Seq.contains "android", scenario.Tags |> Seq.contains "ios" with 
                | true, true -> failwith "Can't run both ios and android. Check your spelling for the tags" 
                | false, false-> failwith "Must run with platform either ios or android. Add one of: @android, @ios, @android_ios" 
                | true, false -> Platform.Android
                | false, true -> Platform.iOS

            AppInitializer.app <- AppInitializer.startApp platform
            scenario.Action.Invoke()
        with
        | ex -> 
            eprintf "Failed: %s\n%s" ex.Message ex.StackTrace
            sprintf "Failed: %s\n%s" ex.Message ex.StackTrace |> Console.WriteLine 
            raise ex


    /// All test scenarios from feature files in current assembly
    static member Scenarios =
    
        let enhanceScenarioName parameters scenarioName =
            let replaceParameterInScenarioName (scenarioName:string) parameter =
                scenarioName.Replace("<" + fst parameter + ">", snd parameter)
            parameters
            |> Seq.fold replaceParameterInScenarioName scenarioName

        let splitTags (tags: string[]) = 
            tags
            |> Array.map (fun x -> x.Split("_")) 
            |> Array.concat
            |> Array.map (fun x -> x.Replace("_", "").Trim().ToLower())

        let isPlatform (name:string) tags (scenario:Scenario) feature = 
            if tags |> Seq.contains (name.ToLower()) then 
                let scenario = 
                    {scenario with 
                        Name = scenario.Name |> sprintf "%s: %s" name
                        Tags = tags |> Array.filter (fun x -> x = (name.ToLower())) }
                (new TestCaseData(scenario))
                    .SetName(enhanceScenarioName scenario.Parameters scenario.Name)
                    .SetProperty("Feature", feature.Name.Substring(9))
                    .SetCategory(name) |> Some
            else None

        let createTestCaseData (feature:Feature) (scenario:Scenario) =
            let tags = splitTags scenario.Tags  

            [isPlatform "Android"; isPlatform "iOS"]
            |> List.choose (fun f -> f tags scenario feature)
            |> Seq.foldBack (fun (tag:string) tests -> 
                tests |> List.map (fun data -> data.SetProperty("Tag", tag))) scenario.Tags

        let createFeatureData (feature:Feature) =
            feature.Scenarios
            |> Seq.map (createTestCaseData feature)
            |> Seq.concat
        
        let assembly = Assembly.GetExecutingAssembly()
        let definitions = new StepDefinitions(assembly.GetTypes())

        assembly.GetManifestResourceNames()
        |> Seq.filter (fun (n:string) -> n.EndsWith(".feature") )
        |> Seq.collect (fun n ->
            definitions.GenerateFeature(n, assembly.GetManifestResourceStream(n))
            |> createFeatureData)

The code snippet above primarily does two things.

• create NUnit tests from the plain text feature files
• create the test for each platform specified in the test

No tests have been added yet, so there won’t be anything to see. Let’s add some.

• add a plain text file to the project with the suffix .feature
• set the build action to EmbeddedResource
• for each test (scenario) add the following on the line above @android_ios

Here is an example of a feature file with one test that will run on both Android and iOS. If you want only one platform then use only that name ie @android or @ios

1: 
2: 
3: 
4: 
5: 
6: 
7: 
8: 

Feature: Login Screen works correctly 

@android_ios
Scenario 1: I can login to the app with the correct credentials
    Given I enter the username admin
    And I enter the password admin
    When I tap login
    Then I should be on the Notes screen

If you compile your code, you should see the test(s) show up in the test windows.

Adding steps

Following the Page-Object-Model, we need a static class name to hold our steps. The following code implements the required steps for the example test above.

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
15: 
16: 
17: 

module LoginDefinitions = 
    
    [<Given>]
    let  ``I enter the username (.*)`` username =
        LoginScreen.enterTextForUsername username AppInitializer.app

    let [<Given>] ``I enter the password (.*)`` password =
        LoginScreen.enterTextForPassword password AppInitializer.app

    let [<When>] ``I tap login`` () =
        LoginScreen.login AppInitializer.app

    let [<Then>] ``I should be on the Notes screen`` () =
        NotesScreen.canSeeGetNotesButton AppInitializer.app

    let [<Then>] ``I am still on the Login page`` () =
        LoginScreen.canSeeLoginButton AppInitializer.app

Note the in F# we can use the double backticks to write the name of a function with spaces. This makes writing the test names really easy and avoids naming wars - PascalCase, CamelCase, SnakeCase - just use English now!

[<given>] is an attribute ([Given] in C#). The first test shows how attributes are used in a C# style fashion. The remaining tests use the inlined style, to make the steps extremely readable.

Each step delegates the work to a page and passes in the app. Here is the NotesScreen as an example:

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 

module NotesScreen = 

    let getNotesButton = "Get Notes"

    let canSeeGetNotesButton (app:IApp) = 
        app.WaitForElement(fun x -> x.Marked getNotesButton) 
        |> Array.filter (fun x -> x.Text = getNotesButton)
        |> function
        | [|x|] -> ()
        | xs -> 
            xs |> Array.iter (fun x -> sprintf "%A %s" x.Class x.Text |> Console.WriteLine)
            sprintf "failed to find button: %s, %A" getNotesButton xs |> failwith

NotesScreen is just a static class (module in F#). We can then define each of the text using let. No need for extra keywords as let bindings are immutable by default.

A simple function canSeeGetNotesButton then does the work to read the app’s UI and check that the item exists. If it is missing, an exception is thrown.

Build and run

With those items in place, you should now be able to build and run. For each tests, there should be an Android and iOS test.

The following gists show full examples of each file:

AppInitializer.fs

Tests.fs

Login.feature

Help - when I build the tests don’t show up

Looking for the wrong file type

This can be caused by a few things. At the start of this blog post, the code to setup things up was added. The end of that code had a the following lines:

1: 
2: 

assembly.GetManifestResourceNames()
|> Seq.filter (fun (n:string) -> n.EndsWith(".feature") )

Make sure that it still says .feature or it won’t find the right file.

Wrong file type name

The plain test file must end with .feature if it doesn’t, then it won’t be found

Build action must be EmbeddedResource

This must be set, as TickSpec scans the assembly looking for feature files (rather than using magic to generate backings files). If not set, then the feature fiel wont’ be in the assembly, and TickSpec won’t be able to find it.

Tag each test with @android_ios or @android or @ios

Each test is named with Scenario. Before that line, there must be a tag with what platform the test will run on. The code added at the top of this post describes how to generate two tests (one for each platform) from the single test. No tag, no tests

Tag each test with @ignore

If you have this tag before your test, TickSpec will ignore the test and not run it.

Have any questions?

Leave a comment or message me on twitter @willsam100 - I’m happy to help Happy [type safe] coding CodingWithSam