framework hell #2

Let's continue this -> FRAMEWORK HELL

First step is simple, add basic dependencies.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>maketestingharder</groupId>
    <artifactId>frameworkHell</artifactId>
    <version>666</version>

    <dependencies>
        <dependency>
            <groupId>org.spockframework</groupId>
            <artifactId>spock-core</artifactId>
            <version>0.7-groovy-2.0</version>
        </dependency>
        <dependency>
            <groupId>org.jbehave</groupId>
            <artifactId>jbehave-core</artifactId>
            <version>3.8</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-core</artifactId>
            <version>1.0.10</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-java</artifactId>
            <version>1.0.10</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-junit</artifactId>
            <version>1.0.10</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-html</artifactId>
            <version>0.2.2</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>de.codecentric</groupId>
            <artifactId>jbehave-junit-runner</artifactId>
            <version>1.0.2</version>
        </dependency>
        <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>6.8.5</version>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.11</version>
        </dependency>
    </dependencies>
    <build>
     //
    </build>
</project>

Above section contains only basic dependencies, does not include parent section, which may contain common build steps shared between multiple child poms, does not contain dependency management section which could delegate dependency version management to other pom files (I guess this topic will have a separate entry as there are few ways to manage dependencies across multiple projects). For now we'll keep it simple (well.. maybe not exactly, since we'd like to share complex problems instead of simple ones :P).

Have you tried to build you're code? If you did, you have noticed that only testNG test were triggered. One more thing that you should notice is content of

target/test-classes/

All Java classes were compiled, and made it to that folder... What about Spock, Groovy to be precise? Java has no idea about Groovy but Groovy is one of JVM languages and Groovy compiler is able to compile Java so we'll use gmaven-plugin to compile those. To do this we have to fill our build section with below entries.

<build>
    <plugins>
        <plugin>
            <groupId>org.codehaus.gmaven</groupId>
            <artifactId>gmaven-plugin</artifactId>
            <version>1.4</version>
            <executions>
                <execution>
                    <goals>
                        <goal>compile</goal>
                        <goal>testCompile</goal>
                    </goals>
                </execution>
            </executions>
            <dependencies>
                <dependency>
                    <groupId>org.codehaus.gmaven.runtime</groupId>
                    <artifactId>gmaven-runtime-2.0</artifactId>
                    <version>1.4</version>
                    <exclusions>
                        <exclusion>
                            <groupId>org.codehaus.groovy</groupId>
                            <artifactId>groovy-all</artifactId>
                        </exclusion>
                    </exclusions>
                </dependency>
                <dependency>
                    <groupId>org.codehaus.groovy</groupId>
                    <artifactId>groovy-all</artifactId>
                    <version>2.1.5</version>
                </dependency>
            </dependencies>
        </plugin>
    </plugins>
</build>

Build you're code now... What happens? Still only TestNG entries on console... but target/test-classes now contains also groovy/spock classes. So we're half way there (well... 1/5 because we still have plenty to configure).

Now we have to setup test phases, so that unit test are triggered during test phase and IT tests in integration-tests phase. First one is simple. Just add this section.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.15</version>
        <configuration>
            <excludes>
                <exclude>**/*IT.java</exclude>
            </excludes>
        </configuration>
</plugin>

Why there is ".java" in this config? Because groovy and java classes are at binary level treated alike and surefire-plugin treats them also alike.
But this time instead of running more tests less were triggered. Where the hell are JUnit tests? In order to do this we need to setup two types of executions. One that will trigger TestNG and second that will kick off JUnit. So above section at the end should like like this.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.15</version>
    <configuration>
        <excludes>
            <exclude>**/*IT.java</exclude>
        </excludes>
    </configuration>
    <executions>
        <execution>
            <id>JUnitUnitTest</id>
            <phase>test</phase>
            <goals>
                <goal>test</goal>
            </goals>
            <configuration>
                <excludes>
                    <exclude>**/*IT.java</exclude>
                </excludes>
                <testNGArtifactName>none:none</testNGArtifactName>
            </configuration>
        </execution>
    </executions>
</plugin>

How about now? Surprised? I guess not... All we did is forcing test execution twice, first one with JUnit, second one with TestNG. And now we're half way there. As you've noticed only Unit tests were triggered. Now we need to add another plugin that will trigger Integration Tests. So we need to add below section.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>2.16</version>
    <dependencies>
        <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-testng</artifactId>
            <version>2.16</version>
        </dependency>
        <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-junit47</artifactId>
            <version>2.16</version>
        </dependency>
    </dependencies>
    <executions>
        <execution>
            <id>TestNGIT</id>
            <phase>integration-test</phase>
            <goals>
                <goal>integration-test</goal>
            </goals>
            <configuration>
                <threadCount>1</threadCount>
            </configuration>
        </execution>
    </executions>
</plugin>

Last section does basically the same as previous one, but is triggered in integration test phase.

Of course there are some drawbacks... I have no idea how to separate properly IT tests, for some reason failsafe-plugin does not work similarly to surefire-plugin that's why there are so many print lines despite the fact that those tests are not triggered (take a look at TestSummary for integration-test phase...).

Next time I'll show you how to test Java web apps/services (*.war) without deploying them into test environment.

framework Hell #1.5

Next post was supposed to be about pom.xml details... but it won't.
Since I used Spock for Groovy tests why should't I also include Cucumber and JBehave

So today I'm going to fix this omission.

First let's add dependencies to pom file.

        <dependency>
            <groupId>org.jbehave</groupId>
            <artifactId>jbehave-core</artifactId>
            <version>3.8</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-core</artifactId>
            <version>1.0.10</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-java</artifactId>
            <version>1.0.10</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-junit</artifactId>
            <version>1.0.10</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-html</artifactId>
            <version>0.2.2</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>de.codecentric</groupId>
            <artifactId>jbehave-junit-runner</artifactId>
            <version>1.0.2</version>
        </dependency>

Now freaked file/folder structure with names that assure we'll trigger tests when needed.

You may ask why cucumber folder structure is such a mess.
It's because this was the only way I was able to force Cucumber to use Unit Test Steps with Unit Tests and Integration Test Steps with IT Tests. This solutions allows you to have same steps with different implementation for Unit Tests and different for IT Tests.

Class implementation details will be a part of next post (Yep... before pom.xml post).

Impatient readers can take a look at GitHub...

framework hell

In most cases setting test for a project end up quite quick, unit test classes have "Test" suffix, while integration tests have "IT" suffix.

But that's easy...

Today we'll show you how to use Java with JUnit, TestNG and Groovy with Spock tightly coupled with Maven within one project/module.

You may ask WHY??

!!!! BECAUSE F*** YOU THAT'S WHY !!!!

Why doing it the easy way when hard way is more fun !!!!!!!

First we need to distinguish tests on language level.
Easiest way is to keep them in dedicated folders

/src/test/java/
/src/test/groovy/

Now we need to distinguish tests by level and framework, best way is to do this by using suffixes and prefixes on classes.

HINT!!
Since Spock tests are handled by JUnit runner class you can treat those equally. What more... dividing tests based on framework will be done on Maven level so all we need to do is distinguish tests based on when they should be triggered within maven build lifecycle. Personally I like below convention.

SampleClassNameTest.java   <-Java Unit Tests
SampleClassNameIT.java     <-Java Integration Tests
SampleClassNameTest.groovy <-Groovy Unit Tests
SampleClassNameSpec.groovy <-Groovy Integration Tests

Be careful with class names, since Groovy classes are compiled to same .class as Java classes and land in the same target location make sure you don't have same classes in Java and Groovy locations (if they are in different packages then it's fine). But most of IDE's will prompt you if such situation occurs. 

And now last and simplest part of this setup... pom.xml

Hah, just kidding, this is the most tricky part.
But this will come in separate post in a few days.