testing without test environment

It's always a problem... you develop a code, write unit tests, you build your project locally so unit tests can give you a quick feedback. What do you do if you need to test a web-app? You build it, run unit tests, everything seems to work fine, CI server builds your code, run tests and everything is ok... until you deploy your war/ear file into application server. Then it turns out it does not deploy, and your head blows, s**** and f*** are flying around...

Wouldn't it be nice if you wouldn't have to wait for whole CI/CD pipeline or manual deploy to be notified that your app can't be deployed or you can't communicate with it due to incorrect dependencies.

Turns out it's quite easy to do it. All you have to do is use embedded application server. I'll show you how to pick one, and set it according to your needs.

It always comes down to five phases/steps tightly coupled to maven life cycle.


There are few different maven plugins that can help you with it.

<plugin>
    <groupId>org.eclipse.jetty</groupId>
    <artifactId>jetty-maven-plugin</artifactId>
    <version>9.0.0.M4</version>
</plugin>

<plugin>
    <groupId>org.glassfish</groupId>
    <artifactId>maven-embedded-glassfish-plugin</artifactId>
    <version>3.1.1</version>
</plugin>

<plugin>
    <groupId>org.apache.tomcat.maven</groupId>
    <artifactId>tomcat7-maven-plugin</artifactId>
    <version>2.0</version>
</plugin>

First one - Jetty - seems to be the most lightweight solution, if you don't use JMX (maybe there is a way to turn it on and configure it, but I wasn't able to figure it out) this one is for you. Setup is quick, easy and what's most important it does not take much time to set it up, run tests, tear down and get a report.

How to do this... will come next week. Stay tuned.

framework hell #2

Let's continue this -> FRAMEWORK HELL

First step is simple, add basic dependencies.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>maketestingharder</groupId>
    <artifactId>frameworkHell</artifactId>
    <version>666</version>

    <dependencies>
        <dependency>
            <groupId>org.spockframework</groupId>
            <artifactId>spock-core</artifactId>
            <version>0.7-groovy-2.0</version>
        </dependency>
        <dependency>
            <groupId>org.jbehave</groupId>
            <artifactId>jbehave-core</artifactId>
            <version>3.8</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-core</artifactId>
            <version>1.0.10</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-java</artifactId>
            <version>1.0.10</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-junit</artifactId>
            <version>1.0.10</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-html</artifactId>
            <version>0.2.2</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>de.codecentric</groupId>
            <artifactId>jbehave-junit-runner</artifactId>
            <version>1.0.2</version>
        </dependency>
        <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>6.8.5</version>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.11</version>
        </dependency>
    </dependencies>
    <build>
     //
    </build>
</project>

Above section contains only basic dependencies, does not include parent section, which may contain common build steps shared between multiple child poms, does not contain dependency management section which could delegate dependency version management to other pom files (I guess this topic will have a separate entry as there are few ways to manage dependencies across multiple projects). For now we'll keep it simple (well.. maybe not exactly, since we'd like to share complex problems instead of simple ones :P).

Have you tried to build you're code? If you did, you have noticed that only testNG test were triggered. One more thing that you should notice is content of

target/test-classes/

All Java classes were compiled, and made it to that folder... What about Spock, Groovy to be precise? Java has no idea about Groovy but Groovy is one of JVM languages and Groovy compiler is able to compile Java so we'll use gmaven-plugin to compile those. To do this we have to fill our build section with below entries.

<build>
    <plugins>
        <plugin>
            <groupId>org.codehaus.gmaven</groupId>
            <artifactId>gmaven-plugin</artifactId>
            <version>1.4</version>
            <executions>
                <execution>
                    <goals>
                        <goal>compile</goal>
                        <goal>testCompile</goal>
                    </goals>
                </execution>
            </executions>
            <dependencies>
                <dependency>
                    <groupId>org.codehaus.gmaven.runtime</groupId>
                    <artifactId>gmaven-runtime-2.0</artifactId>
                    <version>1.4</version>
                    <exclusions>
                        <exclusion>
                            <groupId>org.codehaus.groovy</groupId>
                            <artifactId>groovy-all</artifactId>
                        </exclusion>
                    </exclusions>
                </dependency>
                <dependency>
                    <groupId>org.codehaus.groovy</groupId>
                    <artifactId>groovy-all</artifactId>
                    <version>2.1.5</version>
                </dependency>
            </dependencies>
        </plugin>
    </plugins>
</build>

Build you're code now... What happens? Still only TestNG entries on console... but target/test-classes now contains also groovy/spock classes. So we're half way there (well... 1/5 because we still have plenty to configure).

Now we have to setup test phases, so that unit test are triggered during test phase and IT tests in integration-tests phase. First one is simple. Just add this section.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.15</version>
        <configuration>
            <excludes>
                <exclude>**/*IT.java</exclude>
            </excludes>
        </configuration>
</plugin>

Why there is ".java" in this config? Because groovy and java classes are at binary level treated alike and surefire-plugin treats them also alike.
But this time instead of running more tests less were triggered. Where the hell are JUnit tests? In order to do this we need to setup two types of executions. One that will trigger TestNG and second that will kick off JUnit. So above section at the end should like like this.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.15</version>
    <configuration>
        <excludes>
            <exclude>**/*IT.java</exclude>
        </excludes>
    </configuration>
    <executions>
        <execution>
            <id>JUnitUnitTest</id>
            <phase>test</phase>
            <goals>
                <goal>test</goal>
            </goals>
            <configuration>
                <excludes>
                    <exclude>**/*IT.java</exclude>
                </excludes>
                <testNGArtifactName>none:none</testNGArtifactName>
            </configuration>
        </execution>
    </executions>
</plugin>

How about now? Surprised? I guess not... All we did is forcing test execution twice, first one with JUnit, second one with TestNG. And now we're half way there. As you've noticed only Unit tests were triggered. Now we need to add another plugin that will trigger Integration Tests. So we need to add below section.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>2.16</version>
    <dependencies>
        <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-testng</artifactId>
            <version>2.16</version>
        </dependency>
        <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-junit47</artifactId>
            <version>2.16</version>
        </dependency>
    </dependencies>
    <executions>
        <execution>
            <id>TestNGIT</id>
            <phase>integration-test</phase>
            <goals>
                <goal>integration-test</goal>
            </goals>
            <configuration>
                <threadCount>1</threadCount>
            </configuration>
        </execution>
    </executions>
</plugin>

Last section does basically the same as previous one, but is triggered in integration test phase.

Of course there are some drawbacks... I have no idea how to separate properly IT tests, for some reason failsafe-plugin does not work similarly to surefire-plugin that's why there are so many print lines despite the fact that those tests are not triggered (take a look at TestSummary for integration-test phase...).

Next time I'll show you how to test Java web apps/services (*.war) without deploying them into test environment.

framework Hell #1.6

As I promised this post will show basic, maybe not simplest but definitely most effective setup of both Cucumber and JBehave, which you can in most cases copy-paste into your project.

Project structure was published in last post.
Now for class implementation.

Cucumber first.
This is the simplest ever setup. Of course it won't be the simplest one, required to run Cucumber, but this one allows you to define which steps file can be used with desired feature file.

package cucumber.unit;

import cucumber.junit.Cucumber;
import org.junit.runner.RunWith;

@RunWith(Cucumber.class)
@Cucumber.Options(
        features={"src/test/resources/cucumber/unit/UnitTests.feature"},
        glue="cucumber.unit.steps")
public class CucumberUnitTest {
} 

And now few words of explanation:

@RunWith(Cucumber.class)

tells JUnit which class should be used to run Cucumber tests, so it's mandatory

features={"src/test/resources/cucumber/unit/UnitTests.feature"}

tells Cucumber where features are located, I'm precisely pointing file, but location is also fine

glue="cucumber.unit.steps"

tells Cucumber where to look for classes with steps definitions/implementation

@Cucumber.Options()

tells Cucumber how it should behave, what should it do before tests, during them, and after. You can specify how report should look like, and more. In minimal setup you don't need this annotation at all.

With JBehave it's a bit more code.

@RunWith(JUnitReportingRunner.class)
public class JBehaveUnitTest extends JUnitStories {
    private Configuration configuration;
    public JBehaveUnitTest() {
        super();
        configuration = new Configuration() {
        };

        configuration.useFailureStrategy(new RethrowingFailure());
        configuration.useKeywords( new LocalizedKeywords( Locale.ENGLISH ) );
        configuration.usePathCalculator( new AbsolutePathCalculator() );
        configuration.useParameterControls( new ParameterControls() );
        configuration.useParameterConverters( new ParameterConverters() );
        configuration.useParanamer( new NullParanamer() );
        configuration.usePendingStepStrategy( new PassingUponPendingStep() );
        configuration.useStepCollector(new MarkUnmatchedStepsAsPending());
        configuration.useStepdocReporter(new PrintStreamStepdocReporter());
        configuration.useStepFinder(new StepFinder());
        configuration.useStepMonitor(new SilentStepMonitor());
        configuration
                .useStepPatternParser(new RegexPrefixCapturingPatternParser());
        configuration.useStoryControls(new StoryControls());
        configuration.useStoryLoader(new LoadFromClasspath());
        configuration.useStoryParser(new RegexStoryParser(configuration
                .keywords()));
        configuration.useStoryPathResolver(new UnderscoredCamelCaseResolver());
        configuration.useStoryReporterBuilder(new StoryReporterBuilder());
        configuration.useViewGenerator(new FreemarkerViewGenerator());

        EmbedderControls embedderControls = configuredEmbedder()
                .embedderControls();
        embedderControls.doBatch(false);
        embedderControls.doGenerateViewAfterStories(false);
        embedderControls.doIgnoreFailureInStories(false);
        embedderControls.doIgnoreFailureInView(false);
        embedderControls.doSkip(false);
        embedderControls.doVerboseFailures(false);
        embedderControls.doVerboseFiltering(false);
        embedderControls.useStoryTimeoutInSecs(300);
        embedderControls.useThreads(1);
    }

    @Override
    public Configuration configuration() {
        return configuration;
    }

    @Override
    public InjectableStepsFactory stepsFactory() {
        return new InstanceStepsFactory(configuration(), new JBehaveUnitTestSteps());
    }
    @Override
    protected List storyPaths() {
        return Arrays.asList( "jbehave/UnitTests.story" );
    }
}

Again few words of explanation (I've skipped 99,9% of lines...):

@RunWith(JUnitReportingRunner.class)

tells JUnit which class should be used to run JBehave tests, so it's mandatory

@Override
    protected List storyPaths() {
        return Arrays.asList( "jbehave/UnitTests.story" );
    }

tells JBehave which story files should be used in this test phase.

And that's basically all for Cucumber and JBehave setup. Sure there is still some customization within pom.xml file... but this will come... hopefully soon.

framework Hell #1.5

Next post was supposed to be about pom.xml details... but it won't.
Since I used Spock for Groovy tests why should't I also include Cucumber and JBehave

So today I'm going to fix this omission.

First let's add dependencies to pom file.

        <dependency>
            <groupId>org.jbehave</groupId>
            <artifactId>jbehave-core</artifactId>
            <version>3.8</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-core</artifactId>
            <version>1.0.10</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-java</artifactId>
            <version>1.0.10</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-junit</artifactId>
            <version>1.0.10</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-html</artifactId>
            <version>0.2.2</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>de.codecentric</groupId>
            <artifactId>jbehave-junit-runner</artifactId>
            <version>1.0.2</version>
        </dependency>

Now freaked file/folder structure with names that assure we'll trigger tests when needed.

You may ask why cucumber folder structure is such a mess.
It's because this was the only way I was able to force Cucumber to use Unit Test Steps with Unit Tests and Integration Test Steps with IT Tests. This solutions allows you to have same steps with different implementation for Unit Tests and different for IT Tests.

Class implementation details will be a part of next post (Yep... before pom.xml post).

Impatient readers can take a look at GitHub...

framework hell

In most cases setting test for a project end up quite quick, unit test classes have "Test" suffix, while integration tests have "IT" suffix.

But that's easy...

Today we'll show you how to use Java with JUnit, TestNG and Groovy with Spock tightly coupled with Maven within one project/module.

You may ask WHY??

!!!! BECAUSE F*** YOU THAT'S WHY !!!!

Why doing it the easy way when hard way is more fun !!!!!!!

First we need to distinguish tests on language level.
Easiest way is to keep them in dedicated folders

/src/test/java/
/src/test/groovy/

Now we need to distinguish tests by level and framework, best way is to do this by using suffixes and prefixes on classes.

HINT!!
Since Spock tests are handled by JUnit runner class you can treat those equally. What more... dividing tests based on framework will be done on Maven level so all we need to do is distinguish tests based on when they should be triggered within maven build lifecycle. Personally I like below convention.

SampleClassNameTest.java   <-Java Unit Tests
SampleClassNameIT.java     <-Java Integration Tests
SampleClassNameTest.groovy <-Groovy Unit Tests
SampleClassNameSpec.groovy <-Groovy Integration Tests

Be careful with class names, since Groovy classes are compiled to same .class as Java classes and land in the same target location make sure you don't have same classes in Java and Groovy locations (if they are in different packages then it's fine). But most of IDE's will prompt you if such situation occurs. 

And now last and simplest part of this setup... pom.xml

Hah, just kidding, this is the most tricky part.
But this will come in separate post in a few days.