testing without test environment - jetty

So how it actually goes?
And the answer is, as always in IT world, it depends...

For Jetty it's pretty simple.

start Jetty -> deploy war -> run tests -> stop Jetty

In order to do so we need to make sure that our application is build prior to test execution.
This is one of drawbacks, but main advantage is that if those tests will fail, main build does not.
All you need to do is add one more plugin (yep, one more!!) to your build section in pom file.

<plugin>
    <groupId>org.eclipse.jetty</groupId>
    <artifactId>jetty-maven-plugin</artifactId>
    <version>9.0.0.M4</version>
    <configuration>
        <war>${project.basedir}/webapp/frameworkHell.war</war>
        <webAppConfig>
            <contextPath>/frameworkHell</contextPath>
        </webAppConfig>
    </configuration>
    <executions>
        <execution>
            <id>start-jetty</id>
            <phase>pre-integration-test</phase>
            <goals>
                <goal>run-war</goal>
            </goals>
            <configuration>
                <scanIntervalSeconds>0</scanIntervalSeconds>
                <daemon>true</daemon>
            </configuration>
        </execution>
        <execution>
            <id>stop-jetty</id>
            <phase>post-integration-test</phase>
            <goals>
                <goal>stop</goal>
            </goals>
            <configuration>
                <stopKey>stopJetty</stopKey>
                <stopPort>9966</stopPort>
            </configuration>
        </execution>
    </executions>
</plugin>

Above section does all you need. All you have to do is to properly set path to your war file, and decide under what context path your app will be available.

Just in case you'd like to test it and you don't have war file you can use, take a look at my github, there's a simple web app that does nothing, it's a "hello world" example.

Why Jetty?
Because it's light, easy to use, and in most cases will be enough.
Sure, you can use Tomcat (will show next time how to do this) but do you really need it? It's quite heavy, and configuration sometimes can give you headache.
So if you don't use any fancy stuff like JMX, Jetty will do just best.

Next time GlassFish... heavy server... but worth trying.

testing without test environment

It's always a problem... you develop a code, write unit tests, you build your project locally so unit tests can give you a quick feedback. What do you do if you need to test a web-app? You build it, run unit tests, everything seems to work fine, CI server builds your code, run tests and everything is ok... until you deploy your war/ear file into application server. Then it turns out it does not deploy, and your head blows, s**** and f*** are flying around...

Wouldn't it be nice if you wouldn't have to wait for whole CI/CD pipeline or manual deploy to be notified that your app can't be deployed or you can't communicate with it due to incorrect dependencies.

Turns out it's quite easy to do it. All you have to do is use embedded application server. I'll show you how to pick one, and set it according to your needs.

It always comes down to five phases/steps tightly coupled to maven life cycle.


There are few different maven plugins that can help you with it.

<plugin>
    <groupId>org.eclipse.jetty</groupId>
    <artifactId>jetty-maven-plugin</artifactId>
    <version>9.0.0.M4</version>
</plugin>

<plugin>
    <groupId>org.glassfish</groupId>
    <artifactId>maven-embedded-glassfish-plugin</artifactId>
    <version>3.1.1</version>
</plugin>

<plugin>
    <groupId>org.apache.tomcat.maven</groupId>
    <artifactId>tomcat7-maven-plugin</artifactId>
    <version>2.0</version>
</plugin>

First one - Jetty - seems to be the most lightweight solution, if you don't use JMX (maybe there is a way to turn it on and configure it, but I wasn't able to figure it out) this one is for you. Setup is quick, easy and what's most important it does not take much time to set it up, run tests, tear down and get a report.

How to do this... will come next week. Stay tuned.

State of testing?

Have you ever wondered what's the current state of testing?
I have no idea, that's why I took part in State of Testing Survey 2013.

You can do it too, and once results are published we're all be satisfied. 

framework hell #2

Let's continue this -> FRAMEWORK HELL

First step is simple, add basic dependencies.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>maketestingharder</groupId>
    <artifactId>frameworkHell</artifactId>
    <version>666</version>

    <dependencies>
        <dependency>
            <groupId>org.spockframework</groupId>
            <artifactId>spock-core</artifactId>
            <version>0.7-groovy-2.0</version>
        </dependency>
        <dependency>
            <groupId>org.jbehave</groupId>
            <artifactId>jbehave-core</artifactId>
            <version>3.8</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-core</artifactId>
            <version>1.0.10</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-java</artifactId>
            <version>1.0.10</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-junit</artifactId>
            <version>1.0.10</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-html</artifactId>
            <version>0.2.2</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>de.codecentric</groupId>
            <artifactId>jbehave-junit-runner</artifactId>
            <version>1.0.2</version>
        </dependency>
        <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>6.8.5</version>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.11</version>
        </dependency>
    </dependencies>
    <build>
     //
    </build>
</project>

Above section contains only basic dependencies, does not include parent section, which may contain common build steps shared between multiple child poms, does not contain dependency management section which could delegate dependency version management to other pom files (I guess this topic will have a separate entry as there are few ways to manage dependencies across multiple projects). For now we'll keep it simple (well.. maybe not exactly, since we'd like to share complex problems instead of simple ones :P).

Have you tried to build you're code? If you did, you have noticed that only testNG test were triggered. One more thing that you should notice is content of

target/test-classes/

All Java classes were compiled, and made it to that folder... What about Spock, Groovy to be precise? Java has no idea about Groovy but Groovy is one of JVM languages and Groovy compiler is able to compile Java so we'll use gmaven-plugin to compile those. To do this we have to fill our build section with below entries.

<build>
    <plugins>
        <plugin>
            <groupId>org.codehaus.gmaven</groupId>
            <artifactId>gmaven-plugin</artifactId>
            <version>1.4</version>
            <executions>
                <execution>
                    <goals>
                        <goal>compile</goal>
                        <goal>testCompile</goal>
                    </goals>
                </execution>
            </executions>
            <dependencies>
                <dependency>
                    <groupId>org.codehaus.gmaven.runtime</groupId>
                    <artifactId>gmaven-runtime-2.0</artifactId>
                    <version>1.4</version>
                    <exclusions>
                        <exclusion>
                            <groupId>org.codehaus.groovy</groupId>
                            <artifactId>groovy-all</artifactId>
                        </exclusion>
                    </exclusions>
                </dependency>
                <dependency>
                    <groupId>org.codehaus.groovy</groupId>
                    <artifactId>groovy-all</artifactId>
                    <version>2.1.5</version>
                </dependency>
            </dependencies>
        </plugin>
    </plugins>
</build>

Build you're code now... What happens? Still only TestNG entries on console... but target/test-classes now contains also groovy/spock classes. So we're half way there (well... 1/5 because we still have plenty to configure).

Now we have to setup test phases, so that unit test are triggered during test phase and IT tests in integration-tests phase. First one is simple. Just add this section.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.15</version>
        <configuration>
            <excludes>
                <exclude>**/*IT.java</exclude>
            </excludes>
        </configuration>
</plugin>

Why there is ".java" in this config? Because groovy and java classes are at binary level treated alike and surefire-plugin treats them also alike.
But this time instead of running more tests less were triggered. Where the hell are JUnit tests? In order to do this we need to setup two types of executions. One that will trigger TestNG and second that will kick off JUnit. So above section at the end should like like this.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.15</version>
    <configuration>
        <excludes>
            <exclude>**/*IT.java</exclude>
        </excludes>
    </configuration>
    <executions>
        <execution>
            <id>JUnitUnitTest</id>
            <phase>test</phase>
            <goals>
                <goal>test</goal>
            </goals>
            <configuration>
                <excludes>
                    <exclude>**/*IT.java</exclude>
                </excludes>
                <testNGArtifactName>none:none</testNGArtifactName>
            </configuration>
        </execution>
    </executions>
</plugin>

How about now? Surprised? I guess not... All we did is forcing test execution twice, first one with JUnit, second one with TestNG. And now we're half way there. As you've noticed only Unit tests were triggered. Now we need to add another plugin that will trigger Integration Tests. So we need to add below section.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>2.16</version>
    <dependencies>
        <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-testng</artifactId>
            <version>2.16</version>
        </dependency>
        <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-junit47</artifactId>
            <version>2.16</version>
        </dependency>
    </dependencies>
    <executions>
        <execution>
            <id>TestNGIT</id>
            <phase>integration-test</phase>
            <goals>
                <goal>integration-test</goal>
            </goals>
            <configuration>
                <threadCount>1</threadCount>
            </configuration>
        </execution>
    </executions>
</plugin>

Last section does basically the same as previous one, but is triggered in integration test phase.

Of course there are some drawbacks... I have no idea how to separate properly IT tests, for some reason failsafe-plugin does not work similarly to surefire-plugin that's why there are so many print lines despite the fact that those tests are not triggered (take a look at TestSummary for integration-test phase...).

Next time I'll show you how to test Java web apps/services (*.war) without deploying them into test environment.

framework Hell #1.6

As I promised this post will show basic, maybe not simplest but definitely most effective setup of both Cucumber and JBehave, which you can in most cases copy-paste into your project.

Project structure was published in last post.
Now for class implementation.

Cucumber first.
This is the simplest ever setup. Of course it won't be the simplest one, required to run Cucumber, but this one allows you to define which steps file can be used with desired feature file.

package cucumber.unit;

import cucumber.junit.Cucumber;
import org.junit.runner.RunWith;

@RunWith(Cucumber.class)
@Cucumber.Options(
        features={"src/test/resources/cucumber/unit/UnitTests.feature"},
        glue="cucumber.unit.steps")
public class CucumberUnitTest {
} 

And now few words of explanation:

@RunWith(Cucumber.class)

tells JUnit which class should be used to run Cucumber tests, so it's mandatory

features={"src/test/resources/cucumber/unit/UnitTests.feature"}

tells Cucumber where features are located, I'm precisely pointing file, but location is also fine

glue="cucumber.unit.steps"

tells Cucumber where to look for classes with steps definitions/implementation

@Cucumber.Options()

tells Cucumber how it should behave, what should it do before tests, during them, and after. You can specify how report should look like, and more. In minimal setup you don't need this annotation at all.

With JBehave it's a bit more code.

@RunWith(JUnitReportingRunner.class)
public class JBehaveUnitTest extends JUnitStories {
    private Configuration configuration;
    public JBehaveUnitTest() {
        super();
        configuration = new Configuration() {
        };

        configuration.useFailureStrategy(new RethrowingFailure());
        configuration.useKeywords( new LocalizedKeywords( Locale.ENGLISH ) );
        configuration.usePathCalculator( new AbsolutePathCalculator() );
        configuration.useParameterControls( new ParameterControls() );
        configuration.useParameterConverters( new ParameterConverters() );
        configuration.useParanamer( new NullParanamer() );
        configuration.usePendingStepStrategy( new PassingUponPendingStep() );
        configuration.useStepCollector(new MarkUnmatchedStepsAsPending());
        configuration.useStepdocReporter(new PrintStreamStepdocReporter());
        configuration.useStepFinder(new StepFinder());
        configuration.useStepMonitor(new SilentStepMonitor());
        configuration
                .useStepPatternParser(new RegexPrefixCapturingPatternParser());
        configuration.useStoryControls(new StoryControls());
        configuration.useStoryLoader(new LoadFromClasspath());
        configuration.useStoryParser(new RegexStoryParser(configuration
                .keywords()));
        configuration.useStoryPathResolver(new UnderscoredCamelCaseResolver());
        configuration.useStoryReporterBuilder(new StoryReporterBuilder());
        configuration.useViewGenerator(new FreemarkerViewGenerator());

        EmbedderControls embedderControls = configuredEmbedder()
                .embedderControls();
        embedderControls.doBatch(false);
        embedderControls.doGenerateViewAfterStories(false);
        embedderControls.doIgnoreFailureInStories(false);
        embedderControls.doIgnoreFailureInView(false);
        embedderControls.doSkip(false);
        embedderControls.doVerboseFailures(false);
        embedderControls.doVerboseFiltering(false);
        embedderControls.useStoryTimeoutInSecs(300);
        embedderControls.useThreads(1);
    }

    @Override
    public Configuration configuration() {
        return configuration;
    }

    @Override
    public InjectableStepsFactory stepsFactory() {
        return new InstanceStepsFactory(configuration(), new JBehaveUnitTestSteps());
    }
    @Override
    protected List storyPaths() {
        return Arrays.asList( "jbehave/UnitTests.story" );
    }
}

Again few words of explanation (I've skipped 99,9% of lines...):

@RunWith(JUnitReportingRunner.class)

tells JUnit which class should be used to run JBehave tests, so it's mandatory

@Override
    protected List storyPaths() {
        return Arrays.asList( "jbehave/UnitTests.story" );
    }

tells JBehave which story files should be used in this test phase.

And that's basically all for Cucumber and JBehave setup. Sure there is still some customization within pom.xml file... but this will come... hopefully soon.

framework Hell #1.5

Next post was supposed to be about pom.xml details... but it won't.
Since I used Spock for Groovy tests why should't I also include Cucumber and JBehave

So today I'm going to fix this omission.

First let's add dependencies to pom file.

        <dependency>
            <groupId>org.jbehave</groupId>
            <artifactId>jbehave-core</artifactId>
            <version>3.8</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-core</artifactId>
            <version>1.0.10</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-java</artifactId>
            <version>1.0.10</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-junit</artifactId>
            <version>1.0.10</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-html</artifactId>
            <version>0.2.2</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>de.codecentric</groupId>
            <artifactId>jbehave-junit-runner</artifactId>
            <version>1.0.2</version>
        </dependency>

Now freaked file/folder structure with names that assure we'll trigger tests when needed.

You may ask why cucumber folder structure is such a mess.
It's because this was the only way I was able to force Cucumber to use Unit Test Steps with Unit Tests and Integration Test Steps with IT Tests. This solutions allows you to have same steps with different implementation for Unit Tests and different for IT Tests.

Class implementation details will be a part of next post (Yep... before pom.xml post).

Impatient readers can take a look at GitHub...

GitHub repository

Since we're waiting for another part of THIS post I would like to share something with you.

Every IT geek blog I also have GitHub account...


In this place I will be sharing code from this blog. Hope you'll enjoy it.