Karate is the only open-source tool to combine API test-automation, mocks, performance-testing and even UI automation into a single, unified framework. The BDD syntax popularized by Cucumber is language-neutral, and easy for even non-programmers. Assertions and HTML reports are built-in, and you can run tests in parallel for speed.
There’s also a cross-platform stand-alone executable for teams not comfortable with Java. You don’t have to compile code. Just write tests in a simple, readable syntax - carefully designed for HTTP, JSON, GraphQL and XML. And you can mix API and UI test-automation within the same test script.
A Java API also exists for those who prefer to programmatically integrate Karate’s rich automation and data-assertion capabilities.
Hello World
readme
For API Testing
If you are familiar with Cucumber / Gherkin, the big difference here is that you don’t need to write extra "glue" code or Java "step definitions" !
It is worth pointing out that JSON is a 'first class citizen' of the syntax such that you can express payload and expected data without having to use double-quotes and without having to enclose JSON field names in quotes. There is no need to 'escape' characters like you would have had to in Java or other programming languages.
And you don’t need to create additional Java classes for any of the payloads that you need to work with.
Java knowledge is not required and even non-programmers can write tests
Scripts are plain-text, require no compilation step or IDE, and teams can collaborate using Git / standard SCM
Based on the popular Cucumber / Gherkin standard - with IDE support and syntax-coloring options
Elegant DSL syntax 'natively' supports JSON and XML - including JsonPath and XPath expressions
Eliminate the need for 'Java Beans' or 'helper code' to represent payloads and HTTP end-points, and dramatically reduce the lines of code needed for a test
Ideal for testing the highly dynamic responses from GraphQL API-s because of Karate’s built-in text-manipulation and JsonPath capabilities
Express expected results as readable, well-formed JSON or XML, and assert in a single step that the entire response payload (no matter how complex or deeply nested) - is as expected
Comprehensive assertion capabilities - and failures clearly report which data element (and path) is not as expected, for easy troubleshooting of even large payloads
Scripts can call other scripts - which means that you can easily re-use and maintain authentication and 'set up' flows efficiently, across multiple tests
Embedded JavaScript engine that allows you to build a library of re-usable functions that suit your specific environment or organization
Re-use of payload-data and user-defined functions across tests is so easy - that it becomes a natural habit for the test-developer
Built-in support for switching configuration across different environments (e.g. dev, QA, pre-prod)
Support for data-driven tests and being able to tag or group tests is built-in, no need to rely on an external framework
Native support for reading YAML and even CSV files - and you can use them for data-driven tests
Standard Java / Maven project structure, and seamless integration into CI / CD pipelines - and support for JUnit 5
Option to use as a light-weight stand-alone executable - convenient for teams not comfortable with Java
Multi-threaded parallel execution, which is a huge time-saver, especially for integration and end-to-end tests
Built-in test-reports compatible with Cucumber so that you have the option of using third-party (open-source) maven-plugins for even better-looking reports
Async support that allows you to seamlessly integrate the handling of custom events or listening to message-queues
Mock HTTP Servlet that enables you to test any controller servlet such as Spring Boot / MVC or Jersey / JAX-RS - without having to boot an app-server, and you can use your HTTP integration tests un-changed
Built-in HTML templating so that you can extend your test-reports into readable specifications
Comprehensive support for different flavors of HTTP calls:
A set of real-life examples can be found here: Karate Demos
Comparison with REST-assured
For teams familiar with or currently using REST-assured, this detailed comparison of Karate vs REST-assured - can help you evaluate Karate. Do note that if you prefer a pure Java API - Karate has that covered, and with far more capabilities.
You can find a lot more references, tutorials and blog-posts in the wiki. Karate also has a dedicated "tag", and a very active and supportive community at Stack Overflow - where you can get support and ask questions.
Getting Started
If you are a Java developer - Karate requires at least Java 8 and then either Maven, Gradle, Eclipse or IntelliJ to be installed. Note that Karate works fine on OpenJDK.
If you don’t want to use Java, you have the option of just downloading and extracting the ZIP release. Try this especially if you don’t have much experience with programming or test-automation. We recommend that you use the Karate extension for Visual Studio Code - and with that, JavaScript, .NET and Python programmers will feel right at home.
Both the official Visual Studio Code and IntelliJ plugins support step-through debugging of Karate tests.
If you mix Karate into a Maven or Gradle project with many other dependendies, you may run into problems because of dependency conflicts. For example a lot of Java projects directly (or indirectly) depend on Netty or Thymeleaf or ANTLR, etc.
If you face issues such as "class not found", just pull in the karate-core dependency, and use the allclassifier in your pom.xml (or build.gradle).
Note that for very complicated projects you can consider using a Maven profile so that testing-related dependencies don’t collide with your development-time dependencies. Of course it is an option to have Karate tests in a separate stand-alone maven project and folder, while still being in the same Git repository.
Quickstart
It may be easier for you to use the Karate Maven archetype to create a skeleton project with one command. You can then skip the next few sections, as the pom.xml, recommended directory structure, sample test and JUnit 5 runners - will be created for you.
If you are behind a corporate proxy, or especially if your local Maven installation has been configured to point to a repository within your local network, the command below may not work. One workaround is to temporarily disable or rename your Maven settings.xml file, and try again.
You can replace the values of com.mycompany and myproject as per your needs.
A Karate test script has the file extension .feature which is the standard followed by Cucumber. You are free to organize your files using regular Java package conventions.
The Maven tradition is to have non-Java source files in a separate src/test/resources folder structure - but we recommend that you keep them side-by-side with your .java files. When you have a large and complex project, you will end up with a few data files (e.g. .js, .json, .txt) as well and it is much more convenient to see the .java and .feature files and all related artifacts in the same place.
This can be easily achieved with the following tweak to your maven <build> section.
With the above in place, you don’t have to keep switching between your src/test/java and src/test/resources folders, you can have all your test-code and artifacts under src/test/java and everything will work as expected.
Once you get used to this, you may even start wondering why projects need a src/test/resources folder at all !
Spring Boot Example
Soumendra Daas has created a nice example and guide that you can use as a reference here: hello-karate. This demonstrates a Java Maven + JUnit 5 project set up to test a Spring Boot app.
Naming Conventions
Since these are tests and not production Java code, you don’t need to be bound by the com.mycompany.foo.bar convention and the un-necessary explosion of sub-folders that ensues. We suggest that you have a folder hierarchy only one or two levels deep - where the folder names clearly identify which 'resource', 'entity' or API is the web-service under test.
Assuming you use JUnit, there are some good reasons for the recommended (best practice) naming convention and choice of file-placement shown above:
Not using the Test.java convention for the JUnit classes (e.g. CatsRunner.java) in the cats and dogs folder ensures that these tests will *not be picked up when invoking mvn test (for the whole project) from the command line. But you can still invoke these tests from the IDE, which is convenient when in development mode.
AnimalsTest.java (the only file that follows the Test.java naming convention) acts as the 'test suite' for the entire project. By default, Karate will load all .feature files from sub-directories as well. But since some-reusable.feature is aboveAnimalsTest.java in the folder hierarchy, it will not be picked-up. Which is exactly what we want, because some-reusable.feature is designed to be called only from one of the other test scripts (perhaps with some parameters being passed). You can also use tags to skip files.
some-classpath-function.js and some-classpath-payload.json are in the 'root' of the Java 'classpath' which means they can be easily read (and re-used) from any test-script by using the classpath: prefix, for e.g: read('classpath:some-classpath-function.js'). Relative paths will also work.
For details on what actually goes into a script or *.feature file, refer to the syntax guide.
file.encoding
In some cases, for large payloads and especially when the default system encoding is not UTF-8 (Windows or non-US locales), you may run into issues where a java.io.ByteArrayInputStream is encountered instead of a string. Other errors could be a java.net.URISyntaxException and match not working as expected because of special or foreign characters, e.g. German or ISO-8859-15. Typical symptoms are your tests working fine via the IDE but not when running via Maven or Gradle. The solution is to ensure that when Karate tests run, the JVM file.encoding is set to UTF-8. This can be done via the maven-surefire-plugin configuration. Add the plugin to the <build>/<plugins> section of your pom.xml if not already present:
To run a script .feature file from your Java IDE, you just need the following empty test-class in the same package. The name of the class doesn’t matter, and it will automatically run any .feature file in the same package. This comes in useful because depending on how you organize your files and folders - you can have multiple feature files executed by a single JUnit test-class.
package animals.cats;
import com.intuit.karate.junit4.Karate;
import org.junit.runner.RunWith;
@RunWith(Karate.class)
public class CatsRunner {
}
Refer to your IDE documentation for how to run a JUnit class. Typically right-clicking on the file in the project browser or even within the editor view would bring up the "Run as JUnit Test" menu option.
Karate will traverse sub-directories and look for .feature files. For example if you have the JUnit class in the com.mycompany package, .feature files in com.mycompany.foo and com.mycompany.bar will also be run. This is one reason why you may want to prefer a 'flat' directory structure as explained above.
JUnit 5
Karate supports JUnit 5 and the advantage is that you can have multiple methods in a test-class. Only 1 import is needed, and instead of a class-level annotation, you use a nice DRY and fluent-api to express which tests and tags you want to use.
Note that the Java class does not need to be public and even the test methods do not need to be public - so tests end up being very concise.
Note that more "builder" methods are available from the Runner.Builder class such as reportDir() etc.
You should be able to right-click and run a single method using your IDE - which should be sufficient when you are in development mode. But to be able to run JUnit 5 tests from the command-line, you need to ensure that the latest version of the maven-surefire-plugin is present in your project pom.xml (within the <build>/<plugins> section):
When you use a JUnit runner - after the execution of each feature, an HTML report is output to the target/karate-reports folder and the full path will be printed to the console (see video).
html report: (paste into browser to view)
-----------------------------------------
file:///projects/myproject/target/karate-reports/mypackage.myfeature.html
You can easily select (double-click), copy and paste this file: URL into your browser address bar. This report is useful for troubleshooting and debugging a test because all requests and responses are shown in-line with the steps, along with error messages and the output of print statements. Just re-fresh your browser window if you re-run the test.
Dry Run
This will give you the usual HTML report showing what features will be run, including all steps shown (including comments) so that it can be reviewed. Of course the actual time-durations, and logs will be missing, and everything will pass.
The “dry run” report is useful to review the tag "coverage" of what will be run. For example you can get a nice feature “coverage” report, provided you have a rich set of tags. e.g. @smoke @module=one @module=two etc.
The Runner.Builder API has a dryRun() method to switch this on. Note that this mode can be also triggered via the command-line by adding -D or --dryrun to the karate.options.
Command Line
Normally in dev mode, you will use your IDE to run a *.feature file directly or via the companion 'runner' JUnit Java class. When you have a 'runner' class in place, it would be possible to run it from the command-line as well.
Note that the mvn test command only runs test classes that follow the *Test.javanaming convention by default. But you can choose a single test to run like this:
mvn test -Dtest=CatsRunner
karate.options
When your Java test "runner" is linked to multiple feature files, which will be the case when you use the recommended parallel runner, you can narrow down your scope to a single feature, scenario or directory via the command-line, useful in dev-mode. Note how even tags to exclude (or include) can be specified:
Note that any Feature or Scenario with the special @ignore tag will be skipped by default.
mvn test "-Dkarate.options=--tags ~@skipme classpath:demo/cats/cats.feature" -Dtest=DemoTestParallel
Multiple feature files (or paths) can be specified, de-limited by the space character. They should be at the end of the karate.options. To run only a single scenario, append the line number on which the scenario is defined, de-limited by :.
mvn test "-Dkarate.options=PathToFeatureFiles/order.feature:12" -Dtest=DemoTestParallel
Since paths are expected at the end of the command-line options - if you want to only over-ride tags, use the = sign to make argument values clear. For example:
mvn test -Dkarate.options='-t=@dev -t=@src' -Dtest=ExamplesTest
Command Line - Gradle
For Gradle, you must extend the test task to allow the karate.options to be passed to the runtime (otherwise they get consumed by Gradle itself). To do that, add the following:
test {
// pull karate options into the runtime
systemProperty "karate.options", System.properties.getProperty("karate.options")
// pull karate env into the runtime
systemProperty "karate.env", System.properties.getProperty("karate.env")
// ensure tests are always run
outputs.upToDateWhen { false }
}
And then the above command in Gradle would look like:
./gradlew test --tests *CatsRunner
or
./gradlew test -Dtest.single=CatsRunner
Test Suites
The recommended way to define and run test-suites and reporting in Karate is to use the parallel runner, described in the next section. The approach in this section is more suited for troubleshooting in dev-mode, using your IDE.
One way to define 'test-suites' in Karate is to have a JUnit class at a level 'above' (in terms of folder hierarchy) all the *.feature files in your project. So if you take the previous folder structure example, you can do this on the command-line:
mvn test "-Dkarate.options=--tags ~@skipme" -Dtest=AnimalsTest
Here, AnimalsTest is the name of the Java class we designated to run the multiple *.feature files that make up your test-suite. There is a neat way to tag your tests and the above example demonstrates how to run all tests except the ones tagged @skipme.
Note that the special, built-in tag @ignore will always be skipped by default, and you don’t need to specify ~@ignore anywhere.
You can 'lock down' the fact that you only want to execute the single JUnit class that functions as a test-suite - by using the following maven-surefire-plugin configuration:
Note how the karate.options can be specified using the <systemProperties> configuration.
For Gradle, you simply specify the test which is to be include-d:
test {
include 'animals/AnimalsTest.java'
// pull karate options into the runtime
systemProperty "karate.options", System.properties.getProperty("karate.options")
// pull karate env into the runtime
systemProperty "karate.env", System.properties.getProperty("karate.env")
// ensure tests are always run
outputs.upToDateWhen { false }
}
The big drawback of the approach above is that you cannot run tests in parallel. The recommended approach for Karate reporting in a Continuous Integration set-up is described in the next section which can generate the JUnit XML format that most CI tools can consume. The Cucumber JSON format can be also emitted, which gives you plenty of options for generating pretty reports using third-party maven plugins.
And most importantly - you can run tests in parallel without having to depend on third-party hacks that introduce code-generation and config 'bloat' into your pom.xml or build.gradle.
Parallel Execution
Karate can run tests in parallel, and dramatically cut down execution time. This is a 'core' feature and does not depend on JUnit, Maven or Gradle.
You can easily "choose" features and tags to run and compose test-suites in a very flexible manner.
You can use the returned Results object to check if any scenarios failed, and to even summarize the errors
JUnit XML reports can be generated in the "reportDir" path you specify, and you can easily configure your CI to look for these files after a build (for e.g. in **/*.xml or **/karate-reports/*.xml). Note that you have to call the outputJunitXml(true) method on the Runner "builder".
Cucumber JSON reports can be generated, except that the extension will be .json instead of .xml. Note that you have to call the outputCucumberJson(true) method on the Runner "builder".
do not use the @RunWith(Karate.class) annotation. This is a normal JUnit 4 test class ! If you want to use JUnit 4, use the karate-junit4 Maven dependency instead of karate-junit5.
import com.intuit.karate.Results;
import com.intuit.karate.Runner;
import static org.junit.Assert.*;
import org.junit.Test;
public class TestParallel {
@Test
public void testParallel() {
Results results = Runner.path("classpath:some/package").tags("@smoke").parallel(5);
assertTrue(results.getErrorMessages(), results.getFailCount() == 0);
}
}
You don’t use a JUnit runner (no @RunWith annotation), and you write a plain vanilla JUnit test (it could even be a normal Java class with a main method)
The Runner.path() "builder" method in karate-core is how you refer to the package you want to execute, and all feature files within sub-directories will be picked up
Runner.path() takes multiple string parameters, so you can refer to multiple packages or even individual *.feature files and easily "compose" a test-suite
e.g. Runner.path("classpath:animals", "classpath:some/other/package.feature")
To choose tags, call the tags() API, note that by default, any *.feature file tagged with the special (built-in) tag: @ignore will be skipped. You can also specify tags on the command-line. The tags() method also takes multiple arguments, for e.g.
this is an "AND" operation: tags("@customer", "@smoke")
and this is an "OR" operation: tags("@customer,@smoke")
There is an optional reportDir() method if you want to customize the directory to which the HTML, XML and JSON files will be output, it defaults to target/karate-reports
If you want to dynamically and programmatically determine the tags and features to be included - the API also accepts List<String> as the path() and tags() methods arguments
parallel()has to be the last method called, and you pass the number of parallel threads needed. It returns a Results object that has all the information you need - such as the number of passed or failed tests.
JUnit 5 Parallel Execution
For JUnit 5 you can omit the public modifier for the class and method, and there are some changes to import package names. The method signature of the assertTrue has flipped around a bit. Also note that you don’t use @Karate.Test for the method, and you just use the normal JUnit 5 @Test annotation.
Else the Runner.path() "builder" API is the same, refer the description above for JUnit 4.
The parallel runner will always run Feature-s in parallel. Karate will also run Scenario-s in parallel by default. So if you have a Feature with multiple Scenario-s in it - they will execute in parallel, and even each Examples row in a Scenario Outline will do so !
A karate-timeline.html file will also be saved to the report output directory mentioned above (target/karate-reports by default) - which is useful for visually verifying or troubleshooting the effectiveness of the test-run (see video).
@parallel=false
In rare cases you may want to suppress the default of Scenario-s executing in parallel and the special tag@parallel=false can be used. If you place it above the Feature keyword, it will apply to all Scenario-s. And if you just want one or two Scenario-s to NOT run in parallel, you can place this tag above only thoseScenario-s. See example.
Note that forcing Scenario-s to run in a particular sequence is an anti-pattern, and should be avoided as far as possible.
Test Reports
As mentioned above, most CI tools would be able to process the JUnit XML output of the parallel runner and determine the status of the build as well as generate reports.
The Karate Demo has a working example of the recommended parallel-runner set up. It also details how a third-party library can be easily used to generate some very nice-looking reports, from the JSON output of the parallel runner.
For example, here below is an actual report generated by the cucumber-reporting open-source library.
Another example for a popular Maven reporting plugin that is compatible with Karate JSON is Cluecumber.
The demo also features code-coverage using Jacoco, and some tips for even non-Java back-ends. Some third-party report-server solutions integrate with Karate such as ReportPortal.io.
Logging
This is optional, and Karate will work without the logging config in place, but the default console logging may be too verbose for your needs.
Karate uses LOGBack which looks for a file called logback-test.xml on the 'classpath'.
You can change the com.intuit.karate logger level to INFO to reduce the amount of logging. When the level is DEBUG the entire request and response payloads are logged. If you use the above config, logs will be captured in target/karate.log.
If you want to keep the level as DEBUG (for HTML reports) but suppress logging to the console, you can comment out the STDOUT "root" appender-ref:
If you want to exclude the logs from your CI/CD pipeline but keep them in the execution of your users in their locals you can configure your logback using Janino. In such cases it might be desirable to have your tests using karate.logger.debug('your additional info') instead of the print keyword so you can keep logs in your pipeline in INFO.
For suppressing sensitive information such as secrets and passwords from the log and reports, see Log Masking and Report Verbosity.
Configuration
You can skip this section and jump straight to the Syntax Guide if you are in a hurry to get started with Karate. Things will work even if the karate-config.js file is not present.
Classpath
The 'classpath' is a Java concept and is where some configuration files such as the one for logging are expected to be by default. If you use the Maven <test-resources> tweak described earlier (recommended), the 'root' of the classpath will be in the src/test/java folder, or else would be src/test/resources.
karate-config.js
The only 'rule' is that on start-up Karate expects a file called karate-config.js to exist on the 'classpath' and contain a JavaScript function. The function is expected to return a JSON object and all keys and values in that JSON object will be made available as script variables.
And that’s all there is to Karate configuration ! You can easily get the value of the current 'environment' or 'profile', and then set up 'global' variables using some simple JavaScript. Here is an example:
function fn() {
var env = karate.env; // get java system property 'karate.env'
karate.log('karate.env system property was:', env);
if (!env) {
env = 'dev'; // a custom 'intelligent' default
}
var config = { // base config JSON
appId: 'my.app.id',
appSecret: 'my.secret',
someUrlBase: 'https://some-host.com/v1/auth/',
anotherUrlBase: 'https://another-host.com/v1/'
};
if (env == 'stage') {
// over-ride only those that need to be
config.someUrlBase = 'https://stage-host/v1/auth';
} else if (env == 'e2e') {
config.someUrlBase = 'https://e2e-host/v1/auth';
}
// don't waste time waiting for a connection or if servers don't respond within 5 seconds
karate.configure('connectTimeout', 5000);
karate.configure('readTimeout', 5000);
return config;
}
Here above, you see the karate.log(), karate.env and karate.configure() "helpers" being used. Note that the karate-config.js is re-processed for everyScenario and in rare cases, you may want to initialize (e.g. auth tokens) only once for all of your tests. This can be achieved using karate.callSingle().
A common requirement is to pass dynamic parameter values via the command line, and you can use the karate.properties['some.name'] syntax for getting a system property passed via JVM options in the form -Dsome.name=foo. Refer to the section on dynamic port numbers for an example.
You can even retrieve operating-system environment variables via Java interop as follows: var systemPath = java.lang.System.getenv('PATH');
This decision to use JavaScript for config is influenced by years of experience with the set-up of complicated test-suites and fighting with Maven profiles, Maven resource-filtering and the XML-soup that somehow gets summoned by the Maven AntRun plugin.
Karate’s approach frees you from Maven, is far more expressive, allows you to eyeball all environments in one place, and is still a plain-text file. If you want, you could even create nested chunks of JSON that 'name-space' your config variables.
One way to appreciate Karate’s approach is to think over what it takes to add a new environment-dependent variable (e.g. a password) into a test. In typical frameworks it could mean changing multiple properties files, maven profiles and placeholders, and maybe even threading the value via a dependency-injection framework - before you can even access the value within your test.
This approach is indeed slightly more complicated than traditional *.properties files - but you need this complexity. Keep in mind that these are tests (not production code) and this config is going to be maintained more by the dev or QE team instead of the 'ops' or operations team.
And there is no more worrying about Maven profiles and whether the 'right' *.properties file has been copied to the proper place.
Switching the Environment
There is only one thing you need to do to switch the environment - which is to set a Java system property.
By default, the value of karate.env when you access it within karate-config.js - would be null.
The recipe for doing this when running Maven from the command line is:
mvn test -DargLine="-Dkarate.env=e2e"
Or in Gradle:
./gradlew test -Dkarate.env=e2e
You can refer to the documentation of the
Maven Surefire Plugin for alternate ways of achieving this, but the argLine approach is the simplest and should be more than sufficient for your Continuous Integration or test-automation needs.
Where CatsRunner is the JUnit class name (in any package) you wish to run.
Karate is flexible, you can easily over-write config variables within the Java or JUnit "runner" - which is very convenient when in dev-mode or rapid-prototyping.
System.setProperty("karate.env", "pre-prod");
But the recommended way is to use the karateEnv(name, value) or systemProperty(name, value) API on the parallel-runner.
For advanced users, note that tags and the karate.env environment-switch can be "linked" using the special environment tags.
Environment Specific Config
When your project gets complex, you can have separate karate-config-<env>.js files that will be processed for that specific value of karate.env. This is especially useful when you want to maintain passwords, secrets or even URL-s specific for your local dev environment.
Make sure you configure your source code management system (e.g. Git) to ignore karate-config-*.js if needed.
There should always be karate-config.js in the "root" folder, even if you don’t have any "common" config. In such cases, the function can do nothing or return an empty JSON. Learn more.
Here are the rules Karate uses on bootstrap (before every Scenario or Examples row in a Scenario Outline):
if the system-property karate.config.dir was set, Karate will look in this folder for karate-config.js - and if found, will process it
else if karate-config.js was not found in the above location (or karate.config.dir was not set), classpath:karate-config.js would be processed (this is the default / common case)
if the karate.env system property was set
if karate.config.dir was set, Karate will also look for file:<karate.config.dir>/karate-config-<env>.js
else (if the karate.config.dir was not set), Karate will look for classpath:karate-config-<env>.js
if the over-ride karate-config-<env>.js exists, it will be processed, and the configuration (JSON entries) returned by this function will over-ride any set by karate-config.js
Advanced users who build frameworks on top of Karate have the option to supply a karate-base.js file that Karate will look for on the classpath:. This is useful when you ship a JAR file containing re-usable features and JavaScript / Java code and want to 'default' a few variables that teams can 'inherit' from. So an additional rule in the above flow of 'rules' (before the first step) is as follows:
if classpath:karate-base.js exists - Karate will process this as a configuration source before anything else
Syntax Guide
Script Structure
Karate scripts are technically in 'Gherkin' format - but all you need to grok as someone who needs to test web-services are the three sections: Feature, Background and Scenario. There can be multiple Scenario-s in a *.feature file, and at least one should be present. The Background is optional.
Variables set using def in the Background will be re-set before everyScenario. If you are looking for a way to do something only once per Feature, take a look at callonce. On the other hand, if you are expecting a variable in the Background to be modified by one Scenario so that later ones can see the updated value - that is not how you should think of them, and you should combine your 'flow' into one scenario. Keep in mind that you should be able to comment-out a Scenario or skip some via tags without impacting any others. Note that the parallel runner will run Scenario-s in parallel, which means they can run in any order. If you are looking for ways to do something only once per feature or across all your tests, see Hooks.
Lines that start with a # are comments.
Feature: brief description of what is being tested
more lines of description if needed.
Background:
= this section is optional !
= steps here are executed before each Scenario in this file
= variables defined here will be 'global' to all scenarios
= and will be re-initialized before every scenario
Scenario: brief description of this scenario
= steps for this scenario
Scenario: a different scenario
= steps for this other scenario
There is also a variant of Scenario called Scenario Outline along with Examples, useful for data-driven tests.
Given-When-Then
The business of web-services testing requires access to low-level aspects such as HTTP headers, URL-paths, query-parameters, complex JSON or XML payloads and response-codes. And Karate gives you control over these aspects with the small set of keywords focused on HTTP such as url, path, param, etc.
Karate does not attempt to have tests be in "natural language" like how Cucumber tests are traditionally expected to be. That said, the syntax is very concise, and the convention of every step having to start with either Given, And, When or Then, makes things very readable. You end up with a decent approximation of BDD even though web-services by nature are "headless", without a UI, and not really human-friendly.
Cucumber vs Karate
Karate was based on Cucumber-JVM until version 0.8.0 but the parser and engine were re-written from scratch in 0.9.0 onwards. So we use the same Gherkin syntax - but the similarity ends there.
If you are familiar with Cucumber (JVM), you may be wondering if you need to write step-definitions. The answer is no.
Karate’s approach is that all the step-definitions you need in order to work with HTTP, JSON and XML have been already implemented. And since you can easily extend Karate using JavaScript, there is no need to compile Java code any more.
The following table summarizes some key differences between Cucumber and Karate.
No. Cucumber’s Scenario Outline expects the Examples to contain a fixed set of rows.
:white_check_mark: Yes. Karate’s support for calling other *.feature files allows you to use a JSON array as the data-source and you can use JSON or even CSV directly in a data-driven Scenario Outline.
Parallel Execution
No. There are some challenges (especially with reporting) and you can find various discussions and third-party projects on the web that attempt to close this gap
:white_check_mark: Yes. Karate runs even Scenario-s in parallel, not just Feature-s.
Run 'Set-Up' Routines Only Once
No. Cucumber has a limitation where Background steps are re-run for every Scenario and worse - even for every Examples row within a Scenario Outline. This has been a highly-requested open issue for a long time.
No. And you have to roll your own approach to environment-specific configuration and worry about dependency-injection.
:white_check_mark: Yes. Easily define all environments in a single file and share variables across all scenarios. Full script-ability via JS or Java interop.
One nice thing about the design of the Gherkin syntax is that script-steps are treated the same no matter whether they start with the keyword Given, And, When or Then. What this means is that you are free to use whatever makes sense for you. You could even have all the steps start with When and Karate won’t care.
You can read more about the Given-When-Then convention at the Cucumber reference documentation. Since Karate uses Gherkin, you can also employ data-driven techniques such as expressing data-tables in test scripts. Another good thing that Karate inherits is the nice IDE support for Cucumber that IntelliJ and Eclipse have. So you can do things like right-click and run a *.feature file (or scenario) without needing to use a JUnit runner.
For a detailed discussion on BDD and how Karate relates to Cucumber, please refer to this blog-post: Yes, Karate is not true BDD. It is the opinion of the author of Karate that true BDD is un-necessary over-kill for API testing, and this is explained more in this answer on Stack Overflow.
With the formalities out of the way, let’s dive straight into the syntax.
Setting and Using Variables
def
Set a named variable
= assigning a string value:
Given def myVar = 'world'
= using a variable
Then print myVar
= assigning a number (you can use '*' instead of Given / When / Then)
* def myNum = 5
Note that def will over-write any variable that was using the same name earlier. Keep in mind that the start-up configuration routine could have already initialized some variables before the script even started. For details of scope and visibility of variables, see Script Structure.
Note that url and request are not allowed as variable names. This is just to reduce confusion for users new to Karate who tend to do * def request = {} and expect the request body or similarly, the url to be set.
The examples above are simple, but a variety of expression 'shapes' are supported on the right hand side of the = symbol. The section on Karate Expressions goes into the details.
assert
Assert if an expression evaluates to true
Once defined, you can refer to a variable by name. Expressions are evaluated using the embedded JavaScript engine. The assert keyword can be used to assert that an expression returns a boolean value.
Given def color = 'red '
And def num = 5
Then assert color + num == 'red 5'
Everything to the right of the assert keyword will be evaluated as a single expression.
Something worth mentioning here is that you would hardly need to use assert in your test scripts. Instead you would typically use the match keyword, that is designed for performing powerful assertions against JSON and XML response payloads.
print
Log to the console
You can use print to log variables to the console in the middle of a script. For convenience, you can have multiple expressions separated by commas, so this is the recommended pattern:
* print 'the value of a is:', a
Similar to assert, the expressions on the right-hand-side of a print have to be valid JavaScript. JsonPath and Karate expressions are not supported.
If you use commas (instead of concatenating strings using +), Karate will 'pretty-print' variables, which is what you typically want when dealing with JSON or XML.
* def myJson = { foo: 'bar', baz: [1, 2, 3] }
* print 'the value of myJson is:', myJson
Which results in the following output:
20:29:11.290 [main] INFO com.intuit.karate - [print] the value of myJson is: {
"foo": "bar",
"baz": [
1,
2,
3
]
}
Since XML is represented internally as a JSON-like or map-like object, if you perform string concatenation when printing, you will not see XML - which can be confusing at first. Use the comma-delimited form (see above) or the JS helper (see below).
The built-in karate object is explained in detail later, but for now, note that this is also injected into print (and even assert) statements, and it has a helpful pretty method, that takes a JSON argument and a prettyXml method that deals with XML. So you could have also done something like:
* print 'the value of myJson is:\n' + karate.pretty(myJson)
Also refer to the configure keyword on how to switch on pretty-printing of all HTTP requests and responses.
'Native' data types
Native data types mean that you can insert them into a script without having to worry about enclosing them in strings and then having to 'escape' double-quotes all over the place. They seamlessly fit 'in-line' within your test script.
JSON
Note that the parser is 'lenient' so that you don’t have to enclose all keys in double-quotes.
Some characters such as the hyphen - are not permitted in 'lenient' JSON keys (because they are interpreted by the JS engine as a 'minus sign'). In such cases, you have to use string quotes: { 'Content-Type': 'application/json' }
When asserting for expected values in JSON or XML, always prefer using match instead of assert. Match failure messages are much more descriptive and useful, and you get the power of embedded expressions and fuzzy matching.
Karate’s native support for JSON means that you can assign parts of a JSON instance into another variable, which is useful when dealing with complex response payloads.
* def first = cats[0]
* match first == { name: 'Billie' }
For manipulating or updating JSON (or XML) using path expressions, refer to the set keyword.
XML
Given def cat = <cat><name>Billie</name><scores><score>2</score><score>5</score></scores></cat>
= sadly, xpath list indexes start from 1
Then match cat/cat/scores/score[2] == '5'
= but karate allows you to traverse xml like json !!
Then match cat.cat.scores.score[1] == 5
Embedded Expressions
Karate has a very useful payload 'templating' approach. Variables can be referred to within JSON, for example:
Given def user = { name: 'john', age: 21 }
And def lang = 'en'
When def session = { name: '#(user.name)', locale: '#(lang)', sessionUser: '#(user)' }
So the rule is - if a string value within a JSON (or XML) object declaration is enclosed between #( and ) - it will be evaluated as a JavaScript expression. And any variables which are alive in the context can be used in this expression. Here’s how it works for XML:
Given def user = <user><name>john</name></user>
And def lang = 'en'
When def session = <session><locale>#(lang)</locale><sessionUser>#(user)</sessionUser></session>
This comes in useful in some cases - and avoids needing to use the set keyword or JavaScript functions to manipulate JSON. So you get the best of both worlds: the elegance of JSON to express complex nested data - while at the same time being able to dynamically plug values (that could even be other JSON or XML 'trees') into a 'template'.
Note that embedded expressions will be evaluated even when you read() from a JSON or XML file. This is super-useful for re-use and data-driven tests.
A few special built-in variables such as $ (which is a reference to the JSON root) - can be mixed into JSON embedded expressions.
A special case of embedded expressions can remove a JSON key (or XML element / attribute) if the expression evaluates to null.
the expression has to start with #( and end with )
Because of the last rule above, note that string-concatenation may not work quite the way you expect:
= wrong !
* def foo = { bar: 'hello #(name)' }
= right !
* def foo = { bar: '#("hello " + name)' }
Observe how you can achieve string concatenation if you really want, because any valid JavaScript expression can be stuffed within an embedded expression. You could always do this in two steps:
* def temp = 'hello ' + name
* def foo = { bar: '#(temp)' }
As a convenience, embedded expressions are supported on the Right Hand Side of a match statement even for "quoted string" literals:
* def foo = 'a1'
* match foo == '#("a" + 1)'
And do note that in Karate 1.0 onwards, ES6 string-interpolation within "backticks" is supported:
* param filter = `ORDER_DATE:"${todaysDate}"`
Enclosed JavaScript
An alternative to embedded expressions (for JSON only) is to enclose the entire payload within parentheses - which tells Karate to evaluate it as pure JavaScript. This can be a lot simpler than embedded expressions in many cases, and JavaScript programmers will feel right at home.
The example below shows the difference between embedded expressions and enclosed JavaScript:
When def user = { name: 'john', age: 21 }
And def lang = 'en'
* def embedded = { name: '#(user.name)', locale: '#(lang)', sessionUser: '#(user)' }
* def enclosed = ({ name: user.name, locale: lang, sessionUser: user })
* match embedded == enclosed
So how would you choose between the two approaches to create JSON ? Embedded expressions are useful when you have complex JSON read from files, because you can auto-replace (or even remove) data-elements with values dynamically evaluated from variables. And the JSON will still be 'well-formed', and editable in your IDE or text-editor. Embedded expressions also make more sense in validation and schema-like short-cut situations. It can also be argued that the # symbol is easy to spot when eyeballing your test scripts - which makes things more readable and clear.
Multi-Line Expressions
The keywords def, set, match, request and eval take multi-line input as the last argument. This is useful when you want to express a one-off lengthy snippet of text in-line, without having to split it out into a separate file. Note how triple-quotes (""") are used to enclose content. Here are some examples:
= instead of:
* def cat = <cat><name>Billie</name><scores><score>2</score><score>5</score></scores></cat>
= this is more readable:
* def cat =
"""
<cat>
<name>Billie</name>
<scores>
<score>2</score>
<score>5</score>
</scores>
</cat>
"""
= example of a request payload in-line
Given request
"""
<?xml version='1.0' encoding='UTF-8'?>
<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/">
<S:Body>
<ns2:QueryUsageBalance xmlns:ns2="http://www.mycompany.com/usage/V1">
<ns2:UsageBalance>
<ns2:LicenseId>12341234</ns2:LicenseId>
</ns2:UsageBalance>
</ns2:QueryUsageBalance>
</S:Body>
</S:Envelope>
"""
= example of a payload assertion in-line
Then match response ==
"""
{ id: { domain: "DOM", type: "entityId", value: "#ignore" },
created: { on: "#ignore" },
lastUpdated: { on: "#ignore" },
entityState: "ACTIVE"
}
"""
table
A simple way to create JSON Arrays
Now that we have seen how JSON is a 'native' data type that Karate understands, there is a very nice way to create JSON using Cucumber’s support for expressing data-tables.
The match keyword is explained later, but it should be clear right away how convenient the table keyword is. JSON can be combined with the ability to call other *.feature files to achieve dynamic data-driven testing in Karate.
Notice that in the above example, string values within the table need to be enclosed in quotes. Otherwise they would be evaluated as expressions - which does come in useful for some dynamic data-driven situations:
* def one = 'hello'
* def two = { baz: 'world' }
* table json
| foo | bar |
| one | { baz: 1 } |
| two.baz | ['baz', 'ban'] |
* match json == [{ foo: 'hello', bar: { baz: 1 } }, { foo: 'world', bar: ['baz', 'ban'] }]
Yes, you can even nest chunks of JSON in tables, and things work as you would expect.
Empty cells or expressions that evaluate to null will result in the key being omitted from the JSON. To force a null value, wrap it in parentheses:
An alternate way to create data is using the set multiple syntax. It is actually a 'transpose' of the table approach, and can be very convenient when there are a large number of keys per row or if the nesting is complex. Here is an example of what is possible:
Not something you would commonly use, but in some cases you need to disable Karate’s default behavior of attempting to parse anything that looks like JSON (or XML) when using multi-line / string expressions. This is especially relevant when manipulating GraphQL queries - because although they look suspiciously like JSON, they are not, and tend to confuse Karate’s internals. And as shown in the example below, having text 'in-line' is useful especially when you use the Scenario Outline: and Examples: for data-driven tests involving Cucumber-style place-holder substitutions in strings.
Scenario Outline:
= note the 'text' keyword instead of 'def'
* text query =
"""
{
hero(name: "<name>") {
height
mass
}
}
"""
Given path 'graphql'
And request { query: '#(query)' }
And header Accept = 'application/json'
When method post
Then status 200
Examples:
| name |
| John |
| Smith |
Note that if you did not need to inject Examples: into 'placeholders' enclosed within < and >, reading from a file with the extension *.txt may have been sufficient.
For placeholder-substitution, the replace keyword can be used instead, but with the advantage that the text can be read from a file or dynamically created.
Karate is a great fit for testing GraphQL because of how easy it is to deal with dynamic and deeply nested JSON responses. Refer to this example for more details: graphql.feature.
replace
Text Placeholder Replacement
Modifying existing JSON and XML is natively supported by Karate via the set keyword, and replace is primarily intended for dealing with raw strings. But when you deal with complex, nested JSON (or XML) - it may be easier in some cases to use replace, especially when you want to substitute multiple placeholders with one value, and when you don’t need array manipulation. Since replace auto-converts the result to a string, make sure you perform type conversion back to JSON (or XML) if applicable.
Karate provides an elegant 'native-like' experience for placeholder substitution within strings or text content. This is useful in any situation where you need to concatenate dynamic string fragments to form content such as GraphQL or SQL.
The placeholder format defaults to angle-brackets, for example: <replaceMe>. Here is how to replace one placeholder at a time:
* def text = 'hello <foo> world'
* replace text.foo = 'bar'
* match text == 'hello bar world'
Karate makes it really easy to substitute multiple placeholders in a single, readable step as follows:
* def text = 'hello <one> world <two> bye'
* replace text
| token | value |
| one | 'cruel' |
| two | 'good' |
* match text == 'hello cruel world good bye'
Note how strings have to be enclosed in quotes. This is so that you can mix expressions into text replacements as shown below. This example also shows how you can use a custom placeholder format instead of the default:
* def text = 'hello <one> world ${two} bye'
* def first = 'cruel'
* def json = { second: 'good' }
* replace text
| token | value |
| one | first |
| ${two} | json.second |
* match text == 'hello cruel world good bye'
For those who may prefer YAML as a simpler way to represent data, Karate allows you to read YAML content from a file - and it will be auto-converted into JSON.
= yaml from a file (the extension matters), and the data-type of 'bar' would be JSON
* def bar = read('data.yaml')
yaml
A very rare need is to be able to convert a string which happens to be in YAML form into JSON, and this can be done via the yaml type cast keyword. For example - if a response data element or downloaded file is YAML and you need to use the data in subsequent steps. Also see type conversion.
* text foo =
"""
name: John
input:
id: 1
subType:
name: Smith
deleted: false
"""
= yaml to json type conversion
* yaml foo = foo
* match foo ==
"""
{
name: 'John',
input: {
id: 1,
subType: { name: 'Smith', deleted: false }
}
}
"""
CSV Files
Karate can read *.csv files and will auto-convert them to JSON. A header row is always expected. See the section on reading files - and also this example dynamic-csv.feature, which shows off the convenience of dynamic Scenario Outline-s.
In rare cases you may want to use a csv-file as-is and not auto-convert it to JSON. A good example is when you want to use a CSV file as the request-body for a file-upload. You could get by by renaming the file-extension to say *.txt but an alternative is to use the karate.readAsString() API.
csv
Just like yaml, you may occasionally need to convert a string which happens to be in CSV form into JSON, and this can be done via the csv keyword.
* text foo =
"""
name,type
Billie,LOL
Bob,Wild
"""
* csv bar = foo
* match bar == [{ name: 'Billie', type: 'LOL' }, { name: 'Bob', type: 'Wild' }]
JavaScript Functions
JavaScript Functions are also 'native'. And yes, functions can take arguments.
Standard JavaScript syntax rules apply, but the right-hand-side should begin with the function keyword if declared in-line. When using stand-alone *.js files, you can have a comment before the function keyword, and you can use fn as the function name, so that your IDE does not complain about JavaScript syntax errors, e.g. function fn(x){ return x + 1 }
* def greeter = function(title, name) { return 'hello ' + title + ' ' + name }
* assert greeter('Mr.', 'Bob') == 'hello Mr. Bob'
When JavaScript executes in Karate, the built-in karate object provides some commonly used utility functions. And with Karate expressions, you can "dive into" JavaScript without needing to define a function - and conditional logic is a good example.
Java Interop
For more complex functions you are better off using the multi-line 'doc-string' approach. This example actually calls into existing Java code, and being able to do this opens up a whole lot of possibilities. The JavaScript interpreter will try to convert types across Java and JavaScript as smartly as possible. For e.g. JSON objects become Java Map-s, JSON arrays become Java List-s, and Java Bean properties are accessible (and update-able) using 'dot notation' e.g. 'object.name'
* def dateStringToLong =
"""
function(s) {
var SimpleDateFormat = Java.type('java.text.SimpleDateFormat');
var sdf = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSZ");
return sdf.parse(s).time; // '.getTime()' would also have worked instead of '.time'
}
"""
* assert dateStringToLong("2016-12-24T03:39:21.081+0000") == 1482550761081
More examples of Java interop and how to invoke custom code can be found in the section on Calling Java.
The call keyword provides an alternate way of calling JavaScript functions that have only one argument. The argument can be provided after the function name, without parentheses, which makes things slightly more readable (and less cluttered) especially when the solitary argument is JSON.
Karate makes re-use of payload data, utility-functions and even other test-scripts as easy as possible. Teams typically define complicated JSON (or XML) payloads in a file and then re-use this in multiple scripts. Keywords such as set and remove allow you to to 'tweak' payload-data to fit the scenario under test. You can imagine how this greatly simplifies setting up tests for boundary conditions. And such re-use makes it easier to re-factor tests when needed, which is great for maintainability.
Note that the set (multiple) keyword can build complex, nested JSON (or XML) from scratch in a data-driven manner, and you may not even need to read from files for many situations. Test data can be within the main flow itself, which makes scripts highly readable.
Reading files is achieved using the built-in JavaScript function called read(). By default, the file is expected to be in the same folder (package) and side-by-side with the *.feature file. But you can prefix the name with classpath: in which case the 'root' folder would be src/test/java (assuming you are using the recommended folder structure).
Prefer classpath: when a file is expected to be heavily re-used all across your project. And yes, relative paths will work.
= json
* def someJson = read('some-json.json')
* def moreJson = read('classpath:more-json.json')
= xml
* def someXml = read('../common/my-xml.xml')
= import yaml (will be converted to json)
* def jsonFromYaml = read('some-data.yaml')
= csv (will be converted to json)
* def jsonFromCsv = read('some-data.csv')
= string
* def someString = read('classpath:messages.txt')
= javascript (will be evaluated)
* def someValue = read('some-js-code.js')
= if the js file evaluates to a function, it can be re-used later using the 'call' keyword (or invoked just like normal js)
* def someFunction = read('classpath:some-reusable-code.js')
* def someCallResult = call someFunction
* def sameCallResult = someFunction()
= the following short-cut is also allowed
* def someCallResult = call read('some-js-code.js')
= perfect for all those common authentication or 'set up' flows
* def result = call read('classpath:some-reusable-steps.feature')
When a called feature depends on some side-by-side resources such as JSON or JS files, you can use the this: prefix to ensure that relative paths work correctly - because by default Karate calculates relative paths from the "root" feature or the top-most "caller".
* def data = read('this:payload.json')
If a file does not end in .json, .xml, .yaml, .js, .csv or .txt, it is treated as a stream - which is typically what you would need for multipart file uploads.
* def someStream = read('some-pdf.pdf')
The .graphql and .gql extensions are also recognized (for GraphQL) but are handled the same way as .txt and treated as a string.
For JSON and XML files, Karate will evaluate any embedded expressions on load. This enables more concise tests, and the file can be re-usable in multiple, data-driven tests.
Since it is internally implemented as a JavaScript function, you can mix calls to read() freely wherever JavaScript expressions are allowed:
you can even use JS expressions to dynamically choose a file based on some condition: * def someConfig = read('my-config-' + someVariable + '.json'). Refer to conditional logic for more ideas.
And a very common need would be to use a file as the request body:
And match response == read('expected-response-payload.json')
The rarely used file: prefix is also supported. You could use it for 'hard-coded' absolute paths in dev mode, but is obviously not recommended for CI test-suites. A good example of where you may need this is if you programmatically write a file to the target folder, and then you can read it like this:
* def payload = read('file:target/large.xml')
Path Prefixes
To summarize the possible prefixes:
Prefix
Description
classpath:
relative to the classpath, recommended for re-usable features
file:
do not use this unless you know what you are doing, see above
this:
when in a called feature, ensure that files are resolved relative to the current feature file
Take a look at the Karate Demos for real-life examples of how you can use files for validating HTTP responses, like this one: read-files.feature.
Read File As String
In some rare cases where you don’t want to auto-convert JSON, XML, YAML or CSV, and just get the raw string content (without having to re-name the file to end with .txt) - you can use the karate.readAsString() API. Here is an example of using a CSV file as the request-body:
Given path 'upload'
And header Content-Type = 'text/csv'
And request karate.readAsString('classpath:my.csv')
When method post
Then status 202
Compare Image
Karate provides a flexible way to compare two images to determine if they are the same or similar. This is especially useful when capturing screenshots during tests and comparing against baseline images that are known to be correct.
Below is a simple example that will compare a baseline image to a more recent latest image. An image comparison UI will also be embedded into the Karate HTML report with detailed information about any differences between the two images.
When true we will scale latest images to match the dimensions of the baseline when they are not equal
engine
string
'resemble'
Comparison engine(s) to use. Valid options are resemble and ssim separated by either , or |
failureThreshold
number
0.0
Precentage of latest image pixels allowed to differ from baseline before we consider the comparison as failed
mismatchShouldPass
boolean
false
When true all image comparisons will pass (even when difference is >= failureThreshold). Note: failures will result in image comparison UI always being embedded in Karate HTML reports regardless of hideUiOnSuccess setting.
onShowRebase
string (js)
null
Function to be called when displaying image comparison rebase in Karate HTML reports (e.g. to customize rebase filename and/or output)
onShowConfig
string (js)
null
Function to be called when displaying image comparison configuration in Karate HTML reports (e.g. to customize configuration output)
hideUiOnSuccess
boolean
false
When true the comparison UI will NOT be embedded in Karate HTML reports for all non-failed image comparisons
Examples:
= use only 'ssim' (structural similarity) engine
* configure imageComparison = { engine: 'ssim' }
= always use both 'resemble' and 'ssim' engines but only evaluate the lowest mismatch percentage against our `failureThreshold`
* configure imageComparison = { engine: 'resemble,ssim' }
= prefer 'resemble' and fallback to 'ssim' engine only if the resemble mismatch percentage is >= `failureThreshold`
* configure imageComparison = { engine: 'resemble|ssim' }
= only consider the comparison as failed when 2% or more pixels are different from the baseline
* configure imageComparison = { failureThreshold: 2 }
= consider image comparisons that fail due to too many mismatched pixels as passed (especially useful when you are first starting without any baseline images)
* configure imageComparison = { mismatchShouldPass: true }
= custom JS function called in Karate HTML image comparison UI when the user clicks the `Rebase` button
* text onShowRebaseFn =
"""
function (config, downloadLatestFn) {
// trigger download of latest image with custom file name
downloadLatestFn('https://raw.githubusercontent.com/karatelabs/karate/main/custom_latest.png')
return 'this text will be displayed to the user when they click the rebase button'
}
"""
* configure imageComparison = { onShowRebase: '#(onShowRebaseFn)' }
= custom JS function called in Karate HTML image comparison UI when the user clicks the `Show config` button
* text onShowConfigFn =
"""
function (customConfigJson, config) {
return 'this text will be displayed above the image comparison config\n' + customConfigJson
}
"""
* configure imageComparison = { onShowConfig: '#(onShowConfigFn)' }
= don't embed the image comparison UI when the latest image is the same / similar to the baseline (e.g. to save space and speed up report loading)
* configure imageComparison = { hideUiOnSuccess: true }
Best practice is to stick to using only def unless there is a very good reason to do otherwise.
Internally, Karate will auto-convert JSON (and even XML) to Java Map objects. And JSON arrays would become Java List-s. But you will never need to worry about this internal data-representation most of the time.
In some rare cases, for e.g. if you acquired a string from some external source, or if you generated JSON (or XML) by concatenating text or using replace, you may want to convert a string to JSON and vice-versa. You can even perform a conversion from XML to JSON if you want.
One example of when you may want to convert JSON (or XML) to a string is when you are passing a payload to custom code via Java interop. Do note that when passing JSON, the default Map and List representations should suffice for most needs (see example), and using them would avoid un-necessary string-conversion.
So you have the following type markers you can use instead of def (or the rarely used text). The first four below are best explained in this example file: type-conv.feature.
string - convert JSON or any other data-type (except XML) to a string
json - convert XML, a map-like or list-like object, a string, or even a Java object into JSON
xml - convert JSON, a map-like object, a string, or even a Java object into XML
xmlstring - specifically for converting the map-like Karate internal representation of XML into a string
bytes - convert to a byte-array, useful for binary payloads or comparisons, see example
copy - to clone a given payload variable reference (JSON, XML, Map or List), refer: copy
The csv and yaml types can be initialized in-line using the "triple quote" or "docstring" multi-line approach as shown here.
If you want to 'pretty print' a JSON or XML value with indenting, refer to the documentation of the print keyword.
Floats and Integers
While converting a number to a string is easy (just concatenate an empty string e.g. myInt + ''), in some rare cases, you may need to convert a string to a number. You can do this by multiplying by 1 or using the built-in JavaScript parseInt() function:
* def foo = '10'
* string json = { bar: '#(1 * foo)' }
* match json == '{"bar":10.0}'
* string json = { bar: '#(parseInt(foo))' }
* match json == '{"bar":10.0}'
As per the JSON spec, all numeric values are treated as doubles, so for integers - it really doesn’t matter if there is a decimal point or not. In fact it may be a good idea to slip doubles instead of integers into some of your tests ! Anyway, there are times when you may want to force integers (perhaps for cosmetic reasons) and you can easily do so using the 'double-tilde' http://rocha.la/JavaScript-bitwise-operators-in-practice~~'].
* def foo = '10'
* string json = { bar: '#(~~foo)' }
* match json == '{"bar":10}'
= JS math can introduce a decimal point in some cases
* def foo = 100
* string json = { bar: '#(foo * 0.1)' }
* match json == '{"bar":10.0}'
= but you can easily coerce to an integer if needed
* string json = { bar: '#(~~(foo * 0.1))' }
* match json == '{"bar":10}'
Large Numbers
Sometimes when dealing with very large numbers, the JS engine may mangle the number into scientific notation:
* def big = 123123123123
* string json = { num: '#(big)' }
* match json == '{"num":1.23123123123E11}'
This can be easily solved by using java.math.BigDecimal:
* def big = new java.math.BigDecimal(123123123123)
* string json = { num: '#(big)' }
* match json == '{"num":123123123123}'
doc
Karate has a built-in HTML templating engine that can be used to insert additional custom HTML into the test-reports. Here is an example:
JsonPath on the variable bar, you can also use get[0] to get the first item if the JsonPath evaluates to an array - especially useful when using wildcards such as [*] or filter-criteria
or callonce, where expressions like read('foo.js') are allowed as the object to be called or the argument
* def foo = bar({ baz: ban })
JS
equivalent to the above, JavaScript function invocation
Core Keywords
They are url, path, request, method and status.
These are essential HTTP operations, they focus on setting one (un-named or 'key-less') value at a time and therefore don’t need an = sign in the syntax.
url
Given url 'https://myhost.com/v1/cats'
A URL remains constant until you use the url keyword again, so this is a good place to set-up the 'non-changing' parts of your REST URL-s.
A URL can take expressions, so the approach below is legal. And yes, variables can come from global config.
Given url 'https://' + e2eHostName + '/v1/api'
If you are trying to build dynamic URLs including query-string parameters in the form: http://myhost/some/path?foo=bar&search=true - please refer to the param keyword.
path
REST-style path parameters. Can be expressions that will be evaluated. Comma delimited values are supported which can be more convenient, and takes care of URL-encoding and appending '/' between path segments as needed.
Given path 'documents', documentId, 'download'
= or you can do the same on multiple lines if you wish
Given path 'documents'
And path documentId
And path 'download'
Note that the path 'resets' after any HTTP request is made but not the url. The Hello World is a great example of 'REST-ful' use of the url when the test focuses on a single REST 'resource'. Look at how the path did not need to be specified for the second HTTP get call since /cats is part of the url.
Important
If you attempt to build a URL in the form ?myparam=value by using path the ? will get encoded into %3F. Use either the param keyword, e.g.: * param myparam = 'value' or url: * url 'http://example.com/v1?myparam'
Because Karate strips trailing slashes if part of a path parameter, if you want to append a forward-slash to the end of the URL in the final HTTP request - make sure that the last path is a single '/'.
Given path 'documents', documentId, '/'
request
In-line JSON:
Given request { name: 'Billie', type: 'LOL' }
In-line XML:
And request <cat><name>Billie</name><type>Ceiling</type></cat>
From a file in the same package. Use the classpath: prefix to load from the classpath instead.
Given request read('my-json.json')
You could always use a variable:
And request myVariable
In most cases you won’t need to set the Content-Typeheader as Karate will automatically do the right thing depending on the data-type of the request.
Defining the request is mandatory if you are using an HTTP method that expects a body such as post. If you really need to have an empty body, you can use an empty string as shown below, and you can force the right Content-Type header by using the header keyword.
Given request ''
And header Content-Type = 'text/html'
Sending a file as the entire binary request body is easy (note that multipart is different):
Given path 'upload'
And request read('https://raw.githubusercontent.com/karatelabs/karate/main/my-image.jpg')
When method put
Then status 200
method
The HTTP verb - get, post, put, delete, patch, options, head, connect, trace.
Lower-case is fine.
When method post
It is worth internalizing that during test-execution, it is upon the method keyword that the actual HTTP request is issued. Which suggests that the step should be in the When form, for example: When method post. And steps that follow should logically be in the Then form. Also make sure that you complete the set up of things like url, param, header, configure etc. before you fire the method.
= set headers or params (if any) BEFORE the method step
Given header Accept = 'application/json'
When method get
= the step that immediately follows the above would typically be:
Then status 200
Although rarely needed, variable references or expressions are also supported:
This is a shortcut to assert the HTTP response code.
Then status 200
And this assertion will cause the test to fail if the HTTP response code is something else.
See also responseStatus if you want to do some complex assertions against the HTTP status code.
Keywords that set key-value pairs
They are param, header, cookie, form field and multipart field.
The syntax will include a '=' sign between the key and the value. The key should not be within quotes.
To make dynamic data-driven testing easier, the following keywords also exist: params, headers, cookies and form fields. They use JSON to build the relevant parts of the HTTP request.
param
Setting query-string parameters:
Given param someKey = 'hello'
And param anotherKey = someVariable
The above would result in a URL like: http://myhost/mypath?someKey=hello&anotherKey=foo. Note that the ? and & will be automatically inserted.
Multi-value params are also supported:
* param myParam = ['foo', 'bar']
For convenience, a null value will be ignored. You can also use JSON to set multiple query-parameters in one-line using params and this is especially useful for dynamic data-driven testing.
Given header Authorization = myAuthFunction()
And header transaction-id = 'test-' + myIdString
It is worth repeating that in most cases you won’t need to set the Content-Type header as Karate will automatically do the right thing depending on the data-type of the request.
Because of how easy it is to set HTTP headers, Karate does not provide any special keywords for things like
the Accept header. You simply do
something like this:
Given path 'some/path'
And request { some: 'data' }
And header Accept = 'application/json'
When method post
Then status 200
A common need is to send the same header(s) for every request, and configure headers (with JSON) is how you can set this up once for all subsequent requests. And if you do this within a Background: section, it would apply to all Scenario: sections within the *.feature file.
Note that Content-Type had to be enclosed in quotes in the JSON above because the "-" (hyphen character) would cause problems otherwise. Also note that "; charset=UTF-8" would be appended to the Content-Type header that Karate sends by default, and in some rare cases, you may need to suppress this behavior completely. You can do so by setting the charset to null via the configure keyword:
* configure charset = null
If you need headers to be dynamically generated for each HTTP request, use a JavaScript function with configure headers instead of JSON.
Multi-value headers (though rarely used in the wild) are also supported:
* header myHeader = ['foo', 'bar']
Also look at the headers keyword which uses JSON and makes some kinds of dynamic data-driven testing easier.
cookie
Setting a cookie:
Given cookie foo = 'bar'
You also have the option of setting multiple cookies in one-step using the cookies keyword.
Note that any cookies returned in the HTTP response would be automatically set for any future requests. This mechanism works by calling configure cookies behind the scenes and if you need to stop auto-adding cookies for future requests, just do this:
* configure cookies = null
Also refer to the built-in variable responseCookies for how you can access and perform assertions on cookie data values.
form field
HTML form fields would be URL-encoded when the HTTP request is submitted (by the method step). You would typically use these to simulate a user sign-in and then grab a security token from the response.
Note that the Content-Type header will be automatically set to: application/x-www-form-urlencoded. You just need to do a normal POST (or GET).
For example:
Given path 'login'
And form field username = 'john'
And form field password = 'secret'
When method post
Then status 200
And def authToken = response.token
A good example of the use of form field for a typical sign-in flow is this OAuth 2 demo: oauth2.feature.
Multi-values are supported the way you would expect (e.g. for simulating check-boxes and multi-selects):
* form field selected = ['apple', 'orange']
You can also dynamically set multiple fields in one step using the form fields keyword.
multipart field
Use this for building multipart named (form) field requests. This is typically combined with multipart file as shown below.
Given multipart file myFile = { read: 'test.pdf', filename: 'upload-name.pdf', contentType: 'application/pdf' }
And multipart field message = 'hello world'
When method post
Then status 200
It is important to note that myFile above is the "field name" within the multipart/form-data request payload. This roughly corresponds to a cURL argument of -F @myFile=test.pdf.
multipart file uploads can be tricky, and hard to get right. If you get stuck and ask a question on Stack Overflow, make sure you provide a cURL command that works - or else it would be very difficult for anyone to troubleshoot what you could be doing wrong. Also see this thread.
Also note that multipart file takes a JSON argument so that you can easily set the filename and the contentType (mime-type) in one step.
read: the name of a file, and the classpath: prefix also is allowed. mandatory unless value is used, see below.
value: alternative to read in rare cases where something like a JSON or XML file is being uploaded and you want to create it dynamically.
filename: optional, if not specified there will be no filename attribute in Content-Disposition
contentType: optional, will default to application/octet-stream
When 'multipart' content is involved, the Content-Type header of the HTTP request defaults to multipart/form-data.
You can over-ride it by using the header keyword before the method step. Look at
multipart entity for an example.
You can also dynamically set multiple files in one step using multipart files.
multipart entity
This is technically not in the key-value form: multipart field name = 'foo', but logically
belongs here in the documentation.
Use this for multipart content items that don’t have field-names. Here below is an example that
also demonstrates using the multipart/related content-type.
Given path 'v2', 'documents'
And multipart entity read('foo.json')
And multipart field image = read('https://raw.githubusercontent.com/karatelabs/karate/main/bar.jpg')
And header Content-Type = 'multipart/related'
When method post
Then status 201
Multi-Param Keywords
Keywords that set multiple key-value pairs in one step
params, headers, cookies, form fields, multipart fields and multipart files take a single JSON argument (which can be in-line or a variable reference), and this enables certain types of dynamic data-driven testing, especially because any JSON key with a null value will be ignored. Here is a good example in the demos: dynamic-params.feature
The single JSON argument needs to be in the form { field1: { read: 'file1.ext' }, field2: { read: 'file2.ext' } } where each nested JSON is in the form expected by multipart file
* def json = {}
* set json.myFile1 = { read: 'test1.pdf', filename: 'upload-name1.pdf', contentType: 'application/pdf' }
= if you have dynamic keys you can do this
* def key = 'myFile2'
* json[key] = { read: 'test2.pdf', filename: 'upload-name2.pdf', contentType: 'application/pdf' }
And multipart files json
Since a SOAP request needs special handling, this is the only case where the
method step is not used to actually fire the request to the server.
soap action
The name of the SOAP action specified is used as the 'SOAPAction' header. Here is an example
which also demonstrates how you could assert for expected values in the response XML.
Given request read('soap-request.xml')
When soap action 'QueryUsageBalance'
Then status 200
And match response /Envelope/Body/QueryUsageBalanceResponse/Result/Error/Code == 'DAT_USAGE_1003'
And match response /Envelope/Body/QueryUsageBalanceResponse == read('expected-response.xml')
A working example of calling a SOAP service can be found within the Karate project test-suite. Refer to the demos for another example: soap.feature.
More examples are available that showcase various ways of parameter-izing and dynamically manipulating SOAP requests in a data-driven fashion. Karate is quite flexible, and provides multiple options for you to evolve patterns that fit your environment, as you can see here: xml.feature.
retry until
Karate has built-in support for re-trying an HTTP request until a certain condition has been met. The default setting for the max retry-attempts is 3 with a poll interval of 3000 milliseconds (3 seconds). If needed, this can be changed by using configure - any time during a test, or set globally via karate-config.js
* configure retry = { count: 10, interval: 5000 }
The retry keyword is designed to extend the existing method syntax (and should appear before a method step) like so:
Given url demoBaseUrl
And path 'greeting'
And retry until response.id > 3
When method get
Then status 200
Any JavaScript expression that uses any variable in scope can be placed after the "retry until" part. So you can refer to the response, responseStatus or even responseHeaders if needed. For example:
Given url demoBaseUrl
And path 'greeting'
And retry until responseStatus == 200 && response.id > 3
When method get
Note that it has to be a pure JavaScript expression - which means that match syntax such as contains will not work. But you can easily achieve any complex logic by using the JS API.
Refer to polling.feature for an example, and also see the alternative way to achieve polling.
configure
Managing Headers, SSL, Timeouts and HTTP Proxy
You can adjust configuration settings for the HTTP client used by Karate using this keyword. The syntax is similar to def but instead of a named variable, you update configuration. Here are the configuration keys supported:
Just like configure headers, but for cookies. You will typically never use this, as response cookies are auto-added to all future requests. If you need to clear cookies at any time, just do configure cookies = null
logPrettyRequest
boolean
Pretty print the request payload JSON or XML with indenting (default false)
logPrettyResponse
boolean
Pretty print the response payload JSON or XML with indenting (default false)
printEnabled
boolean
Can be used to suppress the print output when not in 'dev mode' by setting as false (default true)
Whether the HTTP client automatically follows redirects - (default true), refer to this example.
connectTimeout
integer
Set the connect timeout (milliseconds). The default is 30000 (30 seconds). Note that for karate-apache, this sets the socket timeout to the same value as well.
readTimeout
integer
Set the read timeout (milliseconds). The default is 30000 (30 seconds).
proxy
string
Set the URI of the HTTP proxy to use.
proxy
JSON
For a proxy that requires authentication, set the uri, username and password, see example below. Also a nonProxyHosts key is supported which can take a list for e.g. { uri: 'http://my.proxy.host:8080', nonProxyHosts: ['host1', 'host2']}
The charset that will be sent in the request Content-Type which defaults to utf-8. You typically never need to change this, and you can over-ride (or disable) this per-request if needed via the header keyword (example).
retry
JSON
defaults to { count: 3, interval: 3000 } - see retry until
Converts every key in the responseHeaders to lower-case which makes it easier to validate or re-use
abortedStepsShouldPass
boolean
defaults to false, whether steps after a karate.abort() should be marked as PASSED instead of SKIPPED - this can impact the behavior of 3rd-party reports, see this issue for details
defaults to false, to not attempt to run any more tests upon a failure
Examples:
= pretty print the response payload
* configure logPrettyResponse = true
= enable ssl (and no certificate is required)
* configure ssl = true
= enable ssl and force the algorithm to TLSv1.2
* configure ssl = 'TLSv1.2'
= time-out if the response is not received within 10 seconds (after the connection is established)
* configure readTimeout = 10000
= set the uri of the http proxy server to use
* configure proxy = 'http://my.proxy.host:8080'
= proxy which needs authentication
* configure proxy = { uri: 'http://my.proxy.host:8080', username: 'john', password: 'secret' }
In rare cases where you need to add nested non-JSON data to the configure value, you have to play by the rules that apply within karate-config.js. Here is an example of performing a configure driver step in JavaScript:
By default, Karate will add logs to the report output so that HTTP requests and responses appear in-line in the HTML reports. There may be cases where you want to suppress this to make the reports "lighter" and easier to read.
The configure key here is report and it takes a JSON value. For example:
HTTP requests and responses (including headers) will appear in the HTML report, default true
showAllSteps
boolean
If false, any step that starts with * instead of Given, When, Then etc. will not appear in the HTML report. The print step is an exception. Default true.
You can 'reset' default settings by using the following short-cut:
= reset to defaults
* configure report = true
Since you can use configure any time within a test, you have control over which requests or steps you want to show / hide. This can be convenient if a particular call results in a huge response payload.
The following short-cut is also supported which will disable all logs:
* configure report = false
@report=false
When you use a re-usable feature that has commonly used utilities, you may want to hide this completely from the HTML reports. The special tag@report=false can be used, and it can even be used only for a single Scenario:
@ignore @report=false
Feature:
Scenario:
= some re-usable steps
Log Masking
In cases where you want to "mask" values which are sensitive from a security point of view from the output files, logs and HTML reports, you can implement the HttpLogModifier and tell Karate to use it via the configure keyword. Here is an example of an implementation. For performance reasons, you can implement enableForUri() so that this "activates" only for some URL patterns.
Instantiating a Java class and using this in a test is easy (see example):
= if this was in karate-config.js, it would apply "globally"
* def LM = Java.type('demo.headers.DemoLogModifier')
* configure logModifier = new LM()
Also referred to as "mutual auth" - if your API requires that clients present an X509 certificate for authentication, Karate supports this via JSON as the configure ssl value. The following parameters are supported:
Key
Type
Required?
Description
keyStore
string
optional
path to file containing public and private keys for your client certificate.
keyStorePassword
string
optional
password for keyStore file.
keyStoreType
string
optional
Format of the keyStore file. Allowed keystore types are as described in the Java KeyStore docs.
trustStore
string
optional
path to file containing the trust chain for your server certificate.
trustStorePassword
string
optional
password for trustStore file.
trustStoreType
string
optional
Format of the trustStore file. Allowed keystore types are as described in the Java KeyStore docs.
trustAll
boolean
optional
if all server certificates should be considered trusted. Default value is false. If true will allow self-signed certificates. If false, will expect the whole chain in the trustStore or use what is available in the environment.
algorithm
string
optional
force the SSL algorithm to one of these values. Default is TLS.
= trust all server certificates, in the feature file
* configure ssl = { trustAll: true }
// trust all server certificates, global configuration in 'karate-config.js'
karate.configure('ssl', { trustAll: true });
For end-to-end examples in the Karate demos, look at the files in this folder.
Payload Assertions
Prepare, Mutate, Assert.
Now it should be clear how Karate makes it easy to express JSON or XML. If you read from a file, the advantage is that multiple scripts can re-use the same data.
Once you have a JSON or XML object, Karate provides multiple ways to manipulate, extract or transform data. And you can easily assert that the data is as expected by comparing it with another JSON or XML object.
match
Payload Assertions / Smart Comparison
The match operation is smart because white-space does not matter, and the order of keys (or data elements) does not matter. Karate is even able to ignore fields you choose - which is very useful when you want to handle server-side dynamically generated fields such as UUID-s, time-stamps, security-tokens and the like.
The match syntax involves a double-equals sign '==' to represent a comparison (and not an assignment '=').
Since match and set go well together, they are both introduced in the examples in the section below.
set
Game, set and match - Karate !
JS for JSON
Before you consider the set keyword - note that for simple JSON update operations, you can use eval - especially useful when the path you are trying to mutate is dynamic. Since the eval keyword can be omitted when operating on variables using JavaScript, this leads to very concise code:
Setting values on JSON documents is simple using the set keyword.
* def myJson = { foo: 'bar' }
* set myJson.foo = 'world'
* match myJson == { foo: 'world' }
= add new keys. you can use pure JsonPath expressions (notice how this is different from the above)
* set myJson $.hey = 'ho'
* match myJson == { foo: 'world', hey: 'ho' }
= and even append to json arrays (or create them automatically)
* set myJson.zee[0] = 5
* match myJson == { foo: 'world', hey: 'ho', zee: [5] }
= omit the array index to append
* set myJson.zee[] = 6
* match myJson == { foo: 'world', hey: 'ho', zee: [5, 6] }
= nested json ? no problem
* set myJson.cat = { name: 'Billie' }
* match myJson == { foo: 'world', hey: 'ho', zee: [5, 6], cat: { name: 'Billie' } }
= and for match - the order of keys does not matter
* match myJson == { cat: { name: 'Billie' }, hey: 'ho', foo: 'world', zee: [5, 6] }
= you can ignore fields marked with '#ignore'
* match myJson == { cat: '#ignore', hey: 'ho', foo: 'world', zee: [5, 6] }
XML and XPath works just like you’d expect.
* def cat = <cat><name>Billie</name></cat>
* set cat /cat/name = 'Jean'
* match cat / == <cat><name>Jean</name></cat>
= you can even set whole fragments of xml
* def xml = <foo><bar>baz</bar></foo>
* set xml/foo/bar = <hello>world</hello>
* match xml == <foo><bar><hello>world</hello></bar></foo>
Refer to the section on XPath Functions for examples of advanced XPath usage.
match and variables
In case you were wondering, variables (and even expressions) are supported on the right-hand-side. So you can compare 2 JSON (or XML) payloads if you wanted to:
* def foo = { hello: 'world', baz: 'ban' }
* def bar = { baz: 'ban', hello: 'world' }
* match foo == bar
If you are wondering about the finer details of the match syntax, the Left-Hand-Side has to be either a
variable name - e.g. foo
a 'named' JsonPath or XPath expression - e.g. foo[0].bar or foo[*].bar
note that this cannot be "dynamic" (with in-line variables) so use an extra step if needed
any valid function or method call - e.g. foo.bar() or foo.bar('hello').baz
or anything wrapped in parentheses which will be evaluated as JavaScript - e.g. (foo + bar) or (42) - and in this case, variables can be used
And the right-hand-side can be any valid Karate expression. Refer to the section on JsonPath short-cuts for a deeper understanding of 'named' JsonPath expressions in Karate.
match != (not equals)
The 'not equals' operator != works as you would expect:
* def test = { foo: 'bar' }
* match test != { foo: 'baz' }
You typically will never need to use the != (not-equals) operator ! Use it sparingly, and only for string, number or simple payload comparisons.
set multiple
Karate has an elegant way to set multiple keys (via path expressions) in one step. For convenience, non-existent keys (or array elements) will be created automatically. You can find more JSON examples here: js-arrays.feature.
* def cat = { name: '' }
* set cat
| path | value |
| name | 'Bob' |
| age | 5 |
* match cat == { name: 'Bob', age: 5 }
One extra convenience for JSON is that if the variable itself (which was cat in the above example) does not exist, it will be created automatically. You can even create (or modify existing) JSON arrays by using multiple columns.
* set foo
| path | 0 | 1 |
| bar | 'baz' | 'ban' |
* match foo == [{ bar: 'baz' }, { bar: 'ban' }]
If you have to set a bunch of deeply nested keys, you can move the parent path to the top, next to the set keyword and save a lot of typing ! Note that this is not supported for "arrays" like above, and you can have only one value column.
* set foo.bar
| path | value |
| one | 1 |
| two[0] | 2 |
| two[1] | 3 |
* match foo == { bar: { one: 1, two: [2, 3] } }
The same concept applies to XML and you can build complicated payloads from scratch in just a few, extremely readable lines. The value column can take expressions, even XML chunks. You can find more examples here: xml.feature.
This is like the opposite of set if you need to remove keys or data elements from JSON or XML instances. You can even remove JSON array elements by index.
* def xml = <foo><bar><hello>world</hello></bar></foo>
* remove xml/foo/bar/hello
* match xml == <foo><bar/></foo>
* remove xml /foo/bar
* match xml == <foo/>
When expressing expected results (in JSON or XML) you can mark some fields to be ignored when the match (comparison) is performed. You can even use a regular-expression so that instead of checking for equality, Karate will just validate that the actual value conforms to the expected pattern.
This means that even when you have dynamic server-side generated values such as UUID-s and time-stamps appearing in the response, you can still assert that the full-payload matched in one step.
Note that #present and #notpresent only make sense when you are matching within a JSON or XML context or using a JsonPath or XPath on the left-hand-side.
The rest can also be used even in 'primitive' data matches like so:
* match foo == '#string'
= convenient (and recommended) way to check for array length
* match bar == '#[2]'
Optional Fields
If two cross-hatch symbols are used as the prefix (for example: #number), it means that the key is optional or that the value can be null.
* def foo = { bar: 'baz' }
* match foo == { bar: '#string', ban: '##string' }
Remove If Null
A very useful behavior when you combine the optional marker with an embedded expression is as follows: if the embedded expression evaluates to null - the JSON key (or XML element or attribute) will be deleted from the payload (the equivalent of remove).
* def data = { a: 'hello', b: null, c: null }
* def json = { foo: '#(data.a)', bar: '#(data.b)', baz: '##(data.c)' }
* match json == { foo: 'hello', bar: null }
If you are just trying to pre-define schema snippets to use in a fuzzy-match, you can use enclosed Javascript to suppress the default behavior of replacing placeholders. For example:
* def dogSchema = { id: '#string', color: '#string' }
= here we enclose in round-brackets to preserve the optional embedded expression
= so that it can be used later in a "match"
* def schema = ({ id: '#string', name: '#string', dog: '##(dogSchema)' })
* def response1 = { id: '123', name: 'foo' }
* match response1 == schema
And if you need to suppress placeholder substitution for read(), but still need a JSON snippet, you can do this. Note how we read as a string, but "cast" to JSON:
If you want to use the triple-quote / multi-line way of defining JSON or if you have to use XML - you can use text and "cast" to JSON or XML as a second step - before using in a match:
* text schema =
"""
<root>
<a>#string</a>
<b>##(subSchema)</b>
</root>
"""
* xml schema = schema
#null and #notpresent
Karate’s match is strict, and the case where a JSON key exists but has a null value (#null) is considered different from the case where the key is not present at all (#notpresent) in the payload.
But note that null can be used to represent a convention that many teams adopt, which is that keys with null values are stripped from the JSON payload. In other words, { a: 1, b: null } is considered 'equal' to { a: 1 } and { a: 1, b: 'null' } will match both cases.
These examples (all exact matches) can make things more clear:
* def foo = { }
* match foo == { a: '##null' }
* match foo == { a: '##notnull' }
* match foo == { a: '#notpresent' }
* match foo == { a: '#ignore' }
* def foo = { a: null }
* match foo == { a: '#null' }
* match foo == { a: '##null' }
* match foo == { a: '#present' }
* match foo == { a: '#ignore' }
* def foo = { a: 1 }
* match foo == { a: '#notnull' }
* match foo == { a: '##notnull' }
* match foo == { a: '#present' }
* match foo == { a: '#ignore' }
Note that you can alternatively use JsonPath on the left-hand-side:
* def foo = { a: 1 }
* match foo.a == '#present'
* match foo.nope == '#notpresent'
But of course it is preferable to match whole objects in one step as far as possible.
'Self' Validation Expressions
The special 'predicate' marker ? EXPR in the table above is an interesting one. It is best explained via examples. Any valid JavaScript expression that evaluates to a Truthy or Falsy value is expected after the ?.
Observe how the value of the field being validated (or 'self') is injected into the 'underscore' expression variable: '_'
* def date = { month: 3 }
* match date == { month: '#? _ > 0 && _ < 13' }
What is even more interesting is that expressions can refer to variables:
* def date = { month: 3 }
* def min = 1
* def max = 12
* match date == { month: '#? _ >= min && _ <= max' }
And functions work as well ! You can imagine how you could evolve a nice set of utilities that validate all your domain objects.
* def date = { month: 3 }
* def isValidMonth = function(m) { return m >= 1 && m <= 12 }
* match date == { month: '#? isValidMonth(_)' }
Especially since strings can be easily coerced to numbers (and vice-versa) in Javascript, you can combine built-in validators with the self-validation 'predicate' form like this: '#number? _ > 0'
= given this invalid input (string instead of number)
* def date = { month: '3' }
= this will pass
* match date == { month: '#? _ > 0' }
= but this 'combined form' will fail, which is what we want
= * match date == { month: '#number? _ > 0' }
Referring to the JSON root
You can actually refer to any JsonPath on the document via $ and perform cross-field or conditional validations ! This example uses contains and the #? 'predicate' syntax, and situations where this comes in useful will be apparent when we discuss match each.
Given def temperature = { celsius: 100, fahrenheit: 212 }
Then match temperature == { celsius: '#number', fahrenheit: '#? _ == $.celsius * 1.8 + 32' }
= when validation logic is an 'equality' check, an embedded expression works better
Then match temperature contains { fahrenheit: '#($.celsius * 1.8 + 32)' }
match text or binary
= when the response is plain-text
Then match response == 'Health Check OK'
And match response != 'Error'
= when the response is binary (byte-array)
Then match responseBytes == read('test.pdf')
= incidentally, match and assert behave exactly the same way for strings
* def hello = 'Hello World!'
* match hello == 'Hello World!'
* assert hello == 'Hello World!'
Checking if a string is contained within another string is a very common need and match (name) contains works just like you’d expect:
* def hello = 'Hello World!'
* match hello contains 'World'
* match hello !contains 'blah'
Since asserting against header values in the response is a common task - match header has a special meaning. It short-cuts to the pre-defined variable responseHeaders and reduces some complexity - because strictly, HTTP headers are a 'multi-valued map' or a 'map of lists' - the Java-speak equivalent being Map<String, List<String>>. And since header names are case-insensitive - it ignores the case when finding the header to match.
= so after a http request
Then match header Content-Type == 'application/json'
= 'contains' works as well
Then match header Content-Type contains 'application'
Note the extra convenience where you don’t have to enclose the LHS key in quotes.
You can always directly access the variable called responseHeaders if you wanted to do more checks, but you typically won’t need to.
match and XML
All the fuzzy matching markers will work in XML as well. Here are some examples:
* def xml = <root><hello>world</hello><foo>bar</foo></root>
* match xml == <root><hello>world</hello><foo>#ignore</foo></root>
* def xml = <root><hello foo="bar">world</hello></root>
* match xml == <root><hello foo="#ignore">world</hello></root>
Refer to this file for a comprehensive set of XML examples: xml.feature.
Matching Sub-Sets of JSON Keys and Arrays
match contains
JSON Keys
In some cases where the response JSON is wildly dynamic, you may want to only check for the existence of some keys. And match (name) contains is how you can do so:
* def foo = { bar: 1, baz: 'hello', ban: 'world' }
* match foo contains { bar: 1 }
* match foo contains { baz: 'hello' }
* match foo contains { bar:1, baz: 'hello' }
= this will fail
= * match foo == { bar:1, baz: 'hello' }
Note that match contains will not "recurse" any nested JSON chunks so use match contains deep instead.
It is sometimes useful to be able to check if a key-value-pair does not exist. This is possible by prefixing contains with a ! (with no space in between).
* def foo = { bar: 1, baz: 'hello', ban: 'world' }
* match foo !contains { bar: 2 }
* match foo !contains { huh: '#notnull' }
Here’s a reminder that the #notpresent marker can be mixed into an equality match (==) to assert that some keys exist and at the same time ensure that some keys do not exist:
* def foo = { a: 1 }
* match foo == { a: '#number', b: '#notpresent' }
= if b can be present (optional) but should always be null
* match foo == { a: '#number', b: '##null' }
The ! (not) operator is especially useful for contains and JSON arrays.
* def foo = [1, 2, 3]
* match foo !contains 4
* match foo !contains [5, 6]
JSON Arrays
This is a good time to deep-dive into JsonPath, which is perfect for slicing and dicing JSON into manageable chunks. It is worth taking a few minutes to go through the documentation and examples here: JsonPath Examples.
Here are some example assertions performed while scraping a list of child elements out of the JSON below. Observe how you can match the result of a JsonPath expression with your expected data.
Given def cat =
"""
{
name: 'Billie',
kittens: [
{ id: 23, name: 'Bob' },
{ id: 42, name: 'Wild' }
]
}
"""
= normal 'equality' match. note the wildcard '*' in the JsonPath (returns an array)
Then match cat.kittens[*].id == [23, 42]
= when inspecting a json array, 'contains' just checks if the expected items exist
= and the size and order of the actual array does not matter
Then match cat.kittens[*].id contains 23
Then match cat.kittens[*].id contains [42]
Then match cat.kittens[*].id contains [23, 42]
Then match cat.kittens[*].id contains [42, 23]
= the .. operator is great because it matches nodes at any depth in the JSON "tree"
Then match cat..name == ['Billie', 'Bob', 'Wild']
= and yes, you can assert against nested objects within JSON arrays !
Then match cat.kittens contains [{ id: 42, name: 'Wild' }, { id: 23, name: 'Bob' }]
= ... and even ignore fields at the same time !
Then match cat.kittens contains { id: 42, name: '#string' }
It is worth mentioning that to do the equivalent of the last line in Java, you would typically have to traverse 2 Java Objects, one of which is within a list, and you would have to check for nulls as well.
When you use Karate, all your data assertions can be done in pure JSON and without needing a thick forest of companion Java objects. And when you read your JSON objects from (re-usable) files, even complex response payload assertions can be accomplished in just a single line of Karate-script.
Refer to this case study for how dramatic the reduction of lines of code can be.
match contains only
For those cases where you need to assert that all array elements are present but in any order
you can do this:
* def data = { foo: [1, 2, 3] }
* match data.foo contains 1
* match data.foo contains [2]
* match data.foo contains [3, 2]
* match data.foo contains only [3, 2, 1]
* match data.foo contains only [2, 3, 1]
= this will fail
= * match data.foo contains only [2, 3]
match contains any
To assert that any of the given array elements are present.
* def data = { foo: [1, 2, 3] }
* match data.foo contains any [9, 2, 8]
And this happens to work as expected for JSON object keys as well:
* def data = { a: 1, b: 'x' }
* match data contains any { b: 'x', c: true }
match contains deep
This modifies the behavior of match contains so that nested lists or objects are processed for a "deep contains" match instead of a "deep equals" one which is the default. This is convenient for complex nested payloads where you are sure that you only want to check for some values in the various "trees" of data.
the NOT operator e.g. !contains deep is not yet supported, please contribute code if you can.
match contains only deep
This is exactly like match == but the order of arrays does not matter. All arrays no matter the "depth" will be checked in this way.
* def response = { foo: [ 'a', 'b' ] }
* match response contains only deep { foo: [ 'b', 'a' ] }
Validate every element in a JSON array
match each
The match keyword can be made to iterate over all elements in a JSON array using the each modifier. Here’s how it works:
* def data = { foo: [{ bar: 1, baz: 'a' }, { bar: 2, baz: 'b' }, { bar: 3, baz: 'c' }]}
* match each data.foo == { bar: '#number', baz: '#string' }
= and you can use 'contains' the way you'd expect
* match each data.foo contains { bar: '#number' }
* match each data.foo contains { bar: '#? _ != 4' }
= some more examples of validation macros
* match each data.foo contains { baz: "#? _ != 'z'" }
* def isAbc = function(x) { return x == 'a' || x == 'b' || x == 'c' }
* match each data.foo contains { baz: '#? isAbc(_)' }
= this is also possible, see the subtle difference from the above
* def isXabc = function(x) { return x.baz == 'a' || x.baz == 'b' || x.baz == 'c' }
* match each data.foo == '#? isXabc(_)'
Here is a contrived example that uses match each, contains and the #? 'predicate' marker to validate that the value of totalPrice is always equal to the roomPrice of the first item in the roomInformation array.
Given def json =
"""
{
"hotels": [
{ "roomInformation": [{ "roomPrice": 618.4 }], "totalPrice": 618.4 },
{ "roomInformation": [{ "roomPrice": 679.79}], "totalPrice": 679.79 }
]
}
"""
Then match each json.hotels contains { totalPrice: '#? _ == _$.roomInformation[0].roomPrice' }
= when validation logic is an 'equality' check, an embedded expression works better
Then match each json.hotels contains { totalPrice: '#(_$.roomInformation[0].roomPrice)' }
Referring to self
While $ always refers to the JSON 'root', note the use of _$ above to represent the 'current' node of a match each iteration. Here is a recap of symbols that can be used in JSON embedded expressions:
Karate provides a far more simpler and more powerful way than JSON-schema to validate the structure of a given payload. You can even mix domain and conditional validations and perform all assertions in a single step.
But first, a special short-cut for array validation needs to be introduced:
* def foo = ['bar', 'baz']
= should be an array
* match foo == '#[]'
= should be an array of size 2
* match foo == '#[2]'
= should be an array of strings with size 2
* match foo == '#[2] #string'
= each array element should have a 'length' property with value 3
* match foo == '#[]? _.length == 3'
= should be an array of strings each of length 3
* match foo == '#[] #string? _.length == 3'
= should be null or an array of strings
* match foo == '##[] #string'
This 'in-line' short-cut for validating JSON arrays is similar to how match each works. So now, complex payloads (that include arrays) can easily be validated in one step by combining validation markers like so:
Especially note the re-use of the oddSchema both as an embedded-expression and as an array validation (on the last line).
And you can perform conditional / cross-field validations and even business-logic validations at the same time.
= optional (can be null) and if present should be an array of size greater than zero
* match $.odds == '##[_ > 0]'
= should be an array of size equal to $.count
* match $.odds == '#[$.count]'
= use a predicate function to validate each array element
* def isValidOdd = function(o){ return o.name.length == 1 }
* match $.odds == '#[]? isValidOdd(_)'
And there is another example in the karate-demos: schema.feature where you can compare Karate’s approach with an actual JSON-schema example. You can also find a nice visual comparison and explanation here.
contains short-cuts
Especially when payloads are complex (or highly dynamic), it may be more practical to use contains semantics. Karate has the following short-cut symbols designed to be mixed into embedded expressions:
For completeness, == and != also belong in the above list.
Here’a table of the alternative 'in-line' forms compared with the 'standard' form. Note that all the short-cut forms on the right-side of the table resolve to 'equality' (==) matches, which enables them to be 'in-lined' into a full (single-step) payload match, using embedded expressions.
A very useful capability is to be able to check that an array contains an object that contains the provided sub-set of keys instead of having to specify the complete JSON - which can get really cumbersome for large objects. This turns out to be very useful in practice, and this particular matchjsonArray+contains '#(^+`partialObject)'` form has no 'in-line' equivalent (see the third-from-last row above).
The last row in the table is a little different from the rest, and this short-cut form is the recommended way to validate the length of a JSON array. As a rule of thumb, prefer match over assert, because match failure messages are more detailed and descriptive.
In real-life tests, these are very useful when the order of items in arrays returned from the server are not guaranteed. You can easily assert that all expected elements are present, even in nested parts of your JSON - while doing a match on the full payload.
There’s a lot going on in the last line above ! It validates the entire payload in one step and checks if the kittens array *contains all* the expected items but in *any order*.
get
By now, it should be clear that link:(JsonPath can be very useful for extracting JSON 'trees' out of a given object. The get keyword allows you to save the results of a JsonPath expression for later use - which is especially useful for dynamic data-driven testing.
The 'short cut' $variableName form is also supported. Refer to JsonPath short-cuts for a detailed explanation. So the above could be re-written as follows:
It is worth repeating that the above can be condensed into 2 lines. Note that since only JsonPath is expected on the left-hand-side of the == sign of a match statement, you don’t need to prefix the variable reference with $:
* match cat.kittens[*].id == [23, 42]
* match cat.kittens[*].name == ['Bob', 'Wild']
= if you prefer using 'pure' JsonPath, you can do this
* match cat $.kittens[*].id == [23, 42]
* match cat $.kittens[*].name == ['Bob', 'Wild']
get plus index
A convenience that the get syntax supports (but not the $ short-cut form) is to return a single element if the right-hand-side evaluates to a list-like result (e.g. a JSON array). This is useful because the moment you use a wildcard [*] or search filter in JsonPath (see the next section), you get an array back - even though typically you would only be interested in the first item.
* def actual = 23
= so instead of this
* def kitnums = get cat.kittens[*].id
* match actual == kitnums[0]
= you can do this in one line
* match actual == get[0] cat.kittens[*].id
JsonPath filters
JsonPath filter expressions are very useful for extracting elements that meet some filter criteria out of arrays.
* def cat =
"""
{
name: 'Billie',
kittens: [
{ id: 23, name: 'Bob' },
{ id: 42, name: 'Wild' }
]
}
"""
= find single kitten where id == 23
* def bob = get[0] cat.kittens[?(@.id==23)]
* match bob.name == 'Bob'
= using the karate object if the expression is dynamic
* def temp = karate.jsonPath(cat, "$.kittens[?(@.name=='" + bob.name + "')]")
* match temp[0] == bob
= or alternatively
* def temp = karate.jsonPath(cat, "$.kittens[?(@.name=='" + bob.name + "')]")[0]
* match temp == bob
You usually won’t need this, but the second-last line above shows how the karate object can be used to evaluate JsonPath if the filter expression depends on a variable. If you find yourself struggling to write dynamic JsonPath filters, look at karate.filter() as an alternative, described just below.
JSON Transforms
Karate supports the following functional-style operations via the JS API - karate.map(), karate.filter() and karate.forEach(). They can be very useful in some situations. A good example is when you have the expected data available as ready-made JSON but it is in a different "shape" from the actual data or HTTP response. There is also a karate.mapWithKey() for a common need - which is to convert an array of primitives into an array of objects, which is the form that data driven features expect.
A few more useful "transforms" are to select a sub-set of key-value pairs using karate.filterKeys(), merging 2 or more JSON-s using karate.merge() and combining 2 or more arrays (or objects) into a single array using karate.append(). And karate.appendTo() is for updating an existing variable (the equivalent of array.push() in JavaScript), which is especially useful in the body of a karate.forEach().
You can also sort arrays of arbitrary JSON using karate.sort(). Simple arrays of strings or numbers can be stripped of duplicates using karate.distinct(). All JS "native" array operations can be used, such as someName.reverse().
Note that a single JS function is sufficient to transform a given JSON object into a completely new one, and you can use complex conditional logic if needed.
Scenario: karate map operation
* def fun = function(x){ return x * x }
* def list = [1, 2, 3]
* def res = karate.map(list, fun)
* match res == [1, 4, 9]
Scenario: convert an array into a different shape
* def before = [{ foo: 1 }, { foo: 2 }, { foo: 3 }]
* def fun = function(x){ return { bar: x.foo } }
* def after = karate.map(before, fun)
* match after == [{ bar: 1 }, { bar: 2 }, { bar: 3 }]
Scenario: convert array of primitives into array of objects
* def list = [ 'Bob', 'Wild', 'Nyan' ]
* def data = karate.mapWithKey(list, 'name')
* match data == [{ name: 'Bob' }, { name: 'Wild' }, { name: 'Nyan' }]
Scenario: karate filter operation
* def fun = function(x){ return x % 2 == 0 }
* def list = [1, 2, 3, 4]
* def res = karate.filter(list, fun)
* match res == [2, 4]
Scenario: forEach works even on object key-values, not just arrays
* def keys = []
* def vals = []
* def idxs = []
* def fun =
"""
function(x, y, i) {
karate.appendTo(keys, x);
karate.appendTo(vals, y);
karate.appendTo(idxs, i);
}
"""
* def map = { a: 2, b: 4, c: 6 }
* karate.forEach(map, fun)
* match keys == ['a', 'b', 'c']
* match vals == [2, 4, 6]
* match idxs == [0, 1, 2]
Scenario: filterKeys
* def schema = { a: '#string', b: '#number', c: '#boolean' }
* def response = { a: 'x', c: true }
= very useful for validating a response against a schema "super-set"
* match response == karate.filterKeys(schema, response)
* match karate.filterKeys(response, 'b', 'c') == { c: true }
* match karate.filterKeys(response, ['a', 'b']) == { a: 'x' }
Scenario: merge
* def foo = { a: 1 }
* def bar = karate.merge(foo, { b: 2 })
* match bar == { a: 1, b: 2 }
Scenario: append
* def foo = [{ a: 1 }]
* def bar = karate.append(foo, { b: 2 })
* match bar == [{ a: 1 }, { b: 2 }]
Scenario: sort
* def foo = [{a: { b: 3 }}, {a: { b: 1 }}, {a: { b: 2 }}]
* def fun = function(x){ return x.a.b }
* def bar = karate.sort(foo, fun)
* match bar == [{a: { b: 1 }}, {a: { b: 2 }}, {a: { b: 3 }}]
* match bar.reverse() == [{a: { b: 3 }}, {a: { b: 2 }}, {a: { b: 1 }}]
Loops
Given the examples above, it has to be said that a best practice with Karate is to avoid JavaScript for loops as far as possible. A common requirement is to build an array with n elements or do something n times where n is an integer (that could even be a variable reference). This is easily achieved with the karate.repeat() API:
* def fun = function(i){ return i * 2 }
* def foo = karate.repeat(5, fun)
* match foo == [0, 2, 4, 6, 8]
* def foo = []
* def fun = function(i){ karate.appendTo(foo, i) }
* karate.repeat(5, fun)
* match foo == [0, 1, 2, 3, 4]
= generate test data easily
* def fun = function(i){ return { name: 'User ' + (i + 1) } }
* def foo = karate.repeat(3, fun)
* match foo == [{ name: 'User 1' }, { name: 'User 2' }, { name: 'User 3' }]
= generate a range of numbers as a json array
* def foo = karate.range(4, 9)
* match foo == [4, 5, 6, 7, 8, 9]
And there’s also karate.range() which can be useful to generate test-data.
When handling XML, you sometimes need to call XPath functions, for example to get the count of a node-set. Any valid XPath expression is allowed on the left-hand-side of a match statement.
Some XPath expressions return a list of nodes (instead of a single node). But since you can express a list of data-elements as a JSON array - even these XPath expressions can be used in match statements.
If your XPath is dynamic and has to be formed 'on the fly' perhaps by using some variable derived from previous steps, you can use the karate.xmlPath() helper:
* def xml = <query><name><foo>bar</foo></name></query>
* def elementName = 'name'
* def name = karate.xmlPath(xml, '/query/' + elementName + '/foo')
* match name == 'bar'
* def queryName = karate.xmlPath(xml, '/query/' + elementName)
* match queryName == <name><foo>bar</foo></name>
You can refer to this file (which is part of the Karate test-suite) for more XML examples: xml-and-xpath.feature
Special Variables
These are 'built-in' variables, there are only a few and all of them give you access to the HTTP response.
response
After every HTTP call this variable is set with the response body, and is available until the next HTTP request over-writes it. You can easily assign the whole response (or just parts of it using Json-Path or XPath) to a variable, and use it in later steps.
The response is automatically available as a JSON, XML or String object depending on what the response contents are.
As a short-cut, when running JsonPath expressions - $ represents the response. This has the advantage that you can use pure JsonPath and be more concise. For example:
= the three lines below are equivalent
Then match response $ == { name: 'Billie' }
Then match response == { name: 'Billie' }
Then match $ == { name: 'Billie' }
= the three lines below are equivalent
Then match response.name == 'Billie'
Then match response $.name == 'Billie'
Then match $.name == 'Billie'
And similarly for XML and XPath, '/' represents the response
= the four lines below are equivalent
Then match response / == <cat><name>Billie</name></cat>
Then match response/ == <cat><name>Billie</name></cat>
Then match response == <cat><name>Billie</name></cat>
Then match / == <cat><name>Billie</name></cat>
= the three lines below are equivalent
Then match response /cat/name == 'Billie'
Then match response/cat/name == 'Billie'
Then match /cat/name == 'Billie'
JsonPath short-cuts
The $varName form is used on the right-hand-side of Karate expressions and is slightly different from pure JsonPath expressions which always begin with $. or $[. Here is a summary of what the different 'shapes' mean in Karate:
Shape
Description
$.bar
Pure JsonPath equivalent of $response.bar where response is a JSON object
$[0]
Pure JsonPath equivalent of $response[0] where response is a JSON array
$foo.bar
Evaluates the JsonPath $.bar on the variable foo which is a JSON object or map-like
$foo[0]
Evaluates the JsonPath $[0] on the variable foo which is a JSON array or list-like
There is no need to prefix variable names with $ on the left-hand-side of match statements because it is implied. You can if you want to, but since only JsonPath (on variables) is allowed here, Karate ignores the $ and looks only at the variable name. None of the examples in the documentation use the $varName form on the LHS, and this is the recommended best-practice.
responseBytes
This will always hold the contents of the response as a byte-array. This is rarely used, unless you are expecting binary content returned by the server. The match keyword will work as you expect. Here is an example: binary.feature.
responseCookies
The responseCookies variable is set upon any HTTP response and is a map-like (or JSON-like) object. It can be easily inspected or used in expressions.
* assert responseCookies['my.key'].value == 'someValue'
= karate's unified data handling means that even 'match' works
* match responseCookies contains { time: '#notnull' }
= ... which means that checking if a cookie does NOT exist is a piece of cake
* match responseCookies !contains { blah: '#notnull' }
= save a response cookie for later use
* def time = responseCookies.time.value
As a convenience, cookies from the previous response are collected and passed as-is as part of the next HTTP request. This is what is normally expected and simulates a web-browser - which makes it easy to script things like HTML-form based authentication into test-flows. Refer to the documentation for cookie for details and how you can disable this if need be.
Each item within responseCookies is itself a 'map-like' object. Typically you would examine the value property as in the example above, but domain and path are also available.
responseHeaders
See also match header which is what you would normally need.
But if you need to use values in the response headers - they will be in a variable named responseHeaders. Note that it is a 'map of lists' so you will need to do things like this:
And just as in the responseCookies example above, you can use match to run complex validations on the responseHeaders.
Finally, using karate.response.header(name) can be simpler to just get a header value string by name, and it will ignore-case for the name passed as the argument:
* match karate.response.header('content-type') == 'application/json'
responseStatus
You would normally only need to use the status keyword. But if you really need to use the HTTP response code in an expression or save it for later, you can get it as an integer:
* def uploadStatusCode = responseStatus
= check if the response status is either of two values
Then assert responseStatus == 200 || responseStatus == 204
Note that match can give you some extra readable options:
* match [200, 201, 204] contains responseStatus
= this may be sufficient to check a range of values
* assert responseStatus >= 200
* assert responseStatus < 300
= but using karate.range() you can even do this !
* match karate.range(200, 299) contains responseStatus
responseTime
The response time (in milliseconds) for the current response would be available in a variable called responseTime. You can use this to assert that it was returned within the expected time like so:
When method post
Then status 201
And assert responseTime < 1000
responseType
Karate will attempt to parse the raw HTTP response body as JSON or XML and make it available as the response value. If parsing fails, Karate will log a warning and the value of response will then be a plain string. You can still perform string comparisons such as a match contains and look for error messages etc. In rare cases, you may want to check what the "type" of the response is and it can be one of 3 different values: json, xml and string.
So if you really wanted to assert that the HTTP response body is well-formed JSON or XML you can do this:
When method post
Then status 201
And match responseType == 'json'
requestTimeStamp
Very rarely used - but you can get the Java system-time (for the current response) at the point when the HTTP request was initiated (the value of System.currentTimeMillis()) which can be used for detailed logging or custom framework / stats calculations.
HTTP Header Manipulation
configure headers
Custom header manipulation for every HTTP request is something that Karate makes very easy and pluggable. For every HTTP request made from Karate, the internal flow is as follows:
did the function invocation return a map-like (or JSON) object ?
all the key-value pairs are added to the HTTP headers.
or is the configured value a JSON object ?
all the key-value pairs are added to the HTTP headers.
This makes setting up of complex authentication schemes for your test-flows really easy. It typically ends up being a one-liner that appears in the Background section at the start of your test-scripts. You can re-use the function you create across your whole project.
Here is an example JavaScript function that uses some variables in the context (which have been possibly set as the result of a sign-in) to build the Authorization header. Note how even calls to Java code can be made if needed.
In the example below, note the use of the karate.get() helper for getting the value of a dynamic variable (which was not set at the time this JS function was declared). This is preferred because it takes care of situations such as if the value is undefined in JavaScript. In rare cases you may need to set a variable from this routine, and a good example is to make the generated UUID "visible" to the currently executing script or feature. You can easily do this via karate.set('someVarName', value).
function fn() {
var uuid = '' + java.util.UUID.randomUUID(); // convert to string
var out = { // so now the txid_header would be a unique uuid for each request
txid_header: uuid,
ip_header: '123.45.67.89', // hard coded here, but also can be as dynamic as you want
};
var authString = '';
var authToken = karate.get('authToken'); // use the 'karate' helper to do a 'safe' get of a 'dynamic' variable
if (authToken) { // and if 'authToken' is not null ...
authString = ',auth_type=MyAuthScheme'
+ ',auth_key=' + authToken.key
+ ',auth_user=' + authToken.userId
+ ',auth_project=' + authToken.projectId;
}
// the 'appId' variable here is expected to have been set via karate-config.js (bootstrap init) and will never change
out['Authorization'] = 'My_Auth app_id=' + appId + authString;
return out;
}
Assuming the above code is in a file called my-headers.js, the next section on calling other feature files shows how it looks like in action at the beginning of a test script.
Notice how once the authToken variable is initialized, it is used by the above function to generate headers for every HTTP call made as part of the test flow.
If a few steps in your flow need to temporarily change (or completely bypass) the currently-set header-manipulation scheme, just update configure headers to a new value (or set it to null) in the middle of a script. Then use the header keyword to do a custom 'over-ride' if needed.
useful to append to a list-like variable (that has to exist) in scope, see JSON transforms - the first argument can be a reference to an array-like variable or even the name (string) of an existing variable which is list-like
compare two images the same way that compareImage works (with an optional options argument), returns an object with the following keys: baseline, latest, mismatchPercentage, engine, failureThreshold and optionally: error, isBaselineMissing, isScaleMismatch, isMismatch, resembleMismatchPercentage, ssimMismatchPercentage
does the same thing as the configure keyword, and a very useful example is to do karate.configure('connectTimeout', 5000); in karate-config.js - which has the 'global' effect of not wasting time if a connection cannot be established within 5 seconds
convenient way to execute an OS specific command and return the console output e.g. karate.exec('some.exe -h') (or karate.exec(['some.exe', '-h'])) useful for calling non-Java code (that can even return data) or for starting user-interfaces to be automated, this command will block until the process terminates, also see karate.fork()
functional-style 'filter' operation useful to filter list-like objects (e.g. JSON arrays), see example, the second argument has to be a JS function (item, [index]) that returns a boolean
extracts a sub-set of key-value pairs from the first argument, the second argument can be a list (or varargs) of keys - or even another JSON where only the keys would be used for extraction, example
karate.forEach(list, function)
functional-style 'loop' operation useful to traverse list-like (or even map-like) objects (e.g. JSON / arrays), see example, the second argument has to be a JS function (item, [index]) for lists and (key, [value], [index]) for JSON / maps
executes an OS command, but forks a process in parallel and will not block the test like karate.exec() e.g. karate.fork({ args: ['some.exe', '-h'] }) or karate.fork(['some.exe', '-h']) - you can use a composite string as line (or the solitary argument e.g. karate.fork('some.exe -h')) instead of args, and an optional workingDir string property and env JSON / map is also supported - this returns a Command object which has operations such as waitSync() and close() if you need more control, more details here
for advanced conditional logic for e.g. when a string coming from an external process is dynamic - and whether it is JSON or XML is not known in advance, see example
get the value of a variable by name (or JsonPath expression), if not found - this returns null which is easier to handle in JavaScript (than undefined), and an optional (literal / constant) second argument can be used to return a "default" value, very useful to set variables in called features that have not been pre-defined
log to the same logger (and log file) being used by the parent process, logging can be suppressed with configure printEnabled set to false, and just like print - use comma-separated values to "pretty print" JSON or XML
access to the Karate logger directly and log in debug. Might be desirable instead of karate.log or print when looking to reduce the logs in console in your CI/CD pipeline but still retain the information for reports. See Logging for additional details.
functional-style 'map' operation useful to transform list-like objects (e.g. JSON arrays), see example, the second argument has to be a JS function (item, [index])
brings the power of the fuzzymatch syntax into Karate-JS, returns a JSON in the form { pass: '#boolean', message: '#string' } and you can find an example here - you can even place a full match expression like this: karate.match("each foo contains { a: '#number' }")
returns the operating system details as JSON, for e.g. { type: 'macosx', name: 'Mac OS X' } - useful for writing conditional logic, the possible type-s being: macosx, windows, linux and unknown
for advanced users, you can inspect the actual HTTP request after it happens, useful if you are writing a framework over Karate, refer to this example: request.feature
returns a JSON array of integers (inclusive), the optional third argument must be a positive integer and defaults to 1, and if start < end the order of values is reversed
returns the last HTTP response as a JS object that enables advanced use-cases such as getting a header ignoring case: karate.response.header('some-header')
returns the last HTTP request as a JS object that enables advanced use-cases such as getting a header ignoring case: karate.request.header('some-header'), which works even in mocks
where the single argument is expected to be a Map or JSON-like, and will perform the above karate.set() operation for all key-value pairs in one-shot, see example
only needed when you need to conditionally build payload elements, especially XML. This is best explained via an example, and it behaves the same way as the set keyword. Also see eval.
sorts the list using the provided custom function called for each item in the list (and the optional second argument is the item index) e.g. karate.sort(myList, x => x.val), and the second / function argument is not needed if the list is of plain strings or numbers
will pause the test execution until a socket connection (even HTTP GET) is made to the port logged to the console, useful for troubleshooting UI tests without using a de-bugger, of course - NEVER forget to remove this after use !
for even more advanced users - Karate natively supports tags in a @name=val1,val2 format, and there is an inheritance mechanism where Scenario level tags can over-ride Feature level tags, refer to this example: tags.feature
converts a JSON array (of objects) or a list-like object into a CSV string, writing this to a file is your responsibility or you could use karate.write()
rarely used, when you need to pass a JS function to custom Java code, typically for Async, and another edge case is to convert a JSON array or object to a Java List or Map, see example
converts a Java object into JSON, and karate.toJson(object, true) will strip all keys that have null values from the resulting JSON, convenient for unit-testing Java code, see example
normally not recommended, please read this first - writes the bytes of object to a path which will always be relative to the "build" directory (typically target), see this example: embed-pdf.js - and this method returns a java.io.File reference to the file created / written to
Just like karate.jsonPath() - but for XML, and allows you to use dynamic XPath if needed, see example.
Code Reuse / Common Routines
call
In any complex testing endeavor, you would find yourself needing 'common' code that needs to be re-used across multiple test scripts. A typical need would be to perform a 'sign in', or create a fresh user as a pre-requisite for the scenarios being tested.
There are two types of code that can be call-ed. *.feature files and JavaScript functions.
Calling other *.feature files
When you have a sequence of HTTP calls that need to be repeated for multiple test scripts, Karate allows you to treat a .feature file as a re-usable unit. You can also pass parameters into the .feature file being called, and extract variables out of the invocation result.
Here is an example of using the call keyword to invoke another feature file, loaded using the read function:
If you find this hard to understand at first, try looking at this set of examples.
Feature: which makes a 'call' to another re-usable feature
Background:
* configure headers = read('classpath:my-headers.js')
* def signIn = call read('classpath:my-signin.feature') { username: 'john', password: 'secret' }
* def authToken = signIn.authToken
Scenario: some scenario
= main test steps
Note that def can be used to assign a feature to a variable. For example look at how "creator" has been defined in the Background in this example, and used later in a call statement. This is very close to how "custom keywords" work in other frameworks. See this other example for more ideas: dsl.feature.
The contents of my-signin.feature are shown below. A few points to note:
Karate creates a new 'context' for the feature file being invoked but passes along all variables and configuration. This means that all your config variables and configure settings would be available to use, for example loginUrlBase in the example below.
When you use def in the 'called' feature, it will not over-write variables in the 'calling' feature (unless you explicitly choose to use shared scope). But note that JSON, XML, Map-like or List-like variables are 'passed by reference' which means that 'called' feature steps can update or 'mutate' them using the set keyword. Use the copy keyword to 'clone' a JSON or XML payload if needed, and refer to this example for more details: copy.feature.
You can add (or over-ride) variables by passing a call 'argument' as shown above. Only one JSON argument is allowed, but this does not limit you in any way as you can use any complex JSON structure. You can even initialize the JSON in a separate step and pass it by name, especially if it is complex. Observe how using JSON for parameter-passing makes things super-readable. In the 'called' feature, the argument can also be accessed using the built-in variable: __arg.
All variables that were defined (using def) in the 'called' script would be returned as 'keys' within a JSON-like object. Note that this includes 'built-in' variables, which means that things like the last value of response would also be present. In the example above you can see that the JSON 'envelope' returned - is assigned to the variable named signIn. And then getting hold of any data that was generated by the 'called' script is as simple as accessing it by name, for example signIn.authToken as shown above. This design has the following advantages:
'called' Karate scripts don’t need to use any special keywords to 'return' data and can behave like 'normal' Karate tests in 'stand-alone' mode if needed
the data 'return' mechanism is 'safe', there is no danger of the 'called' script over-writing any variables in the 'calling' (or parent) script (unless you use shared scope)
the need to explicitly 'unpack' variables by name from the returned 'envelope' keeps things readable and maintainable in the 'caller' script
Note that only variables and configuration settings will be passed. You can’t do things such as * url 'http://foo.bar' and expect the URL to be set in the "called" feature. Use a variable in the "called" feature instead, for e.g. * url myUrl.
Feature: here are the contents of 'my-signin.feature'
Scenario:
Given url loginUrlBase
And request { userId: '#(username)', userPass: '#(password)' }
When method post
Then status 200
And def authToken = response
= second HTTP call, to get a list of 'projects'
Given path 'users', authToken.userId, 'projects'
When method get
Then status 200
= logic to 'choose' first project
And set authToken.projectId = response.projects[0].projectId;
The above example actually makes two HTTP requests - the first is a standard 'sign-in' POST and then (for illustrative purposes) another HTTP call (a GET) is made for retrieving a list of projects for the signed-in user, and the first one is 'selected' and added to the returned 'auth token' JSON object.
So you get the picture, any kind of complicated 'sign-in' flow can be scripted and re-used.
If the second HTTP call above expects headers to be set by my-headers.js - which in turn depends on the authToken variable being updated, you will need to duplicate the line * configure headers = read('classpath:my-headers.js') from the 'caller' feature here as well. The above example does not use shared scope, which means that the variables in the 'calling' (parent) feature are not shared by the 'called' my-signin.feature. The above example can be made more simpler with the use of call (or callonce) without a def-assignment to a variable, and is the recommended pattern for implementing re-usable authentication setup flows.
Do look at the documentation and example for configure headers also as it goes hand-in-hand with call. In the above example, the end-result of the call to my-signin.feature resulted in the authToken variable being initialized. Take a look at how the configure headers example uses the authToken variable.
Call Tag Selector
You can "select" a single Scenario (or Scenario-s or Scenario Outline-s or even specific Examples rows) by appending a "tag selector" at the end of the feature-file you are calling. For example:
While the tag does not need to be in the @key=value form, it is recommended for readability when you start getting into the business of giving meaningful names to your Scenario-s.
This "tag selection" capability is designed for you to be able to "compose" flows out of existing test-suites when using the Karate Gatling integration. Normally we recommend that you keep your "re-usable" features lightweight - by limiting them to just one Scenario.
Call Same Feature
As a convenience, you can call a tag directly, which is a short-cut to call another Scenario within the same feature file. Note that you would typically want to use the @ignore tag for such cases.
Scenario: one
* call read('@two')
@ignore @two
Scenario: two
* print 'called'
Data-Driven Features
If the argument passed to the call of a *.feature file is a JSON array, something interesting happens. The feature is invoked for each item in the array. Each array element is expected to be a JSON object, and for each object - the behavior will be as described above.
But this time, the return value from the call step will be a JSON array of the same size as the input array. And each element of the returned array will be the 'envelope' of variables that resulted from each iteration where the *.feature got invoked.
Here is an example that combines the table keyword with calling a *.feature. Observe how the getshortcut is used to 'distill' the result array of variable 'envelopes' into an array consisting only of response payloads.
* table kittens
| name | age |
| 'Bob' | 2 |
| 'Wild' | 1 |
| 'Nyan' | 3 |
* def result = call read('cat-create.feature') kittens
* def created = $result[*].response
* match each created == { id: '#number', name: '#string', age: '#number' }
* match created[*].name contains only ['Bob', 'Wild', 'Nyan']
And here is how cat-create.feature could look like:
@ignore
Feature:
Scenario:
Given url someUrlFromConfig
And path 'cats'
And request { name: '#(name)', age: '#(age)' }
When method post
Then status 200
If you replace the table with perhaps a JavaScript function call that gets some JSON data from some data-source, you can imagine how you could go about dynamic data-driven testing.
Although it is just a few lines of code, take time to study the above example carefully. It is a great example of how to effectively use the unique combination of Cucumber and JsonPath that Karate provides.
Although all properties in the passed JSON-like argument are 'unpacked' into the current scope as separate 'named' variables, it sometimes makes sense to access the whole argument and this can be done via arg. And if being called in a loop, a built-in variable called loop will also be available that will hold the value of the current loop index. So you can do things like this: * def name = name + __loop - or you can use the loop index value for looking up other values that may be in scope - in a data-driven style.
Variable
Refers To
__arg
the single call (or callonce) argument, will be null if there was none
__loop
the current iteration index (starts from 0) if being called in a loop, will be -1 if not
Some users need "callable" features that are re-usable even when variables have not been defined by the calling feature. Normally an undefined variable results in nasty JavaScript errors. But there is an elegant way you can specify a default value using the karate.get() API:
= if foo is not defined, it will default to 42
* def foo = karate.get('foo', 42)
A word of caution: we recommend that you should not over-use Karate’s capability of being able to re-use features. Re-use can sometimes result in negative benefits - especially when applied to test-automation. Prefer readability over re-use. See this for an example.
copy
For a call (or callonce) - payload / data structures (JSON, XML, Map-like or List-like) variables are 'passed by reference' which means that steps within the 'called' feature can update or 'mutate' them, for e.g. using the set keyword. This is actually the intent most of the time and is convenient. If you want to pass a 'clone' to a 'called' feature, you can do so using the rarely used copy keyword that works very similar to type conversion. This is best explained in this example: copy.feature.
Calling JavaScript Functions
Examples of defining and using JavaScript functions appear in earlier sections of this document. Being able to define and re-use JavaScript functions is a powerful capability of Karate. For example, you can:
call re-usable functions that take complex data as an argument and return complex data that can be stored in a variable
share and re-use test utilities or 'helper' functionality across your organization
For an advanced example of how you can build and re-use a common set of JS functions, refer to this answer on Stack Overflow.
In real-life scripts, you would typically also use this capability of Karate to configure headers where the specified JavaScript function uses the variables that result from a sign in to manipulate headers for all subsequent HTTP requests. And it is worth mentioning that the Karate configuration 'bootstrap' routine is itself a JavaScript function.
Also refer to the eval keyword for a simpler way to execute arbitrary JavaScript that can be useful in some situations.
JS function argument rules for call
When using call (or callonce), only one argument is allowed. But this does not limit you in any way, because similar to how you can call *.feature files, you can pass a whole JSON object as the argument. In the case of the call of a JavaScript function, you can also pass a JSON array or a primitive (string, number, boolean) as the solitary argument, and the function implementation is expected to handle whatever is passed.
Instead of using call (or callonce) you are always free to call JavaScript functions 'normally' and then you can use more than one argument.
* def adder = function(a, b){ return a + b }
* assert adder(1, 2) == 3
Return types
Naturally, only one value can be returned. But again, you can return a JSON object. There are two things that can happen to the returned value.
Either - it can be assigned to a variable like so.
* def returnValue = call myFunction
Or - if a call is made without an assignment, and if the function returns a map-like object, it will add each key-value pair returned as a new variable into the execution context.
= while this looks innocent ...
= ... behind the scenes, it could be creating (or over-writing) a bunch of variables !
* call someFunction
While this sounds dangerous and should be used with care (and limits readability), the reason this feature exists is to quickly set (or over-write) a bunch of config variables when needed. In fact, this is the mechanism used when karate-config.js is processed on start-up.
Shared Scope
This behavior where all key-value pairs in the returned map-like object get automatically added as variables - applies to the calling of *.feature files as well. In other words, when call or callonce is used without a def, the 'called' script not only shares all variables (and configure settings) but can update the shared execution context. This is very useful to boil-down those 'common' steps that you may have to perform at the start of multiple test-scripts - into one-liners. But use wisely, because called scripts will now over-write variables that may have been already defined.
* def config = { user: 'john', password: 'secret' }
= this next line may perform many steps and result in multiple variables set for the rest of the script
* call read('classpath:common-setup.feature') config
You can use callonce instead of call within the Background in case you have multiple Scenario sections or Examples. Note the 'inline' use of the read function as a short-cut above. This applies to JS functions as well:
* call read('my-function.js')
These heavily commented demo examples can help you understand 'shared scope' better, and are designed to get you started with creating re-usable 'sign-in' or authentication flows:
Once you get comfortable with Karate, you can consider moving your authentication flow into a 'global' one-time flow using karate.callSingle(), think of it as 'callonce on steroids'.
call vs read()
Since this is a frequently asked question, the different ways of being able to re-use code (or data) are summarized below.
Code
Description
* def login = read('login.feature') * call login
Shared Scope, and the login variable can be re-used
* call read('login.feature')
short-cut for the above
without needing a variable
using the call keyword makes
passing an in-line JSON argument
more "readable"
* call read 'credentials.json'
Since "read" happens to be a function (that takes a single
string argument), this has the effect
of loading all keys in the JSON file
into Shared Scope as variables !
This can be sometimes handy.
* call read ('credentials.json')
A common mistake. First, there
is no meaning in call for JSON.
Second, the space after the "read"
makes this equal to the above.
* karate.set(read('credentials.json'))
For completeness - this has exactly the same effect as the above two rows !
Calling any Java code is that easy. Given this custom, user-defined Java class:
package com.mycompany;
import java.util.HashMap;
import java.util.Map;
public class JavaDemo {
public Map<String, Object> doWork(String fromJs) {
Map<String, Object> map = new HashMap<>();
map.put("someKey", "hello " + fromJs);
return map;
}
public static String doWorkStatic(String fromJs) {
return "hello " + fromJs;
}
}
This is how it can be called from a test-script via JavaScript, and yes, even static methods can be invoked:
* def doWork =
"""
function(arg) {
var JavaDemo = Java.type('com.mycompany.JavaDemo');
var jd = new JavaDemo();
return jd.doWork(arg);
}
"""
= in this case the solitary 'call' argument is of type string
* def result = call doWork 'world'
* match result == { someKey: 'hello world' }
= using a static method - observe how java interop is truly seamless !
* def JavaDemo = Java.type('com.mycompany.JavaDemo')
* def result = JavaDemo.doWorkStatic('world')
* assert result == 'hello world'
Note that JSON gets auto-converted to Map (or List) when making the cross-over to Java. Refer to the cats-java.feature demo for an example.
An additional-level of auto-conversion happens when objects cross the boundary between JS and Java. In the rare case that you need to mutate a Map or List returned from Java but while still within a JS block, use karate.toJson() to convert.
Another example is dogs.feature - which actually makes JDBC (database) calls, and since the data returned from the Java code is JSON, the last section of the test is able to use matchvery effectively for data assertions.
A great example of how you can extend Karate, even bypass the HTTP client but still use Karate’s test-automation effectively, is this gRPC example by @thinkerou: karate-grpc. And you can even handle asynchronous flows such as listening to message-queues.
HTTP Basic Authentication Example
This should make it clear why Karate does not provide 'out of the box' support for any particular HTTP authentication scheme. Things are designed so that you can plug-in what you need, without needing to compile Java code. You get to choose how to manage your environment-specific configuration values such as user-names and passwords.
First the JavaScript file, basic-auth.js:
function fn(creds) {
var temp = creds.username + ':' + creds.password;
var Base64 = Java.type('java.util.Base64');
var encoded = Base64.getEncoder().encodeToString(temp.toString().getBytes());
return 'Basic ' + encoded;
}
And here’s how it works in a test-script using the header keyword.
You can set this up for all subsequent requests or dynamically generate headers for each HTTP request if you configure headers.
callonce
Cucumber has a limitation where Background steps are re-run for every Scenario. And if you have a Scenario Outline, this happens for every row in the Examples. This is a problem especially for expensive, time-consuming HTTP calls, and this has been an open issue for a long time.
Karate’s callonce keyword behaves exactly like call but is guaranteed to execute only once. The results of the first call are cached, and any future calls will simply return the cached result instead of executing the JavaScript function (or feature) again and again.
This does require you to move 'set-up' into a separate *.feature (or JavaScript) file. But this totally makes sense for things not part of the 'main' test flow and which typically need to be re-usable anyway.
So when you use the combination of callonce in a Background, you can indeed get the same effect as using a @BeforeClass annotation, and you can find examples in the karate-demo, such as this one: callonce.feature.
A callonce is ideally used for only "pure" JSON. You may face issues if you attempt to mix in JS functions or Java code. See karate.callSingle().
eval
This is for evaluating arbitrary JavaScript and you are advised to use this only as a last resort ! Conditional logic is not recommended especially within test scripts because tests should be deterministic.
There are a few situations where this comes in handy:
you really don’t need to assign a result to a variable
= just perform an action, we don't care about saving the result
* eval myJavaScriptFunction()
= do something only if a condition is true
* eval if (zone == 'zone1') karate.set('temp', 'after')
As a convenience, you can omit the eval keyword and so you can shorten the above to:
* myJavaScriptFunction()
* if (zone == 'zone1') karate.set('temp', 'after')
This is very convenient especially if you are calling a method on a variable that has been defined such as the karate object, and for general-purpose scripting needs such as UI automation. Note how karate.set() and karate.remove() below are used directly as a script "statement".
= you can use multiple lines of JavaScript if needed
* eval
"""
var foo = function(v){ return v * v };
var nums = [0, 1, 2, 3, 4];
var squares = [];
for (var n in nums) {
squares.push(foo(n));
}
karate.set('temp', squares);
"""
* match temp == [0, 1, 4, 9, 16]
* def json = { a: 1 }
* def key = 'b'
= use dynamic path expressions to mutate json
* json[key] = 2
* match json == { a: 1, b: 2 }
* karate.remove('json', key)
* match json == { a: 1 }
* karate.set('json', '$.c[]', { d: 'e' })
* match json == { a: 1, c: [{ d: 'e' }] }
Advanced / Tricks
Polling
The built-in retry until syntax should suffice for most needs, but if you have some specific needs, this demo example (using JavaScript) should get you up and running: polling.feature.
Conditional Logic
The keywords GivenWhenThen are only for decoration and should not be thought of as similar to an if - then - else statement. And as a testing framework, Karate discourages tests that give different results on every run.
That said, if you really need to implement 'conditional' checks, this can be one pattern:
* def filename = zone == 'zone1' ? 'test1.feature' : 'test2.feature'
* def result = call read(filename)
And this is another, using karate.call(). Here we want to call a file only if a condition is satisfied:
* def expected = zone == 'zone1' ? { foo: '#string' } : { bar: '#number' }
* match response == expected
JSON Lookup
You can always use a JavaScript switch case within an eval or function block. But one pattern that you should be aware of is that JSON is actually a great data-structure for looking up data.
* def data =
"""
{
foo: 'hello',
bar: 'world'
}
"""
= in real-life key can be dynamic
* def key = 'bar'
= and used to lookup data
* match (data[key]) == 'world'
You can find more details here. Also note how you can wrap the LHS of the match in parentheses in the rare cases where the parser expects JsonPath by default.
Abort and Fail
In some rare cases you need to exit a Scenario based on some condition. You can use karate.abort() like so:
* if (responseStatus == 404) karate.abort()
Using karate.abort() will not fail the test. Conditionally making a test fail is easy with karate.fail()
* if (condition) karate.fail('a custom message')
But normally a match statement is preferred unless you want a really descriptive error message.
Since it is so easy to dive into Java-interop, Karate does not include any random-number functions, uuid generator or date / time utilities out of the box. You simply roll your own.
Here is an example of how to get the current date, and formatted the way you want:
* def getDate =
"""
function() {
var SimpleDateFormat = Java.type('java.text.SimpleDateFormat');
var sdf = new SimpleDateFormat('yyyy/MM/dd');
var date = new java.util.Date();
return sdf.format(date);
}
"""
* def temp = getDate()
* print temp
And the above will result in something like this being logged: [print] 2017/10/16.
The first three are good enough for random string generation for most situations. Note that if you need to do a lot of case-insensitive string checks, karate.lowerCase() is what you are looking for.
Multiple Functions in One File
If you find yourself needing a complex helper or utility function, we strongly recommend that you use Java because it is much easier to maintain and even debug if needed. And if you need multiple functions, you can easily organize them into a single Java class with multiple static methods.
That said, if you want to stick to JavaScript, but find yourself accumulating a lot of helper functions that you need to use in multiple feature files, the following pattern is recommended.
You can organize multiple "common" utilities into a single re-usable feature file as follows e.g. common.feature
And then you have two options. The first option using shared scope should be fine for most projects, but if you want to "name space" your functions, use "isolated scope":
You can even move commonly used routines into karate-config.js which means that they become "global". But we recommend that you do this only if you are sure that these routines are needed in almost all*.feature files. Bloating your configuration can lead to loss of performance, and maintainability may suffer.
Async
The JS API has a karate.signal(result) method that is useful for involving asynchronous flows into a test.
listen
You use the listen keyword (with a timeout) to wait until that event occurs. The listenResult magic variable will hold the value passed to the call to karate.signal().
This is best explained in this example that involves listening to an ActiveMQ / JMS queue.
Note how JS functions defined at run-time can be mixed with custom Java code to get things done. You need to use karate.toJava() to "wrap" JS functions passed to custom Java code.
Background:
* def QueueConsumer = Java.type('mock.contract.QueueConsumer')
* def queue = new QueueConsumer(queueName)
* def handler = function(msg){ karate.signal(msg) }
* queue.listen(karate.toJava(handler))
* url paymentServiceUrl + '/payments'
Scenario: create, get, update, list and delete payments
Given request { amount: 5.67, description: 'test one' }
When method post
Then status 200
And match response == { id: '#number', amount: 5.67, description: 'test one' }
And def id = response.id
* listen 5000
* json shipment = listenResult
* print '
=== received:', shipment
* match shipment == { paymentId: '#(id)', status: 'shipped' }
WebSocket
Karate also has built-in support for websocket that is based on the async capability and the listen keyword. The following method signatures are available on the karate JS object to obtain a websocket client:
karate.webSocket(url)
karate.webSocket(url, handler)
karate.webSocket(url, handler, options) - where options is an optional JSON (or map-like) object that takes the following optional keys:
subProtocol - in case the server expects it
headers - another JSON of key-value pairs
maxPayloadSize - this defaults to 4194304 (bytes, around 4 MB)
These will init a websocket client for the given url and optional subProtocol. You can call send() on the returned object to send a message.
If a handlerfunction (returning a boolean) is provided - it will be used to complete the listen "wait" if true is returned. A handler function is needed only if you have to ignore some incoming traffic and stop the "wait" when a certain payload arrives. If you don’t pass a handler (or it is null), the first message is returned.
Note that karate.signal() (described as part of the listen keyword) will be called internally and the listenResult will be the payload contents of the "selected" message.
Here is an example, where the same websocket connection is used to send as well as receive a message.
For handling binary messages, the same karate.webSocket() method signatures exist for karate.webSocketBinary(). Refer to these examples for more: echo.feature
websocket.feature. Note that any websocket instances created will be auto-closed at the end of the Scenario.
Java Function References
JavaScript functions have some limitations when combined with multi-threaded Java code. So it is recommended that you directly use a Java Function when possible instead of using the karate.toJava() "wrapper" as shown above.
One pattern you can adopt is to create a "factory" method that returns a Java function - where you can easily delegate to the logic you want. For example, see the sayHelloFactory() method below:
public class Hello {
public static String sayHello(String message) {
return "hello " + message;
}
public static Function<String, String> sayHelloFactory() {
return s -> sayHello(s);
}
}
And now, to get a reference to that "function" you can do this:
This can be convenient when using shared scope because you can just call sayHello('myname') where needed.
Tags
Gherkin has a great way to sprinkle meta-data into test-scripts - which gives you some interesting options when running tests in bulk. The most common use-case would be to partition your tests into 'smoke', 'regression' and the like - which enables being able to selectively execute a sub-set of tests.
The documentation on how to run tests via the command line has an example of how to use tags to decide which tests to not run (or ignore). Also see first.feature and second.feature in the demos. If you find yourself juggling multiple tags with logical AND and OR complexity, refer to this Stack Overflow answer.
For advanced users, Karate supports being able to query for tags within a test, and even tags in a @name=value form. Refer to karate.tags and karate.tagValues.
Special Tags
For completeness, the "built-in" tags are the following:
Tag
Description
@ignore
Any Scenario with (or that has inherited) this tag will be skipped at run-time. This does not apply to anything that is "called" though
There are two special tags that allow you to "select" or "un-select" a Scenario depending on the value of karate.env. This can be really convenient, for example to never run some tests in a certain "production like" or sensitive environment.
@env=foo,bar - will run only when the value of karate.env is not-null and equal to fooorbar
@envnot=foo - will run when the value of karate.env is null or anything other thanfoo
Here is an example:
@env=dev
Scenario: runs only when karate.env is 'dev'
* print 'karate.env is:', karate.env
Since multiple values are supported, you can also do this:
@envnot=perf,prod
Scenario: never runs in perf or prod
* print 'karate.env is:', karate.env
Tags And Examples
A little-known capability of the Cucumber / Gherkin syntax is to be able to tag even specific rows in a bunch of examples ! You have to repeat the Examples section for each tag. The example below combines this with the advanced features described above.
Scenario Outline: examples partitioned by tag
* def vals = karate.tagValues
* match vals.region[0] == expected
@region=US
Examples:
| expected |
| US |
@region=GB
Examples:
| expected |
| GB |
Note that if you tag Examples like this, and if a tag selector is used when running a given Feature - only the Examples that match the tag selector will be executed. There is no concept of a "default" where for e.g. if there is no matching tag - that the Examples without a tag will be executed. But note that you can use the negative form of a tag selector: ~@region=GB.
Dynamic Port Numbers
In situations where you start an (embedded) application server as part of the test set-up phase, a typical challenge is that the HTTP port may be determined at run-time. So how can you get this value injected into the Karate configuration ?
It so happens that the karate object has a field called properties which can read a Java system-property by name like this: karate.properties['myName']. Since the karate object is injected within karate-config.js on start-up, it is a simple and effective way for other processes within the same JVM to pass configuration values to Karate at run-time. Refer to the 'demo' karate-config.js for an example and how the demo.server.port system-property is set-up in the test runner: TestBase.java.
Java API
Karate has a set of Java API-s that expose the HTTP, JSON, data-assertion and UI automation capabilities. The primary classes are described below.
Http - build and execute any HTTP request and retrieve responses
Json - build and manipulate JSON data using JsonPath expressions, convert to and from Java Map-s and List-s, parse strings into JSON and convert Java objects into JSON
Match - exposes all of Karate’s match capabilities, and this works for Java Map and List objects
Do note that if you choose the Java API, you will naturally lose some of the test-automation framework benefits such as HTML reports, parallel execution and JavaScript / configuration. You may have to rely on unit-testing frameworks or integrate additional dependencies.
jbang
jbang is a great way for you to install and execute scripts that use Karate’s Java API on any machine with minimal setup. Note that jbang itself is super-easy to install and there is even a "Zero Install" option.
Here below is an example jbang script that uses the Karate Java API to do some useful work. Name the file as javadsl.java and run using the command: jbang javadsl.java.
please replace RELEASE with the exact version of Karate you intend to use if applicable
///usr/bin/env jbang "$0" "$@" ; exit $?
//DEPS com.intuit.karate:karate-core:RELEASE:all
import com.intuit.karate.*;
import java.util.List;
public class javadsl {
public static void main(String[] args) {
List users = Http.to("https://jsonplaceholder.typicode.com/users")
.get().json().asList();
Match.that(users.get(0)).contains("{ name: 'Leanne Graham' }");
String city = Json.of(users).get("$[0].address.city");
Match.that("Gwenborough").isEqualTo(city);
System.out.println("\n*** second user: " + Json.of(users.get(1)).toString());
}
}
Read the documentation of the stand-alone JAR for more - such as how you can even install custom command-line applications using jbang !
Invoking feature files using the Java API
It is also possible to invoke a feature file via a Java API which can be useful in some test-automation situations.
A common use case is to mix API-calls into a larger test-suite, for example a Selenium or WebDriver UI test. So you can use Karate to set-up data via API calls, then run the UI test-automation, and finally again use Karate to assert that the system-state is as expected. Note that you can even include calls to a database from Karate using Java interop. And this example may make it clear why using Karate itself to drive even your UI-tests may be a good idea.
The static method com.intuit.karate.Runner.runFeature() is best explained in this demo unit-test: JavaApiTest.java.
You can optionally pass in variable values or over-ride config via a HashMap or leave the second-last argument as null. The variable state after feature execution would be returned as a Map<String, Object>. The last boolean argument is whether the karate-config.js should be processed or not. Refer to the documentation on type-conversion to make sure you can 'unpack' data returned from Karate correctly, especially when dealing with XML.
Hooks
If you are looking for Cucumber 'hooks' Karate does not support them, mainly because they depend on Java code, which goes against the Karate Way™.
Instead, Karate gives you all you need as part of the syntax. Here is a summary:
Use a callonce in the Background. The advantage is that you can set up variables (using def if needed) which can be used in all Scenario-s within that Feature.
You can use karate.callSingle() directly in a *.feature file, but it logically fits better in the global "bootstrap". Ideally it should return "pure JSON" and note that you always get a "deep clone" of the cached result object.
Important
There are some restrictions when using callonce or karate.callSingle() especially within karate-config.js. Ideally you should return only pure JSON data (or a primitive string, number etc.). Keep in mind that the reason this exists is to "cache" data, and not behavior. So if you return complex objects such as a custom Java instance or a JS function that depends on complex objects, this may cause issues when you run in parallel. If you really need to re-use a Java function, see Java Function References.
Multiple calls using karate.callSingle()
The first argument to karate.callSingle() is used as the "cache key". So if you tried to re-use the same feature but with multiple arguments, things will not work as you expect. But you can suffix a ?name to the feature to de-dupe it, like so:
var adminResponse = karate.callSingle('classpath:get-token.feature?admin', {'username': 'admin', 'password': 'password123' });
var userResponse = karate.callSingle('classpath:get-token.feature?user', {'username': 'user', 'password': 'password456' });
Now adminResponse and userResponse will be different, even though the same feature file is being used for a callSingle().
configure callSingleCache
When re-running tests in development mode and when your test suite depends on say an Authorization header set by karate.callSingle(), you can cache the results locally to a file, which is very convenient when your "auth token" is valid for a period of a few minutes - which typically is the case. This means that as long as the token "on file" is valid, you can save time by not having to make the one or two HTTP calls needed to "sign-in" or create "throw-away" users in your SSO store.
So in "dev mode" you can easily set this behavior like this. Just ensure that this is "configured" before you use karate.callSingle():
This caching behavior will work only if the result of karate.callSingle() is a JSON-like object, and any JS functions or Java objects mixed in will be lost.
Data Driven Tests
The Cucumber Way
Cucumber has a concept of Scenario Outlines where you can re-use a set of data-driven steps and assertions, and the data can be declared in a very user-friendly fashion. Observe the usage of Scenario Outline: instead of Scenario:, and the new Examples: section.
You should take a minute to compare this with the exact same example implemented in REST-assured and TestNG. Note that this example only does a "string equals" check on parts of the JSON, but with Karate you are always encouraged to match the entire payload in one step.
Feature: karate answers 2
Background:
* url 'http://localhost:8080'
Scenario Outline: given circuit name, validate country
Given path 'api/f1/circuits/<name>.json'
When method get
Then match $.MRData.CircuitTable.Circuits[0].Location.country == '<country>'
Examples:
| name | country |
| monza | Italy |
| spa | Belgium |
| sepang | Malaysia |
Scenario Outline: given race number, validate number of pitstops for Max Verstappen in 2015
Given path 'api/f1/2015/<race>/drivers/max_verstappen/pitstops.json'
When method get
Then assert response.MRData.RaceTable.Races[0].PitStops.length == <stops>
Examples:
| race | stops |
| 1 | 1 |
| 2 | 3 |
| 3 | 2 |
| 4 | 2 |
This is great for testing boundary conditions against a single end-point, with the added bonus that your test becomes even more readable. This approach can certainly enable product-owners or domain-experts who are not programmer-folk, to review, and even collaborate on test-scenarios and scripts.
Scenario Outline Enhancements
Karate has enhanced the Cucumber Scenario Outline as follows:
Type Hints: if the Examples column header has a ! appended, each value will be evaluated as a JavaScript data-type (number, boolean, or even in-line JSON) - else it defaults to string.
Magic Variables: row gives you the entire row as a JSON object, and num gives you the row index (the first row is 0).
Auto Variables: in addition to __row, each column key-value will be available as a separate variable, which greatly simplifies JSON manipulation - especially when you want to re-use JSON files containing embedded expressions.
Any empty cells will result in a null value for that column-key, and this can be useful to remove nodes from JSON or XML documents
These are best explained with examples. You can choose between the string-placeholder style <foo> or directly refer to the variablefoo (or even the whole row JSON as __row) in JSON-friendly expressions.
Note that even the scenario name can accept placeholders - which is very useful in reports.
Scenario Outline: name is <name> and age is <age>
* def temp = '<name>'
* match temp == name
* match temp == __row.name
* def expected = __num == 0 ? 'name is Bob and age is 5' : 'name is Nyan and age is 6'
* match expected == karate.scenario.name
Examples:
| name | age |
| Bob | 5 |
| Nyan | 6 |
Scenario Outline: magic variables with type hints
* def expected = [{ name: 'Bob', age: 5 }, { name: 'Nyan', age: 6 }]
* match __row == expected[__num]
Examples:
| name | age! |
| Bob | 5 |
| Nyan | 6 |
Scenario Outline: embedded expressions and type hints
* match __row == { name: '#(name)', alive: '#boolean' }
Examples:
| name | alive! |
| Bob | false |
| Nyan | true |
Scenario Outline: inline json
* match __row == { first: 'hello', second: { a: 1 } }
* match first == 'hello'
* match second == { a: 1 }
Examples:
| first | second! |
| hello | { a: 1 } |
If you’re looking for more complex ways of dynamically naming your scenarios you can use JS string interpolation by including placeholders in your scenario name.
Scenario Outline: name is ${name.first} ${name.last} and age is ${age}
* match name.first == "#? _ == 'Bob' || _ == 'Nyan'"
* match name.last == "#? _ == 'Dylan' || _ == 'Cat'"
* match title == karate.scenario.name
Examples:
| name! | age | title |
| { "first": "Bob", "last": "Dylan" } | 10 | name is Bob Dylan and age is 10 |
| { "first": "Nyan", "last": "Cat" } | 5 | name is Nyan Cat and age is 5 |
String interpolation will support variables in scope and / or the Examples (including functions defined globally, but not functions defined in the background). Even Java interop and access to the karate JS API would work.
The limitation of the Cucumber Scenario Outline: (seen above) is that the number of rows in the Examples: is fixed. But take a look at how Karate can loop over a *.feature file for each object in a JSON array - which gives you dynamic data-driven testing, if you need it. For advanced examples, refer to some of the scenarios within this demo: dynamic-params.feature.
Also see the option below, where you can data-drive an Examples: table using JSON.
Dynamic Scenario Outline
You can feed an Examples table from a custom data-source, which is great for those situations where the table-content is dynamically resolved at run-time. This capability is triggered when the table consists of a single "cell", i.e. there is exactly one row and one column in the table.
JSON Array Data Source
The "scenario expression" result is expected to be an array of JSON objects. Here is an example:
Feature: scenario outline using a JSON array as the data-source
Scenario Outline: cat name: ${name}
Given url demoBaseUrl
And path 'cats'
And request { name: '#(name)' }
When method post
Then status 200
And match response == { id: '#number', name: '#(name)' }
= the single cell can be any valid karate expression
Examples:
| read('../callarray/kittens.json') |
You can see the structure of the data here: kittens.json
Any Karate expression can be used in the "cell expression", and you can even use Java-interop to use external data-sources such as a database.
In cases where the data-source needs multiple steps, for e.g. if an API needs to be called to get a JSON array, you can call a separate Scenario to "set up" this data. The @setup tag is built-in for this purpose and any Scenario tagged with this will behave like @ignore. So the only way to call this Scenario is by using the karate.setup() JS API.
Here is the above example re-written to do so:
Feature: scenario outline using a JSON array as the data-source
@setup
Scenario:
* def kittens = read('../callarray/kittens.json')
Scenario Outline: cat name: ${name}
Given url demoBaseUrl
And path 'cats'
And request { name: '#(name)' }
When method post
Then status 200
And match response == { id: '#number', name: '#(name)' }
Examples:
| karate.setup().kittens |
The result of karate.setup() will be a JSON of all the variables created within the Scenario tagged with @setup. Note how we "unpack" the kittens and use it to "data drive" the Scenario Outline. You can get really creative and use JS functions to filter data for different needs.
Though not really recommended, you can have multiple Scenario-s within a Feature tagged with @setup. But in that case you should de-dupe them using a name:
And since it is common to run a @setupScenario only once per-feature you can call karate.setupOnce(). In the feature below, the * print 'in setup' step will run only once. Also note how the Background will run 4 times (twice per Scenario).
An advanced option is where the "scenario expression" returns a JavaScript "generator" function. This is a very powerful way to generate test-data without having to load a large number of data rows into memory. The function has to return a JSON object. To signal the end of the data, just return null. The function argument is the row-index, so you can easily determine when to stop the generation of data. Here is an example:
Feature: scenario outline using a dynamic generator function
@setup
Scenario:
* def generator = function(i){ if (i == 20) return null; return { name: 'cat' + i, age: i } }
Scenario Outline: cat name: ${name}
Given url demoBaseUrl
And path 'cats'
And request { name: '#(name)', age: '#(age)' }
When method post
Then status 200
And match response == { id: '#number', name: '#(name)' }
Examples:
| karate.setup().generator |
@karatelabs
Test Automation Made
Simple.
Karate is the only open-source tool to combine API test-automation, mocks, performance-testing and even UI automation into a single, unified framework. The BDD syntax popularized by Cucumber is language-neutral, and easy for even non-programmers. Assertions and HTML reports are built-in, and you can run tests in parallel for speed.
There’s also a cross-platform stand-alone executable for teams not comfortable with Java. You don’t have to compile code. Just write tests in a simple, readable syntax - carefully designed for HTTP, JSON, GraphQL and XML. And you can mix API and UI test-automation within the same test script.
A Java API also exists for those who prefer to programmatically integrate Karate’s rich automation and data-assertion capabilities.
Hello World
readme
For API Testing
It is worth pointing out that JSON is a 'first class citizen' of the syntax such that you can express payload and expected data without having to use double-quotes and without having to enclose JSON field names in quotes. There is no need to 'escape' characters like you would have had to in Java or other programming languages.
And you don’t need to create additional Java classes for any of the payloads that you need to work with.
Index
Features
Java knowledge is not required and even non-programmers can write tests
Scripts are plain-text, require no compilation step or IDE, and teams can collaborate using Git / standard SCM
Based on the popular Cucumber / Gherkin standard - with IDE support and syntax-coloring options
Elegant DSL syntax 'natively' supports JSON and XML - including JsonPath and XPath expressions
Eliminate the need for 'Java Beans' or 'helper code' to represent payloads and HTTP end-points, and dramatically reduce the lines of code needed for a test
Ideal for testing the highly dynamic responses from GraphQL API-s because of Karate’s built-in text-manipulation and JsonPath capabilities
Tests are super-readable - as scenario data can be expressed in-line, in human-friendly JSON, XML, Cucumber Scenario Outline tables, or a payload builder approach unique to Karate
Express expected results as readable, well-formed JSON or XML, and assert in a single step that the entire response payload (no matter how complex or deeply nested) - is as expected
Comprehensive assertion capabilities - and failures clearly report which data element (and path) is not as expected, for easy troubleshooting of even large payloads
Fully featured debugger that can step backwards and even re-play a step while editing it - a huge time-saver
Simpler and more powerful alternative to JSON-schema for validating payload structure and format - that even supports cross-field / domain validation logic
Scripts can call other scripts - which means that you can easily re-use and maintain authentication and 'set up' flows efficiently, across multiple tests
Embedded JavaScript engine that allows you to build a library of re-usable functions that suit your specific environment or organization
Re-use of payload-data and user-defined functions across tests is so easy - that it becomes a natural habit for the test-developer
Built-in support for switching configuration across different environments (e.g. dev, QA, pre-prod)
Support for data-driven tests and being able to tag or group tests is built-in, no need to rely on an external framework
Native support for reading YAML and even CSV files - and you can use them for data-driven tests
Standard Java / Maven project structure, and seamless integration into CI / CD pipelines - and support for JUnit 5
Option to use as a light-weight stand-alone executable - convenient for teams not comfortable with Java
Multi-threaded parallel execution, which is a huge time-saver, especially for integration and end-to-end tests
Built-in test-reports compatible with Cucumber so that you have the option of using third-party (open-source) maven-plugins for even better-looking reports
Reports include HTTP request and response logs in-line, which makes troubleshooting and debugging easier
Easily invoke JDK classes, Java libraries, or re-use custom Java code if needed, for ultimate extensibility
Simple plug-in system for authentication and HTTP header management that will handle any complex, real-world scenario
Cross-browser Web UI automation so that you can test all layers of your application with the same framework
Visual Validation via the built-in image comparison capabilities
Cross platform Desktop Automation that can be mixed into Web Automation flows if needed
Option to invoke via a Java API, which means that you can easily mix Karate into Java projects or legacy UI-automation suites
Save significant effort by re-using Karate test-suites as Gatling performance tests that deeply assert that server responses are accurate under load
Gatling integration can hook into any custom Java code - which means that you can perf-test even non-HTTP protocols such as gRPC
Built-in distributed-testing capability that works for API, UI and even load-testing - without needing any complex "grid" infrastructure
API mocks or test-doubles that even maintain CRUD 'state' across multiple calls - enabling TDD for micro-services and Consumer Driven Contracts
Async support that allows you to seamlessly integrate the handling of custom events or listening to message-queues
Mock HTTP Servlet that enables you to test any controller servlet such as Spring Boot / MVC or Jersey / JAX-RS - without having to boot an app-server, and you can use your HTTP integration tests un-changed
Built-in HTML templating so that you can extend your test-reports into readable specifications
Comprehensive support for different flavors of HTTP calls:
SOAP / XML requests
HTTPS / SSL - without needing certificates, key-stores or trust-stores
HTTP proxy server support
URL-encoded HTML-form data
Multi-part file-upload - including
multipart/mixed
andmultipart/related
Browser-like cookie handling
Full control over HTTP headers, path and query parameters
Re-try until condition
Websocket support
Real World Examples
A set of real-life examples can be found here: Karate Demos
Comparison with REST-assured
For teams familiar with or currently using REST-assured, this detailed comparison of Karate vs REST-assured - can help you evaluate Karate. Do note that if you prefer a pure Java API - Karate has that covered, and with far more capabilities.
References
Intro to all features of Karate - video + demos (with subtitles) by Peter Thomas (creator / lead dev of Karate)
Karate entered the ThoughtWorks Tech Radar in 2019 and was upgraded in ranking in May 2020
マイクロサービスにおけるテスト自動化 with Karate - (Microservices Test Automation with Karate) presentation by Takanori Suzuki
7 New Features in Karate Test Automation Version 1.0 - by Peter Quiel
You can find a lot more references, tutorials and blog-posts in the wiki. Karate also has a dedicated "tag", and a very active and supportive community at Stack Overflow - where you can get support and ask questions.
Getting Started
If you are a Java developer - Karate requires at least Java 8 and then either Maven, Gradle, Eclipse or IntelliJ to be installed. Note that Karate works fine on OpenJDK.
If you are new to programming or test-automation, refer to the options for IDE support and the official IntelliJ plugin is recommended. Other options are the quickstart or the standalone executable.
If you don’t want to use Java, you have the option of just downloading and extracting the ZIP release. Try this especially if you don’t have much experience with programming or test-automation. We recommend that you use the Karate extension for Visual Studio Code - and with that, JavaScript, .NET and Python programmers will feel right at home.
Both the official Visual Studio Code and IntelliJ plugins support step-through debugging of Karate tests.
Maven
All you need is available in the
karate-core
artifact. You can run tests with this directly, but teams can choose the JUnit variant (shown below) that pulls in JUnit 5 and slightly improves the in-IDE experience.If you want to use JUnit 4, use
karate-junit4
instead ofkarate-junit5
.Gradle
Alternatively for Gradle:
Also refer to the wiki for using Karate with Gradle.
Karate Core "Fat JAR"
If you mix Karate into a Maven or Gradle project with many other dependendies, you may run into problems because of dependency conflicts. For example a lot of Java projects directly (or indirectly) depend on Netty or Thymeleaf or ANTLR, etc.
If you face issues such as "class not found", just pull in the
karate-core
dependency, and use theall
classifier in yourpom.xml
(orbuild.gradle
).For example when using Maven:
Note that for very complicated projects you can consider using a Maven profile so that testing-related dependencies don’t collide with your development-time dependencies. Of course it is an option to have Karate tests in a separate stand-alone maven project and folder, while still being in the same Git repository.
Quickstart
It may be easier for you to use the Karate Maven archetype to create a skeleton project with one command. You can then skip the next few sections, as the
pom.xml
, recommended directory structure, sample test and JUnit 5 runners - will be created for you.You can replace the values of
com.mycompany
andmyproject
as per your needs.This will create a folder called
myproject
(or whatever you set the name to).IDE Support
Refer to the wiki - IDE Support.
Folder Structure
A Karate test script has the file extension
.feature
which is the standard followed by Cucumber. You are free to organize your files using regular Java package conventions.The Maven tradition is to have non-Java source files in a separate
src/test/resources
folder structure - but we recommend that you keep them side-by-side with your.java
files. When you have a large and complex project, you will end up with a few data files (e.g..js
,.json
,.txt
) as well and it is much more convenient to see the.java
and.feature
files and all related artifacts in the same place.This can be easily achieved with the following tweak to your maven
<build>
section.This is very common in the world of Maven users and keep in mind that these are tests and not production code.
Alternatively, if using Gradle then add the following
sourceSets
definitionWith the above in place, you don’t have to keep switching between your
src/test/java
andsrc/test/resources
folders, you can have all your test-code and artifacts undersrc/test/java
and everything will work as expected.Once you get used to this, you may even start wondering why projects need a
src/test/resources
folder at all !Spring Boot Example
Soumendra Daas has created a nice example and guide that you can use as a reference here:
hello-karate
. This demonstrates a Java Maven + JUnit 5 project set up to test a Spring Boot app.Naming Conventions
Since these are tests and not production Java code, you don’t need to be bound by the
com.mycompany.foo.bar
convention and the un-necessary explosion of sub-folders that ensues. We suggest that you have a folder hierarchy only one or two levels deep - where the folder names clearly identify which 'resource', 'entity' or API is the web-service under test.For example:
Assuming you use JUnit, there are some good reasons for the recommended (best practice) naming convention and choice of file-placement shown above:
Not using the
Test.java
convention for the JUnit classes (e.g.CatsRunner.java
) in thecats
anddogs
folder ensures that these tests will *not be picked up when invokingmvn test
(for the whole project) from the command line. But you can still invoke these tests from the IDE, which is convenient when in development mode.AnimalsTest.java
(the only file that follows theTest.java
naming convention) acts as the 'test suite' for the entire project. By default, Karate will load all.feature
files from sub-directories as well. But sincesome-reusable.feature
is aboveAnimalsTest.java
in the folder hierarchy, it will not be picked-up. Which is exactly what we want, becausesome-reusable.feature
is designed to be called only from one of the other test scripts (perhaps with some parameters being passed). You can also use tags to skip files.some-classpath-function.js
andsome-classpath-payload.json
are in the 'root' of the Java 'classpath' which means they can be easily read (and re-used) from any test-script by using theclasspath:
prefix, for e.g:read('classpath:some-classpath-function.js')
. Relative paths will also work.For details on what actually goes into a script or
*.feature
file, refer to the syntax guide.file.encoding
In some cases, for large payloads and especially when the default system encoding is not
UTF-8
(Windows or non-US locales), you may run into issues where ajava.io.ByteArrayInputStream
is encountered instead of a string. Other errors could be ajava.net.URISyntaxException
andmatch
not working as expected because of special or foreign characters, e.g. German orISO-8859-15
. Typical symptoms are your tests working fine via the IDE but not when running via Maven or Gradle. The solution is to ensure that when Karate tests run, the JVMfile.encoding
is set toUTF-8
. This can be done via themaven-surefire-plugin
configuration. Add the plugin to the<build>/<plugins>
section of yourpom.xml
if not already present:JUnit 4
To run a script
.feature
file from your Java IDE, you just need the following empty test-class in the same package. The name of the class doesn’t matter, and it will automatically run any.feature
file in the same package. This comes in useful because depending on how you organize your files and folders - you can have multiple feature files executed by a single JUnit test-class.Refer to your IDE documentation for how to run a JUnit class. Typically right-clicking on the file in the project browser or even within the editor view would bring up the "Run as JUnit Test" menu option.
JUnit 5
Karate supports JUnit 5 and the advantage is that you can have multiple methods in a test-class. Only 1
import
is needed, and instead of a class-level annotation, you use a nice DRY and fluent-api to express which tests and tags you want to use.Note that the Java class does not need to be
public
and even the test methods do not need to bepublic
- so tests end up being very concise.Here is an example:
Note that more "builder" methods are available from the
Runner.Builder
class such asreportDir()
etc.You should be able to right-click and run a single method using your IDE - which should be sufficient when you are in development mode. But to be able to run JUnit 5 tests from the command-line, you need to ensure that the latest version of the maven-surefire-plugin is present in your project
pom.xml
(within the<build>/<plugins>
section):To run a single test method, for example the
testTags()
in the example above, you can do this:Also look at how to run tests via the command-line and the parallel runner.
JUnit HTML report
When you use a JUnit runner - after the execution of each feature, an HTML report is output to the
target/karate-reports
folder and the full path will be printed to the console (see video).You can easily select (double-click), copy and paste this
file:
URL into your browser address bar. This report is useful for troubleshooting and debugging a test because all requests and responses are shown in-line with the steps, along with error messages and the output ofprint
statements. Just re-fresh your browser window if you re-run the test.Dry Run
This will give you the usual HTML report showing what features will be run, including all steps shown (including comments) so that it can be reviewed. Of course the actual time-durations, and logs will be missing, and everything will pass.
The “dry run” report is useful to review the tag "coverage" of what will be run. For example you can get a nice feature “coverage” report, provided you have a rich set of tags. e.g.
@smoke @module=one @module=two
etc.The
Runner.Builder
API has adryRun()
method to switch this on. Note that this mode can be also triggered via the command-line by adding-D
or--dryrun
to thekarate.options
.Command Line
Normally in dev mode, you will use your IDE to run a
*.feature
file directly or via the companion 'runner' JUnit Java class. When you have a 'runner' class in place, it would be possible to run it from the command-line as well.Note that the
mvn test
command only runs test classes that follow the*Test.java
naming convention by default. But you can choose a single test to run like this:karate.options
When your Java test "runner" is linked to multiple feature files, which will be the case when you use the recommended parallel runner, you can narrow down your scope to a single feature, scenario or directory via the command-line, useful in dev-mode. Note how even tags to exclude (or include) can be specified:
Multiple feature files (or paths) can be specified, de-limited by the space character. They should be at the end of the
karate.options
. To run only a single scenario, append the line number on which the scenario is defined, de-limited by:
.Since paths are expected at the end of the command-line options - if you want to only over-ride tags, use the
=
sign to make argument values clear. For example:Command Line - Gradle
For Gradle, you must extend the test task to allow the
karate.options
to be passed to the runtime (otherwise they get consumed by Gradle itself). To do that, add the following:And then the above command in Gradle would look like:
or
Test Suites
One way to define 'test-suites' in Karate is to have a JUnit class at a level 'above' (in terms of folder hierarchy) all the
*.feature
files in your project. So if you take the previous folder structure example, you can do this on the command-line:Here,
AnimalsTest
is the name of the Java class we designated to run the multiple*.feature
files that make up your test-suite. There is a neat way to tag your tests and the above example demonstrates how to run all tests except the ones tagged@skipme
.Note that the special, built-in tag
@ignore
will always be skipped by default, and you don’t need to specify~@ignore
anywhere.You can 'lock down' the fact that you only want to execute the single JUnit class that functions as a test-suite - by using the following maven-surefire-plugin configuration:
Note how the
karate.options
can be specified using the<systemProperties>
configuration.For Gradle, you simply specify the test which is to be
include
-d:The big drawback of the approach above is that you cannot run tests in parallel. The recommended approach for Karate reporting in a Continuous Integration set-up is described in the next section which can generate the JUnit XML format that most CI tools can consume. The Cucumber JSON format can be also emitted, which gives you plenty of options for generating pretty reports using third-party maven plugins.
And most importantly - you can run tests in parallel without having to depend on third-party hacks that introduce code-generation and config 'bloat' into your
pom.xml
orbuild.gradle
.Parallel Execution
Karate can run tests in parallel, and dramatically cut down execution time. This is a 'core' feature and does not depend on JUnit, Maven or Gradle.
You can easily "choose" features and tags to run and compose test-suites in a very flexible manner.
You can use the returned
Results
object to check if any scenarios failed, and to even summarize the errorsJUnit XML reports can be generated in the "
reportDir
" path you specify, and you can easily configure your CI to look for these files after a build (for e.g. in**/*.xml
or**/karate-reports/*.xml
). Note that you have to call theoutputJunitXml(true)
method on theRunner
"builder".Cucumber JSON reports can be generated, except that the extension will be
.json
instead of.xml
. Note that you have to call theoutputCucumberJson(true)
method on theRunner
"builder".HTML reports can be disabled by calling
outputHtmlReport(false)
. Please note that some user analytics is tracked only when you view the built-in Karate HTML report.JUnit 4 Parallel Execution
@RunWith(Karate.class)
annotation. This is a normal JUnit 4 test class ! If you want to use JUnit 4, use thekarate-junit4
Maven dependency instead ofkarate-junit5
.You don’t use a JUnit runner (no
@RunWith
annotation), and you write a plain vanilla JUnit test (it could even be a normal Java class with amain
method)The
Runner.path()
"builder" method inkarate-core
is how you refer to the package you want to execute, and all feature files within sub-directories will be picked upRunner.path()
takes multiple string parameters, so you can refer to multiple packages or even individual*.feature
files and easily "compose" a test-suitee.g.
Runner.path("classpath:animals", "classpath:some/other/package.feature")
To choose tags, call the
tags()
API, note that by default, any*.feature
file tagged with the special (built-in) tag:@ignore
will be skipped. You can also specify tags on the command-line. Thetags()
method also takes multiple arguments, for e.g.this is an "AND" operation:
tags("@customer", "@smoke")
and this is an "OR" operation:
tags("@customer,@smoke")
There is an optional
reportDir()
method if you want to customize the directory to which the HTML, XML and JSON files will be output, it defaults totarget/karate-reports
If you want to dynamically and programmatically determine the tags and features to be included - the API also accepts
List<String>
as thepath()
andtags()
methods argumentsparallel()
has to be the last method called, and you pass the number of parallel threads needed. It returns aResults
object that has all the information you need - such as the number of passed or failed tests.JUnit 5 Parallel Execution
For JUnit 5 you can omit the
public
modifier for the class and method, and there are some changes toimport
package names. The method signature of theassertTrue
has flipped around a bit. Also note that you don’t use@Karate.Test
for the method, and you just use the normal JUnit 5@Test
annotation.Else the
Runner.path()
"builder" API is the same, refer the description above for JUnit 4.Parallel Stats
For convenience, some stats are logged to the console when execution completes, which should look something like this:
The parallel runner will always run
Feature
-s in parallel. Karate will also runScenario
-s in parallel by default. So if you have aFeature
with multipleScenario
-s in it - they will execute in parallel, and even eachExamples
row in aScenario Outline
will do so !A
karate-timeline.html
file will also be saved to the report output directory mentioned above (target/karate-reports
by default) - which is useful for visually verifying or troubleshooting the effectiveness of the test-run (see video).@parallel=false
In rare cases you may want to suppress the default of
Scenario
-s executing in parallel and the specialtag
@parallel=false
can be used. If you place it above theFeature
keyword, it will apply to allScenario
-s. And if you just want one or twoScenario
-s to NOT run in parallel, you can place this tag above only thoseScenario
-s. See example.Note that forcing
Scenario
-s to run in a particular sequence is an anti-pattern, and should be avoided as far as possible.Test Reports
As mentioned above, most CI tools would be able to process the JUnit XML output of the parallel runner and determine the status of the build as well as generate reports.
The Karate Demo has a working example of the recommended parallel-runner set up. It also details how a third-party library can be easily used to generate some very nice-looking reports, from the JSON output of the parallel runner.
For example, here below is an actual report generated by the cucumber-reporting open-source library.
Another example for a popular Maven reporting plugin that is compatible with Karate JSON is Cluecumber.
The demo also features code-coverage using Jacoco, and some tips for even non-Java back-ends. Some third-party report-server solutions integrate with Karate such as ReportPortal.io.
Logging
Karate uses LOGBack which looks for a file called
logback-test.xml
on the 'classpath'.Here is a sample
logback-test.xml
for you to get started.You can change the
com.intuit.karate
logger level toINFO
to reduce the amount of logging. When the level isDEBUG
the entire request and response payloads are logged. If you use the above config, logs will be captured intarget/karate.log
.If you want to keep the level as
DEBUG
(for HTML reports) but suppress logging to the console, you can comment out theSTDOUT
"root"appender-ref
:Or another option is to use a
ThresholdFilter
, so you still see critical logs on the console:If you want to exclude the logs from your CI/CD pipeline but keep them in the execution of your users in their locals you can configure your logback using Janino. In such cases it might be desirable to have your tests using
karate.logger.debug('your additional info')
instead of theprint
keyword so you can keep logs in your pipeline in INFO.For suppressing sensitive information such as secrets and passwords from the log and reports, see Log Masking and Report Verbosity.
Configuration
Classpath
The 'classpath' is a Java concept and is where some configuration files such as the one for logging are expected to be by default. If you use the Maven
<test-resources>
tweak described earlier (recommended), the 'root' of the classpath will be in thesrc/test/java
folder, or else would besrc/test/resources
.karate-config.js
The only 'rule' is that on start-up Karate expects a file called
karate-config.js
to exist on the 'classpath' and contain a JavaScript function. The function is expected to return a JSON object and all keys and values in that JSON object will be made available as script variables.And that’s all there is to Karate configuration ! You can easily get the value of the current 'environment' or 'profile', and then set up 'global' variables using some simple JavaScript. Here is an example:
A common requirement is to pass dynamic parameter values via the command line, and you can use the
karate.properties['some.name']
syntax for getting a system property passed via JVM options in the form-Dsome.name=foo
. Refer to the section on dynamic port numbers for an example.This decision to use JavaScript for config is influenced by years of experience with the set-up of complicated test-suites and fighting with Maven profiles, Maven resource-filtering and the XML-soup that somehow gets summoned by the Maven AntRun plugin.
Karate’s approach frees you from Maven, is far more expressive, allows you to eyeball all environments in one place, and is still a plain-text file. If you want, you could even create nested chunks of JSON that 'name-space' your config variables.
This approach is indeed slightly more complicated than traditional
*.properties
files - but you need this complexity. Keep in mind that these are tests (not production code) and this config is going to be maintained more by the dev or QE team instead of the 'ops' or operations team.And there is no more worrying about Maven profiles and whether the 'right'
*.properties
file has been copied to the proper place.Switching the Environment
There is only one thing you need to do to switch the environment - which is to set a Java system property.
The recipe for doing this when running Maven from the command line is:
Or in Gradle:
You can refer to the documentation of the Maven Surefire Plugin for alternate ways of achieving this, but the
argLine
approach is the simplest and should be more than sufficient for your Continuous Integration or test-automation needs.Here’s a reminder that running any single JUnit test via Maven can be done by:
Where
CatsRunner
is the JUnit class name (in any package) you wish to run.Karate is flexible, you can easily over-write config variables within the Java or JUnit "runner" - which is very convenient when in dev-mode or rapid-prototyping.
But the recommended way is to use the
karateEnv(name, value)
orsystemProperty(name, value)
API on the parallel-runner.For advanced users, note that tags and the
karate.env
environment-switch can be "linked" using the special environment tags.Environment Specific Config
When your project gets complex, you can have separate
karate-config-<env>.js
files that will be processed for that specific value ofkarate.env
. This is especially useful when you want to maintain passwords, secrets or even URL-s specific for your local dev environment.Here are the rules Karate uses on bootstrap (before every
Scenario
orExamples
row in aScenario Outline
):if the system-property
karate.config.dir
was set, Karate will look in this folder forkarate-config.js
- and if found, will process itelse if
karate-config.js
was not found in the above location (orkarate.config.dir
was not set),classpath:karate-config.js
would be processed (this is the default / common case)if the
karate.env
system property was setif
karate.config.dir
was set, Karate will also look forfile:<karate.config.dir>/karate-config-<env>.js
else (if the
karate.config.dir
was not set), Karate will look forclasspath:karate-config-<env>.js
if the over-ride
karate-config-<env>.js
exists, it will be processed, and the configuration (JSON entries) returned by this function will over-ride any set bykarate-config.js
Refer to the karate demo for an example.
karate-base.js
Advanced users who build frameworks on top of Karate have the option to supply a
karate-base.js
file that Karate will look for on theclasspath:
. This is useful when you ship a JAR file containing re-usable features and JavaScript / Java code and want to 'default' a few variables that teams can 'inherit' from. So an additional rule in the above flow of 'rules' (before the first step) is as follows:if
classpath:karate-base.js
exists - Karate will process this as a configuration source before anything elseSyntax Guide
Script Structure
Karate scripts are technically in 'Gherkin' format - but all you need to grok as someone who needs to test web-services are the three sections:
Feature
,Background
andScenario
. There can be multiple Scenario-s in a*.feature
file, and at least one should be present. TheBackground
is optional.Lines that start with a
#
are comments.Given-When-Then
The business of web-services testing requires access to low-level aspects such as HTTP headers, URL-paths, query-parameters, complex JSON or XML payloads and response-codes. And Karate gives you control over these aspects with the small set of keywords focused on HTTP such as
url
,path
,param
, etc.Karate does not attempt to have tests be in "natural language" like how Cucumber tests are traditionally expected to be. That said, the syntax is very concise, and the convention of every step having to start with either
Given
,And
,When
orThen
, makes things very readable. You end up with a decent approximation of BDD even though web-services by nature are "headless", without a UI, and not really human-friendly.Cucumber vs Karate
If you are familiar with Cucumber (JVM), you may be wondering if you need to write step-definitions. The answer is no.
Karate’s approach is that all the step-definitions you need in order to work with HTTP, JSON and XML have been already implemented. And since you can easily extend Karate using JavaScript, there is no need to compile Java code any more.
The following table summarizes some key differences between Cucumber and Karate.
Step Definitions Built-In
No. You need to keep implementing them as your functionality grows. This can get very tedious, especially since for dependency-injection, you are on your own.
:white_check_mark: Yes. No extra Java code needed.
Single Layer of Code To Maintain
No. There are 2 Layers. The Gherkin spec or
*.feature
files make up one layer, and you will also have the corresponding Java step-definitions.:white_check_mark: Yes. Only 1 layer of Karate-script (based on Gherkin).
Readable Specification
Yes. Cucumber will read like natural language if you implement the step-definitions right.
:x: No. Although Karate is simple, and a true DSL, it is ultimately a mini-programming language. But it is perfect for testing web-services at the level of HTTP requests and responses.
Re-Use Feature Files
No. Cucumber does not support being able to call (and thus re-use) other
*.feature
files from a test-script.:white_check_mark: Yes.
Dynamic Data-Driven Testing
No. Cucumber’s
Scenario Outline
expects theExamples
to contain a fixed set of rows.:white_check_mark: Yes. Karate’s support for calling other
*.feature
files allows you to use a JSON array as the data-source and you can use JSON or even CSV directly in a data-drivenScenario Outline
.Parallel Execution
No. There are some challenges (especially with reporting) and you can find various discussions and third-party projects on the web that attempt to close this gap
:white_check_mark: Yes. Karate runs even
Scenario
-s in parallel, not justFeature
-s.Run 'Set-Up' Routines Only Once
No. Cucumber has a limitation where
Background
steps are re-run for everyScenario
and worse - even for everyExamples
row within aScenario Outline
. This has been a highly-requested open issue for a long time.:white_check_mark: Yes.
Embedded JavaScript Engine
No. And you have to roll your own approach to environment-specific configuration and worry about dependency-injection.
:white_check_mark: Yes. Easily define all environments in a single file and share variables across all scenarios. Full script-ability via JS or Java interop.
One nice thing about the design of the Gherkin syntax is that script-steps are treated the same no matter whether they start with the keyword
Given
,And
,When
orThen
. What this means is that you are free to use whatever makes sense for you. You could even have all the steps start withWhen
and Karate won’t care.In fact Gherkin supports the https://www.relishapp.com/cucumber/cucumber/docs/gherkin/using-star-notation-instead-of-given-when-then
'] - instead of forcing you to use
Given
,When
orThen
. This is perfect for those cases where it really doesn’t make sense - for example theBackground
section or when you use thedef
orset
syntax. When eyeballing a test-script, think of theas a 'bullet-point'.
You can read more about the Given-When-Then convention at the Cucumber reference documentation. Since Karate uses Gherkin, you can also employ data-driven techniques such as expressing data-tables in test scripts. Another good thing that Karate inherits is the nice IDE support for Cucumber that IntelliJ and Eclipse have. So you can do things like right-click and run a
*.feature
file (or scenario) without needing to use a JUnit runner.For a detailed discussion on BDD and how Karate relates to Cucumber, please refer to this blog-post: Yes, Karate is not true BDD. It is the opinion of the author of Karate that true BDD is un-necessary over-kill for API testing, and this is explained more in this answer on Stack Overflow.
With the formalities out of the way, let’s dive straight into the syntax.
Setting and Using Variables
def
Set a named variable
Note that
def
will over-write any variable that was using the same name earlier. Keep in mind that the start-up configuration routine could have already initialized some variables before the script even started. For details of scope and visibility of variables, see Script Structure.The examples above are simple, but a variety of expression 'shapes' are supported on the right hand side of the
=
symbol. The section on Karate Expressions goes into the details.assert
Assert if an expression evaluates to
true
Once defined, you can refer to a variable by name. Expressions are evaluated using the embedded JavaScript engine. The assert keyword can be used to assert that an expression returns a boolean value.
Everything to the right of the
assert
keyword will be evaluated as a single expression.Something worth mentioning here is that you would hardly need to use
assert
in your test scripts. Instead you would typically use thematch
keyword, that is designed for performing powerful assertions against JSON and XML response payloads.print
Log to the console
You can use
print
to log variables to the console in the middle of a script. For convenience, you can have multiple expressions separated by commas, so this is the recommended pattern:Similar to
assert
, the expressions on the right-hand-side of aprint
have to be valid JavaScript. JsonPath and Karate expressions are not supported.If you use commas (instead of concatenating strings using
+
), Karate will 'pretty-print' variables, which is what you typically want when dealing with JSON or XML.Which results in the following output:
The built-in
karate
object is explained in detail later, but for now, note that this is also injected intoprint
(and evenassert
) statements, and it has a helpfulpretty
method, that takes a JSON argument and aprettyXml
method that deals with XML. So you could have also done something like:Also refer to the
configure
keyword on how to switch on pretty-printing of all HTTP requests and responses.'Native' data types
Native data types mean that you can insert them into a script without having to worry about enclosing them in strings and then having to 'escape' double-quotes all over the place. They seamlessly fit 'in-line' within your test script.
JSON
Note that the parser is 'lenient' so that you don’t have to enclose all keys in double-quotes.
When asserting for expected values in JSON or XML, always prefer using
match
instead ofassert
. Match failure messages are much more descriptive and useful, and you get the power of embedded expressions and fuzzy matching.Karate’s native support for JSON means that you can assign parts of a JSON instance into another variable, which is useful when dealing with complex
response
payloads.For manipulating or updating JSON (or XML) using path expressions, refer to the
set
keyword.XML
Embedded Expressions
Karate has a very useful payload 'templating' approach. Variables can be referred to within JSON, for example:
So the rule is - if a string value within a JSON (or XML) object declaration is enclosed between
#(
and)
- it will be evaluated as a JavaScript expression. And any variables which are alive in the context can be used in this expression. Here’s how it works for XML:This comes in useful in some cases - and avoids needing to use the
set
keyword or JavaScript functions to manipulate JSON. So you get the best of both worlds: the elegance of JSON to express complex nested data - while at the same time being able to dynamically plug values (that could even be other JSON or XML 'trees') into a 'template'.Note that embedded expressions will be evaluated even when you
read()
from a JSON or XML file. This is super-useful for re-use and data-driven tests.A few special built-in variables such as
$
(which is a reference to the JSON root) - can be mixed into JSON embedded expressions.A special case of embedded expressions can remove a JSON key (or XML element / attribute) if the expression evaluates to
null
.Rules for Embedded Expressions
They work only within JSON or XML
and when on the Right Hand Side of a
def
match
configure
and when you
read()
a JSON or XML filethe expression has to start with
#(
and end with)
Because of the last rule above, note that string-concatenation may not work quite the way you expect:
Observe how you can achieve string concatenation if you really want, because any valid JavaScript expression can be stuffed within an embedded expression. You could always do this in two steps:
As a convenience, embedded expressions are supported on the Right Hand Side of a
match
statement even for "quoted string" literals:And do note that in Karate 1.0 onwards, ES6 string-interpolation within "backticks" is supported:
Enclosed JavaScript
An alternative to embedded expressions (for JSON only) is to enclose the entire payload within parentheses - which tells Karate to evaluate it as pure JavaScript. This can be a lot simpler than embedded expressions in many cases, and JavaScript programmers will feel right at home.
The example below shows the difference between embedded expressions and enclosed JavaScript:
Multi-Line Expressions
The keywords
def
,set
,match
,request
andeval
take multi-line input as the last argument. This is useful when you want to express a one-off lengthy snippet of text in-line, without having to split it out into a separate file. Note how triple-quotes ("""
) are used to enclose content. Here are some examples:table
A simple way to create JSON Arrays
Now that we have seen how JSON is a 'native' data type that Karate understands, there is a very nice way to create JSON using Cucumber’s support for expressing data-tables.
The
match
keyword is explained later, but it should be clear right away how convenient thetable
keyword is. JSON can be combined with the ability to call other*.feature
files to achieve dynamic data-driven testing in Karate.Notice that in the above example, string values within the table need to be enclosed in quotes. Otherwise they would be evaluated as expressions - which does come in useful for some dynamic data-driven situations:
Yes, you can even nest chunks of JSON in tables, and things work as you would expect.
Empty cells or expressions that evaluate to
null
will result in the key being omitted from the JSON. To force anull
value, wrap it in parentheses:An alternate way to create data is using the
set
multiple syntax. It is actually a 'transpose' of thetable
approach, and can be very convenient when there are a large number of keys per row or if the nesting is complex. Here is an example of what is possible:text
Don’t parse, treat as raw text
Not something you would commonly use, but in some cases you need to disable Karate’s default behavior of attempting to parse anything that looks like JSON (or XML) when using multi-line / string expressions. This is especially relevant when manipulating GraphQL queries - because although they look suspiciously like JSON, they are not, and tend to confuse Karate’s internals. And as shown in the example below, having text 'in-line' is useful especially when you use the
Scenario Outline:
andExamples:
for data-driven tests involving Cucumber-style place-holder substitutions in strings.Note that if you did not need to inject
Examples:
into 'placeholders' enclosed within<
and>
, reading from a file with the extension*.txt
may have been sufficient.For placeholder-substitution, the
replace
keyword can be used instead, but with the advantage that the text can be read from a file or dynamically created.Karate is a great fit for testing GraphQL because of how easy it is to deal with dynamic and deeply nested JSON responses. Refer to this example for more details:
graphql.feature
.replace
Text Placeholder Replacement
Karate provides an elegant 'native-like' experience for placeholder substitution within strings or text content. This is useful in any situation where you need to concatenate dynamic string fragments to form content such as GraphQL or SQL.
The placeholder format defaults to angle-brackets, for example:
<replaceMe>
. Here is how to replace one placeholder at a time:Karate makes it really easy to substitute multiple placeholders in a single, readable step as follows:
Note how strings have to be enclosed in quotes. This is so that you can mix expressions into text replacements as shown below. This example also shows how you can use a custom placeholder format instead of the default:
Refer to this file for a detailed example:
replace.feature
YAML Files
For those who may prefer YAML as a simpler way to represent data, Karate allows you to read YAML content from a file - and it will be auto-converted into JSON.
yaml
A very rare need is to be able to convert a string which happens to be in YAML form into JSON, and this can be done via the
yaml
type cast keyword. For example - if a response data element or downloaded file is YAML and you need to use the data in subsequent steps. Also see type conversion.CSV Files
Karate can read
*.csv
files and will auto-convert them to JSON. A header row is always expected. See the section on reading files - and also this exampledynamic-csv.feature
, which shows off the convenience of dynamicScenario Outline
-s.In rare cases you may want to use a csv-file as-is and not auto-convert it to JSON. A good example is when you want to use a CSV file as the request-body for a file-upload. You could get by by renaming the file-extension to say
*.txt
but an alternative is to use thekarate.readAsString()
API.csv
Just like
yaml
, you may occasionally need to convert a string which happens to be in CSV form into JSON, and this can be done via thecsv
keyword.JavaScript Functions
JavaScript Functions are also 'native'. And yes, functions can take arguments.
Java Interop
For more complex functions you are better off using the multi-line 'doc-string' approach. This example actually calls into existing Java code, and being able to do this opens up a whole lot of possibilities. The JavaScript interpreter will try to convert types across Java and JavaScript as smartly as possible. For e.g. JSON objects become Java
Map
-s, JSON arrays become JavaList
-s, and Java Bean properties are accessible (and update-able) using 'dot notation' e.g. 'object.name
'The
call
keyword provides an alternate way of calling JavaScript functions that have only one argument. The argument can be provided after the function name, without parentheses, which makes things slightly more readable (and less cluttered) especially when the solitary argument is JSON.Reading Files
Karate makes re-use of payload data, utility-functions and even other test-scripts as easy as possible. Teams typically define complicated JSON (or XML) payloads in a file and then re-use this in multiple scripts. Keywords such as
set
andremove
allow you to to 'tweak' payload-data to fit the scenario under test. You can imagine how this greatly simplifies setting up tests for boundary conditions. And such re-use makes it easier to re-factor tests when needed, which is great for maintainability.Reading files is achieved using the built-in JavaScript function called
read()
. By default, the file is expected to be in the same folder (package) and side-by-side with the*.feature
file. But you can prefix the name withclasspath:
in which case the 'root' folder would besrc/test/java
(assuming you are using the recommended folder structure).Prefer
classpath:
when a file is expected to be heavily re-used all across your project. And yes, relative paths will work.You can also re-use other
*.feature
files from test-scripts:When a called feature depends on some side-by-side resources such as JSON or JS files, you can use the
this:
prefix to ensure that relative paths work correctly - because by default Karate calculates relative paths from the "root" feature or the top-most "caller".If a file does not end in
.json
,.xml
,.yaml
,.js
,.csv
or.txt
, it is treated as a stream - which is typically what you would need formultipart
file uploads.For JSON and XML files, Karate will evaluate any embedded expressions on load. This enables more concise tests, and the file can be re-usable in multiple, data-driven tests.
Since it is internally implemented as a JavaScript function, you can mix calls to
read()
freely wherever JavaScript expressions are allowed:* def someConfig = read('my-config-' + someVariable + '.json')
. Refer to conditional logic for more ideas.And a very common need would be to use a file as the
request
body:Or in a
match
:The rarely used
file:
prefix is also supported. You could use it for 'hard-coded' absolute paths in dev mode, but is obviously not recommended for CI test-suites. A good example of where you may need this is if you programmatically write a file to thetarget
folder, and then you can read it like this:Path Prefixes
To summarize the possible prefixes:
classpath:
relative to the classpath, recommended for re-usable features
file:
do not use this unless you know what you are doing, see above
this:
when in a called feature, ensure that files are resolved relative to the current feature file
Take a look at the Karate Demos for real-life examples of how you can use files for validating HTTP responses, like this one:
read-files.feature
.Read File As String
In some rare cases where you don’t want to auto-convert JSON, XML, YAML or CSV, and just get the raw string content (without having to re-name the file to end with
.txt
) - you can use thekarate.readAsString()
API. Here is an example of using a CSV file as the request-body:Compare Image
Karate provides a flexible way to compare two images to determine if they are the same or similar. This is especially useful when capturing screenshots during tests and comparing against baseline images that are known to be correct.
Below is a simple example that will compare a
baseline
image to a more recentlatest
image. An image comparison UI will also be embedded into the Karate HTML report with detailed information about any differences between the two images.You can also compare images using Karate path prefixes (e.g.
classpath:
,this:
,file:
) or byte arrays:You may configure the following image comparison options using the
configure
action:Image comparison configuration options:
allowScaling
boolean
false
When
true
we will scale latest images to match the dimensions of the baseline when they are not equalengine
string
'resemble'
Comparison engine(s) to use. Valid options are
resemble
andssim
separated by either,
or|
failureThreshold
number
0.0
Precentage of
latest
image pixels allowed to differ frombaseline
before we consider the comparison as failedmismatchShouldPass
boolean
false
When
true
all image comparisons will pass (even when difference is >=failureThreshold
). Note: failures will result in image comparison UI always being embedded in Karate HTML reports regardless ofhideUiOnSuccess
setting.onShowRebase
string (js)
null
Function to be called when displaying image comparison rebase in Karate HTML reports (e.g. to customize rebase filename and/or output)
onShowConfig
string (js)
null
Function to be called when displaying image comparison configuration in Karate HTML reports (e.g. to customize configuration output)
hideUiOnSuccess
boolean
false
When
true
the comparison UI will NOT be embedded in Karate HTML reports for all non-failed image comparisonsExamples:
Image comparison engines can also be customized:
Image comparison configuration options:
ignoredBoxes
resemble, ssim
array of object
null
Array of rectangles that should be ignored during image comparison
ignore
resemble
string
'less'
Resemble ignore preset. Valid options are
nothing
,less
,antialiasing
,colors
,alpha
(see presets).ignoreAreasColoredWith
resemble
object
null
Resemble option to ignore a specific color
ignoreColors
resemble
boolean
false
When
true
only pixel brightness is comparedignoreAntialiasing
resemble
boolean
false
When
true
only pixel brightness is compared for pixels determined to be antialiasedtolerances
resemble
object
null
Resemble option to override preset tolerances for color and brightness
ssim
ssim
string
'WEBER'
SSIM algorithm. Valid options are
FAST
orWEBER
rgb2grayVersion
ssim
string
'INTEGER'
SSIM grayscale algorithm. Valid options are
ORIGINAL
orINTEGER
k1
ssim
number
0.01
SSIM first stability constant (see ssim.pdf)
k2
ssim
number
0.03
SSIM second stability constant (see ssim.pdf)
windowSize
ssim
integer
11
Window size for the SSIM map (see ssim.pdf)
bitDepth
ssim
integer
8
The number of bits used to encode each pixel
maxSize
ssim
integer
256
The maximum size on the smallest dimension before downsampling
Examples:
Type Conversion
Internally, Karate will auto-convert JSON (and even XML) to Java
Map
objects. And JSON arrays would become JavaList
-s. But you will never need to worry about this internal data-representation most of the time.In some rare cases, for e.g. if you acquired a string from some external source, or if you generated JSON (or XML) by concatenating text or using
replace
, you may want to convert a string to JSON and vice-versa. You can even perform a conversion from XML to JSON if you want.One example of when you may want to convert JSON (or XML) to a string is when you are passing a payload to custom code via Java interop. Do note that when passing JSON, the default
Map
andList
representations should suffice for most needs (see example), and using them would avoid un-necessary string-conversion.So you have the following type markers you can use instead of
def
(or the rarely usedtext
). The first four below are best explained in this example file:type-conv.feature
.string
- convert JSON or any other data-type (except XML) to a stringjson
- convert XML, a map-like or list-like object, a string, or even a Java object into JSONxml
- convert JSON, a map-like object, a string, or even a Java object into XMLxmlstring
- specifically for converting the map-like Karate internal representation of XML into a stringcsv
- convert a CSV string into JSON, seecsv
yaml
- convert a YAML string into JSON, seeyaml
bytes
- convert to a byte-array, useful for binary payloads or comparisons, see examplecopy
- to clone a given payload variable reference (JSON, XML, Map or List), refer:copy
The
csv
andyaml
types can be initialized in-line using the "triple quote" or "docstring" multi-line approach as shown here.If you want to 'pretty print' a JSON or XML value with indenting, refer to the documentation of the
print
keyword.Floats and Integers
While converting a number to a string is easy (just concatenate an empty string e.g.
myInt + ''
), in some rare cases, you may need to convert a string to a number. You can do this by multiplying by1
or using the built-in JavaScriptparseInt()
function:As per the JSON spec, all numeric values are treated as doubles, so for integers - it really doesn’t matter if there is a decimal point or not. In fact it may be a good idea to slip doubles instead of integers into some of your tests ! Anyway, there are times when you may want to force integers (perhaps for cosmetic reasons) and you can easily do so using the 'double-tilde' http://rocha.la/JavaScript-bitwise-operators-in-practice
~~
'].Large Numbers
Sometimes when dealing with very large numbers, the JS engine may mangle the number into scientific notation:
This can be easily solved by using
java.math.BigDecimal
:doc
Karate has a built-in HTML templating engine that can be used to insert additional custom HTML into the test-reports. Here is an example:
Any Karate variable will be available to the template, which is
users.html
in this example.You can see what the result looks like here.
Since templates can be loaded using the
classpath:
prefix, you can even re-use templates across your projects via Java JAR files.Karate Expressions
Before we get to the HTTP keywords, it is worth doing a recap of the various 'shapes' that the right-hand-side of an assignment statement can take:
* def foo = 'bar'
JS
simple strings, numbers or booleans
* def foo = 'bar' + baz[0]
JS
any valid JavaScript expression, and variables can be mixed in, another example:
bar.length + 1
* def foo = { bar: '#(baz)' }
JSON
anything that starts with a
{
or a[
is parsed as JSON, usetext
instead ofdef
if you need to suppress the default behavior* def foo = ({ bar: baz })
JS
enclosed JavaScript, the result of which is exactly equivalent to the above
* def foo = <foo>bar</foo>
XML
anything that starts with a
<
is parsed as XML, usetext
instead ofdef
if you need to suppress the default behavior* def foo = function(arg){ return arg + bar }
JS Fn
anything that starts with
function(...){
is parsed as a JS function.* def foo = read('bar.json')
JS
using the built-in
read()
function* def foo = $.bar[0]
JsonPath
short-cut JsonPath on the
response
* def foo = /bar/baz
XPath
short-cut XPath on the
response
* def foo = get bar $..baz[?(@.ban)]
get
JsonPathJsonPath on the variable
bar
, you can also useget[0]
to get the first item if the JsonPath evaluates to an array - especially useful when using wildcards such as[*]
or filter-criteria* def foo = $bar..baz[?(@.ban)]
$var.JsonPath
convenience short-cut for the above
* def foo = get bar count(/baz//ban)
get
XPathXPath on the variable
bar
* def foo = karate.pretty(bar)
JS
using the built-in
karate
object in JS expressions* def Foo = Java.type('com.mycompany.Foo')
JS-Java
Java Interop, and even package-name-spaced one-liners like
java.lang.System.currentTimeMillis()
are possible* def foo = call bar { baz: '#(ban)' }
call
or
callonce
, where expressions likeread('foo.js')
are allowed as the object to be called or the argument* def foo = bar({ baz: ban })
JS
equivalent to the above, JavaScript function invocation
Core Keywords
They are
url
,path
,request
,method
andstatus
.These are essential HTTP operations, they focus on setting one (un-named or 'key-less') value at a time and therefore don’t need an
=
sign in the syntax.url
A URL remains constant until you use the
url
keyword again, so this is a good place to set-up the 'non-changing' parts of your REST URL-s.A URL can take expressions, so the approach below is legal. And yes, variables can come from global config.
If you are trying to build dynamic URLs including query-string parameters in the form:
http://myhost/some/path?foo=bar&search=true
- please refer to theparam
keyword.path
REST-style path parameters. Can be expressions that will be evaluated. Comma delimited values are supported which can be more convenient, and takes care of URL-encoding and appending '/' between path segments as needed.
Note that the
path
'resets' after any HTTP request is made but not theurl
. The Hello World is a great example of 'REST-ful' use of theurl
when the test focuses on a single REST 'resource'. Look at how thepath
did not need to be specified for the second HTTPget
call since/cats
is part of theurl
.?myparam=value
by usingpath
the?
will get encoded into%3F
. Use either theparam
keyword, e.g.:* param myparam = 'value'
orurl
:* url 'http://example.com/v1?myparam'
Because Karate strips trailing slashes if part of a
path
parameter, if you want to append a forward-slash to the end of the URL in the final HTTP request - make sure that the lastpath
is a single '/'.request
In-line JSON:
In-line XML:
From a file in the same package. Use the
classpath:
prefix to load from the classpath instead.You could always use a variable:
In most cases you won’t need to set the
Content-Type
header
as Karate will automatically do the right thing depending on the data-type of therequest
.Defining the
request
is mandatory if you are using an HTTPmethod
that expects a body such aspost
. If you really need to have an empty body, you can use an empty string as shown below, and you can force the rightContent-Type
header by using theheader
keyword.Sending a file as the entire binary request body is easy (note that
multipart
is different):method
The HTTP verb -
get
,post
,put
,delete
,patch
,options
,head
,connect
,trace
.Lower-case is fine.
It is worth internalizing that during test-execution, it is upon the
method
keyword that the actual HTTP request is issued. Which suggests that the step should be in theWhen
form, for example:When method post
. And steps that follow should logically be in theThen
form. Also make sure that you complete the set up of things likeurl
,param
,header
,configure
etc. before you fire themethod
.Although rarely needed, variable references or expressions are also supported:
status
This is a shortcut to assert the HTTP response code.
And this assertion will cause the test to fail if the HTTP response code is something else.
See also
responseStatus
if you want to do some complex assertions against the HTTP status code.Keywords that set key-value pairs
They are
param
,header
,cookie
,form field
andmultipart field
.The syntax will include a '=' sign between the key and the value. The key should not be within quotes.
param
Setting query-string parameters:
The above would result in a URL like:
http://myhost/mypath?someKey=hello&anotherKey=foo
. Note that the?
and&
will be automatically inserted.Multi-value params are also supported:
For convenience, a
null
value will be ignored. You can also use JSON to set multiple query-parameters in one-line usingparams
and this is especially useful for dynamic data-driven testing.header
You can use functions or expressions:
It is worth repeating that in most cases you won’t need to set the
Content-Type
header as Karate will automatically do the right thing depending on the data-type of therequest
.Because of how easy it is to set HTTP headers, Karate does not provide any special keywords for things like the
Accept
header. You simply do something like this:A common need is to send the same header(s) for every request, and
configure headers
(with JSON) is how you can set this up once for all subsequent requests. And if you do this within aBackground:
section, it would apply to allScenario:
sections within the*.feature
file.Note that
Content-Type
had to be enclosed in quotes in the JSON above because the "-
" (hyphen character) would cause problems otherwise. Also note that "; charset=UTF-8
" would be appended to theContent-Type
header that Karate sends by default, and in some rare cases, you may need to suppress this behavior completely. You can do so by setting thecharset
to null via theconfigure
keyword:If you need headers to be dynamically generated for each HTTP request, use a JavaScript function with
configure headers
instead of JSON.Multi-value headers (though rarely used in the wild) are also supported:
Also look at the
headers
keyword which uses JSON and makes some kinds of dynamic data-driven testing easier.cookie
Setting a cookie:
You also have the option of setting multiple cookies in one-step using the
cookies
keyword.Note that any cookies returned in the HTTP response would be automatically set for any future requests. This mechanism works by calling
configure cookies
behind the scenes and if you need to stop auto-adding cookies for future requests, just do this:Also refer to the built-in variable
responseCookies
for how you can access and perform assertions on cookie data values.form field
HTML form fields would be URL-encoded when the HTTP request is submitted (by the
method
step). You would typically use these to simulate a user sign-in and then grab a security token from theresponse
.Note that the
Content-Type
header will be automatically set to:application/x-www-form-urlencoded
. You just need to do a normalPOST
(orGET
).For example:
A good example of the use of
form field
for a typical sign-in flow is this OAuth 2 demo:oauth2.feature
.Multi-values are supported the way you would expect (e.g. for simulating check-boxes and multi-selects):
You can also dynamically set multiple fields in one step using the
form fields
keyword.multipart field
Use this for building multipart named (form) field requests. This is typically combined with
multipart file
as shown below.multipart file
It is important to note that
myFile
above is the "field name" within themultipart/form-data
request payload. This roughly corresponds to acURL
argument of-F @myFile=test.pdf
.Also note that
multipart file
takes a JSON argument so that you can easily set thefilename
and thecontentType
(mime-type) in one step.read
: the name of a file, and theclasspath:
prefix also is allowed. mandatory unlessvalue
is used, see below.value
: alternative toread
in rare cases where something like a JSON or XML file is being uploaded and you want to create it dynamically.filename
: optional, if not specified there will be nofilename
attribute inContent-Disposition
contentType
: optional, will default toapplication/octet-stream
When 'multipart' content is involved, the
Content-Type
header of the HTTP request defaults tomultipart/form-data
. You can over-ride it by using theheader
keyword before themethod
step. Look atmultipart entity
for an example.Also refer to this demo example for a working example of multipart file uploads:
upload.feature
.You can also dynamically set multiple files in one step using
multipart files
.multipart entity
Use this for multipart content items that don’t have field-names. Here below is an example that also demonstrates using the
multipart/related
content-type.Multi-Param Keywords
Keywords that set multiple key-value pairs in one step
params
,headers
,cookies
,form fields
,multipart fields
andmultipart files
take a single JSON argument (which can be in-line or a variable reference), and this enables certain types of dynamic data-driven testing, especially because any JSON key with anull
value will be ignored. Here is a good example in the demos:dynamic-params.feature
params
See also
param
.headers
See also
header
.cookies
See also
cookie
.form fields
See also
form field
.multipart fields
See also
multipart field
.multipart files
The single JSON argument needs to be in the form
{ field1: { read: 'file1.ext' }, field2: { read: 'file2.ext' } }
where each nested JSON is in the form expected bymultipart file
For an example, refer:
upload-multiple-files.feature
.SOAP
Since a SOAP request needs special handling, this is the only case where the
method
step is not used to actually fire the request to the server.soap action
The name of the SOAP action specified is used as the 'SOAPAction' header. Here is an example which also demonstrates how you could assert for expected values in the response XML.
A working example of calling a SOAP service can be found within the Karate project test-suite. Refer to the demos for another example:
soap.feature
.More examples are available that showcase various ways of parameter-izing and dynamically manipulating SOAP requests in a data-driven fashion. Karate is quite flexible, and provides multiple options for you to evolve patterns that fit your environment, as you can see here:
xml.feature
.retry until
Karate has built-in support for re-trying an HTTP request until a certain condition has been met. The default setting for the max retry-attempts is 3 with a poll interval of 3000 milliseconds (3 seconds). If needed, this can be changed by using
configure
- any time during a test, or set globally viakarate-config.js
The
retry
keyword is designed to extend the existingmethod
syntax (and should appear before amethod
step) like so:Any JavaScript expression that uses any variable in scope can be placed after the "
retry until
" part. So you can refer to theresponse
,responseStatus
or evenresponseHeaders
if needed. For example:Refer to
polling.feature
for an example, and also see the alternative way to achieve polling.configure
Managing Headers, SSL, Timeouts and HTTP Proxy
You can adjust configuration settings for the HTTP client used by Karate using this keyword. The syntax is similar to
def
but instead of a named variable, you update configuration. Here are the configuration keys supported:headers
JSON / JS function
See
configure headers
cookies
JSON / JS function
Just like
configure headers
, but for cookies. You will typically never use this, as response cookies are auto-added to all future requests. If you need to clear cookies at any time, just doconfigure cookies = null
logPrettyRequest
boolean
Pretty print the request payload JSON or XML with indenting (default
false
)logPrettyResponse
boolean
Pretty print the response payload JSON or XML with indenting (default
false
)printEnabled
boolean
Can be used to suppress the
print
output when not in 'dev mode' by setting asfalse
(defaulttrue
)report
JSON / boolean
see report verbosity
afterScenario
JS function
Will be called after every
Scenario
(orExample
within aScenario Outline
), refer to this example:hooks.feature
afterFeature
JS function
Will be called after every
Feature
, refer to this example:hooks.feature
ssl
boolean
Enable HTTPS calls without needing to configure a trusted certificate or key-store.
ssl
string
Like above, but force the SSL algorithm to one of these values. (The above form internally defaults to
TLS
if simply set totrue
).ssl
JSON
see X509 certificate authentication
followRedirects
boolean
Whether the HTTP client automatically follows redirects - (default
true
), refer to this example.connectTimeout
integer
Set the connect timeout (milliseconds). The default is 30000 (30 seconds). Note that for
karate-apache
, this sets the socket timeout to the same value as well.readTimeout
integer
Set the read timeout (milliseconds). The default is 30000 (30 seconds).
proxy
string
Set the URI of the HTTP proxy to use.
proxy
JSON
For a proxy that requires authentication, set the
uri
,username
andpassword
, see example below. Also anonProxyHosts
key is supported which can take a list for e.g.{ uri: 'http://my.proxy.host:8080', nonProxyHosts: ['host1', 'host2']}
localAddress
string
see
karate-gatling
charset
string
The charset that will be sent in the request
Content-Type
which defaults toutf-8
. You typically never need to change this, and you can over-ride (or disable) this per-request if needed via theheader
keyword (example).retry
JSON
defaults to
{ count: 3, interval: 3000 }
- seeretry until
callSingleCache
JSON
defaults to
{ minutes: 0, dir: 'target' }
- seeconfigure callSingleCache
lowerCaseResponseHeaders
boolean
Converts every key in the
responseHeaders
to lower-case which makes it easier to validate or re-useabortedStepsShouldPass
boolean
defaults to
false
, whether steps after akarate.abort()
should be marked asPASSED
instead ofSKIPPED
- this can impact the behavior of 3rd-party reports, see this issue for detailslogModifier
Java Object
See Log Masking
responseHeaders
JSON / JS function
See
karate-netty
cors
boolean
See
karate-netty
driver
JSON
See UI Automation
driverTarget
JSON / Java Object
See
configure driverTarget
pauseIfNotPerf
boolean
defaults to
false
, relevant only for performance-testing, seekarate.pause()
andkarate-gatling
xmlNamespaceAware
boolean
defaults to
false
, to handle XML namespaces in some special circumstancesabortSuiteOnFailure
boolean
defaults to
false
, to not attempt to run any more tests upon a failureExamples:
configure
globallyIf you need to set any of these "globally" you can easily do so using the
karate
object inkarate-config.js
- for e.g:In rare cases where you need to add nested non-JSON data to the
configure
value, you have to play by the rules that apply withinkarate-config.js
. Here is an example of performing aconfigure driver
step in JavaScript:Report Verbosity
By default, Karate will add logs to the report output so that HTTP requests and responses appear in-line in the HTML reports. There may be cases where you want to suppress this to make the reports "lighter" and easier to read.
The configure key here is
report
and it takes a JSON value. For example:report
showLog
boolean
HTTP requests and responses (including headers) will appear in the HTML report, default
true
showAllSteps
boolean
If
false
, any step that starts with*
instead ofGiven
,When
,Then
etc. will not appear in the HTML report. Theprint
step is an exception. Defaulttrue
.You can 'reset' default settings by using the following short-cut:
Since you can use
configure
any time within a test, you have control over which requests or steps you want to show / hide. This can be convenient if a particular call results in a huge response payload.The following short-cut is also supported which will disable all logs:
@report=false
When you use a re-usable feature that has commonly used utilities, you may want to hide this completely from the HTML reports. The special tag
@report=false
can be used, and it can even be used only for a singleScenario
:Log Masking
In cases where you want to "mask" values which are sensitive from a security point of view from the output files, logs and HTML reports, you can implement the
HttpLogModifier
and tell Karate to use it via theconfigure
keyword. Here is an example of an implementation. For performance reasons, you can implementenableForUri()
so that this "activates" only for some URL patterns.Instantiating a Java class and using this in a test is easy (see example):
Or globally in
karate-config.js
Since
karate-config.js
is processed for everyScenario
, you can use a singleton instead of callingnew
every time. Something like this:System Properties for SSL and HTTP proxy
For HTTPS / SSL, you can also specify a custom certificate or trust store by setting Java system properties. And similarly - for specifying the HTTP proxy.
X509 Certificate Authentication
Also referred to as "mutual auth" - if your API requires that clients present an X509 certificate for authentication, Karate supports this via JSON as the
configure ssl
value. The following parameters are supported:keyStore
string
optional
path to file containing public and private keys for your client certificate.
keyStorePassword
string
optional
password for keyStore file.
keyStoreType
string
optional
Format of the keyStore file. Allowed keystore types are as described in the Java
KeyStore
docs.trustStore
string
optional
path to file containing the trust chain for your server certificate.
trustStorePassword
string
optional
password for trustStore file.
trustStoreType
string
optional
Format of the trustStore file. Allowed keystore types are as described in the Java
KeyStore
docs.trustAll
boolean
optional
if all server certificates should be considered trusted. Default value is
false
. Iftrue
will allow self-signed certificates. Iffalse
, will expect the whole chain in thetrustStore
or use what is available in the environment.algorithm
string
optional
force the SSL algorithm to one of these values. Default is
TLS
.Example:
For end-to-end examples in the Karate demos, look at the files in this folder.
Payload Assertions
Prepare, Mutate, Assert.
Now it should be clear how Karate makes it easy to express JSON or XML. If you read from a file, the advantage is that multiple scripts can re-use the same data.
Once you have a JSON or XML object, Karate provides multiple ways to manipulate, extract or transform data. And you can easily assert that the data is as expected by comparing it with another JSON or XML object.
match
Payload Assertions / Smart Comparison
The
match
operation is smart because white-space does not matter, and the order of keys (or data elements) does not matter. Karate is even able to ignore fields you choose - which is very useful when you want to handle server-side dynamically generated fields such as UUID-s, time-stamps, security-tokens and the like.The match syntax involves a double-equals sign '==' to represent a comparison (and not an assignment '=').
Since
match
andset
go well together, they are both introduced in the examples in the section below.set
Game,
set
andmatch
- Karate !JS for JSON
Before you consider the
set
keyword - note that for simple JSON update operations, you can useeval
- especially useful when the path you are trying to mutate is dynamic. Since theeval
keyword can be omitted when operating on variables using JavaScript, this leads to very concise code:Refer to
eval
for more / advanced examples.Manipulating Data
Setting values on JSON documents is simple using the
set
keyword.XML and XPath works just like you’d expect.
Refer to the section on XPath Functions for examples of advanced XPath usage.
match
and variablesIn case you were wondering, variables (and even expressions) are supported on the right-hand-side. So you can compare 2 JSON (or XML) payloads if you wanted to:
If you are wondering about the finer details of the
match
syntax, the Left-Hand-Side has to be either avariable name - e.g.
foo
a 'named' JsonPath or XPath expression - e.g.
foo[0].bar
orfoo[*].bar
note that this cannot be "dynamic" (with in-line variables) so use an extra step if needed
any valid function or method call - e.g.
foo.bar()
orfoo.bar('hello').baz
or anything wrapped in parentheses which will be evaluated as JavaScript - e.g.
(foo + bar)
or(42)
- and in this case, variables can be usedAnd the right-hand-side can be any valid Karate expression. Refer to the section on JsonPath short-cuts for a deeper understanding of 'named' JsonPath expressions in Karate.
match !=
(not equals)The 'not equals' operator
!=
works as you would expect:set
multipleKarate has an elegant way to set multiple keys (via path expressions) in one step. For convenience, non-existent keys (or array elements) will be created automatically. You can find more JSON examples here:
js-arrays.feature
.One extra convenience for JSON is that if the variable itself (which was
cat
in the above example) does not exist, it will be created automatically. You can even create (or modify existing) JSON arrays by using multiple columns.If you have to set a bunch of deeply nested keys, you can move the parent path to the top, next to the
set
keyword and save a lot of typing ! Note that this is not supported for "arrays" like above, and you can have only onevalue
column.The same concept applies to XML and you can build complicated payloads from scratch in just a few, extremely readable lines. The
value
column can take expressions, even XML chunks. You can find more examples here:xml.feature
.remove
This is like the opposite of
set
if you need to remove keys or data elements from JSON or XML instances. You can even remove JSON array elements by index.remove
works for XML elements as well:Also take a look at how a special case of embedded-expressions can remove key-value pairs from a JSON (or XML) payload: Remove if Null.
See also
delete
, below.delete
For JSON, you can also use the JS
delete
operator viaeval
, useful when the path you are trying to mutate is dynamic.As a convenience, you can omit the
eval
:Fuzzy Matching
Ignore or Validate
When expressing expected results (in JSON or XML) you can mark some fields to be ignored when the match (comparison) is performed. You can even use a regular-expression so that instead of checking for equality, Karate will just validate that the actual value conforms to the expected pattern.
This means that even when you have dynamic server-side generated values such as UUID-s and time-stamps appearing in the response, you can still assert that the full-payload matched in one step.
The supported markers are the following:
#ignore
Skip comparison for this field even if the data element or JSON key is present
#null
Expects actual value to be
null
, and the data element or JSON key must be present#notnull
Expects actual value to be not-
null
#present
Actual value can be any type or even
null
, but the key must be present (only for JSON / XML, see below)#notpresent
Expects the key to be not present at all (only for JSON / XML, see below)
#array
Expects actual value to be a JSON array
#object
Expects actual value to be a JSON object
#boolean
Expects actual value to be a boolean
true
orfalse
#number
Expects actual value to be a number
#string
Expects actual value to be a string
#uuid
Expects actual (string) value to conform to the UUID format
#regex STR
Expects actual (string) value to match the regular-expression 'STR' (see examples above)
#? EXPR
Expects the JavaScript expression 'EXPR' to evaluate to true, see self-validation expressions below
#[NUM] EXPR
Advanced array validation, see schema validation
#(EXPR)
For completeness, embedded expressions belong in this list as well
Note that
#present
and#notpresent
only make sense when you are matching within a JSON or XML context or using a JsonPath or XPath on the left-hand-side.The rest can also be used even in 'primitive' data matches like so:
Optional Fields
If two cross-hatch
symbols are used as the prefix (for example:
#number
), it means that the key is optional or that the value can be null.Remove If Null
A very useful behavior when you combine the optional marker with an embedded expression is as follows: if the embedded expression evaluates to
null
- the JSON key (or XML element or attribute) will be deleted from the payload (the equivalent ofremove
).If you are just trying to pre-define schema snippets to use in a fuzzy-match, you can use enclosed Javascript to suppress the default behavior of replacing placeholders. For example:
And if you need to suppress placeholder substitution for
read()
, but still need a JSON snippet, you can do this. Note how we read as a string, but "cast" to JSON:If you want to use the triple-quote / multi-line way of defining JSON or if you have to use XML - you can use
text
and "cast" to JSON or XML as a second step - before using in amatch
:#null
and#notpresent
Karate’s
match
is strict, and the case where a JSON key exists but has anull
value (#null
) is considered different from the case where the key is not present at all (#notpresent
) in the payload.But note that
null
can be used to represent a convention that many teams adopt, which is that keys withnull
values are stripped from the JSON payload. In other words,{ a: 1, b: null }
is considered 'equal' to{ a: 1 }
and{ a: 1, b: 'null' }
willmatch
both cases.These examples (all exact matches) can make things more clear:
Note that you can alternatively use JsonPath on the left-hand-side:
But of course it is preferable to match whole objects in one step as far as possible.
'Self' Validation Expressions
The special 'predicate' marker
? EXPR
in the table above is an interesting one. It is best explained via examples. Any valid JavaScript expression that evaluates to a Truthy or Falsy value is expected after the?
.Observe how the value of the field being validated (or 'self') is injected into the 'underscore' expression variable: '
_
'What is even more interesting is that expressions can refer to variables:
And functions work as well ! You can imagine how you could evolve a nice set of utilities that validate all your domain objects.
Especially since strings can be easily coerced to numbers (and vice-versa) in Javascript, you can combine built-in validators with the self-validation 'predicate' form like this:
'#number? _ > 0'
Referring to the JSON root
You can actually refer to any JsonPath on the document via
$
and perform cross-field or conditional validations ! This example usescontains
and the#?
'predicate' syntax, and situations where this comes in useful will be apparent when we discussmatch each
.match
text or binaryChecking if a string is contained within another string is a very common need and
match
(name)contains
works just like you’d expect:For case-insensitive string comparisons, see how to create custom utilities or
karate.lowerCase()
. And for dealing with binary content - seebytes
.match header
Since asserting against header values in the response is a common task -
match header
has a special meaning. It short-cuts to the pre-defined variableresponseHeaders
and reduces some complexity - because strictly, HTTP headers are a 'multi-valued map' or a 'map of lists' - the Java-speak equivalent beingMap<String, List<String>>
. And since header names are case-insensitive - it ignores the case when finding the header to match.Note the extra convenience where you don’t have to enclose the LHS key in quotes.
You can always directly access the variable called
responseHeaders
if you wanted to do more checks, but you typically won’t need to.match
and XMLAll the fuzzy matching markers will work in XML as well. Here are some examples:
Refer to this file for a comprehensive set of XML examples:
xml.feature
.Matching Sub-Sets of JSON Keys and Arrays
match contains
JSON Keys
In some cases where the response JSON is wildly dynamic, you may want to only check for the existence of some keys. And
match
(name)contains
is how you can do so:Note that
match contains
will not "recurse" any nested JSON chunks so usematch contains deep
instead.Also note that
match contains any
is possible for JSON objects as well as JSON arrays.(not)
!contains
It is sometimes useful to be able to check if a key-value-pair does not exist. This is possible by prefixing
contains
with a!
(with no space in between).Here’s a reminder that the
#notpresent
marker can be mixed into an equalitymatch
(==
) to assert that some keys exist and at the same time ensure that some keys do not exist:The
!
(not) operator is especially useful forcontains
and JSON arrays.JSON Arrays
This is a good time to deep-dive into JsonPath, which is perfect for slicing and dicing JSON into manageable chunks. It is worth taking a few minutes to go through the documentation and examples here: JsonPath Examples.
Here are some example assertions performed while scraping a list of child elements out of the JSON below. Observe how you can
match
the result of a JsonPath expression with your expected data.It is worth mentioning that to do the equivalent of the last line in Java, you would typically have to traverse 2 Java Objects, one of which is within a list, and you would have to check for nulls as well.
When you use Karate, all your data assertions can be done in pure JSON and without needing a thick forest of companion Java objects. And when you
read
your JSON objects from (re-usable) files, even complex response payload assertions can be accomplished in just a single line of Karate-script.Refer to this case study for how dramatic the reduction of lines of code can be.
match contains only
For those cases where you need to assert that all array elements are present but in any order you can do this:
match contains any
To assert that any of the given array elements are present.
And this happens to work as expected for JSON object keys as well:
match contains deep
This modifies the behavior of
match contains
so that nested lists or objects are processed for a "deep contains" match instead of a "deep equals" one which is the default. This is convenient for complex nested payloads where you are sure that you only want to check for some values in the various "trees" of data.Here is an example:
match contains only deep
This is exactly like
match ==
but the order of arrays does not matter. All arrays no matter the "depth" will be checked in this way.Validate every element in a JSON array
match each
The
match
keyword can be made to iterate over all elements in a JSON array using theeach
modifier. Here’s how it works:Here is a contrived example that uses
match each
,contains
and the#?
'predicate' marker to validate that the value oftotalPrice
is always equal to theroomPrice
of the first item in theroomInformation
array.Referring to self
While
$
always refers to the JSON 'root', note the use of_$
above to represent the 'current' node of amatch each
iteration. Here is a recap of symbols that can be used in JSON embedded expressions:$
The 'root' of the JSON document in scope
_
The value of 'self'
_$
The 'parent' of 'self' or 'current' item in the list, relevant when using
match each
There is a shortcut for
match each
explained in the next section that can be quite useful, especially for 'in-line' schema-like validations.match each contains deep
match each
can be combined withcontains deep
so that for each JSON object a “deep contains” match is performed within nested lists or objects.This is useful for testing payloads with JSON arrays whose members have a few essential keys that you wish to validate.
Schema Validation
Karate provides a far more simpler and more powerful way than JSON-schema to validate the structure of a given payload. You can even mix domain and conditional validations and perform all assertions in a single step.
But first, a special short-cut for array validation needs to be introduced:
This 'in-line' short-cut for validating JSON arrays is similar to how
match each
works. So now, complex payloads (that include arrays) can easily be validated in one step by combining validation markers like so:Especially note the re-use of the
oddSchema
both as an embedded-expression and as an array validation (on the last line).And you can perform conditional / cross-field validations and even business-logic validations at the same time.
Refer to this for the complete example:
schema-like.feature
And there is another example in the karate-demos:
schema.feature
where you can compare Karate’s approach with an actual JSON-schema example. You can also find a nice visual comparison and explanation here.contains
short-cutsEspecially when payloads are complex (or highly dynamic), it may be more practical to use
contains
semantics. Karate has the following short-cut symbols designed to be mixed intoembedded expressions
:^
contains
^^
contains only
^*
contains any
^+
contains deep
!^
not contains
Here’a table of the alternative 'in-line' forms compared with the 'standard' form. Note that all the short-cut forms on the right-side of the table resolve to 'equality' (
==
) matches, which enables them to be 'in-lined' into a full (single-step) payloadmatch
, using embedded expressions.A very useful capability is to be able to check that an array
contains
an object thatcontains
the provided sub-set of keys instead of having to specify the complete JSON - which can get really cumbersome for large objects. This turns out to be very useful in practice, and this particularmatch
jsonArray+contains '#(^+`partialObject
)'` form has no 'in-line' equivalent (see the third-from-last row above).In real-life tests, these are very useful when the order of items in arrays returned from the server are not guaranteed. You can easily assert that all expected elements are present, even in nested parts of your JSON - while doing a
match
on the full payload.There’s a lot going on in the last line above ! It validates the entire payload in one step and checks if the
kittens
array *contains all* theexpected
items but in *any order*.get
By now, it should be clear that link:(JsonPath can be very useful for extracting JSON 'trees' out of a given object. The
get
keyword allows you to save the results of a JsonPath expression for later use - which is especially useful for dynamic data-driven testing.get
short-cutThe 'short cut'
$variableName
form is also supported. Refer to JsonPath short-cuts for a detailed explanation. So the above could be re-written as follows:It is worth repeating that the above can be condensed into 2 lines. Note that since only JsonPath is expected on the left-hand-side of the
==
sign of amatch
statement, you don’t need to prefix the variable reference with$
:get
plus indexA convenience that the
get
syntax supports (but not the$
short-cut form) is to return a single element if the right-hand-side evaluates to a list-like result (e.g. a JSON array). This is useful because the moment you use a wildcard[*]
or search filter in JsonPath (see the next section), you get an array back - even though typically you would only be interested in the first item.JsonPath filters
JsonPath filter expressions are very useful for extracting elements that meet some filter criteria out of arrays.
You usually won’t need this, but the second-last line above shows how the
karate
object can be used to evaluate JsonPath if the filter expression depends on a variable. If you find yourself struggling to write dynamic JsonPath filters, look atkarate.filter()
as an alternative, described just below.JSON Transforms
Karate supports the following functional-style operations via the JS API -
karate.map()
,karate.filter()
andkarate.forEach()
. They can be very useful in some situations. A good example is when you have the expected data available as ready-made JSON but it is in a different "shape" from the actual data or HTTPresponse
. There is also akarate.mapWithKey()
for a common need - which is to convert an array of primitives into an array of objects, which is the form that data driven features expect.A few more useful "transforms" are to select a sub-set of key-value pairs using
karate.filterKeys()
, merging 2 or more JSON-s usingkarate.merge()
and combining 2 or more arrays (or objects) into a single array usingkarate.append()
. Andkarate.appendTo()
is for updating an existing variable (the equivalent ofarray.push()
in JavaScript), which is especially useful in the body of akarate.forEach()
.You can also sort arrays of arbitrary JSON using
karate.sort()
. Simple arrays of strings or numbers can be stripped of duplicates usingkarate.distinct()
. All JS "native" array operations can be used, such assomeName.reverse()
.Loops
Given the examples above, it has to be said that a best practice with Karate is to avoid JavaScript
for
loops as far as possible. A common requirement is to build an array withn
elements or do somethingn
times wheren
is an integer (that could even be a variable reference). This is easily achieved with thekarate.repeat()
API:And there’s also
karate.range()
which can be useful to generate test-data.Don’t forget that Karate’s data-driven testing capabilities can loop over arrays of JSON objects automatically.
XPath Functions
When handling XML, you sometimes need to call XPath functions, for example to get the count of a node-set. Any valid XPath expression is allowed on the left-hand-side of a
match
statement.Advanced XPath
Some XPath expressions return a list of nodes (instead of a single node). But since you can express a list of data-elements as a JSON array - even these XPath expressions can be used in
match
statements.If your XPath is dynamic and has to be formed 'on the fly' perhaps by using some variable derived from previous steps, you can use the
karate.xmlPath()
helper:You can refer to this file (which is part of the Karate test-suite) for more XML examples:
xml-and-xpath.feature
Special Variables
These are 'built-in' variables, there are only a few and all of them give you access to the HTTP response.
response
After every HTTP call this variable is set with the response body, and is available until the next HTTP request over-writes it. You can easily assign the whole
response
(or just parts of it using Json-Path or XPath) to a variable, and use it in later steps.The response is automatically available as a JSON, XML or String object depending on what the response contents are.
As a short-cut, when running JsonPath expressions -
$
represents theresponse
. This has the advantage that you can use pure JsonPath and be more concise. For example:And similarly for XML and XPath, '/' represents the
response
JsonPath short-cuts
The
$varName
form is used on the right-hand-side of Karate expressions and is slightly different from pure JsonPath expressions which always begin with$.
or$[
. Here is a summary of what the different 'shapes' mean in Karate:$.bar
Pure JsonPath equivalent of
$response.bar
whereresponse
is a JSON object$[0]
Pure JsonPath equivalent of
$response[0]
whereresponse
is a JSON array$foo.bar
Evaluates the JsonPath
$.bar
on the variablefoo
which is a JSON object or map-like$foo[0]
Evaluates the JsonPath
$[0]
on the variablefoo
which is a JSON array or list-likeresponseBytes
This will always hold the contents of the response as a byte-array. This is rarely used, unless you are expecting binary content returned by the server. The
match
keyword will work as you expect. Here is an example:binary.feature
.responseCookies
The
responseCookies
variable is set upon any HTTP response and is a map-like (or JSON-like) object. It can be easily inspected or used in expressions.As a convenience, cookies from the previous response are collected and passed as-is as part of the next HTTP request. This is what is normally expected and simulates a web-browser - which makes it easy to script things like HTML-form based authentication into test-flows. Refer to the documentation for
cookie
for details and how you can disable this if need be.Each item within
responseCookies
is itself a 'map-like' object. Typically you would examine thevalue
property as in the example above, butdomain
andpath
are also available.responseHeaders
See also
match header
which is what you would normally need.But if you need to use values in the response headers - they will be in a variable named
responseHeaders
. Note that it is a 'map of lists' so you will need to do things like this:And just as in the
responseCookies
example above, you can usematch
to run complex validations on theresponseHeaders
.Finally, using
karate.response.header(name)
can be simpler to just get a header value string by name, and it will ignore-case for the name passed as the argument:responseStatus
You would normally only need to use the
status
keyword. But if you really need to use the HTTP response code in an expression or save it for later, you can get it as an integer:Note that
match
can give you some extra readable options:responseTime
The response time (in milliseconds) for the current
response
would be available in a variable calledresponseTime
. You can use this to assert that it was returned within the expected time like so:responseType
Karate will attempt to parse the raw HTTP response body as JSON or XML and make it available as the
response
value. If parsing fails, Karate will log a warning and the value ofresponse
will then be a plain string. You can still perform string comparisons such as amatch contains
and look for error messages etc. In rare cases, you may want to check what the "type" of theresponse
is and it can be one of 3 different values:json
,xml
andstring
.So if you really wanted to assert that the HTTP response body is well-formed JSON or XML you can do this:
requestTimeStamp
Very rarely used - but you can get the Java system-time (for the current
response
) at the point when the HTTP request was initiated (the value ofSystem.currentTimeMillis()
) which can be used for detailed logging or custom framework / stats calculations.HTTP Header Manipulation
configure headers
Custom header manipulation for every HTTP request is something that Karate makes very easy and pluggable. For every HTTP request made from Karate, the internal flow is as follows:
did we
configure
the value ofheaders
?if so, is the configured value a JavaScript function ?
if so, a
call
is made to that function.did the function invocation return a map-like (or JSON) object ?
all the key-value pairs are added to the HTTP headers.
or is the configured value a JSON object ?
all the key-value pairs are added to the HTTP headers.
This makes setting up of complex authentication schemes for your test-flows really easy. It typically ends up being a one-liner that appears in the
Background
section at the start of your test-scripts. You can re-use the function you create across your whole project.Here is an example JavaScript function that uses some variables in the context (which have been possibly set as the result of a sign-in) to build the
Authorization
header. Note how even calls to Java code can be made if needed.Assuming the above code is in a file called
my-headers.js
, the next section on calling other feature files shows how it looks like in action at the beginning of a test script.Notice how once the
authToken
variable is initialized, it is used by the above function to generate headers for every HTTP call made as part of the test flow.If a few steps in your flow need to temporarily change (or completely bypass) the currently-set header-manipulation scheme, just update
configure headers
to a new value (or set it tonull
) in the middle of a script. Then use theheader
keyword to do a custom 'over-ride' if needed.The karate-demo has an example showing various ways to
configure
or set headers:headers.feature
The
karate
objectA JavaScript function or Karate expression at runtime has access to a utility object in a variable named:
karate
. This provides the following methods:karate.abort()
you can prematurely exit a
Scenario
by combining this with conditional logic like so:* if (condition) karate.abort()
- please use sparingly ! and also seeconfigure abortedStepsShouldPass
karate.append(... items)
useful to create lists out of items (which can be lists as well), see JSON transforms
karate.appendTo(name, ... items)
useful to append to a list-like variable (that has to exist) in scope, see JSON transforms - the first argument can be a reference to an array-like variable or even the name (string) of an existing variable which is list-like
karate.compareImage(baseline, latest, [options])
compare two images the same way that
compareImage
works (with an optionaloptions
argument), returns an object with the following keys:baseline
,latest
,mismatchPercentage
,engine
,failureThreshold
and optionally:error
,isBaselineMissing
,isScaleMismatch
,isMismatch
,resembleMismatchPercentage
,ssimMismatchPercentage
karate.call(fileName, [arg])
invoke a
*.feature
file or a JavaScript function the same way thatcall
works (with an optional solitary argument), seecall()
vsread()
for detailskarate.callSingle(fileName, [arg])
like the above, but guaranteed to run only once even across multiple features - see
karate.callSingle()
karate.configure(key, value)
does the same thing as the
configure
keyword, and a very useful example is to dokarate.configure('connectTimeout', 5000);
inkarate-config.js
- which has the 'global' effect of not wasting time if a connection cannot be established within 5 secondskarate.distinct(list)
returns only unique items out of an array of strings or numbers
karate.doc(arg)
just like
karate.render()
but will insert the HTML into the reportkarate.embed(object, mimeType)
embeds the object (can be raw bytes or an image) into the JSON report output, see this example
karate.env
gets the value (read-only) of the environment property 'karate.env', and this is typically used for bootstrapping configuration
karate.eval(expression)
for really advanced needs, you can programmatically generate a snippet of JavaScript which can be evaluated at run-time, you can find an example here
karate.exec(command)
convenient way to execute an OS specific command and return the console output e.g.
karate.exec('some.exe -h')
(orkarate.exec(['some.exe', '-h'])
) useful for calling non-Java code (that can even return data) or for starting user-interfaces to be automated, this command will block until the process terminates, also seekarate.fork()
karate.extract(text, regex, group)
useful to "scrape" text out of non-JSON or non-XML text sources such as HTML,
group
follows the Java regex rules, see this examplekarate.extractAll(text, regex, group)
like the above, but returns a list of text-matches
karate.fail(message)
if you want to conditionally stop a test with a descriptive error message, e.g.
* if (condition) karate.fail('we expected something else')
karate.feature
get metadata about the currently executing feature within a test
karate.filter(list, predicate)
functional-style 'filter' operation useful to filter list-like objects (e.g. JSON arrays), see example, the second argument has to be a JS function (item, [index]) that returns a
boolean
karate.filterKeys(map, keys)
extracts a sub-set of key-value pairs from the first argument, the second argument can be a list (or varargs) of keys - or even another JSON where only the keys would be used for extraction, example
karate.forEach(list, function)
functional-style 'loop' operation useful to traverse list-like (or even map-like) objects (e.g. JSON / arrays), see example, the second argument has to be a JS function (item, [index]) for lists and (key, [value], [index]) for JSON / maps
karate.fork(map)
executes an OS command, but forks a process in parallel and will not block the test like
karate.exec()
e.g.karate.fork({ args: ['some.exe', '-h'] })
orkarate.fork(['some.exe', '-h'])
- you can use a composite string asline
(or the solitary argument e.g.karate.fork('some.exe -h')
) instead ofargs
, and an optionalworkingDir
string property andenv
JSON / map is also supported - this returns aCommand
object which has operations such aswaitSync()
andclose()
if you need more control, more details herekarate.fromString(string)
for advanced conditional logic for e.g. when a string coming from an external process is dynamic - and whether it is JSON or XML is not known in advance, see example
karate.get(name, [default])
get the value of a variable by name (or JsonPath expression), if not found - this returns
null
which is easier to handle in JavaScript (thanundefined
), and an optional (literal / constant) second argument can be used to return a "default" value, very useful to set variables in called features that have not been pre-definedkarate.http(url)
returns a convenience
Http
request builder class, only recommended for advanced usekarate.jsonPath(json, expression)
brings the power of JsonPath into JavaScript, and you can find an example here.
karate.keysOf(object)
returns only the keys of a map-like object
karate.log(... args)
log to the same logger (and log file) being used by the parent process, logging can be suppressed with
configure printEnabled
set tofalse
, and just likeprint
- use comma-separated values to "pretty print" JSON or XMLkarate.logger.debug(... args)
access to the Karate logger directly and log in debug. Might be desirable instead of
karate.log
orprint
when looking to reduce the logs in console in your CI/CD pipeline but still retain the information for reports. See Logging for additional details.karate.lowerCase(object)
useful to brute-force all keys and values in a JSON or XML payload to lower-case, useful in some cases, see example
karate.map(list, function)
functional-style 'map' operation useful to transform list-like objects (e.g. JSON arrays), see example, the second argument has to be a JS function (item, [index])
karate.mapWithKey(list, string)
convenient for the common case of transforming an array of primitives into an array of objects, see JSON transforms
karate.match(actual, expected)
brings the power of the fuzzy
match
syntax into Karate-JS, returns a JSON in the form{ pass: '#boolean', message: '#string' }
and you can find an example here - you can even place a full match expression like this:karate.match("each foo contains { a: '#number' }")
karate.merge(... maps)
useful to merge the key-values of two (or more) JSON (or map-like) objects, see JSON transforms
karate.os
returns the operating system details as JSON, for e.g.
{ type: 'macosx', name: 'Mac OS X' }
- useful for writing conditional logic, the possibletype
-s being:macosx
,windows
,linux
andunknown
karate.pause(number)
sleep time in milliseconds, relevant only for performance-testing - and will be a no-op otherwise unless
configure pauseIfNotPerf
istrue
karate.pretty(value)
return a 'pretty-printed', nicely indented string representation of the JSON value, also see:
print
karate.prettyXml(value)
return a 'pretty-printed', nicely indented string representation of the XML value, also see:
print
karate.prevRequest
for advanced users, you can inspect the actual HTTP request after it happens, useful if you are writing a framework over Karate, refer to this example:
request.feature
karate.properties[key]
get the value of any Java system-property by name, useful for advanced custom configuration
karate.range(start, end, [interval])
returns a JSON array of integers (inclusive), the optional third argument must be a positive integer and defaults to 1, and if start < end the order of values is reversed
karate.read(filename)
the same
read()
function - which is pre-defined even within JS blocks, so there is no need to ever dokarate.read()
, and justread()
is sufficientkarate.readAsBytes(filename)
rarely used, like
karate.readAsString
- but returns a byte arraykarate.readAsStream(filename)
rarely used, like
karate.readAsString
- but returns a JavaInputStream
karate.readAsString(filename)
rarely used, behaves exactly like
read
- but does not auto convert to JSON or XMLkarate.remove(name, path)
very rarely used - when needing to perform conditional removal of JSON keys or XML nodes. Behaves the same way as the
remove
keyword.karate.render(arg)
renders an HTML template, the
arg
can be a string (prefixable path to the HTML) or a JSON that takes either apath
orhtml
property, seedoc
karate.repeat(count, function)
useful for building an array with
count
items or doing somethingcount
times, refer this example. Also see loops.karate.response
returns the last HTTP response as a JS object that enables advanced use-cases such as getting a header ignoring case:
karate.response.header('some-header')
karate.request
returns the last HTTP request as a JS object that enables advanced use-cases such as getting a header ignoring case:
karate.request.header('some-header')
, which works even in mockskarate.scenario
get metadata about the currently executing
Scenario
(orOutline
-Example
) within a testkarate.set(name, value)
sets the value of a variable (immediately), which may be needed in case any other routines (such as the configured headers) depend on that variable
karate.set(object)
where the single argument is expected to be a
Map
or JSON-like, and will perform the abovekarate.set()
operation for all key-value pairs in one-shot, see examplekarate.set(name, path, value)
only needed when you need to conditionally build payload elements, especially XML. This is best explained via an example, and it behaves the same way as the
set
keyword. Also seeeval
.karate.setXml(name, xmlString)
rarely used, refer to the example above
karate.setup([name])
call a
Scenario
tagged with the built-in@setup
annotationkarate.setupOnce([name])
like
karate.setup()
above, but cache the result so that the "setup" runs only oncekarate.signal(result)
trigger an event that
karate.listen(timeout)
is waiting for, and pass the data, see asynckarate.sizeOf(object)
returns the size of the map-like or list-like object
karate.sort(list, function)
sorts the list using the provided custom function called for each item in the list (and the optional second argument is the item index) e.g.
karate.sort(myList, x => x.val)
, and the second / function argument is not needed if the list is of plain strings or numberskarate.start()
only for starting a mock from within a test / feature file see mocks
karate.stop(port)
will pause the test execution until a socket connection (even HTTP
GET
) is made to the port logged to the console, useful for troubleshooting UI tests without using a de-bugger, of course - NEVER forget to remove this after use !karate.target(object)
currently for web-ui automation only, see target lifecycle
karate.tags
for advanced users - scripts can introspect the tags that apply to the current scope, refer to this example:
tags.feature
karate.tagValues
for even more advanced users - Karate natively supports tags in a
@name=val1,val2
format, and there is an inheritance mechanism whereScenario
level tags can over-rideFeature
level tags, refer to this example:tags.feature
karate.toAbsolutePath(relativePath)
when you want to get the absolute OS path to the argument which could even have a prefix such as
classpath:
, e.g.karate.toAbsolutePath('some.json')
karate.toBean(json, className)
converts a JSON string or map-like object into a Java object, given the Java class name as the second argument, refer to this file for an example
karate.toCsv(list)
converts a JSON array (of objects) or a list-like object into a CSV string, writing this to a file is your responsibility or you could use
karate.write()
karate.toJava(function)
rarely used, when you need to pass a JS function to custom Java code, typically for Async, and another edge case is to convert a JSON array or object to a Java
List
orMap
, see examplekarate.toJavaFile(path)
in case you need a
java.io.File
instance to pass to Java interop, takes the Karate prefixes such asclasspath:
for conveniencekarate.toJson(object)
converts a Java object into JSON, and
karate.toJson(object, true)
will strip all keys that havenull
values from the resulting JSON, convenient for unit-testing Java code, see examplekarate.typeOf(any)
for advanced conditional logic when object types are dynamic and not known in advance, see example
karate.urlDecode(string)
URL decode
karate.urlEncode(string)
URL encode
karate.valuesOf(object)
returns only the values of a map-like object (or itself if a list-like object)
karate.waitForHttp(url)
will wait until the URL is ready to accept HTTP connections
karate.waitForPort(host, port)
will wait until the host:port is ready to accept socket connections
karate.webSocket(url, handler)
see websocket
karate.write(object, path)
normally not recommended, please read this first - writes the bytes of
object
to a path which will always be relative to the "build" directory (typicallytarget
), see this example:embed-pdf.js
- and this method returns ajava.io.File
reference to the file created / written tokarate.xmlPath(xml, expression)
Just like
karate.jsonPath()
- but for XML, and allows you to use dynamic XPath if needed, see example.Code Reuse / Common Routines
call
In any complex testing endeavor, you would find yourself needing 'common' code that needs to be re-used across multiple test scripts. A typical need would be to perform a 'sign in', or create a fresh user as a pre-requisite for the scenarios being tested.
There are two types of code that can be
call
-ed.*.feature
files and JavaScript functions.Calling other
*.feature
filesWhen you have a sequence of HTTP calls that need to be repeated for multiple test scripts, Karate allows you to treat a
.feature
file as a re-usable unit. You can also pass parameters into the.feature
file being called, and extract variables out of the invocation result.Here is an example of using the
call
keyword to invoke another feature file, loaded using theread
function:The contents of
my-signin.feature
are shown below. A few points to note:Karate creates a new 'context' for the feature file being invoked but passes along all variables and configuration. This means that all your config variables and
configure
settings would be available to use, for exampleloginUrlBase
in the example below.When you use
def
in the 'called' feature, it will not over-write variables in the 'calling' feature (unless you explicitly choose to use shared scope). But note that JSON, XML, Map-like or List-like variables are 'passed by reference' which means that 'called' feature steps can update or 'mutate' them using theset
keyword. Use thecopy
keyword to 'clone' a JSON or XML payload if needed, and refer to this example for more details:copy.feature
.You can add (or over-ride) variables by passing a call 'argument' as shown above. Only one JSON argument is allowed, but this does not limit you in any way as you can use any complex JSON structure. You can even initialize the JSON in a separate step and pass it by name, especially if it is complex. Observe how using JSON for parameter-passing makes things super-readable. In the 'called' feature, the argument can also be accessed using the built-in variable:
__arg
.All variables that were defined (using
def
) in the 'called' script would be returned as 'keys' within a JSON-like object. Note that this includes 'built-in' variables, which means that things like the last value ofresponse
would also be present. In the example above you can see that the JSON 'envelope' returned - is assigned to the variable namedsignIn
. And then getting hold of any data that was generated by the 'called' script is as simple as accessing it by name, for examplesignIn.authToken
as shown above. This design has the following advantages:'called' Karate scripts don’t need to use any special keywords to 'return' data and can behave like 'normal' Karate tests in 'stand-alone' mode if needed
the data 'return' mechanism is 'safe', there is no danger of the 'called' script over-writing any variables in the 'calling' (or parent) script (unless you use shared scope)
the need to explicitly 'unpack' variables by name from the returned 'envelope' keeps things readable and maintainable in the 'caller' script
The above example actually makes two HTTP requests - the first is a standard 'sign-in' POST and then (for illustrative purposes) another HTTP call (a GET) is made for retrieving a list of projects for the signed-in user, and the first one is 'selected' and added to the returned 'auth token' JSON object.
So you get the picture, any kind of complicated 'sign-in' flow can be scripted and re-used.
Do look at the documentation and example for
configure headers
also as it goes hand-in-hand withcall
. In the above example, the end-result of thecall
tomy-signin.feature
resulted in theauthToken
variable being initialized. Take a look at how theconfigure headers
example uses theauthToken
variable.Call Tag Selector
You can "select" a single
Scenario
(orScenario
-s orScenario Outline
-s or even specificExamples
rows) by appending a "tag selector" at the end of the feature-file you are calling. For example:While the tag does not need to be in the
@key=value
form, it is recommended for readability when you start getting into the business of giving meaningful names to yourScenario
-s.This "tag selection" capability is designed for you to be able to "compose" flows out of existing test-suites when using the Karate Gatling integration. Normally we recommend that you keep your "re-usable" features lightweight - by limiting them to just one
Scenario
.Call Same Feature
As a convenience, you can call a tag directly, which is a short-cut to call another
Scenario
within the same feature file. Note that you would typically want to use the@ignore
tag for such cases.Data-Driven Features
If the argument passed to the call of a
*.feature
file is a JSON array, something interesting happens. The feature is invoked for each item in the array. Each array element is expected to be a JSON object, and for each object - the behavior will be as described above.But this time, the return value from the
call
step will be a JSON array of the same size as the input array. And each element of the returned array will be the 'envelope' of variables that resulted from each iteration where the*.feature
got invoked.Here is an example that combines the
table
keyword with calling a*.feature
. Observe how theget
shortcut is used to 'distill' the result array of variable 'envelopes' into an array consisting only ofresponse
payloads.And here is how
cat-create.feature
could look like:If you replace the
table
with perhaps a JavaScript function call that gets some JSON data from some data-source, you can imagine how you could go about dynamic data-driven testing.Although it is just a few lines of code, take time to study the above example carefully. It is a great example of how to effectively use the unique combination of Cucumber and JsonPath that Karate provides.
Also look at the demo examples, especially
dynamic-params.feature
- to compare the above approach with how the CucumberScenario Outline:
can be alternatively used for data-driven tests.Built-in variables for
call
Although all properties in the passed JSON-like argument are 'unpacked' into the current scope as separate 'named' variables, it sometimes makes sense to access the whole argument and this can be done via
arg
. And if being called in a loop, a built-in variable calledloop
will also be available that will hold the value of the current loop index. So you can do things like this:* def name = name + __loop
- or you can use the loop index value for looking up other values that may be in scope - in a data-driven style.__arg
the single
call
(orcallonce
) argument, will benull
if there was none__loop
the current iteration index (starts from 0) if being called in a loop, will be
-1
if notRefer to this demo feature for an example:
kitten-create.feature
Default Values
Some users need "callable" features that are re-usable even when variables have not been defined by the calling feature. Normally an undefined variable results in nasty JavaScript errors. But there is an elegant way you can specify a default value using the
karate.get()
API:copy
For a
call
(orcallonce
) - payload / data structures (JSON, XML, Map-like or List-like) variables are 'passed by reference' which means that steps within the 'called' feature can update or 'mutate' them, for e.g. using theset
keyword. This is actually the intent most of the time and is convenient. If you want to pass a 'clone' to a 'called' feature, you can do so using the rarely usedcopy
keyword that works very similar to type conversion. This is best explained in this example:copy.feature
.Calling JavaScript Functions
Examples of defining and using JavaScript functions appear in earlier sections of this document. Being able to define and re-use JavaScript functions is a powerful capability of Karate. For example, you can:
call re-usable functions that take complex data as an argument and return complex data that can be stored in a variable
call and interoperate with Java code if needed
share and re-use test utilities or 'helper' functionality across your organization
In real-life scripts, you would typically also use this capability of Karate to
configure headers
where the specified JavaScript function uses the variables that result from a sign in to manipulate headers for all subsequent HTTP requests. And it is worth mentioning that the Karate configuration 'bootstrap' routine is itself a JavaScript function.JS function argument rules for
call
When using
call
(orcallonce
), only one argument is allowed. But this does not limit you in any way, because similar to how you can call*.feature files
, you can pass a whole JSON object as the argument. In the case of thecall
of a JavaScript function, you can also pass a JSON array or a primitive (string, number, boolean) as the solitary argument, and the function implementation is expected to handle whatever is passed.Instead of using
call
(orcallonce
) you are always free to call JavaScript functions 'normally' and then you can use more than one argument.Return types
Naturally, only one value can be returned. But again, you can return a JSON object. There are two things that can happen to the returned value.
Either - it can be assigned to a variable like so.
Or - if a
call
is made without an assignment, and if the function returns a map-like object, it will add each key-value pair returned as a new variable into the execution context.While this sounds dangerous and should be used with care (and limits readability), the reason this feature exists is to quickly set (or over-write) a bunch of config variables when needed. In fact, this is the mechanism used when
karate-config.js
is processed on start-up.Shared Scope
This behavior where all key-value pairs in the returned map-like object get automatically added as variables - applies to the calling of
*.feature
files as well. In other words, whencall
orcallonce
is used without adef
, the 'called' script not only shares all variables (andconfigure
settings) but can update the shared execution context. This is very useful to boil-down those 'common' steps that you may have to perform at the start of multiple test-scripts - into one-liners. But use wisely, because called scripts will now over-write variables that may have been already defined.You can use
callonce
instead ofcall
within theBackground
in case you have multipleScenario
sections orExamples
. Note the 'inline' use of the read function as a short-cut above. This applies to JS functions as well:These heavily commented demo examples can help you understand 'shared scope' better, and are designed to get you started with creating re-usable 'sign-in' or authentication flows:
Isolated
call-isolated-headers.feature
common-multiple.feature
Shared
call-updates-config.feature
common.feature
call
vsread()
Since this is a frequently asked question, the different ways of being able to re-use code (or data) are summarized below.
* def login = read('login.feature')
* call login
Shared Scope, and the
login
variable can be re-used* call read('login.feature')
short-cut for the above
without needing a variable
* def credentials = read('credentials.json')
* def login = read('login.feature')
* call login credentials
Note how using
read()
for a JSON file returns data -
not "callable" code, and here it is
used as the
call
argument* call read('login.feature') read('credentials.json')
You can do this in theory,
but it is not as readable as the above
* karate.call('login.feature')
The JS API allows you to do this,
but this will not be Shared Scope
* def result = call read('login.feature')
call
result assigned to a variableand not Shared Scope
* def result = karate.call('login.feature')
exactly equivalent to the above !
* if (cond) karate.call(true, 'login.feature')
if you need conditional logic
and Shared Scope, add a
boolean
true
first argument* def credentials = read('credentials.json')
* def result = call read('login.feature') credentials
like the above,
but with a
call
argument* def credentials = read('credentials.json')
* def result = karate.call('login.feature', credentials)
like the above, but in JS API form,
the advantage of the above form is
that using an in-line argument is less
"cluttered" (see next row)
* def login = read('login.feature')
* def result = call login { user: 'john', password: 'secret' }
using the
call
keyword makespassing an in-line JSON argument
more "readable"
* call read 'credentials.json'
Since "
read
" happens to be afunction (that takes a single
string argument), this has the effect
of loading all keys in the JSON file
into Shared Scope as variables !
This can be sometimes handy.
* call read ('credentials.json')
A common mistake. First, there
is no meaning in
call
for JSON.Second, the space after the "
read
"makes this equal to the above.
* karate.set(read('credentials.json'))
For completeness - this has exactly the same effect as the above two rows !
Calling Java
There are examples of calling JVM classes in the section on Java Interop and in the file-upload demo. Also look at the section on commonly needed utilities for more ideas.
Calling any Java code is that easy. Given this custom, user-defined Java class:
This is how it can be called from a test-script via JavaScript, and yes, even static methods can be invoked:
Note that JSON gets auto-converted to
Map
(orList
) when making the cross-over to Java. Refer to thecats-java.feature
demo for an example.Another example is
dogs.feature
- which actually makes JDBC (database) calls, and since the data returned from the Java code is JSON, the last section of the test is able to usematch
very effectively for data assertions.A great example of how you can extend Karate, even bypass the HTTP client but still use Karate’s test-automation effectively, is this gRPC example by @thinkerou:
karate-grpc
. And you can even handle asynchronous flows such as listening to message-queues.HTTP Basic Authentication Example
This should make it clear why Karate does not provide 'out of the box' support for any particular HTTP authentication scheme. Things are designed so that you can plug-in what you need, without needing to compile Java code. You get to choose how to manage your environment-specific configuration values such as user-names and passwords.
First the JavaScript file,
basic-auth.js
:And here’s how it works in a test-script using the
header
keyword.You can set this up for all subsequent requests or dynamically generate headers for each HTTP request if you
configure headers
.callonce
Cucumber has a limitation where
Background
steps are re-run for everyScenario
. And if you have aScenario Outline
, this happens for every row in theExamples
. This is a problem especially for expensive, time-consuming HTTP calls, and this has been an open issue for a long time.Karate’s
callonce
keyword behaves exactly likecall
but is guaranteed to execute only once. The results of the first call are cached, and any future calls will simply return the cached result instead of executing the JavaScript function (or feature) again and again.This does require you to move 'set-up' into a separate
*.feature
(or JavaScript) file. But this totally makes sense for things not part of the 'main' test flow and which typically need to be re-usable anyway.So when you use the combination of
callonce
in aBackground
, you can indeed get the same effect as using a@BeforeClass
annotation, and you can find examples in the karate-demo, such as this one:callonce.feature
.A
callonce
is ideally used for only "pure" JSON. You may face issues if you attempt to mix in JS functions or Java code. Seekarate.callSingle()
.eval
There are a few situations where this comes in handy:
you really don’t need to assign a result to a variable
statements in the
if
form (also see conditional logic)'one-off' logic (or Java interop) where you don’t need the 'ceremony' of a re-usable function
JavaScript / JSON-style mutation of existing variables as a dynamic alternative to
set
andremove
- by usingkarate.set()
andkarate.remove()
.As a convenience, you can omit the
eval
keyword and so you can shorten the above to:This is very convenient especially if you are calling a method on a variable that has been defined such as the
karate
object, and for general-purpose scripting needs such as UI automation. Note howkarate.set()
andkarate.remove()
below are used directly as a script "statement".Advanced / Tricks
Polling
The built-in
retry until
syntax should suffice for most needs, but if you have some specific needs, this demo example (using JavaScript) should get you up and running:polling.feature
.Conditional Logic
The keywords
Given
When
Then
are only for decoration and should not be thought of as similar to anif - then - else
statement. And as a testing framework, Karate discourages tests that give different results on every run.That said, if you really need to implement 'conditional' checks, this can be one pattern:
And this is another, using
karate.call()
. Here we want tocall
a file only if a condition is satisfied:Or if we don’t care about the result, we can
eval
anif
statement:And this may give you more ideas. You can always use a JavaScript function or call Java for more complex logic.
JSON Lookup
You can always use a JavaScript
switch case
within aneval
or function block. But one pattern that you should be aware of is that JSON is actually a great data-structure for looking up data.You can find more details here. Also note how you can wrap the LHS of the
match
in parentheses in the rare cases where the parser expects JsonPath by default.Abort and Fail
In some rare cases you need to exit a
Scenario
based on some condition. You can usekarate.abort()
like so:Using
karate.abort()
will not fail the test. Conditionally making a test fail is easy withkarate.fail()
But normally a
match
statement is preferred unless you want a really descriptive error message.Also refer to polling for more ideas.
Commonly Needed Utilities
Since it is so easy to dive into Java-interop, Karate does not include any random-number functions, uuid generator or date / time utilities out of the box. You simply roll your own.
Here is an example of how to get the current date, and formatted the way you want:
And the above will result in something like this being logged:
[print] 2017/10/16
.Here below are a few more common examples:
System Time (as a string)
function(){ return java.lang.System.currentTimeMillis() + '' }
UUID
function(){ return java.util.UUID.randomUUID() + '' }
Random Number (
0
tomax-1
)function(max){ return Math.floor(Math.random() * max) }
Case Insensitive Comparison
function(a, b){ return a.equalsIgnoreCase(b) }
Sleep or Wait for
pause
millisecondsfunction(pause){ java.lang.Thread.sleep(pause) }
The first three are good enough for random string generation for most situations. Note that if you need to do a lot of case-insensitive string checks,
karate.lowerCase()
is what you are looking for.Multiple Functions in One File
If you find yourself needing a complex helper or utility function, we strongly recommend that you use Java because it is much easier to maintain and even debug if needed. And if you need multiple functions, you can easily organize them into a single Java class with multiple static methods.
That said, if you want to stick to JavaScript, but find yourself accumulating a lot of helper functions that you need to use in multiple feature files, the following pattern is recommended.
You can organize multiple "common" utilities into a single re-usable feature file as follows e.g.
common.feature
And then you have two options. The first option using shared scope should be fine for most projects, but if you want to "name space" your functions, use "isolated scope":
You can even move commonly used routines into
karate-config.js
which means that they become "global". But we recommend that you do this only if you are sure that these routines are needed in almost all*.feature
files. Bloating your configuration can lead to loss of performance, and maintainability may suffer.Async
The JS API has a
karate.signal(result)
method that is useful for involving asynchronous flows into a test.listen
You use the
listen
keyword (with a timeout) to wait until that event occurs. ThelistenResult
magic variable will hold the value passed to the call tokarate.signal()
.This is best explained in this example that involves listening to an ActiveMQ / JMS queue.
Note how JS functions defined at run-time can be mixed with custom Java code to get things done. You need to use
karate.toJava()
to "wrap" JS functions passed to custom Java code.WebSocket
Karate also has built-in support for websocket that is based on the async capability and the
listen
keyword. The following method signatures are available on thekarate
JS object to obtain a websocket client:karate.webSocket(url)
karate.webSocket(url, handler)
karate.webSocket(url, handler, options)
- whereoptions
is an optional JSON (or map-like) object that takes the following optional keys:subProtocol
- in case the server expects itheaders
- another JSON of key-value pairsmaxPayloadSize
- this defaults to 4194304 (bytes, around 4 MB)These will init a websocket client for the given
url
and optionalsubProtocol
. You can callsend()
on the returned object to send a message.If a
handler
function (returning a boolean) is provided - it will be used to complete thelisten
"wait" iftrue
is returned. A handler function is needed only if you have to ignore some incoming traffic and stop the "wait" when a certain payload arrives. If you don’t pass ahandler
(or it isnull
), the first message is returned.Note that
karate.signal()
(described as part of thelisten
keyword) will be called internally and thelistenResult
will be the payload contents of the "selected" message.Here is an example, where the same websocket connection is used to send as well as receive a message.
For handling binary messages, the same
karate.webSocket()
method signatures exist forkarate.webSocketBinary()
. Refer to these examples for more:echo.feature
websocket.feature
. Note that any websocket instances created will be auto-closed at the end of theScenario
.Java Function References
JavaScript functions have some limitations when combined with multi-threaded Java code. So it is recommended that you directly use a Java
Function
when possible instead of using thekarate.toJava()
"wrapper" as shown above.One pattern you can adopt is to create a "factory" method that returns a Java function - where you can easily delegate to the logic you want. For example, see the
sayHelloFactory()
method below:And now, to get a reference to that "function" you can do this:
This can be convenient when using shared scope because you can just call
sayHello('myname')
where needed.Tags
Gherkin has a great way to sprinkle meta-data into test-scripts - which gives you some interesting options when running tests in bulk. The most common use-case would be to partition your tests into 'smoke', 'regression' and the like - which enables being able to selectively execute a sub-set of tests.
The documentation on how to run tests via the command line has an example of how to use tags to decide which tests to not run (or ignore). Also see
first.feature
andsecond.feature
in the demos. If you find yourself juggling multiple tags with logicalAND
andOR
complexity, refer to this Stack Overflow answer.Special Tags
For completeness, the "built-in" tags are the following:
@ignore
Any
Scenario
with (or that has inherited) this tag will be skipped at run-time. This does not apply to anything that is "called" though@parallel
See
@parallel=false
@report
See
@report=false
@setup
See
@setup
@env
See below
@envnot
See below
Environment Tags
There are two special tags that allow you to "select" or "un-select" a
Scenario
depending on the value ofkarate.env
. This can be really convenient, for example to never run some tests in a certain "production like" or sensitive environment.@env=foo,bar
- will run only when the value ofkarate.env
is not-null and equal tofoo
orbar
@envnot=foo
- will run when the value ofkarate.env
isnull
or anything other thanfoo
Here is an example:
Since multiple values are supported, you can also do this:
Tags And Examples
A little-known capability of the Cucumber / Gherkin syntax is to be able to tag even specific rows in a bunch of examples ! You have to repeat the
Examples
section for each tag. The example below combines this with the advanced features described above.Note that if you tag
Examples
like this, and if a tag selector is used when running a givenFeature
- only theExamples
that match the tag selector will be executed. There is no concept of a "default" where for e.g. if there is no matching tag - that theExamples
without a tag will be executed. But note that you can use the negative form of a tag selector:~@region=GB
.Dynamic Port Numbers
In situations where you start an (embedded) application server as part of the test set-up phase, a typical challenge is that the HTTP port may be determined at run-time. So how can you get this value injected into the Karate configuration ?
It so happens that the
karate
object has a field calledproperties
which can read a Java system-property by name like this:karate.properties['myName']
. Since thekarate
object is injected withinkarate-config.js
on start-up, it is a simple and effective way for other processes within the same JVM to pass configuration values to Karate at run-time. Refer to the 'demo'karate-config.js
for an example and how thedemo.server.port
system-property is set-up in the test runner:TestBase.java
.Java API
Karate has a set of Java API-s that expose the HTTP, JSON, data-assertion and UI automation capabilities. The primary classes are described below.
Http
- build and execute any HTTP request and retrieve responsesJson
- build and manipulate JSON data using JsonPath expressions, convert to and from JavaMap
-s andList
-s, parse strings into JSON and convert Java objects into JSONMatch
- exposes all of Karate’smatch
capabilities, and this works for JavaMap
andList
objectsDriver
- perform web-browser automationDo note that if you choose the Java API, you will naturally lose some of the test-automation framework benefits such as HTML reports, parallel execution and JavaScript / configuration. You may have to rely on unit-testing frameworks or integrate additional dependencies.
jbang
jbang is a great way for you to install and execute scripts that use Karate’s Java API on any machine with minimal setup. Note that jbang itself is super-easy to install and there is even a "Zero Install" option.
Here below is an example jbang script that uses the Karate Java API to do some useful work. Name the file as
javadsl.java
and run using the command:jbang javadsl.java
.Read the documentation of the stand-alone JAR for more - such as how you can even install custom command-line applications using jbang !
Invoking feature files using the Java API
It is also possible to invoke a feature file via a Java API which can be useful in some test-automation situations.
A common use case is to mix API-calls into a larger test-suite, for example a Selenium or WebDriver UI test. So you can use Karate to set-up data via API calls, then run the UI test-automation, and finally again use Karate to assert that the system-state is as expected. Note that you can even include calls to a database from Karate using Java interop. And this example may make it clear why using Karate itself to drive even your UI-tests may be a good idea.
The static method
com.intuit.karate.Runner.runFeature()
is best explained in this demo unit-test:JavaApiTest.java
.You can optionally pass in variable values or over-ride config via a
HashMap
or leave the second-last argument asnull
. The variable state after feature execution would be returned as aMap<String, Object>
. The lastboolean
argument is whether thekarate-config.js
should be processed or not. Refer to the documentation on type-conversion to make sure you can 'unpack' data returned from Karate correctly, especially when dealing with XML.Hooks
If you are looking for Cucumber 'hooks' Karate does not support them, mainly because they depend on Java code, which goes against the Karate Way™.
Instead, Karate gives you all you need as part of the syntax. Here is a summary:
Before everything (or 'globally' once)
See
karate.callSingle()
Before every
Scenario
Use the
Background
. Note thatkarate-config.js
is processed before everyScenario
- so you can choose to put "global" config here, for example usingkarate.configure()
.Once (or at the start of) every
Feature
Use a
callonce
in theBackground
. The advantage is that you can set up variables (usingdef
if needed) which can be used in allScenario
-s within thatFeature
.After every
Scenario
configure afterScenario
(see example)At the end of the
Feature
configure afterFeature
(see example)karate.callSingle()
Only recommended for advanced users, but this guarantees a routine is run only once, even when running tests in parallel. You can use
karate.callSingle()
inkarate-config.js
like this:It can take a second JSON argument following the same rules as
call
. Once you get a result, you typically use it to set global variables.Refer to this example:
karate-config.js
headers-single.feature
You can use
karate.callSingle()
directly in a*.feature
file, but it logically fits better in the global "bootstrap". Ideally it should return "pure JSON" and note that you always get a "deep clone" of the cached result object.callonce
orkarate.callSingle()
especially withinkarate-config.js
. Ideally you should return only pure JSON data (or a primitive string, number etc.). Keep in mind that the reason this exists is to "cache" data, and not behavior. So if you return complex objects such as a custom Java instance or a JS function that depends on complex objects, this may cause issues when you run in parallel. If you really need to re-use a Java function, see Java Function References.Multiple calls using
karate.callSingle()
The first argument to
karate.callSingle()
is used as the "cache key". So if you tried to re-use the same feature but with multiple arguments, things will not work as you expect. But you can suffix a?name
to the feature to de-dupe it, like so:Now
adminResponse
anduserResponse
will be different, even though the same feature file is being used for acallSingle()
.configure callSingleCache
When re-running tests in development mode and when your test suite depends on say an
Authorization
header set bykarate.callSingle()
, you can cache the results locally to a file, which is very convenient when your "auth token" is valid for a period of a few minutes - which typically is the case. This means that as long as the token "on file" is valid, you can save time by not having to make the one or two HTTP calls needed to "sign-in" or create "throw-away" users in your SSO store.So in "dev mode" you can easily set this behavior like this. Just ensure that this is "configured" before you use
karate.callSingle()
:By default Karate will use
target
(orbuild
) as the "cache" folder, which you can over-ride by adding adir
key:Data Driven Tests
The Cucumber Way
Cucumber has a concept of Scenario Outlines where you can re-use a set of data-driven steps and assertions, and the data can be declared in a very user-friendly fashion. Observe the usage of
Scenario Outline:
instead ofScenario:
, and the newExamples:
section.You should take a minute to compare this with the exact same example implemented in REST-assured and TestNG. Note that this example only does a "string equals" check on parts of the JSON, but with Karate you are always encouraged to match the entire payload in one step.
This is great for testing boundary conditions against a single end-point, with the added bonus that your test becomes even more readable. This approach can certainly enable product-owners or domain-experts who are not programmer-folk, to review, and even collaborate on test-scenarios and scripts.
Scenario Outline Enhancements
Karate has enhanced the Cucumber
Scenario Outline
as follows:Type Hints: if the
Examples
column header has a!
appended, each value will be evaluated as a JavaScript data-type (number, boolean, or even in-line JSON) - else it defaults to string.Magic Variables:
row
gives you the entire row as a JSON object, andnum
gives you the row index (the first row is0
).Auto Variables: in addition to
__row
, each column key-value will be available as a separate variable, which greatly simplifies JSON manipulation - especially when you want to re-use JSON files containing embedded expressions.Any empty cells will result in a
null
value for that column-key, and this can be useful to remove nodes from JSON or XML documentsThese are best explained with examples. You can choose between the string-placeholder style
<foo>
or directly refer to the variablefoo
(or even the whole row JSON as__row
) in JSON-friendly expressions.Note that even the scenario name can accept placeholders - which is very useful in reports.
For another example, see:
examples.feature
.If you’re looking for more complex ways of dynamically naming your scenarios you can use JS string interpolation by including placeholders in your scenario name.
String interpolation will support variables in scope and / or the
Examples
(including functions defined globally, but not functions defined in the background). Even Java interop and access to thekarate
JS API would work.For some more examples check
test-outline-name-js.feature
.The Karate Way
The limitation of the Cucumber
Scenario Outline:
(seen above) is that the number of rows in theExamples:
is fixed. But take a look at how Karate can loop over a*.feature
file for each object in a JSON array - which gives you dynamic data-driven testing, if you need it. For advanced examples, refer to some of the scenarios within this demo:dynamic-params.feature
.Also see the option below, where you can data-drive an
Examples:
table using JSON.Dynamic Scenario Outline
You can feed an
Examples
table from a custom data-source, which is great for those situations where the table-content is dynamically resolved at run-time. This capability is triggered when the table consists of a single "cell", i.e. there is exactly one row and one column in the table.JSON Array Data Source
The "scenario expression" result is expected to be an array of JSON objects. Here is an example:
You can see the structure of the data here:
kittens.json
Any Karate expression can be used in the "cell expression", and you can even use Java-interop to use external data-sources such as a database.
Note that Karate has built-in support for CSV files and here is an example:
dynamic-csv.feature
.@setup
In cases where the data-source needs multiple steps, for e.g. if an API needs to be called to get a JSON array, you can call a separate
Scenario
to "set up" this data. The@setup
tag is built-in for this purpose and anyScenario
tagged with this will behave like@ignore
. So the only way to call thisScenario
is by using thekarate.setup()
JS API.Here is the above example re-written to do so:
The result of
karate.setup()
will be a JSON of all the variables created within theScenario
tagged with@setup
. Note how we "unpack" thekittens
and use it to "data drive" theScenario Outline
. You can get really creative and use JS functions to filter data for different needs.Though not really recommended, you can have multiple
Scenario
-s within aFeature
tagged with@setup
. But in that case you should de-dupe them using a name:And since it is common to run a
@setup
Scenario
only once per-feature you can callkarate.setupOnce()
. In the feature below, the* print 'in setup'
step will run only once. Also note how theBackground
will run 4 times (twice perScenario
).JSON Function Data Source
An advanced option is where the "scenario expression" returns a JavaScript "generator" function. This is a very powerful way to generate test-data without having to load a large number of data rows into memory. The function has to return a JSON object. To signal the end of the data, just return
null
. The function argument is the row-index, so you can easily determine when to stop the generation of data. Here is an example:Sponsored by