Support attachments, also known as embeddings

[simondean]

Hi. This pull request implements:

• Embedding screenshots in JSON and HTML reports #124 Embedding screenshots in JSON and HTML reports
• success and failed properties in after and around hooks #129 success and failed properties in after and around hooks
• Hooks as steps #179 Hooks as steps
• Callback.fail() not supported in hooks. #159 Callback.fail() not supported in hooks.

It also fixes:

• Errors within Before and After hooks should make the scenario fail #110 Errors within Before and After hooks should make the scenario fail
• visitRow function is unused #181 visitRow function is unused

The pull request includes several new cucumber tests under features/ in the codebase which provide examples of i) how attachments can be used from hooks and from step definitions, ii) how hooks are now reported as steps by the JSON formatter and iii) how you can fail a step using callback("fail") and callback.fail().

Here's a quick example of how the hooks appear as steps in the JSON (taken from features/hooks.feature):

[
  {
    "id": "some-feature",
    "name": "some feature",
    "description": "",
    "line": 1,
    "keyword": "Feature",
    "uri": "<current-directory>/features/a.feature",
    "elements": [
      {
        "name": "I've declared one step and it is passing",
        "id": "some-feature;i've-declared-one-step-and-it-is-passing",
        "line": 3,
        "keyword": "Scenario",
        "description": "",
        "type": "scenario",
        "steps": [
          {
            "name": "scenario",
            "keyword": "Around ",
            "result": {
              "duration": "<duration>",
              "status": "passed"
            },
            "match": {}
          },
          {
            "name": "scenario",
            "keyword": "Before ",
            "result": {
              "duration": "<duration>",
              "status": "passed"
            },
            "match": {}
          },
          {
            "name": "This step is passing",
            "line": 4,
            "keyword": "Given ",
            "result": {
              "duration": "<duration>",
              "status": "passed"
            },
            "match": {}
          },
          {
            "name": "scenario",
            "keyword": "After ",
            "result": {
              "duration": "<duration>",
              "status": "passed"
            },
            "match": {}
          },
          {
            "name": "scenario",
            "keyword": "Around ",
            "result": {
              "duration": "<duration>",
              "status": "passed"
            },
            "match": {}
          }
        ]
      }
    ]
  }
]

Here's an example of how to attach an attachment from a before hook (these attachment examples are taken from features/attachments.feature):

var hooks = function () {
  this.Before(function(scenario, callback) {
    scenario.attach("text");
    callback();
  });
};

module.exports = hooks;

Attaching from an around hook:

var hooks = function () {
  this.Around(function(scenario, runScenario) {
    scenario.attach("text");

    runScenario(function(scenario, callback) {
      scenario.attach("more text");
      callback();
    });
  });
};

module.exports = hooks;

I've implemented #179 "Hooks as steps" as part of this pull request to avoid over complicating the way reporters/listeners/formatters work when attaching from a hook. See the getCurrentStep method in https://github.com/cucumber/gherkin/blob/master/java/src/main/java/gherkin/formatter/JSONFormatter.java to contrast with the other way this could have been implemented (Gerkin's Java JSONFormatter has to rewind time to earlier steps in order to add attachments to the last step that executed/failed).

[simondean]

Travis CI seems to be failing with npm install issues. It's something to do with bower. Any ideas? It doesn't seem to be related to this pull request (there are no changes to the package.json in the pull request).

[jbpros]

Wow, thanks for this big piece of work. I'd like to fix this Travis issue that has been bugging us for some time now before going any further with the PRs.

[simondean]

@jbpros have the Travis issues been happening for long? Is there anything I can do to help?

[simondean]

I've answered my own question :-) It looks like the dev dependency on Bower is causing the Travis CI build failures. I've submitted a pull request that removes the dependency: #191

[nicholaspufal]

Please merge this to master :)

[snatchblock]

Hi @simondean,

I have used the the following commit - git+https://github.com/simondean/cucumber-js#47ecb1a1a1fe0e4421d89a3ae7df919be768248e in my package.json dependencies

my structure code is as follows -

//file: features/stepdefinitions/registration.step.js
var registrationWrapper = function registrationWrapper() {

    this.World = require('../support/world').World;

    this.Before(function (next) {
        this.initBrowser(next);
    });

    this.After(function (next) {
        this.browser.quit(next);
    });

    this.Given(/^I am on Signup page$/, function (next) {
        this.browser.get(this.baseUrl + '/register', next);

    });

    this.When(/^I enter email id "([^"]*)"$/, function (email, next) {
        this.browser.elementById("reg-email", function (err, element) {
            element.type(email, function () {
                next();
            });
        });
    });

    this.When(/^click REGISTER button$/, function (next) {
        this.browser.elementById("register-submit", function (err, element) {
            element.click(function () {
                setTimeout(next, 6000);
            });
        });
    });

    this.Then(/^I should see validation message "([^"]*)"$/, function (message, next) {
        this.browser.elementByCssSelector("p.error", function (err, element) {
            element.text(function (err, text) {
                text.should.equal(message);
                next();
            });
        });
    });
};
module.exports = registrationWrapper;

the feature file is as following -

//file: features/support/registration.feature
Feature: Signup page
    As a user of ourCompany
    I want to sign up to my account
    So that I can view and make changes to my booking

Scenario: To check invalid email id
    Given I am on Signup page
    When I enter email id "invalid.email.com"
        And click REGISTER button
    Then I should see validation message "Please enter a valid email."

When I try to change the Before() or After() to add the scenario as 1st argument, I see that it is still passed with the next details rather than the scenario.

What am I missing here or infact - doing it wrong?

thank you for your time and help.

oh! and for reference, this is what the world.js contains -

var fs = require('fs');
var wd = require('wd');
var nconf = require('nconf');
var chai = require('chai');
chai.should();

var testEnv = process.env.TEST_ENV || 'qa';

console.log('Test environment : ', testEnv);

nconf.argv().env()
    .file({
        file: 'config/config-' + testEnv + '.json'
    });


var World = function World(callback) {
    var webdriverHubHost = process.env.WebDriverHubHost || '127.0.0.1';
    var webdriverHubPort = process.env.WebDriverHubPort || 4444;

    this.browser = wd.remote(webdriverHubHost, webdriverHubPort);
    var baseUrl = nconf.get('BASE_URL') || 'localhost';
    var Url = nconf.get('URL') || 'localhost';
    console.log('Test base URL : ', baseUrl);
    this.baseUrl = baseUrl;
    this.Url = Url;
    callback();
};

World.prototype.initBrowser = function initBrowser(callback) {
    var browserName = nconf.get('BROWSER_NAME') || 'firefox';
    console.log('Initializing browser : ', browserName);
    this.browser.init({
        browserName: browserName
    }, callback);
};


World.prototype.takeScreenshot = function takeScreenshot(filename) {
    if (testEnv !== 'qa') {
        this.browser.takeScreenshot(function (error, screenshot) {
            fs.writeFile('test/acceptance/images/' + filename + "_" + testEnv + "_" + nconf.get('BROWSER_NAME') + "_" + Date.now() + ".png", screenshot, 'base64', function (err) {
                if (err) {
                    console.log(err);
                }
            });
        });
    }
};

module.exports = {
    World: World
};

[samccone]

Hi @simondean can you drop your last 3 commits and rebase off of master. Hopefully then we can get this sucker merged in.

I will do a deep review as soon as you rebase, just ping me 👍

[simondean]

Thanks @samccone. I'll hopefully be able to drop the commits and do the rebase before the end of the week.

[simondean]

Hi @samccone. I've removed the two unnecessary commits. I've kept the other commit as it fixes an issue with node v0.6 and v0.8. I've also rebased with master.

[jbpros]

I've reviewed the whole thing, it's impressive. Thanks for this great work @simondean. I'm playing a bit more with it and then we'll merge if everything looks ok.

[jbpros]

I think it should be undefined as well, this arbitrary value seems artificial.

[jbpros]

Hooks should not be output by the pretty formatter and I think it would be nice to flag steps that are coming from hooks. Adding a isHidden() method to the three AST step kinds (Step, OutlineStep and HookStep with only the latter's returning true) would be a good solution.

[simondean]

@jbpros I've just pushed a change which removes the names from hook steps.

isHidden sounds good. I'll have a look at that.

[simondean]

@jbpros I've just pushed a commit that adds the isHidden functionality. In the end I had to change summary formatter and stats journal as well as pretty formatter. As part of the commit I've created feature tests for the pretty formatter and summary formatter.

[simondean]

Hi @jbpros. The building is passing now. I had to remove the strip-ansi dependency I'd added. I found that the cucumber-js tests already had some regex to strip ANSI color codes so I used that in step.

As part of the commit, I fixed some undefined steps for the Ruby cucumber tests. The undefined steps where in features/cli.feature

[simondean]

@jbpros do you think it's worth adding a "hidden": true attribute to the output of the JSON formatter for hidden steps?

[jbpros]

@simondean thanks mate, this is amazing :)

Yes, I think a hidden property added to the JSON output would do no harm and be useful.

[simondean]

Hi @jbpros. I realised that the Pretty Formatter would print "undefined" as the step name for the hook steps. I've just pushed a commit that fixes that. The commit also changes the JSON formatter to add the "hidden": true attribute to hidden steps (e.g. hook steps).

Hopefully that's everything 😄

[jbpros]

@simondean this has been merged, thanks again! It'd be nice to document this new feature in the README. Can you do that?

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.