当testing失败时,TravisCI并没有失败

我有用 JavaScript编写的testing,我使用TravisCI进行testing。

build立

我的package.json是这样的:

 "scripts": { "test": "node testsRunner.js" } 

而我的.travis.yml是:

 language: node_js node_js: - '0.12.7' 

'testsRunner.js'是:

 var nodeunit = require('nodeunit'); var path = require('path'); nodeunit.reporters.default.run([ path.join(__dirname, 'suite1/test.js') ]); 

suite1/test.js最后是:

 module.exports = { setUp: function(callback) { // Initialization code... callback(); }, tearDown: function(callback) { // Cleanup... callback(); }, test1: function(test) { test.expect(10); // This test expects 10 assertions to be run // Doing stuff... test.done(); }, test2: function(test) { test.expect(10); // This test expects 20 assertions to be run // Doing stuff... test.done(); } }; 

从特拉维斯login

以下是Travis的构build日志和testing执行:

 Using worker: worker-linux-docker-19fc8ef0.prod.travis-ci.org:travis-linux-6 system_info Build system information Build language: node_js Build image provisioning date and time Thu Feb 5 15:09:33 UTC 2015 Operating System Details Distributor ID: Ubuntu Description: Ubuntu 12.04.5 LTS Release: 12.04 Codename: precise Linux Version 3.13.0-29-generic Cookbooks Version a68419e https://github.com/travis-ci/travis-cookbooks/tree/a68419e GCC version gcc (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3 Copyright (C) 2011 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. LLVM version clang version 3.4 (tags/RELEASE_34/final) Target: x86_64-unknown-linux-gnu Thread model: posix Pre-installed Ruby versions ruby-1.9.3-p551 Pre-installed Node.js versions v0.10.36 Pre-installed Go versions 1.4.1 Redis version redis-server 2.8.19 riak version 2.0.2 MongoDB version MongoDB 2.4.12 CouchDB version couchdb 1.6.1 Neo4j version 1.9.4 RabbitMQ Version 3.4.3 ElasticSearch version 1.4.0 Installed Sphinx versions 2.0.10 2.1.9 2.2.6 Default Sphinx version 2.2.6 Installed Firefox version firefox 31.0esr PhantomJS version 1.9.8 ant -version Apache Ant(TM) version 1.8.2 compiled on December 3 2011 mvn -version Apache Maven 3.2.5 (12a6b3acb947671f09b81f49094c53f426d8cea1; 2014-12-14T17:29:23+00:00) Maven home: /usr/local/maven Java version: 1.7.0_76, vendor: Oracle Corporation Java home: /usr/lib/jvm/java-7-oracle/jre Default locale: en_US, platform encoding: ANSI_X3.4-1968 OS name: "linux", version: "3.13.0-29-generic", arch: "amd64", family: "unix" git.checkout 0.61s$ git clone --depth=50 --branch=master git://github.com/myuser/myproj.git myuser/myproj Cloning into 'myuser/myproj'... remote: Counting objects: 1473, done. remote: Compressing objects: 100% (1053/1053), done. remote: Total 1473 (delta 300), reused 1401 (delta 250), pack-reused 0 Receiving objects: 100% (1473/1473), 8.93 MiB | 0 bytes/s, done. Resolving deltas: 100% (300/300), done. Checking connectivity... done. $ cd myuser/myproj $ git checkout -qf d603836a30ea1bc213f7b97682df507c91af8404 This job is running on container-based infrastructure, which does not allow use of 'sudo', setuid and setguid executables. If you require sudo, add 'sudo: required' to your .travis.yml See http://docs.travis-ci.com/user/workers/container-based-infrastructure/ for details. 2.99s$ nvm install 0.12.7 ######################################################################## 100.0% Now using node v0.12.7 $ node --version v0.12.7 $ npm --version 2.11.3 $ nvm --version 0.23.3 install 0.84s$ npm install npm WARN package.json myproj@0.1.0 No license field. 0.64s$ npm test > myproj@0.1.0 test /home/travis/build/myuser/myproj > node testsRunner.js test.js ✔ test1 ✖ test2 TypeError: Cannot read property... at extract (...) at Object.at (...) at Object.module.exports... ... <stacktrace continues> FAILURES: 1/45 assertions failed (19ms) The command "npm test" exited with 0. Done. Your build exited with 0. 

问题

问题是test1成功, test2不通过。 在test2之前抛出exception,这使得Travis报告我的构build失败。

但是现在我修复了这个testing,所以test2不会抛出exception,但是它的断言失败了。 然而,Travis报告说,构build为通过,而它应该报告为失败的构build。

如何处理这个?

编辑

我已经改变了我的跑步者:

 var nodeunit = require('nodeunit'); var path = require('path'); nodeunit.reporters.default.run([ path.join(__dirname, 'suite1/test.js') ]); console.log('Done!'); // ADDED THIS LINE 

那么,在日志中我可以看到Done! 在执行之前显示,之后我可以看到每个testing日志和堆栈跟踪。 是否有可能这些testing正在执行asynchronous,这是导致主进程退出0状态?

固定

按照克里斯·贝克的build议,我改变了这样的跑步者:

 var nodeunit = require('nodeunit'); var path = require('path'); nodeunit.reporters.default.run([ path.join(__dirname, 'suite1/test.js') ], null function(data) { // `data` has a value if errors occurred if (data) { throw 'Error1'; } }); 

我用travis-ci进行unit testing。

如果您希望testing失败代表构build失败,则调用testing的命令行实用程序需要报告非零的错误代码结果(在运行testing之后,在bash值$?将会显示这个结果)。

如果你的js环境不会导致断言失败这样做,最简单的事情很可能是编写你自己的assert函数,抛出一个未捕获的exception,或者,设置一个全局variables来表示失败,并存在你的testing运行器脚本检查全局variables并引发未捕获的exception(如果已设置)。

(后者的优点是,即使某些断言失败,也可以运行所有的testing,但是如果testing循环无限,那么即使在较早的断言失败之后,构build也会暂停一段时间。

通常有通过JUnit xml进行unit testing集成的插件。 在Travis ci中这是默认的吗? 你在用Junit报告你的testing吗?

nodeunit有一个内置的junit记者:

junit – 创buildjUnit兼容的XML报告,可以与Hudson等持续集成工具一起使用。