Skip to content

ng test --code-coverage in 6.1 improperly detecting branches #11672

New issue
New issue
Closed
#12291
Closed
ng test --code-coverage in 6.1 improperly detecting branches#11672
#12291
Labels
P1Impacts a large percentage of users; if a workaround exists it is partial or overly painfularea: @angular-devkit/build-angularfreq3: highseverity5: regressiontype: bug/fix
Milestone
7.0.0

Description

@jhage-accesso
jhage-accesso
opened on Jul 26, 2018

Bug Report or Feature Request (mark with an x)

- [ X ] bug report -> please search issues before submitting
- [ ] feature request

Command (mark with an x)

- [ ] new
- [ ] build
- [ ] serve
- [ X ] test
- [ ] e2e
- [ ] generate
- [ ] add
- [ ] update
- [ ] lint
- [ ] xi18n
- [ ] run
- [ ] config
- [ ] help
- [ ] version
- [ ] doc

Versions

OS: macOS Sierra
Node: v8.9.4
NPM: 6.2.0

Ng Version:

Angular CLI: 6.1.0
Node: 8.9.4
OS: darwin x64
Angular: 6.1.0
... animations, cli, common, compiler, compiler-cli, core, forms
... http, language-service, platform-browser
... platform-browser-dynamic, router

Package                           Version
-----------------------------------------------------------
@angular-devkit/architect         0.7.0
@angular-devkit/build-angular     0.7.0
@angular-devkit/build-optimizer   0.7.0
@angular-devkit/build-webpack     0.7.0
@angular-devkit/core              0.7.0
@angular-devkit/schematics        0.7.0
@angular/cdk                      6.4.1
@angular/flex-layout              6.0.0-beta.16
@angular/material                 6.4.1
@ngtools/webpack                  6.1.0
@schematics/angular               0.7.0
@schematics/update                0.7.0
rxjs                              6.2.2
typescript                        2.9.2
webpack                           4.9.2

Repro steps

Run ng test --code-coverage
Notice that things like import statements are counted as if else branches.
image

The log given by the failure

No log file

Desired functionality

I expect to see things like import statements and decorator metadata to not be marked as if else branches in the code coverage report.

Mention any other details that might be useful

Activity

  1. hansl
    added
    area: @angular-devkit/build-angular
    on Jul 26, 2018

  2. wkurniawan07 commented on Jul 27, 2018

    @wkurniawan07
    wkurniawan07
    on Jul 27, 2018

    Not providing any solution here, but the problem is likely to be in Devkit 0.7 instead of CLI 6.1.

  3. dimitriy-k commented on Jul 27, 2018

    @dimitriy-k
    dimitriy-k
    on Jul 27, 2018

    same issue after upgrade to 6.1.1. coverage dropped from 100%
    Statements : 98.13% ( 3667/3737 )
    Functions : 99.47% ( 752/756 )

  4. kevinbosman commented on Jul 28, 2018

    @kevinbosman
    kevinbosman
    on Jul 28, 2018

    Had the same problem after upgrading to cli 6.1.1 + devkit 0.7.1. Fixed by downgrading @angular-devkit/build-angular to 0.6.8

    Before (with coverage issues):

    Angular CLI: 6.1.1
Node: 8.11.2
OS: win32 x64
Angular: 6.1.0
... animations, common, compiler, compiler-cli, core, forms
... http, language-service, platform-browser
... platform-browser-dynamic, router

Package                           Version
-----------------------------------------------------------
@angular-devkit/architect         0.7.1
@angular-devkit/build-angular     0.7.1
@angular-devkit/build-optimizer   0.7.1
@angular-devkit/build-webpack     0.7.1
@angular-devkit/core              0.7.1
@angular-devkit/schematics        0.7.1
@angular/cli                      6.1.1
@ngtools/webpack                  6.1.1
@schematics/angular               0.7.1
@schematics/update                0.7.1
rxjs                              6.2.2
typescript                        2.9.2
webpack                           4.16.3

    After (coverage working properly again):

    Angular CLI: 6.1.1
Node: 8.11.2
OS: win32 x64
Angular: 6.1.0
... animations, common, compiler, compiler-cli, core, forms
... http, language-service, platform-browser
... platform-browser-dynamic, router

Package                           Version
-----------------------------------------------------------
@angular-devkit/architect         0.7.1
@angular-devkit/build-angular     0.6.8
@angular-devkit/build-optimizer   0.6.8
@angular-devkit/core              0.7.1
@angular-devkit/schematics        0.7.1
@angular/cli                      6.1.1
@ngtools/webpack                  6.0.8
@schematics/angular               0.7.1
@schematics/update                0.7.1
rxjs                              6.2.2
typescript                        2.9.2
webpack                           4.16.3

  5. brian428 commented on Jul 31, 2018

    @brian428
    brian428
    on Jul 31, 2018

    I'm guessing something is wrong in the creation or handling source maps?

  6. alexw10 commented on Aug 2, 2018

    @alexw10
    alexw10
    on Aug 2, 2018

    Hope this gets fixed soon need to be on latest version of the @angular-devkit/build-angular because of the issue in 0.6.8 of it not exiting with proper error code but now this coverage report issue is a problem.
    Willing to try to help fix it as well if anyone can point me where to start looking.

  7. clydin
    added
    type: bug/fix
    P1Impacts a large percentage of users; if a workaround exists it is partial or overly painful
    freq3: high
    severity5: regression
    on Aug 2, 2018

  8. jfpicard1 commented on Aug 3, 2018

    @jfpicard1
    jfpicard1
    on Aug 3, 2018

    Maybe it's the way that the sourceMaps are handled. If I add "sourceMap": true, in the test section, with @angular-devkit/build-angular 0.7.0 it seems to work.

    See: https://stackoverflow.com/questions/51257459/if-else-path-not-taken-in-imports-in-angular-istanbul-code-coverage-report/51676221#51676221

  9. brian428 commented on Aug 3, 2018

    @brian428
    brian428
    on Aug 3, 2018

    I'm on 0.7.1, and adding "sourceMap": true to the test section (or via the command line) doesn't seem to affect the messed up coverage report output.

  10. jfpicard1 commented on Aug 3, 2018

    @jfpicard1
    jfpicard1
    on Aug 3, 2018

    Maybe you have a tsconfig.json with "sourceMap": false ?

    Here my DevDependecies:

      "devDependencies": {
    "@angular-devkit/build-angular": "~0.7.0",
    "@angular/cli": "^6.1.1",
    "@angular/compiler-cli": "^6.1.0",
    "@angular/language-service": "^6.1.1",
    "@types/jasmine": "^2.8.8",
    "@types/jasminewd2": "~2.0.2",
    "@types/node": "^10.5.5",
    "codelyzer": "^4.4.2",
    "jasmine-core": "^2.99.1",
    "jasmine-spec-reporter": "^4.2.1",
    "karma": "^2.0.5",
    "karma-chrome-launcher": "^2.2.0",
    "karma-cli": "^1.0.1",
    "karma-coverage": "^1.1.2",
    "karma-coverage-istanbul-reporter": "^2.0.1",
    "karma-html-reporter": "^0.2.7",
    "karma-ie-launcher": "^1.0.0",
    "karma-intl-shim": "^1.0.3",
    "karma-jasmine": "^1.1.2",
    "karma-jasmine-html-reporter": "^0.2.2",
    "karma-phantomjs-launcher": "^1.0.4",
    "karma-typescript": "^3.0.12",
    "karma-typescript-angular2-transform": "^1.0.2",
    "karma-typescript-plugin": "^0.1.2",
    "lite-server": "^2.2.2",
    "rxjs-tslint-rules": "^4.7.2",
    "ts-node": "^7.0.0",
    "tslint": "^5.11.0",
    "tslint-angular": "^1.1.2",
    "tslint-consistent-codestyle": "^1.13.3",
    "tslint-microsoft-contrib": "^5.1.0",
    "typescript": "^2.7.2"
  }

  11. alexw10 commented on Aug 3, 2018

    @alexw10
    alexw10
    on Aug 3, 2018

    I can confirm that setting sourceMap: true fixed the issue for me I am on 0.7.2 @brian428 @jfpicard1
    Guessing this shouldn't need to be on since it worked without it before?

  12. brian428 commented on Aug 3, 2018

    @brian428
    brian428
    on Aug 3, 2018

    Hmm, updated to 0.7.2, but still no luck. tsconfig.spec.json doesn't set sourceMap: false. I even added sourceMap: true just to try it (even though it's already set to true in the base tsconfig.json), but had no effect. The test section of my angular.json also includes this, but no luck there either:

    "test": {
  "builder": "@angular-devkit/build-angular:karma",
  "options": {
    "main": "src/test.ts",
    "karmaConfig": "./karma.conf.js",
    "sourceMap": true,
    ...
  }
}

    None of these tweaks make any difference. The coverage report is generated but the highlighting is all wrong.

  13. 19 remaining items

  14. alexeagle
    added this to the 7.0.0 milestone on Sep 14, 2018
  15. hansl
    closed this as completedin #12291on Sep 18, 2018

  16. brian428 commented on Sep 18, 2018

    @brian428
    brian428
    on Sep 18, 2018

    FYI, I made the local change that was done in https://github.com/angular/angular-cli/pull/12291/files, but the coverage report highlighting is still messed up. Not sure if that's expected for now (I'm on @angular-devkit/build-angular 0.8.1, and don't know if other things have changed beyond that one schema change), but figured I'd mention it.

  17. lanegoolsby commented on Sep 20, 2018

    @lanegoolsby
    lanegoolsby
    on Sep 20, 2018

    For anyone here from Google, I may have found a more permanent fix than using the CLI flag. Set "sourceMap": true in angular.json under projects -> [yourProject] -> architect -> test -> options.

    Making the change in the tsconfig.spec.json or tsconfig.json had no effect for me.

    My environment:

    Angular CLI: 6.1.3
    Node: 8.11.3
    OS: win32 x64
    Angular: 6.1.2
    ... animations, common, compiler, compiler-cli, core, forms
    ... http, language-service, platform-browser
    ... platform-browser-dynamic, router

    Package Version

    @angular-devkit/architect 0.7.3
    @angular-devkit/build-angular 0.7.3
    @angular-devkit/build-optimizer 0.7.3
    @angular-devkit/build-webpack 0.7.3
    @angular-devkit/core 0.7.3
    @angular-devkit/schematics 0.7.3
    @angular/cli 6.1.3
    @ngtools/webpack 6.1.3
    @schematics/angular 0.7.3
    @schematics/update 0.7.3
    rxjs 6.2.2
    typescript 2.7.2
    webpack 4.9.2

  18. pfbrowning
    added a commit that references this issue on Oct 5, 2018

    Add --source-map test param as a workaround for broken test coverage

    9ee2c92

  20. PikaChokeMe commented on Nov 7, 2018

    @PikaChokeMe
    PikaChokeMe
    on Nov 7, 2018

    Thought it might be worth noting that I'm still seeing this issue...
    I'm currently using Angular 6.2.7
    I currently have "@angular-devkit/build-angular": "~0.8.7",

    After adding "sourceMap": true to my angular.json test options, checking that it was included in tsconfig.json, and manually trying to run ng test --code-coverage=true --watch=false --source-map=true I still have had zero success getting code coverage working and detecting branches properly.

  21. richmwatts commented on Nov 16, 2018

    @richmwatts
    richmwatts
    on Nov 16, 2018

    Is there any update on this? I have also set the sourceMaps property to true and believe I am now seeing more "believable" code coverage metrics. However, the highlighting in the reporting is horribly horribly wrong.

    How can I have any confidence over the quoted coverage metrics?

  22. jhage-accesso commented on Nov 16, 2018

    @jhage-accesso
    jhage-accesso
    on Nov 16, 2018
    Author

    This is fixed in ng7

  23. richmwatts commented on Nov 16, 2018

    @richmwatts
    richmwatts
    on Nov 16, 2018

    I am still having the same issue with all dependencies completely up to date

  24. ash4aque commented on Feb 21, 2019

    @ash4aque
    ash4aque
    on Feb 21, 2019

    This is fixed in ng7

    Does this means it is not fixed for ng6

  25. marthinus-engelbrecht commented on Apr 13, 2019

    @marthinus-engelbrecht
    marthinus-engelbrecht
    on Apr 13, 2019

    This is still broken, running these dependencies

    "devDependencies": {
    "@angular-devkit/build-angular": "~0.13.0",
    "@angular/cli": "~7.3.8",
    "@angular/compiler-cli": "~7.2.0",
    "@angular/language-service": "~7.2.0",
    "@types/node": "~8.9.4",
    "@types/jasmine": "~2.8.8",
    "@types/jasminewd2": "~2.0.3",
    "codelyzer": "~4.5.0",
    "jasmine-core": "~2.99.1",
    "jasmine-spec-reporter": "~4.2.1",
    "karma": "~4.0.0",
    "karma-chrome-launcher": "~2.2.0",
    "karma-coverage-istanbul-reporter": "~2.0.1",
    "karma-jasmine": "~1.1.2",
    "karma-jasmine-html-reporter": "^0.2.2",
    "protractor": "~5.4.0",
    "ts-node": "~7.0.0",
    "tslint": "~5.11.0",
    "typescript": "~3.2.2"
  }

    With --source-map=true it gives me 100% branch coverage, but with branches 0/0. And I don't have 100% branch coverage. With false I get the right numbers but the report is all mixed up.

  26. thorstenherbst commented on Jul 11, 2019

    @thorstenherbst
    thorstenherbst
    on Jul 11, 2019

    FYI, I'm on the latest versions (Angular 8) and I'm still getting a non-accurate coverage report. Even using ng test --code-coverage --source-map. IMHO its a bug which is related to a problem with the MAPPING of the source-maps. Not use of source-maps itself.

    Here is my configuration:

    package.json

    {
  "name": "web",
  "version": "0.0.0",
  "scripts": {
    "ng": "ng",
    "start": "ng serve",
    "build": "grunt && ng build --prod",
    "build-dev": "grunt && ng build --extract-css --watch --aot --output-hashing=bundles",
    "build-debug": "grunt && ng build --source-map --extract-css --output-hashing=bundles",
    "build-svg": "grunt",
    "test": "ng test",
    "test-ci": "ng test --watch=false --browsers=ChromeHeadlessIuE --code-coverage",
    "lint": "ng lint",
    "e2e": "ng e2e",
    "livereload": "livereload ./dist -e html -u true -w 250"
  },
  "private": true,
  "dependencies": {
    "@angular/animations": "^8.0.3",
    "@angular/common": "^8.0.3",
    "@angular/compiler": "^8.0.3",
    "@angular/core": "^8.0.3",
    "@angular/elements": "^8.0.3",
    "@angular/forms": "^8.0.3",
    "@angular/platform-browser": "^8.0.3",
    "@angular/platform-browser-dynamic": "^8.0.3",
    "@angular/router": "^8.0.3",
    "@ngrx/store": "^8.0.1",
    "@types/leaflet-fullscreen": "^1.0.4",
    "@types/leaflet.markercluster": "^1.4.0",
    "classlist.js": "^1.1.20150312",
    "core-js": "^2.6.9",
    "deep-equal": "^1.0.1",
    "leaflet": "^1.4.0",
    "leaflet.markercluster": "^1.4.1",
    "ngx-cookie-service": "^2.1.0",
    "rxjs": "^6.4.0",
    "s": "^0.1.1",
    "swiper": "^4.5.0",
    "tslib": "^1.10.0",
    "video.js": "^7.5.5",
    "zone.js": "~0.9.1"
  },
  "devDependencies": {
    "@angular-devkit/build-angular": "~0.800.6",
    "@angular/cli": "^8.0.6",
    "@angular/compiler-cli": "^8.0.3",
    "@angular/language-service": "^8.0.3",
    "@types/jasmine": "^3.3.13",
    "@types/jasminewd2": "^2.0.6",
    "@types/leaflet": "^1.4.3",
    "@types/node": "^10.14.10",
    "@types/swiper": "^4.4.3",
    "@types/video.js": "^7.2.10",
    "codelyzer": "^5.0.1",
    "grunt": "^1.0.3",
    "grunt-cli": "^1.3.2",
    "grunt-replace": "^1.0.1",
    "grunt-sass": "^3.0.2",
    "grunt-svgstore": "^2.0.0",
    "jasmine-core": "^3.3.0",
    "jasmine-spec-reporter": "^4.2.1",
    "karma": "^4.0.1",
    "karma-chrome-launcher": "^2.2.0",
    "karma-coverage-istanbul-reporter": "^2.0.5",
    "karma-jasmine": "^2.0.1",
    "karma-jasmine-html-reporter": "^1.4.0",
    "karma-junit-reporter": "^1.2.0",
    "livereload": "^0.8.0",
    "node-sass": "^4.11.0",
    "npm-check-updates": "^3.1.12",
    "protractor": "^5.4.2",
    "sass-loader": "^7.1.0",
    "ts-node": "^8.3.0",
    "tslint": "^5.18.0",
    "typescript": "~3.4.5",
    "webpack": "^4.35.0"
  }
}

    Angular.CLI

        / \   _ __   __ _ _   _| | __ _ _ __     / ___| |   |_ _|
   / △ \ | '_ \ / _` | | | | |/ _` | '__|   | |   | |    | |
  / ___ \| | | | (_| | |_| | | (_| | |      | |___| |___ | |
 /_/   \_\_| |_|\__, |\__,_|_|\__,_|_|       \____|_____|___|
                |___/
    

Angular CLI: 7.3.6
Node: 10.15.2
OS: darwin x64
Angular: 
... 

Package                      Version
------------------------------------------------------
@angular-devkit/architect    0.13.6
@angular-devkit/core         7.3.6
@angular-devkit/schematics   7.3.6
@schematics/angular          7.3.6
@schematics/update           0.13.6
rxjs                         6.3.3
typescript                   3.2.4
    
-bash-3.2$

    angular.json

    {
	"$schema": "./node_modules/@angular/cli/lib/config/schema.json",
	"version": 1,
	"newProjectRoot": "projects",
	"projects": {
		"web": {
			"root": "",
			"sourceRoot": "src",
			"projectType": "application",
			"prefix": "db",
			"schematics": {
				"@schematics/angular:component": {
					"styleext": "scss"
				}
			},
			"architect": {
				"build": {
					"builder": "@angular-devkit/build-angular:browser",
					"options": {
						"outputPath": "dist",
						"index": "src/index.html",
						"main": "src/main.ts",
						"polyfills": "src/polyfills.ts",
						"tsConfig": "src/tsconfig.app.json",
						"assets": [
							"src/favicon.ico",
							"src/assets/fonts",
							"src/assets/icons/sprite.svg",
							"src/assets/images",
							"src/assets/sounds",
							"src/static",
							"src/sw.js"
						],
						"styles": [
							"src/styles.scss"
						],
						"scripts": [
						]
					},
					"configurations": {
						"production": {
							"fileReplacements": [
								{
									"replace": "src/environments/environment.ts",
									"with": "src/environments/environment.prod.ts"
								}
							],
							"optimization": true,
							"outputHashing": "all",
							"sourceMap": true,
							"extractCss": true,
							"namedChunks": false,
							"aot": true,
							"extractLicenses": true,
							"vendorChunk": false,
							"buildOptimizer": true
						}
					}
				},
				"serve": {
					"builder": "@angular-devkit/build-angular:dev-server",
					"options": {
						"browserTarget": "web:build"
					},
					"configurations": {
						"production": {
							"browserTarget": "web:build:production"
						}
					}
				},
				"extract-i18n": {
					"builder": "@angular-devkit/build-angular:extract-i18n",
					"options": {
						"browserTarget": "web:build"
					}
				},
				"test": {
					"builder": "@angular-devkit/build-angular:karma",
					"options": {
						"progress": false,
						"main": "src/test.ts",
						"polyfills": "src/polyfills.ts",
						"tsConfig": "src/tsconfig.spec.json",
						"karmaConfig": "src/karma.conf.js",
						"styles": [
							"src/styles.scss"
						],
						"scripts": [],
						"assets": [
							"src/favicon.ico",
							"src/assets"
						]
					}
				},
				"lint": {
					"builder": "@angular-devkit/build-angular:tslint",
					"options": {
						"tsConfig": [
							"src/tsconfig.app.json",
							"src/tsconfig.spec.json"
						],
						"exclude": [
							"**/node_modules/**"
						]
					}
				}
			}
		},
		"web-e2e": {
			"root": "e2e/",
			"projectType": "application",
			"architect": {
				"e2e": {
					"builder": "@angular-devkit/build-angular:protractor",
					"options": {
						"protractorConfig": "e2e/protractor.conf.js",
						"devServerTarget": "web:serve"
					},
					"configurations": {
						"production": {
							"devServerTarget": "web:serve:production"
						}
					}
				},
				"lint": {
					"builder": "@angular-devkit/build-angular:tslint",
					"options": {
						"tsConfig": "e2e/tsconfig.e2e.json",
						"exclude": [
							"**/node_modules/**"
						]
					}
				}
			}
		}
	},
	"defaultProject": "web"
}

    If some more informations are needed dont hasitate to write me. I am very interested to help solving this problem.

    Thanks in advance

  27. angular-automatic-lock-bot commented on Sep 9, 2019

    @angular-automatic-lock-bot
    angular-automatic-lock-botBot
    on Sep 9, 2019

    This issue has been automatically locked due to inactivity.
    Please file a new issue if you are encountering a similar or related problem.

    Read more about our automatic conversation locking policy.

    This action has been performed automatically by a bot.

  28. angular-automatic-lock-bot
    locked and limited conversation to collaborators on Sep 9, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1Impacts a large percentage of users; if a workaround exists it is partial or overly painfularea: @angular-devkit/build-angularfreq3: highseverity5: regressiontype: bug/fix

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions