Skip to content
  • nagachika's avatar
    1d29740c
    merge revision(s)... · 1d29740c
    nagachika authored
    merge revision(s) 5086c25f,3ff0a0b4: [Backport #16936]
    
    	Properly exclude test cases.
    
    	Lets consider the following scenario:
    
    	~~~
    	irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):001:0> p suite
    	OpenSSL::TestEC
    	=> OpenSSL::TestEC
    
    	irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):002:0> p all_test_methods
    	["test_ECPrivateKey", "test_ECPrivateKey_encrypted", "test_PUBKEY", "test_check_key", "test_derive_key", "test_dh_compute_key", "test_dsa_sign_asn1_FIPS186_3", "test_ec_group", "test_ec_key", "test_ec_point", "test_ec_point_add", "test_ec_point_mul", "test_generate", "test_marshal", "test_sign_verify", "test_sign_verify_raw"]
    	=>
    	["test_ECPrivateKey",
    	 "test_ECPrivateKey_encrypted",
    	 "test_PUBKEY",
    	 "test_check_key",
    	 "test_derive_key",
    	 "test_dh_compute_key",
    	 "test_dsa_sign_asn1_FIPS186_3",
    	 "test_ec_group",
    	 "test_ec_key",
    	 "test_ec_point",
    	 "test_ec_point_add",
    	 "test_ec_point_mul",
    	 "test_generate",
    	 "test_marshal",
    	 "test_sign_verify",
    	 "test_sign_verify_raw"]
    
    	irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):003:0> p filter
    	/\A(?=.*)(?!.*(?-mix:(?-mix:memory_leak)|(?-mix:OpenSSL::TestEC.test_check_key)))/
    	=> /\A(?=.*)(?!.*(?-mix:(?-mix:memory_leak)|(?-mix:OpenSSL::TestEC.test_check_key)))/
    
    	irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):004:0> method = "test_check_key"
    	=> "test_check_key"
    	~~~
    
    	The intention here is to exclude the `test_check_key` test case.
    	Unfortunately this does not work as expected, because the negative filter
    	is never checked:
    
    	~~~
    	irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):005:0> filter === method
    	=> true
    
    	irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):006:0> filter === "#{suite}##{method}"
    	=> false
    
    	irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):007:0> filter === method || filter === "#{suite}##{method}"
    	=> true
    	~~~
    
    	Therefore always filter against the fully qualified method name
    	`#{suite}##{method}`, which should provide the expected result.
    
    	However, if plain string filter is used, keep checking also only the
    	method name.
    
    	This resolves [Bug #16936].
    	---
    	 tool/lib/test/unit.rb | 12 +++++++++---
    	 1 file changed, 9 insertions(+), 3 deletions(-)
    
    	Filter method names only if filtering method name only
    
    	If sole `filter` option doesn't seem including test case name,
    	match with method name only.
    	And if the filter is a Regexp or String, it never matches method
    	name symbols.
    	---
    	 tool/lib/test/unit.rb | 15 ++++++---------
    	 1 file changed, 6 insertions(+), 9 deletions(-)
    1d29740c
    merge revision(s)...
    nagachika authored
    merge revision(s) 5086c25f,3ff0a0b4: [Backport #16936]
    
    	Properly exclude test cases.
    
    	Lets consider the following scenario:
    
    	~~~
    	irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):001:0> p suite
    	OpenSSL::TestEC
    	=> OpenSSL::TestEC
    
    	irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):002:0> p all_test_methods
    	["test_ECPrivateKey", "test_ECPrivateKey_encrypted", "test_PUBKEY", "test_check_key", "test_derive_key", "test_dh_compute_key", "test_dsa_sign_asn1_FIPS186_3", "test_ec_group", "test_ec_key", "test_ec_point", "test_ec_point_add", "test_ec_point_mul", "test_generate", "test_marshal", "test_sign_verify", "test_sign_verify_raw"]
    	=>
    	["test_ECPrivateKey",
    	 "test_ECPrivateKey_encrypted",
    	 "test_PUBKEY",
    	 "test_check_key",
    	 "test_derive_key",
    	 "test_dh_compute_key",
    	 "test_dsa_sign_asn1_FIPS186_3",
    	 "test_ec_group",
    	 "test_ec_key",
    	 "test_ec_point",
    	 "test_ec_point_add",
    	 "test_ec_point_mul",
    	 "test_generate",
    	 "test_marshal",
    	 "test_sign_verify",
    	 "test_sign_verify_raw"]
    
    	irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):003:0> p filter
    	/\A(?=.*)(?!.*(?-mix:(?-mix:memory_leak)|(?-mix:OpenSSL::TestEC.test_check_key)))/
    	=> /\A(?=.*)(?!.*(?-mix:(?-mix:memory_leak)|(?-mix:OpenSSL::TestEC.test_check_key)))/
    
    	irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):004:0> method = "test_check_key"
    	=> "test_check_key"
    	~~~
    
    	The intention here is to exclude the `test_check_key` test case.
    	Unfortunately this does not work as expected, because the negative filter
    	is never checked:
    
    	~~~
    	irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):005:0> filter === method
    	=> true
    
    	irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):006:0> filter === "#{suite}##{method}"
    	=> false
    
    	irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):007:0> filter === method || filter === "#{suite}##{method}"
    	=> true
    	~~~
    
    	Therefore always filter against the fully qualified method name
    	`#{suite}##{method}`, which should provide the expected result.
    
    	However, if plain string filter is used, keep checking also only the
    	method name.
    
    	This resolves [Bug #16936].
    	---
    	 tool/lib/test/unit.rb | 12 +++++++++---
    	 1 file changed, 9 insertions(+), 3 deletions(-)
    
    	Filter method names only if filtering method name only
    
    	If sole `filter` option doesn't seem including test case name,
    	match with method name only.
    	And if the filter is a Regexp or String, it never matches method
    	name symbols.
    	---
    	 tool/lib/test/unit.rb | 15 ++++++---------
    	 1 file changed, 6 insertions(+), 9 deletions(-)
Loading