PHP Developers Network

A community of PHP developers offering assistance, advice, discussion, and friendship.
 
Loading
It is currently Sun Nov 23, 2014 9:27 am

All times are UTC - 5 hours




Post new topic Reply to topic  [ 12 posts ] 
Author Message
PostPosted: Mon Oct 04, 2010 11:41 am 
Offline
DevNet Evangelist

Joined: Tue Dec 21, 2004 6:00 pm
Posts: 6259
Location: Winnipeg
I'm curious, but when you discover a bug due to a missed conditional or whatever and you add that conditional and assumingly you add the test, do you leave a comment alongside the testcase or wherever to indicate the test addressed an issue? Sort of a way of keeping precise track of bugs and issues which might not be logged in a bugtracker?

Cheers,
Alex


Top
 Profile  
 
PostPosted: Mon Oct 04, 2010 11:51 am 
Offline
DevNet Master
User avatar

Joined: Fri Jan 18, 2008 1:36 am
Posts: 3549
Location: Israel, ME
If the issue was logged in an issue tracker or any other documentation, then I add a short comment saying something like "Fixes issue #{issue-number}". If there is no formal issue report, then I just leave it as is since there is nothing to refer to (the test should be enough to explain what it's testing against)


Last edited by Eran on Mon Oct 04, 2010 11:59 am, edited 1 time in total.

Top
 Profile  
 
PostPosted: Mon Oct 04, 2010 11:58 am 
Offline
Site Admin
User avatar

Joined: Tue Dec 23, 2003 3:10 am
Posts: 11470
Location: Toronto
What Pytrin said, however, when ammending the tests I will usually add a comment refering to the issue number.


Top
 Profile  
 
PostPosted: Mon Oct 04, 2010 12:51 pm 
Offline
DevNet Evangelist

Joined: Tue Dec 21, 2004 6:00 pm
Posts: 6259
Location: Winnipeg
Quote:
If the issue was logged in an issue tracker or any other documentation, then I add a short comment saying something like "Fixes issue #{issue-number}". If there is no formal issue report, then I just leave it as is since there is nothing to refer to (the test should be enough to explain what it's testing against)


That is what I thought was going to be the answer.

Cheers,
Alex


Top
 Profile  
 
PostPosted: Wed Oct 06, 2010 6:56 am 
Offline
DevNet Master

Joined: Wed Feb 11, 2004 4:23 pm
Posts: 4872
Location: Palm beach, Florida
Same. I know Jenk (not try to speak for him) tries not to use any issue tracker, he uses incomplete test cases to "track" an issue, or a failing test to "track" a bug. I do use an issue tracker and sometimes I will put the bug number in a comment, but typically only if the test isn't obvious. The "bug" was more like a missing feature, I'll try to make it blend in with my other tests (no comment). If its hard to understand why that test is there, I'll put the comment so I don't decide to remove the test later when I don't recognize it.

Its almost always better to use an assertion message / good method name than a comment. Would you rather get an error about "assertion failed on line X"? Or an assertion that describes in plain English what broke? The goal is to have your assertion messages be so crystal clear on their own that you'll never need to use a debugger while refactoring code, when you edit some code and it breaks a behavior it should be crystal clear what you edited, what it broke, why it broke it, etc.. All the whos/whats/whens/ and wheres should be obvious from the test runner output alone, without requiring a developer to necessarily read the test or production code.

"Anything that is important to be in a test should be in the test. If its not important to be in a test, its important thats its not in the test" - http://www.amazon.com/xUnit-Test-Patter ... 0131495054

^ Golden rule to live by


Top
 Profile  
 
PostPosted: Sat Oct 09, 2010 4:23 pm 
Offline
DevNet Master
User avatar

Joined: Mon Sep 19, 2005 6:24 am
Posts: 3587
Location: London
Yeah, we don't use an issue tracker in my office - though we do use a story board which could be considered an issue tracker but with a subtle difference that it is feature/value driven, not bug. We don't have anything in the tests to indicate what story they are from, however we do use Acceptance tests (which directly link to the Acceptance Criteria of a given story) and Scenario style testing to make the tests readable. Such as the test class could be called WhenUpdatingAJobStatusToComplete and the test methods for JobShouldNotAcceptFurtherUpdates and JobShouldNotShowInIncompleteJobsList etc.

We found that putting "bug 1234" in a comment merely frustrated us as we'd see it, not understand it and now we'd need to open up the bugtracker to reference it. Putting atleast an explanation resolved this. Then we found the need for the bug number just disappeared and was being added on fewer and fewer occasions. Eventually it was one of the first steps to us not needing a bug tracker at all. :)

p.s. my current role is purely .NET and we use the fabulous NBehave and SpecFlow to allow us to write really meaningful acceptance tests for both us and the customer, so our tests are literally readable in the Given ... When ... Then format. :)


Top
 Profile  
 
PostPosted: Sat Oct 09, 2010 5:23 pm 
Offline
DevNet Master

Joined: Wed Feb 11, 2004 4:23 pm
Posts: 4872
Location: Palm beach, Florida
That should read as ".. When ... Then" format. Not "when then format". (there is a school of thought to name your tests along the format of ['this' should happen when 'that' happens])


Top
 Profile  
 
PostPosted: Sun Oct 10, 2010 3:26 pm 
Offline
DevNet Master
User avatar

Joined: Mon Sep 19, 2005 6:24 am
Posts: 3587
Location: London
Not sure I follow what you mean, but as an example of what I mean...

Given I am an Administrator
When I create a new user with the username "Jenk"
Then I should see "Jenk" in the user list

:)


Top
 Profile  
 
PostPosted: Sun Oct 10, 2010 8:33 pm 
Offline
DevNet Master

Joined: Wed Feb 11, 2004 4:23 pm
Posts: 4872
Location: Palm beach, Florida
Right, the tests names are written in the format of "when ___ then ___". As opposed to you saying "when i write a test, then I do this" (just clarifying a grammatical ambiguity)


Top
 Profile  
 
PostPosted: Mon Oct 11, 2010 9:06 am 
Offline
DevNet Master
User avatar

Joined: Mon Sep 19, 2005 6:24 am
Posts: 3587
Location: London
I'm with you now. However that's not how we do it, we treat each test class/case as a scenario, as I described. :)


Top
 Profile  
 
PostPosted: Tue Oct 12, 2010 2:06 pm 
Offline
DevNet Master

Joined: Wed Feb 11, 2004 4:23 pm
Posts: 4872
Location: Palm beach, Florida
Not sure what I said wrong? I said "when then", but you use "given when then"?? Here's how I organize mine, any scenario names should go in the name of the class, not repeated in the test methods. If a test method is like whenIDoThisThatShouldHappen() that is a smell to me; instead I like to see a test method called ShouldDoThat() in a class named WhenIDoThis.

Xunit testing patterns outlines different organization methods and mentions that none is better than others, just different pros/cons. They are test case per class, test case per feature, and test case per scenario. For example Zend Framework uses test case per class. They'll have a lot of test methods in one big test case. I find that easier for someone not familiar with the tests to find something. On the other hand test case per scenario makes it hard to find where a given test might be, but makes the tests easier to manage in my opinion.

Also I think using an issue tracker is not useful for bespoke development, but is useful for "off the shelf software", in my opinion. When you're doing off the shelf software, you have to be prepared to only implement a fraction of all ideas, and its really hard to keep track of which ones are most popular among the users, etc... This is where an issue tracker helps. I agree putting the issue # into the test is worthless in almost all cases. I've done it before, when a test was really hard to understand. Obviously its wiser to just make the test easier to understand...


Top
 Profile  
 
PostPosted: Tue Oct 12, 2010 2:35 pm 
Offline
DevNet Master
User avatar

Joined: Mon Sep 19, 2005 6:24 am
Posts: 3587
Location: London
We're on the same page, apologies for the misunderstanding.

My examples of 'Given ... When ... Then ...' are for SpecFlow scenarios which are written literally as above (which then map to NBehave bindings)

Other tests unit etc are named just as you describe :)


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 12 posts ] 

All times are UTC - 5 hours


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
Powered by phpBB® Forum Software © phpBB Group