QA Meeting Minutes - 2014-12-03

Hiya,

We had a great QA Meeting on Wednesday. For complete meeting minutes,
ACTION items, etc.., as well as our agenda for the next meeting,
please see the wiki pages here:
https://wiki.documentfoundation.org/QA/Meetings/2014/December_03
https://wiki.documentfoundation.org/QA/Meetings/2014/December_17

Key topics from the meeting:

* Our next Bug Hunting Session will be December 19-21:
https://wiki.documentfoundation.org/BugHunting_Session_4.4.0_RC1

* All bugs/questions about building LibreOffice or unit tests should
go to the Developer's Mailing List (*not* to Bugzilla).

Details here about all categories of NOTOURBUG:
https://wiki.documentfoundation.org/QA/BugReport#Not_all_bugs_go_to_Bugzilla

Blurb one can copy-into Bugzilla when resolving as NOTOURBUG:
https://wiki.documentfoundation.org/QA/BugTriage#Step_3._Check_Information

* If a bug is fixed on the Master branch (or in Fresh), but is still
present on Fresh/Still, it can be helpful to tag the bug and bibisect
it, in addition to resolving it as WFM. I'll send out a separate email
to the QA list with some advice from the devs.

Big thanks to all of you guys!

Best,
--R

Hi,

* If a bug is fixed on the Master branch (or in Fresh), but is still
present on Fresh/Still, it can be helpful to tag the bug and bibisect
it, in addition to resolving it as WFM. I'll send out a separate email
to the QA list with some advice from the devs.

IMHO a bug should not be resolved as WFM then, it is still a bug in the
release reported. If it is fixed in master then even better, hopefully
it was one commit that can be cherry-picked, so a bibisect indeed would
be welcomed.

  Eike

if we don't mark it as FIXED WFM how do we distinguish a NEW bug without a
fix from a NEW bug which is FIXED in master but not yet present in the
stable branch?

Sure -- and this highlights a granularity problem we have with current
bug reports: We only have a binary switch to indicate if a bug has
been fixed or not.

We currently use the Version field to point to the oldest LO version
on which we can reproduce a bug. If we had a 2nd Version field, we
could use that to point to the newest LO version on which we can
reproduce, but IMHO that's still not as granular as I'd like. I've
some cursory ideas for how we can more clearly indicate which
environments (LO version + OS + etc..) reproduce a given bug, which we
can revisit after the Bugzilla Migration.

I think the scenario Robinson depicted is logical.

1- QA finds that a NEW bug in stable branch is somehow fixed in master so it
marks it as WFM

2- a bibisect attempt is done to identify the committ that fixed it

3- once the bibisect is done, a backportRequest is added to whiteboard and
devs are contacted

While we do have other work in the QA task pool (triaging incoming
bugs, bibisecting open regressions, etc..), let's see how many fixed
bugs get tagged with bibisectRequest, and see how quickly we're able
to address them. If the number is small, it shouldn't represent much
of an additional burden for QA.

If we just tag a bug with bibisectRequest and resolve it as WFM, I
think bibisecters might ignore it, so I suggest that we tag it
bibisectRequest + backportRequest at the same time. To help us keep
track of these bugs, I've added two entries to the Useful Queries
page:
https://wiki.documentfoundation.org/QA/Bugzilla/Useful_Queries#Common_Lists_of_Bugs
1) Backport requests that need bibisection
2) Backport requests that do not need bibisection (i.e. needs dev attention)

How do we decide which bugs to tag? I suggest that for now we only tag
backportRequests if the bug was reported against an active Still or
Fresh branch.

So to put that into context with our current builds, here's my proposal:

If you test a bug reported against 4.3 and find
1) It's fixed in master, AND
2) It's not fixed in 4.3.5.1 (the latest build on the 4.3 branch)

Then change status -> WFM, and add "bibisectRequest backportRequest"
to the Whiteboard.

For all other cases, if it's fixed in master, then (just) change status -> WFM.

Best,
--R