status of release QA/Litmus

Hi all,

as the 3.4.0 release is coming very soon I just opened Litmus again this
morning and was a bit shocked that there is more or less nothing in
there.

Although there is some text about Litmus on
http://wiki.documentfoundation.org/QA nobody seems to use it yet.

So some questions,

a) who is testing the RCs? Some work seems still to be done in the wiki
[1],[2], although the amount of testers is pretty low.

b) what are we doing with Litmus? Weeks and months pass by and the whole
relase QA process looks like it is getting no momentum at all as opposed
to most of the other parts in the LO community.

My suggestion is bring Litmus to front. Even if it has some flaws and
tests for just some parts and languages of the product open and announce
it publicly and use it as the only tool to get content in it asap. If
people create an account and see that there is really work going on in
this tool they will stay and contribute more likely than at the moment.

I could live without an release QA process as described in the wiki,
dump Litmus and just work with the time based releases and blocker bugs
in bugzilla as the release criteria. But one way or the other there
needs to be some direction and decissions.

Michael

[1] http://wiki.documentfoundation.org/QA/Testing/Manual_Tests
[2] http://wiki.documentfoundation.org/QA/Testing/Manual_Tests/3.4.0/de

Hi Michael,

Hi all,

as the 3.4.0 release is coming very soon I just opened Litmus again this
morning and was a bit shocked that there is more or less nothing in
there.

Although there is some text about Litmus on
http://wiki.documentfoundation.org/QA nobody seems to use it yet.

Yes, I didn't find the time and energy to push the project more. I was also waiting for an answer about admin rights, but never mind, we can go on without.

So some questions,

a) who is testing the RCs? Some work seems still to be done in the wiki
[1],[2], although the amount of testers is pretty low.

RCs are tested within the language groups without reporting on the wiki. So I agree that the track is somewhat lost but, the issues found go to BZ, which is the most important to my eyes.

b) what are we doing with Litmus? Weeks and months pass by and the whole
relase QA process looks like it is getting no momentum at all as opposed
to most of the other parts in the LO community.

Welcome aboard :slight_smile: We need hands to write the tests, translate them, upload them, manage them, set the test time periods, manage the VCL testool tests, analyze the results, etc., etc.
I'm the super admin of Litmus, but I've no super powers to do all the work, so I would be more than happy to share this and work with whom wants to go on with part of what has to be done.

My suggestion is bring Litmus to front. Even if it has some flaws and
tests for just some parts and languages of the product open and announce
it publicly and use it as the only tool to get content in it asap. If
people create an account and see that there is really work going on in
this tool they will stay and contribute more likely than at the moment.

Before that you need to populate it: write the tests in English (or in German but they have to be translated in English because the tests should be the same for all languages and I can help for that), upload the tests on Litmus, set testing periods, announce it and get people to participate.

I could live without an release QA process as described in the wiki,
dump Litmus and just work with the time based releases and blocker bugs
in bugzilla as the release criteria. But one way or the other there
needs to be some direction and decissions.

Well, that will come, but I felt a bit alone on all this, and as said at the beginning of my mail, I found neither the time nor the energy to push everything further. June should be more lighter for me, so if we find a handful of people to work on that, I'm sure I also find the time and motivation :wink:

And thanks for shaking me, it's going to help me to put myself in it :slight_smile:
Kind regards
Sophie

Hello Michael, Sophie and *,

Hi Michael, On 28/05/2011 10:43, Michael Münch wrote:
> Hi all,
>
> as the 3.4.0 release is coming very soon I just opened Litmus again
> this morning and was a bit shocked that there is more or less
> nothing in there.
>
> Although there is some text about Litmus on
> http://wiki.documentfoundation.org/QA nobody seems to use it yet.

That has something to do with "Power and Time" see below.

Yes, I didn't find the time and energy to push the project more. I
was also waiting for an answer about admin rights, but never mind, we
can go on without.
>
> So some questions,
>
> a) who is testing the RCs? Some work seems still to be done in the
> wiki [1],[2], although the amount of testers is pretty low.

Some people (i hope) in every language. For my self i can say, i install every new version on 2 computers here and work with them.
Till i will find some errors i report this on the (german) discuss lists.
Thats because of the less time i have to fill out the wiki-sides - at the moment.

RCs are tested within the language groups without reporting on the
wiki. So I agree that the track is somewhat lost but, the issues
found go to BZ, which is the most important to my eyes.

+1

>
> b) what are we doing with Litmus? Weeks and months pass by and the
> whole relase QA process looks like it is getting no momentum at all
> as opposed to most of the other parts in the LO community.

As I understand Sophie: The testcases are still there, but disabled.

Welcome aboard :slight_smile: We need hands to write the tests, translate them,
upload them, manage them, set the test time periods, manage the VCL
testool tests, analyze the results, etc., etc. I'm the super admin of
Litmus, but I've no super powers to do all the work, so I would be
more than happy to share this and work with whom wants to go on with
part of what has to be done.

I recieved a mail from Rimas with this text inside:

>>> One more thing: I took away everybody's superadmin rights and only left
>>> them to the "Litmus Admin" account and Yifan. I sent the changed admin's
>>> password to Sophie, so she can grant superadmin rights to whomever it is
>>> necessary. Right now, it seems a bit complicated to allow certain users
>>> to create/edit testcases without making them superadmins, but this might
>>> be rather easy to fix, and if/when fixed, we should once again demote
>>> most of the superadmins and instead give them only the rights necessary
>>> to manage testcases.

This sounds to me like: Only one Person can manage the whole side?
But we need minimal one person for each language to do that.
I guess there should be more admin's.
Some that puts the testcases in there,
some that manage testcases,
some more that analyze the results.

If this works as it looks like, i think the report functions will give us more information than the wiki-test sides.
Litmus provide all information at one place, for many languages.

> My suggestion is bring Litmus to front. Even if it has some flaws
> and tests for just some parts and languages of the product open and
> announce it publicly and use it as the only tool to get content in
> it asap. If people create an account and see that there is really
> work going on in this tool they will stay and contribute more
> likely than at the moment.

Before that you need to populate it: write the tests in English (or
in German but they have to be translated in English because the tests
should be the same for all languages and I can help for that), upload
the tests on Litmus, set testing periods, announce it and get people
to participate.

as i wrote before... Are the testcases still there?

> I could live without an release QA process as described in the
> wiki, dump Litmus and just work with the time based releases and
> blocker bugs in bugzilla as the release criteria. But one way or
> the other there needs to be some direction and decissions.

Well, that will come, but I felt a bit alone on all this, and as said
at the beginning of my mail, I found neither the time nor the energy
to push everything further. June should be more lighter for me, so if
we find a handful of people to work on that, I'm sure I also find the
time and motivation :wink:

May we can define a few thinks first:
Put a contact adress to the wiki page like:
"You want to help us building testcases, manage them or analys? Get a admin account? Feel free to write a mail to: xxx@xxx.org"

Define the basic tests we need to do and what to do if that fails.

And thanks for shaking me, it's going to help me to put myself in it

Please make me a Admin to litmus again, if this is possible.

As i see the necessity of linked example files, can you tell us where to upload files to be linked in the testcases?
Is it possible to upload them onto http://wiki.documentfoundation.org/Litmus?
Is that the same server than http://tcm.documentfoundation.org ?

:slight_smile: Kind regards Sophie

Regards
Volker