!!

Welcome to Selenium Webdriver Forum

As a web user you can read and search through forum content. Once you have registered and signed you can post topics, post replies, set up watch lists, modify forum settings, and use private messaging.

Author Topic: How about making a beta release before making an official one  (Read 411 times)

Tarun K

  • Web User
Hello all,
 
This is my understanding of how official Selenium releases are made -
 *There are major bux fixes or feature additions > Unit test case execution
 (or may be some thing else) > All look good > Time for new Selenium release.
 *
 
But despite all efforts at times defects like this slip in -
 http://code.google.com/p/selenium/issues/detail?id=2513
 
I was thinking if Selenium could leverage its vast user community before
 making an official release.
 So we still follow the same cycle -
 
*There are major bux fixes or feature additions > Unit test case execution
 (or may be some thing else) > All look good*
 *But a new phase here -*
 *> And now we ask Selenium users to test new release in a time bound manner
 (May be by announcing it on Selenium User group, may be having beta testing
 last for a week or may be less/more time)*
 *
 *
 This would be as simple as Selenium users replacing existing Selenium
 library with new release > Executing their tests and hopefully not finding
 any critical exception due to new library. and if
 *All look good > Time for new Selenium release*
 *
 *
 But if there were a critical error reported by user -
 *Hot fix > Beta testing by Selenium Users > All look good > Time for new
 Selenium release*
 *
 *
 Though I am not sure how long this cycle could go on.
 Thoughts/comments?
 
~tarun

David Burns

  • Web User
Re: How about making a beta release before making an official one
« Reply #1 on: April 12, 2012, 07:19:00 PM »
As we move forward with the project plan of standardisation of the API,
 issues like this are going to be more of a browser issue.
 
If we do see issues like this, and the browser vendor owns the
 implementation like with Google Chrome, should we hold up a release? I for
 one would be extremely against that. So am -1
 
At the moment we are roughly one weekly releases so unless it's a total
 blocker, and as in the build is red, we should carry on as normal. Everyone
 with the commit bit should run tests as often as they can so we may be
 aware of issues.
 
David
  On Nov 10, 2011 5:34 AM, "Tarun K" <tkum...@gmail.com> wrote:
 

- Show quoted text -

Daniel Wagner-Hall

  • Web User
Re: How about making a beta release before making an official one
« Reply #2 on: April 12, 2012, 07:19:14 PM »
I'm not sure what this gains us in a world where every older version is
 available. I believe the current mechanism is as you describe, with one
 additional step: if users have issues, they down grade until the issue is
 resolved. This may be annoying for them, but it's also a workable solution,
 and means that we have more users doing that testing.
 
As we've moved to ~weekly releases, I'm sure a lot of people are also just
 skipping some updates, too.
 On 10 Nov 2011 05:33, "Tarun K" <tkum...@gmail.com> wrote:

Selenium Webdriver

Re: How about making a beta release before making an official one
« Reply #2 on: April 12, 2012, 07:19:14 PM »

Andreas Tolf Tolfsen

  • Web User
Re: How about making a beta release before making an official one
« Reply #3 on: April 12, 2012, 07:19:28 PM »
* Also sprach Daniel Wagner-Hall <dawag...@gmail.com>:
 
> I'm not sure what this gains us in a world where every older version
 > is available. I believe the current mechanism is as you describe, with
 > one additional step: if users have issues, they down grade until the
 > issue is resolved. This may be annoying for them, but it's also a
 > workable solution, and means that we have more users doing that
 > testing.
 > As we've moved to ~weekly releases, I'm sure a lot of people are also
 > just skipping some updates, too.
 


I agree pretty much agree with what's being said here.
 Maintaining different release channels for Selenium would be _a huge_
 undertaking considering the complexities of this project.  Considering
 that we're not pushing Selenium to any package management systems for
 OSes, this is a workable solution.
 
But:
 
I'm unsure what the “@Beta” thing means in all of this.  Does it mean we
 will continue to maintain and release important fixes for the stable
 (previous) version, effectively introducing two release channels?  Or
 does it mean the users will just have to upgrade to the latest bleeding
 edge to get patches for bugs?
 
This sounds suboptimal to me.  I realize that Selenium is not an
 ordinary project and cannot offer the same LTS release schemes as other
 traditional software (like in the traditions of Debian) without major
 effort, and frankly there is no need to do so for the core project
 either, but there is a chance we'll be communicating things less clearly
 by separating between final- and beta versions.
 
I'd personally like to see Selenium reach such a stable state that a
 user wouldn't have to upgrade in between minor (x.Y.z) releases, but
 with things moving so fast within the browser automation world, I
 realize that might be a far stretch.
 
I'd like to hear the reasons for introducing “@Beta”, as it seems to me
 that Daniel Wagner-Hall's described approach above (“if things doesn't
 work for you, please downgrade and use the previous version until we get
 time to fix it and release a newer version”) seems more than adequate to
 me.  It's true that this isn't optimal in terms of user-friendliness,
 but then this is all just really a symptom of our testing environment
 not being good enough.
 
Whenever we catch a bug like above, we should actively be finding ways
 to test for that in the future.  That said, we also need to make it
 easier and more stable to run the full stack of tests.  Or, in the
 least, ensure that our CI environment is functioning correctly.
 
Thanks!!
 
--
 Andreas Tolf Tolfsen
 Systems Developer, Core Systems
 Opera Software ASA

Simon Stewart

  • Web User
Re: How about making a beta release before making an official one
« Reply #4 on: April 12, 2012, 07:19:41 PM »
The @Beta annotation is a marker that essentially says "this is an
 experimental feature that is liable to change, but we'd love some
 feedback." Our policy for removing APIs that are not marked beta is
 that they are deprecated for a release, and then removed in the next.
 There is no such promise for @Beta features, allowing us to iterate
 through possible APIs until we find that we and our users like.
 As a concrete example, we've marked the window control APIs as @Beta
 in this release as we're not entirely sure we've got the user-facing
 pieces right. Let's see what the reception is and how implementors
 feel about it.
 
Simon
 
On Thu, Nov 10, 2011 at 10:52 AM, Andreas Tolf Tolfsen
 

- Show quoted text -

Tarun K

  • Web User
Re: How about making a beta release before making an official one
« Reply #5 on: April 12, 2012, 07:19:54 PM »
It's definitely dev's call.
 And chrome driver may be one long running exceptional issue. But I hope we
 want users to upgrade to new release every week (or so). And not keep going
 back to previous release because of some issues.

semperos

  • Web User
Re: How about making a beta release before making an official one
« Reply #6 on: April 12, 2012, 07:20:08 PM »
To throw another consideration in...
 
In my opinion, x.x.0 releases of software are expected to be stable. I
 haven't found Selenium's releases to be buggy for my own use cases, but I
 think this isn't a point so easily shrugged off, and should be kept on the
 "back burner" as Selenium goes forward.
 
It is precisely because of the complexity of this project that creating
 pre-release builds on specific timetables (which encourage people to focus
 their testing efforts) are a Good Thing. While these releases aren't in OS
 package managers, they are posted to places like the central Maven
 repository, and telling people to go back to a previous version definitely
 does not instill confidence in the project.
 
-Daniel

Paul Hammant

  • Web User
Re: How about making a beta release before making an official one
« Reply #7 on: April 12, 2012, 07:20:23 PM »
You don't know it, and won't necessarily agree given I'm pointing it out,
 but you're making a case for better regression tests before release. Only.
 
- Paul
 
On Tue, Nov 15, 2011 at 9:18 AM, semperos <daniel.l.grego...@gmail.com>wrote:
 

- Show quoted text -

Tarun K

  • Web User
Re: How about making a beta release before making an official one
« Reply #8 on: April 12, 2012, 07:20:37 PM »
No, I would definitely agree to it. It is indeed for better regression
 testing....

 

Related Topics

  Subject / Started by Replies Last post
0 Replies
105 Views
Last post February 22, 2012, 01:48:05 PM
by Paollo Kuccinni
0 Replies
127 Views
Last post February 22, 2012, 05:34:07 PM
by Paollo Kuccinni
When is Beta 2 out?

Started by pnak...@gmail.com Webdriver - Selenium 2.0

7 Replies
510 Views
Last post April 04, 2012, 04:17:00 PM
by Brian Fields
Selenium 2.0 beta 1 released

Started by Simon Stewart « 1 2 » Webdriver - Selenium 2.0

15 Replies
1223 Views
Last post April 09, 2012, 03:46:28 PM
by gadhet
When is Beta 2 out?

Started by pnak...@gmail.com Webdriver - Selenium 2.0

7 Replies
469 Views
Last post April 09, 2012, 04:46:43 PM
by Brian Fields
4 Replies
1079 Views
Last post April 11, 2012, 02:30:49 PM
by Tristan
8 Replies
337 Views
Last post April 12, 2012, 06:27:32 PM
by Tarun K

Review www.seleniumwebdriver.com on alexa.com