!!

Welcome to Selenium Webdriver Forum

As a web user you can read and search through forum content. Once you have registered and signed you can post topics, post replies, set up watch lists, modify forum settings, and use private messaging.

Author Topic: Memory leak with htmlunit and javascript enabled  (Read 1026 times)

Allen Rohner

  • Web User
Memory leak with htmlunit and javascript enabled
« on: March 28, 2012, 03:15:59 PM »
I'm not sure whether this post belongs on webdriver's group or
 htmlunit's so I'll try here. Using the following code, I get OOM
 exceptions. Sorry, this is clojure code, but it should be
 understandable
 (defn new-htmlunit-driver [ & [{js :js}]]
   (let [driver (new HtmlUnitDriver)]
     (when js
       (.setJavascriptEnabled driver js))
     driver))
 
(dotimes [i 100]
     (with-open [driver (new-htmlunit-driver {:js true})]
       (my-webdriver-test driver)))
 
with-open is a clojure macro, it basically looks like
 try
 {
    body
 
}
 

finally
 {
      driver.close()
 
}
 

I'm certain driver.close() is being called. Without javascript
 enabled, I can get through the loop just fine. With javascript
 enabled, I get an OOM exception at i=5. I would post the full
 exception, but it doesn't appear to be deterministic. The exception is
 always "java.lang.OutOfMemoryError: Java heap space".
 The website I'm testing is pretty basic. I serve prototype.js and
 jquery.js, and visit a page that contains a flash chart. I know this
 report isn't all that useful. Is there anything else I can provide
 that would help?
 
Allen

Ahmed Ashour

  • Web User
Re: Memory leak with htmlunit and javascript enabled
« Reply #1 on: March 28, 2012, 03:16:14 PM »
Dear Allen,
 
The root cause is mostly from HtmlUnit or clojure, but hardly could be from WebDriver.
 
Anyhow, please provide external URL (or your complete web application), along with the clojure/WebDriver code, so we can reproduce the error.
 
Yours,
 Ahmed----
 Blog: http://asashour.blogspot.com
 
________________________________
 From: Allen Rohner <aroh...@gmail.com>
 To: webdriver <webdriver@googlegroups.com>
 Sent: Saturday, June 6, 2009 1:46:12 AM
 Subject: [webdriver] Memory leak with htmlunit and javascript enabled
 
I'm not sure whether this post belongs on webdriver's group or
 htmlunit's so I'll try here. Using the following code, I get OOM
 exceptions. Sorry, this is clojure code, but it should be
 understandable
 
(defn new-htmlunit-driver [ & [{js :js}]]
   (let [driver (new HtmlUnitDriver)]
     (when js
       (.setJavascriptEnabled driver js))
     driver))
 
(dotimes [i 100]
     (with-open [driver (new-htmlunit-driver {:js true})]
       (my-webdriver-test driver)))
 
with-open is a clojure macro, it basically looks like
 try
 {
   body
 
}
 

finally
 {
     driver.close()
 
}
 

I'm certain driver.close() is being called. Without javascript
 enabled, I can get through the loop just fine. With javascript
 enabled, I get an OOM exception at i=5. I would post the full
 exception, but it doesn't appear to be deterministic. The exception is
 always "java.lang..OutOfMemoryError: Java heap space".
 The website I'm testing is pretty basic. I serve prototype.js and
 jquery.js, and visit a page that contains a flash chart. I know this
 report isn't all that useful. Is there anything else I can provide
 that would help?
 
Allen

Simon Stewart

  • Web User
Re: Memory leak with htmlunit and javascript enabled
« Reply #2 on: March 28, 2012, 03:16:24 PM »
Hi Allen,
 "close" merely simulates closing a window. Try using "quit", which
 does more clean up of resources. Even without that, however, if you're
 not keeping a reference to webdriver outside of the loop, you
 shouldn't be leaking memory.
 
I'd be interested to hear if a profiler identified any issues.
 
Regards,
 
Simon
 

- Show quoted text -

Selenium Webdriver

Re: Memory leak with htmlunit and javascript enabled
« Reply #2 on: March 28, 2012, 03:16:24 PM »

Allen Rohner

  • Web User
Re: Memory leak with htmlunit and javascript enabled
« Reply #3 on: March 28, 2012, 03:16:34 PM »
Sorry it took me a while to get back to this problem. I've translated
 the results into regular Java, same results. See below for my code. If
 setJavascriptEnabled is set to false, I make it through the test
 successfully. If set to true, I get OOM heap space around i=7. My
 original clojure code was calling the equivalent of driver.close()
 with the same result. This thread recommended driver.quit(), also same
 result.
 I did notice that the htmlunit faq http://htmlunit.sourceforge.net/faq.html#MemoryLeak,
 it mentions closeAllWindows() should be called; I didn't find this
 anywhere in the webdriver source tree.
 
I haven't had a chance to look at this in a profiler (yet), visualvm
 doesn't work on my OSX laptop. I'll try to look at that soon.
 
import org.openqa.selenium.By;
 import org.openqa.selenium.WebDriver;
 import org.openqa.selenium.WebElement;
 import org.openqa.selenium.htmlunit.HtmlUnitDriver;
 
class WebdriverLeaker
 {
     static WebDriver newDriver(boolean js)
     {
         HtmlUnitDriver driver = new HtmlUnitDriver();
         driver.setJavascriptEnabled(js);
         return driver;
     }
 
    static void run(WebDriver driver)
     {
         driver.get("http://localhost:8080");
         driver.findElement(By.id("recommend-0")).click();
     }
 
    public static void main(String[] args)
     {
         for(int i=0; i < 10000; i++)
         {
             System.out.println(i);
             WebDriver driver = newDriver(true);  // set this to false
 to pass the test
             run(driver);
             // driver.close();
             // driver.quit();
         }
     }
 

}
 

On Jun 8, 4:02 am, Simon Stewart <simon.m.stew...@gmail.com> wrote:
 
- Show quoted text -

Michael Tamm

  • Web User
Re: Memory leak with htmlunit and javascript enabled
« Reply #4 on: March 28, 2012, 03:16:45 PM »
I could not reproduce the problem.
 Could you please provide the HTML of the page which is served at
 http://localhost:8080/ in your code.
 
Regards, Michael
 
2009/6/17 Allen Rohner <aroh...@gmail.com>
 

- Show quoted text -

Simon Stewart

  • Web User
Re: Memory leak with htmlunit and javascript enabled
« Reply #5 on: March 28, 2012, 03:16:57 PM »
Hi,
 I've added a call to "closeAllWindows", just in case we're leaking
 rhino resources from the HtmlUnitDriver. I also wrote a small test
 case that loads a page, performs a simple action that causes JS to run
 and then quits the driver, and ran that in a tight loop for 2 hours.
 Memory usage trended upward a little, but not enough to cause my JVM
 to throw out of memory errors.
 
As Michael says, it'd be interesting to find out what's causing your
 test case to cause so much trouble. I'm pretty sure that Marc would
 also be interested.
 
Regards,
 
Simon
 
On Wed, Jun 17, 2009 at 12:26 PM, Michael
 

- Show quoted text -

Allen Rohner

  • Web User
Re: Memory leak with htmlunit and javascript enabled
« Reply #6 on: March 28, 2012, 03:17:09 PM »
That closeAllWindows patch seems to have made the difference. I just
 updated, and now I'm at i=450. Previously, it was dying around i=7.
 Thanks!
 Sorry, the website I'm testing isn't public yet. I can tell you that
 it's serving prototype, scriptaculus, jQuery, swfobject and some of
 our own javascript. There really isn't a lot of our own JS on the
 page, just tens of lines, and most of that is calls into the JS
 libraries.
 
Thanks for the quick response,
 
Allen
 
On Jun 17, 8:19 am, Simon Stewart <simon.m.stew...@gmail.com> wrote:
 

- Show quoted text -

 

Related Topics

  Subject / Started by Replies Last post
3 Replies
706 Views
Last post February 21, 2012, 01:45:09 PM
by Ana Stavreva
18 Replies
3163 Views
Last post February 21, 2012, 02:03:11 PM
by Mirko Nasato
1 Replies
607 Views
Last post February 21, 2012, 02:10:24 PM
by John and Pip
1 Replies
813 Views
Last post April 02, 2012, 08:20:31 PM
by Mike Riley
2 Replies
631 Views
Last post April 15, 2012, 05:04:43 PM
by bs
1 Replies
1152 Views
Last post June 25, 2012, 07:26:42 AM
by sdarshanam
2 Replies
349 Views
Last post July 24, 2012, 08:21:00 PM
by dlai

Review www.seleniumwebdriver.com on alexa.com