Sunday 27 April 2008

Selenium Grid - Why is this the way of the future?

Update: If you are looking for a Selenium Grid Tutorial I have put it here


I have been working with Selenium Grid since Philippe Hanrigou and others at ThoughtWorks released it at the end of last year. I knew then that it was going to become the standard way for automated GUI web application testing.

As an application it has great potential since it is has all the best bits of Selenium but it does multiple browsers in parallel. Suddenly your testing that used to take n units of time to run are taking 1/n to run. As a practitioner you can get things done quicker, as a manager you can get things done for cheaper! Everyone is a winner.

The other thing that makes the grid work well, and this is a thing that makes Selenium Remote Control work brilliantly, is the ability to use any language to control the test. People use Java, Ruby, .Net, Python and many more to create their test scripts and run it with the appropriate client. If you have the appropriate client application like TestNG for Java or DeepTest for Ruby or PNUnit for .Net you can start running tests with one script in parallel. The one limitation toof using these test tools is that you only really have one browser application, like Firefox, being tested at a time. This is because you only have one instance of the Selenium object in your code.

The way that I have got around this is to have many instances of the Selenium object in my code that each control a certain type. This type could be Internet Explorer 7 on Vista or Firefox 2 on XP.

An example of this would be (Note: I use c# to create my tests and then run them with NUnit)

[Setup]
public void setup(){
//Vista IE7 Instance
VistaIE7selenium =
new DefaultSelenium("hub", 4444, "IE7 on Vista", "http://testserver");

//XP Firefox2 Instance
XPFirefox2selenium = new DefaultSelenium("hub",4444 , "Firefox2 on XP",
"http://testserver");
XPFirefox2selenium.Start();
}


[Test]
public void GridTest(){
VistaIE7selenium.Open("/index.html");
XPFirefox2selenium.Open("/index.html");
}

As you can see it has the ability to create multiple instances of Selenium and then run them. This then leaves the handling of the Selenium Remote Control up to the Grid. As Philippe says, "It allows you to gridify your tests".

The negatives of Selenium Grid is that its still a work in progress. I know, thats hardly a reason to call it a negative. There are also a number of bugs but its growing section of OpenQA.org so give it a chance! I have 7 Selenium Remote Control instances running against my Grid (all virtualised with MS Virtual Server and WMWare) and I haven't had many issues. I have hooked Selenium Grid into our CruiseControl.NET server to test all the different browsers. I still have a lot of work to do with this but I know that it will be worth it in the end!

If you want to automate web testing and you want to do it efficiently use Selenium Grid! It will make your life a lot easier!

21 comments:

Anonymous said...

Hello!
Thankx. May I add the url? (http://selenium-grid.openqa.org/)
Do you use Bromine (http://en.wikipedia.org/wiki/Bromine_for_selenium), too?

David Burns said...

Please do use the URL! I do not mind anyone linking my blog or main site.

I don't use Bromine. The company I work for creates marketing analytics software. We feed our results into our database from RC and then "Dogfood" the results through a Reporting Service add-on to Excel.

Vamsi said...

Hi ,
i'm using selenium rc with python.To reduce the execution time of my test suite i want to move my suite to selenium grid.

But the problem is i was not able to figure it out the way to do it in python.

Is there any way to do it in python?Please share your knowledge if you are aware of it.

David Burns said...

Vamsi,

Once you have set up the grid (http://selenium-grid.openqa.org/) has a quick guide then its just a simple case of changing your tests to point at the Hub. The hub will then push the message out to the RC Nodes.

I don't code in Python (yet) but have found some code and tweaked it. You can find it below.


def setUp(self):
self.selenium1 = selenium('hubaddress', 4444, '*firefox', 'http://testaddress')
self.selenium.start()

self.selenium2 = selenium('hubaddress', 4444, '*iexplore', 'http://testaddress')
self.selenium.start()

def testLinks(self):
selenium1 = self.selenium1
selenium2 = self.selenium2

selenium1.open('/test.html')
selenium2.open('/test.html')


Or if you want just have one Selenium object and use a tool for running your tests in parallel. Doing a quick google search I found http://www.parallelpython.com/

I hope that helps!

Unknown said...

hi!,
i am using selenium IDE ...before 1 months i started...i don't know much about it ...but i want to run the same script created in selenium IDE ..in selenium RC....could u plz help me how i can manage all n how i should start developing my script in both......i developed more than 10 scripts in IDE n want to run same in RC
my best wishes for u

David Burns said...

Amita,

I will be releasing my first RC tutorial in the next few days and it will show you how to use scripts created in Selenium IDE.

If you subscribe to the RSS feed you will see when it is available straight away without having to keep revisiting the site for the change.

David

Pavan said...

hi!,
i am using selenium IDE ...

1. How to pass a variable into the Selenium scripts i.e. passing a case number or username as a variable so that you can replace that with any other value without having to change the script.

2. Ability to read the data files such as flat text files. This ability may or may not be there. I am mainly interested in seeing if we can pass the event data from a flat file that the Selenium script can read and then add those events to a case.

Actually i want to automate some modules of my application.
Could all off u plz help me and give some other information that will help full for me to automate the application.

David Burns said...

Hi Pavan,

Unfortunately you can not do Data-Driven Tests with Selenium IDE.

I have started writing Selenium Remote Control tutorials and once I have created the Data-Driven Tutorial you will be able to start doing what you want.

I suggest you work through the Selenium Remote Control Tutorial that I have just put up so that you can get a base understanding of it.

David

Unknown said...

sir,
i am waiting for ur tutorial ...how i will subscribe to rss?

n plz tell me browser status bar,progress bar, win32 pop up dialogs(save as ,delete),select context menu item ,right click mouse to invoke context menu button r supported by selenium or not .....if not how can i handle that while writing script.

one more problem is there in my script.....four text boxes r there n if u fill all four information correctly...next button should enable...but it's not disable after entering four information through selenium...same thing if i will do manually it will work fine.

well i am waiting for ur reply
have a nice day
regards
amita

David Burns said...

Hi Amita,

The tutorial you originally asked for is on my site (http://www.theautomatedtester.co.uk/)

On writing the script for you, if you would like that please email me at info at theautomatedtester dot co dot uk and we can discuss things further.

David

Anonymous said...

I am afraid the following calls will not allow you to run the test cases concurrently, because all calls to selenium are blocking/synchronize call:

selenium1.open('/test.html')
selenium2.open('/test.html')

Please correct me if I am wrong. Thanks.

David Burns said...

In the example I showed its not 100% concurrent because each step waits for the previous step to finish but its moving between different browsers.

Like I said in the original post you can use a parallel testing app like TestNG,PNUnit or parallel python to do pure concurrency but that then limits you to only using 1 browser.

Anonymous said...

Is Select Flex API works with this?

David Burns said...

I don't test Flex applications so can not comment. Hopefully someone reading this blog will know the answer!

Anonymous said...

I am not sure whether my comment is valid or not but my query is related to Python and Selenium RC.
I can run test case using python and selenium-rc now I have a test suite and have more that 200 test cases.
How do I run a test suite (say more than one) using python and selenium-rc.
To more specific I will take a n example. My first test case is user authentication and I have python script for that. Second I have display listing based on serach paramters provided by user and the precondition is I have to run user authntication test case before executing second test case. How do I do that using Python and Selenium-rc.

Unknown said...

Hi David, this isn't exactly in relation to Grid but I did notice you commenting a lot about running concurrent tests with pnunit on the same browser with selenium RC.

This is exactly what I need but can find no real documentation on pnunit so am a little lost at how to achieve it.

Basically, I need to run concurrent tests with slightly differing parameters (like user login details). I can do this outside of a test framework by kicking off threads which load an appdomain with the required data in the datastore, this is then used in a test.

Trouble is I need a unit test framework to support this type of theng and have no idea how to setup pnunit to do this.

Any advice would be greatly received.

Bill.

David Burns said...

Bill,

I have tried to use PNunit for concurrent testing and well, I broke it in about 10 seconds.

After having a look at the code decided that it would be easier to roll my own. I designed it to work with grid and did it at work so can't share it out.

I would suggest having a look at how you could get some RowTest and PNUnit to work if you can get it going.

Anonymous said...

I have developed scripts using Selenium in PHP and I want to run those scripts using Selenium Grid but not getting how to do this. Can anyone help me in resolving this issue.

Anonymous said...

I have developed scripts using Selenium in c# and I want to run those scripts using Selenium Grid.
I installed it on my computer and iam able to run the demo in selenium grid.Now i dont understand how to link my test suite in selenium RC with selenium Grid na drun the tests concurrently?
Can some one please help?

Unknown said...

Hi!
Your post is useful to get into the basics of Selenium-Grid.

I have a group of selenium tests coded in jsp and I use the html test suite to run them in selenium RC. I use the following command:
java -jar selenium-server.jar -htmlsuite "browsername" "baseurl" "testsuite name" "test result name".

I want to 2 such html suites parallely. I tired using Selenium Grid but could'nt figure out how to run the html suite.
Please share how to do it?

Thanks,
Nivedita

Aman said...

Even i want to know what "Nukalaus "asked...

"I have developed scripts using Selenium in c# and I want to run those scripts using Selenium Grid.
I installed it on my computer and iam able to run the demo in selenium grid.Now i dont understand how to link my test suite in selenium RC with selenium Grid na drun the tests concurrently?
"

Waiting for thr response.

thanks,
aman
amankashyap1223@yahoo.com