Hooking Android up to the (Selenium) Grid

When I set myself a Q4 goal of getting a small suite of Mozilla’s WebQA tests running on Android I didn’t think it would be much work. The AndroidDriver has been around for some time, and from what I understood it was pretty mature – I had even run a couple of tests locally against it and they worked well. What I found was there were a couple of important gotchas…

The first was that the port forwarding that’s necessary to run tests on an Android emulator or device binds to localhost. This isn’t a problem so long as the test commands originate from the same machine as the emulator/device is attached to, however that doesn’t make it very useful for Selenium Grid, where we want to run several emulators/devices remotely.

I was suprised that this issue had not come up before… I didn’t think I could possibly be the first person to want to hook up an AndroidDriver to Selenium Grid..! After searching around for a solution I decided to leave things for a while to take care of other priorities. I was then pulled back into the problem when my fellow Mozillian Raymond Etornam came across the same issue and was looking for a solution.

Raymond and I approached Dounia Berrada, who heads up the AndroidDriver efforts at Google, and as a result of this discussion, Raymond raised an issue in the Selenium project. Dounia got to the bottom of the issue and found a very simple solution, using socat to listen on another port on all interfaces and forward traffic to the localhost bound port forward configured by ADB. The Selenium project’s wiki has been updated with the solution.

The second issue I had was registering the AndroidDriver with Selenium Grid. This is simply a case of posting the correct JSON to the server, and is something that’s done for you when launching a Selenium server. The quick solution would have been to just start a Selenium server with the details for the Android node. This feels like too much of a hack, plus I didn’t really want the server running constantly when it’s not really doing anything other than consuming resources.

I decided to write a very simple Python package that would take a few arguments, construct the necessary JSON and post it to the Selenium Grid. I’ve released this as FlynnID (there’s a reference to ‘The Grid’ if you look hard enough) and it’s available on PyPI, with the source code on github.

A more long term solution will be to have the AndroidDriver register with a Selenium Grid when it’s started. If anyone is keen to look into this please get in touch!

So with these issues resolved, I was able to finally put together the first mobile test suite. I migrated the Input tests to WebDriver, did a lot of cleaning up, and split them into desktop/mobile tests. There’s only one mobile test there at the moment, but now we have everything in place we can go full speed into test development! If you’re interested in helping us out to write more tests or improve our frameworks/infrastructure then the WebQA page on QMO is a great place to start.

Oh, I’ve also added a cute Android icon  to Selenium Grid, which will be available from version 2.16.0.

This entry was posted in Mozilla, QMO, Selenium. Bookmark the permalink.

24 Responses to Hooking Android up to the (Selenium) Grid

  1. stackedsax says:

    Thanks for this info — I got the android emulator registered on the grid thanks to your flynnid python script.

    I had one question about a comment you make here, though. Are you sure this is quick and easy:

    The quick solution would have been to just start a Selenium server with the details for the Android node.

    You want to tell grid where the android emulator is running, and the port is part of that equation. As far as I can see, if you start the selenium server and set the port parameter, it will also run on that exact same port. So there will be a conflict between the emulator running on port 8081 and the selenium server you’re trying to start on port 8081.

    This is what I ran into before I started asking over in the IRC channel, and I just want to see if I’m missing something totally obvious.

    The fix, it seems to me, would be to have the selenium server allow ports per browser definition. I tried setting port per browser, but it didn’t seem to change anything. I didn’t dive into the code to see if it should or shouldn’t work, but since it failed and there’s no mention of that feature anywhere I assumed it’s not actually a feature.

    Here’s an example of what I’m describing:

    https://gist.github.com/1587576

    • Dave says:

      Hey stackedsax,

      Good point. I’m sure I had this running locally, although I can’t see an obvious way around your issue. You could try starting an additional node with only the Android capability, and set the remoteHost property to match the actual host/port of the Android device/emulator. I believe this is what is sent to the hub when registering a node, and although it usually reflects the host/port properties for the node you should be able to override it.

      Also, have you tried using FlynnID?

      Cheers,
      Dave

  2. stackedsax says:

    flynnid is what I’m using, yes. It threw some error with python 3.2, but was fine with 2.7.

    I’ll try setting remoteHost in the json file and see if will work. Thanks for the suggestion.

  3. Alex says:

    Thanks for an article! but I can’t register my android from another machine, only from hub’s owner. I wrote something like this:
    flynnid –nodeport=8080 –browsername=android –browserver=2.3.3 –platform=ANDROID –nodehost=”nodeIP” –hubhost=”hostID”
    Grid says “yes, i see you”, but after a couple of minutes the connection is broken. It’s strange, because when I do the same from grid’s owner it works. Could you help understand it, please?

    • Dave says:

      You need to make sure that the hub host is able to communicate with the Android instance. Try pinging the node IP, or connecting to the port on the node from the hub host.

  4. Y says:

    Thanks for this! Great article. Took some time to setup everything but works great. I run my webdriver tests through Eclipse/TestNG/Maven and now run remotely.

    I use RemoteWebDriver to connect to http://remotehost:8081/wd/hub/

  5. Y says:

    Hi,
    As I said before I could connect to other machine through Remote webdriver. It’s a direct connection to the emulator and not through the Selenium hub itself.

    I ran flynnid with json file and I see Android on the grid.

    Now when I run through TestNG the scripts from remote machine how do I tell the Selenium Grid to redirect test to the emulator?

  6. Timmy says:

    Thank you so much. Helped me set up android and iPhone drivers both running on a VM to a selenium grid hosted elsewhere. Cheers!

  7. Amanuel says:

    Hello Dave,

    I tried using your FlynnID and it works great with an iPhone (port 3001) and Android device (port: 8080).
    However when I try to use the same port 8080 with two Android devices it appears to have a conflict. One device is registered but the other times out and is removed from the Grid2 (localhost:4444/grid/console.)

    do you know a way around this?
    I tried changing the port number in HttpdService wtithin the android app but then am hit with org.openqa.selenium.WebDriverException: java.lang.NoClassDefFoundError: org.openqa.selenium.logging.LogEntries

    Any help would be much appreciated.

    Cheers
    Amanuel

    • Amanuel says:

      I should add that I am trying to attach real devices to Selenium grid running on my localhost.

      I do have access to the devices since when I use Remote Web Driver and the specific device ip addresses it works fine.

      I could just not use Grid and instead run through the devices sequentially but the aim is to have the tests run on actual devices in parallel.

      Thanks again

      • Dave says:

        Hey Amanuel, thanks for commenting. I’m not sure I understand the issue. Are you saying that you’re unable to register two devices running WebDriver? This should be possible, so long as you specify the IP address and port. You should be able to register multiple nodes with the same port just as long as the IP address differs. Perhaps you could raise an issue at https://github.com/davehunt/flynnid/issues with as much detail as possible? Thanks again!

  8. James says:

    I am having difficulty getting this set up. Python script seems to run successfully and
    android node is listed. when I try and make connection to the device using line
    Driver = new RemoteWebDriver(new Uri(“http://remoteMachine:4444/wd/hub”), capability);
    I get the following error

    An exception of type ‘System.InvalidOperationException’ occurred in WebDriver.dll but was not handled in user code

    Additional information: Error forwarding the new session cannot find : {platform=ANDROID, browserName=android, version=}

    Tests are written in c#.
    Connection is made when pointing to local port forwarder
    Driver = new RemoteWebDriver(new Uri(“http://localhost:8080/wd/hub”), capability);

    • Dave says:

      I’ve not seen this issue myself, but perhaps there is a mismatch between your node and capabilities. Perhaps version cannot be empty?

  9. Sergey says:

    Hello! I have a problem with running selenium script on android device using Grid 2. I think that the main reason here is in port forwarding issue as adb tool works only with localhost. Actually, if I put a real node IP into config file, port forwarding doesn’t work for me. RemoteWebDriver starts random browser on the node PC while test execution. Only if I set localhost as a node host, selenium sends commands to my device. Do you have any idea how to fix this problem?

    • Dave says:

      Hi Sergey,

      If you have a device running on the same network with an IP address then you shouldn’t need to do any port forwarding. It’s been a while since I’ve worked on this, so if you’re having issues I would recommend asking on the Selenium users group.

      • Sergey says:

        I use Selenium Grid, device is connected through the guest wi-fi network, so it can’t be accessed directly from PC connected to the internal network. As it described in this doc https://code.google.com/p/selenium/wiki/AndroidDriver there should be used port forwarding to send selenium commands from local tcp port (that is used by grid node) to the real device’s one. The main problem here occurs when we try to run our tests remotely. adb server process binds to the loopback address, so it can’t be accessed via external address. An we have no opportunity to use any IP address (except localhost) in our node configuration. Solution is written here http://rxwen.blogspot.com/2009/11/adb-for-remote-connections.html. adb tool should be fixed to allow remote connections. And I’m wondering how you run your tests using grid without port forwarding and adb fix.

  10. Nic says:

    Hey, great article, helped a lot!

    One thing im still not 100% on… If I want to set up multiple android devices, of the same version, but for e.g one is a tablet and ones a phone, how can I specify which one to run the test on?

    Cheers!

    • Dave says:

      I would recommend asking this in the Selenium users group. As far as I know, there’s no way to specify device type yet. The version is just a string, so the other alternative is to differentiate in the version number.

    • Sergey says:

      You can extend DefaultCapabilityMatcher to add custom comparison parameters (e.g. device id) and run it with your grid hub. Next you can assign your custom parameters to the grid nodes to identify what node should get selenium commands according to device’s id. Finally, you can assign this parameter to your test and set an appropriate capability during driver initialization.

  11. Milind says:

    Hi Dave, I tried to use flynnid AndroidDriver is getting registered, I see the cute little android Icon on hub, however I get mesage on hub “DEfaultRemoteProxy unknown version, JSONObject["value"] not found

    and get the following exception:
    Selenium2LibraryFatalException: org.openqa.selenium.WebDriverException: Error forwarding the new session Error forwarding the request no protocol: /wd/hub/session/57039fec-1bae-4dfa-bdf5-0857bc71a22d

    Regards,
    Milind

  12. Milind says:

    Adding this information for benefit of others,

    I was able to workaround a solution with following combination, can run selenium tests on device Galaxy Tab2(4.2.2) and Galaxy S3(4.1.1)

    selenium-server-standalone-2.31.0.jar ( does not work with 2.33 or 2.35)
    Android server APK 2.21.0

    JSON to register nodes: config.json

    {
    “hub”: {
    “host”: “selenium-host.com”,
    “port”: 4444
    },
    “nodes”: [{
    "host": "selenium-node.com",
    "port": 6666,
    "browser": {
    "name": "android",
    "version": "4.2.2"
    },
    "platform": "ANDROID"
    },{
    "host": "selenium-node",
    "port": 5555,
    "browser": {
    "name": "android",
    "version": "4.1.1"
    },
    "platform": "ANDROID"
    }]
    }

    #setup port forwarding on selenium-node

    >>adb -s 4.1.1_device forward tcp:8080 tcp:8080
    >>socat TCP-LISTEN:5555,fork TCP:localhost:8080

    >>adb -s 4.2.2_device forward tcp:8081 tcp:8080
    >>socat TCP-LISTEN:6666,fork TCP:localhost:8081

    # start hub on selenium-hum

    # register node to grid from selenium-node
    flynnid config.json

    Dave, Could you take a look if flynnid is broken on 2.35? I see message on hub “DEfaultRemoteProxy unknown version, JSONObject["value"] not found

    Regards,
    Milind

    • Dave says:

      I don’t think this is an issue with FlynnID as the registration is successful. Comparing a FlynnID registered node with a self registering remote, the only difference I see if the ‘id’ is missing from the FlynnID configuration. If this is causing an issue then it should be pretty easy to troubleshoot and fix. I don’t currently have a suitable device to test against though.

      I also noticed that the following Selenium issue describes a problem with the latest Selenium Grid and AndroidDriver releases: http://code.google.com/p/selenium/issues/detail?id=5655

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">