Wednesday, July 9, 2014

Python WebDriver proof of concept

As my first Python project, I decided to see if I could read a table containing selenium actions from a confluence page.  This is just a proof of concept and is not an automation framework...but it helped me both learn a little bit about Python and see how it could be used to drive a selenium based automation framework.  Here's the code...yeah, it's just ugly text since I couldnt get the code block to work for this post .  To get a pretty-page look at it, PyCharm is a great Python IDE (see my previous post for quick notes on downloading it):

Code

__author__ = 'rhondam'
#
# barebones proof of concept python project...not pretty but it works
# This project reads a table from a confluence page (page name is hard coded...can only have one table
# on the page).  It then starts webdriver and iterates over the stored table executing webdriver commands based
# on data read from the table.
#
# It is hacked to only access elements via ids and uses simple if/elif.  If I take this further, it would need to be
# objectized and broken out into methods.  There is no exception handling...so it just dies if the element isnt present.
#
# Table needs to be in the following format:
# <table>
#   <tbody>
#     <tr>
#       <th>url</th>
#       <th>element</th>
#       <th colspan="1">element type</th>
#       <th>action</th>
#       <th colspan="1">data</th>
#       <th>expected result</th>
#     </tr>
#     <tr>
#       <td>
#         <a href="http://www.google.com">www.google.com</a>
#       </td>
#       <td> </td>
#       <td colspan="1"> </td>
#       <td> </td>
#       <td colspan="1"> </td>
#       <td> </td>
#     </tr>
#     <tr>
#       <td> </td>
#       <td>gbqfq</td>
#       <td colspan="1">id</td>
#       <td>send_keys</td>
#       <td colspan="1">Tommy Heinsohn</td>
#       <td> </td>
#     </tr>
#     <tr>
#       <td colspan="1"> </td>
#       <td colspan="1">gbqfb</td>
#       <td colspan="1">id</td>
#       <td colspan="1">click</td>
#       <td colspan="1"> </td>
#       <td colspan="1"> </td>
#     </tr>
#   </tbody>
# </table>
#
from selenium import webdriver
from collections import defaultdict
from github import Github
from xmlrpc.client import ServerProxy
import configparser
import lxml.etree
import lxml.html
import os

# execute selenium action
def web_action(web_cmd):
    element = web_cmd[1]
    element_type = web_cmd[2]
    action = web_cmd[3]
    data = web_cmd[4]
    expected_result = web_cmd[5]
    print('in web_action')
    print(element, element_type, action, data, expected_result)

    if(action == 'send_keys'):
        if(element_type == 'id'):
            driver_element = driver.find_element_by_id(element)
            driver_element.send_keys(data)
    elif(action == 'click'):
        if(element_type == 'id'):
            driver_element = driver.find_element_by_id(element)
            driver_element.click()

# enter confluence xmlrpc link and credentials
#    sample xmlrpc link: https://confluence.mycorp.com/rpc/xmlrpc
xmlrpc = input('Enter confluence xmlrpc link: ')
s = ServerProxy(xmlrpc)
user_name = input('Enter confluence username: ')
password = input('Enter confluence password: ')
auth_token = s.confluence2.login(user_name, password)
page = s.confluence2.getPage(auth_token, "~"+user_name, "Test Data Google")
content = page["content"]

# get the confluence page content
doc = lxml.html.fromstring(content)

# create dictionary for reading in table data
table_data = defaultdict(list)

print(table_data)

# read table from page
row_count = 0
for row in doc.iter('tr'):
    td = []
    td_count = 0
    for td_item in row.iter('td'):
        td.append(td_item.text_content())
        td_count += 1
    table_data[row_count].append(td)
    row_count += 1

print('td_count:  ', td_count)
print('row_count: ', row_count)
print(table_data)

# start up webdriver and execute actions from table
driver = 0
for key, value in table_data.items():
    # start webdriver and get past header row
    if(key == 0):
        driver = webdriver.Firefox()
        continue
    print(key)
    print(value[0][0])
    if(value[0][0] != '\xa0'):
        driver.get(value[0][0])
    else:
        web_action(value[0])

driver.quit()

How to run it...

I keep it simple and use PyCharm to edit and run the project.   So that it doesnt run so fast that I cant visually verify the results, I sometimes set a debug point at line 124 (the web_action call) and step over that so I can see that the google page comes up and Tommy Heinsohn search works properly.

Improvements needed

This is a real brute force project and to list all the improvements it would take to make it into a real data driven application would take me the rest of the night...and I'm tired.

Tuesday, July 8, 2014

Pyton - download, install and get Pycharm IDE

 Last week I decided to take a look a Python since it's been around quite awhile and I've never used it.  I have found it to have a simple install (with a little caveat around getting the registry keys properly configured in order to  be able to download helpful modules).  The language itself is one that a programmer can quickly pick up the basics too, although it has advanced features that will take awhile to master (at least that is so for me).  I am a self learned Java/ant hacker and I find I like the scripting ability of Python.  However, what makes it really fun and quick to come up to speed on is having a solid IDE to develop the scripts with.  That is where Pycharm comes in.  With it's debugging, auto type, auto format and other features, it makes coming up to speed with Python a much simpler task.  Below is how I got started....

Download Python

Download Python: https://www.python.org/download/
  • by default, installs to c:\Python## where ## is version number, ex Python34
  • installer updates/add HKEY_LOCAL_MACHINE\SOFTWARE\PYTHON registry entry

Adding HKEY_CURRENT_USER Registry entry

Some Python module installers look in the HKEY_CURRENT_USER registry entry for the Python settings so need to get the info there
  • http://choorucode.com/2012/04/28/python-version-not-found-in-registry-error/
    1. open regedit
    2. navigate to Computer->HKEY_LOCAL_MACHINE->SOFTWARE->Python
    3. right click 3.4 -> Export
    4. save as .reg file on your system, ex. c:\Users\yourname\python3.4.reg
    5. edit .reg file and change all HKEY_LOCAL_MACHINE to HKEY_CURRENT_USER
    6. open cmd box as administrator
    7. cd to where .reg file was saved
    8. install using regedit command
      1. c:> regedit python3.4.reg
        • keys should be successfully added to registry
        • verify using regedit that Computer->HKEY_CURRENT_USER->SOFTWARE->Python exists

Installing Python modules

Module install can be done using pip or by double clicking the module if it is an installer.

Modules/Packages

libxml

XML processing library combining libxml2 and libxslt libraries and providing a safe and convenient access to the libraries using the ElementTree API.
  • need to install manually on windows....dont try installing using PyCharm->Settings since it requires building it which you cant do (ok you might be able to, but it wont be easy)
  • get the libxml installer

PyCharm IDE

PyCharm is a Python IDE put out by JetBrains
Adding packages (ex. PyGitHub) using PyCharm Settings
  • start PyCharm
  • File -> Settings -> Project Interpreter
  • click on the +
  • in search bar, type PyGitHub
  • click Install Package