Automate Data Entry – How to Create a Selenium bot

Hey there! This article is a step by step tutorial where you will learn to create a selenium bot that will automate any data entry task you like. Instead of opening a website and enter your login details, you can create a selenium bot to do this for you. In this instance, you will learn how to log in to a site with a bot.

I will show step by step how to get your bot working. Plus some aspects you should avoid, to prevent a website from blocking your bot. 

You will create the bot using selenium. In case you are not familiar with Selenium, Selenium is a library that will help you to launch and control a web browser. Using Selenium you can automatically fill in forms and simulate mouse movements and clicks. Find out more about selenium

Le’s get started with our bot!

1. How to install Selenium

First step to automate data entry is installing the selenium library. You can install the library using the pip tool, which will manage all installed libraries.

You can install selenium executing the following command from your terminal:

>> pip install selenium

To run selenium you also need a driver, which is basically a browser. You could just use your already installed browser or download a driver. You can download a driver following the link below:

• Chrome: https://sites.google.com/a/chromium.org/chromedriver/downloads
• Edge: https://developer.microsoft.com/en-us/microsoft-edge/tools/webdriver/
• Firefox: https://github.com/mozilla/geckodriver/releases
• Safari: https://webkit.org/blog/6900/webdriver-support-in-safari-10/

Make a note where you save this file, you will need it in the following steps

2. Open the web browser

By now, you should have selenium installed and the driver file. Time to start adding some code to our bot. First step is opening the login site. In this instance, I will use the aliexpress login site: https://login.aliexpress.com/.

from selenium import webdriver

browser = webdriver.Chrome('./chromedriver') #Replace with driver filename
browser.get("https://login.aliexpress.com/")

Please note the driver file and your script should be in the same folder, just for simplicity. The code in line 3 is creating the browser using the Chrome class, and as a parameter you need to pass the location of your driver file. Therefore, you should replace ./chromedriver‘ with the your driver filename.

3. Data Entry into a Web Form

Once you get your bot to open the login site, next step is entering your details, username and password, in the webform. There are two steps to achieve this:

• Find the component or widget where you want to enter the details.
• Enter the details

See these steps in action below:

user_input = browser.find_element_by_id('fm-login-id')

user_input.send_keys("<replace with your email> ")

password_input = browser.find_element_by_id('fm-login-password')

time.sleep(1)

password_input.send_keys("<replace with your password> ")

As you can see, in the code above, I used the method find_element_by_id, and as a parameter you need to pass the html widget id. This method will return the widget from the HTML DOM document with the given passed id.

In case you don’t know the component id, you can find out using the “Inspect” function of the Chrome Browser. You can access it just by double clicking the component, and select Inspect from the context menu:

Is this enough to login? I am afraid not. The reason is that most websites want to avoid bots login in and extracting information. Therefore website owners put mechanisms in place to detect bots and block them. Usually adding captchas and other widget that require specific human intervention.

What can you do then? One option is simulating human behaviour. For example, if you run your script, you will notice that the keys were entered really fast. The website will pick up this, detect that a bot could be accessing the site, and automatically add more widgets to block it. So you need to enter the keys as a human would do, one by one and with a small milliseconds delay between each key. Check out how to do that below:

user_input = browser.find_element_by_id('fm-login-id')

username = "<replace with your username>"

for letter in username:
    user_input.send_keys(letter)
    wait_time = random.randint(0,1000)/1000
    time.sleep(wait_time)

password_input = browser.find_element_by_id('fm-login-password')

time.sleep(1)

password = "<replace with your password>"

for letter in password:
    password_input.send_keys(letter)
    wait_time = random.randint(0,1000)/1000
    time.sleep(wait_time)

5. Simulate Sliding Movements

Keep in mind that each website has different mechanisms, and therefore behaves differently when it comes to block a bot. This specific website will include a slider if it suspects a bot is accessing the website.

However that shouldn’t stop the bot. Using selenium you can also simulate sliding movement, which basically involves press a widget, hold, move and release. Here are the steps to slide the slide bar:

• First you will need to find the slide bar in the document, and get its width, so you know where to release.
• Find the slide bar button that you should press.
• Create the sliding movement. In selenium, you can simulate movement using ActionChains

slidebar_width = browser.find_element_by_css_selector("#nc_1__scale_text .nc-lang-cnt").size['width']

    time.sleep(2)
  
    slider_button = browser.find_element_by_css_selector('#nc_1_wrapper #nc_1_n1z')

    sliding_move = ActionChains(browser)
    sliding_move.click_and_hold(slider_button).move_by_offset(slidebar_width,0).release().perform()
    
    time.sleep(2)

As you can see in the code above, another selenium method available to find components is find_element_by_css_selector. This method allows you to search component by their classes as well, which is really useful when the widget doesn’t have an id.

6. Click Sign In Button

And lastly, to log in in this website, the bot will need to press the sign in button. First, as before, you need to find the button component in the html. Once you found it, call the click() method and selenium will click the button.

signin_button = browser.find_element_by_css_selector(".fm-login .fm-button")

signin_button.click()

Conclusion

To summarise, in this tutorial we have learned how to automate data entry into a log in web form using selenium. The principle to create a bot is quite simple, first find the widget on which you want to perform an action, using find_element_by_id or find_element_by_css method, and then click and enter your text.

Keep in mind some website have mechanisms in place to block bots, so you should simulate human behaviour as closely as you can.

Hope you enjoy the article. Thanks for reading and supporting this blog! 🙂

More Interesting Articles

How to open a web browser in python

What is the purpose of the python class initializer?

Python Indentation Error Simply Explained

How to Return Multiple Values in Java