Python Threading Lock Example

There are a number of ways to implement a lock, but one of the easiest is just to subclass threading.Thread and add an attribute that references the lock.

The example below shows a threading lock that is used to synchronize printing. When the print functions aren’t coordinated, multiple threads may try to print at the same time, resulting in garbled output. For instance, in this program, the line breaks are sometimes interleaved incorrectly when the lock is taken out:

Hi, I'm Thread-3 and I'm done processing! Value=3
Hi, I'm Thread-1 and I'm done processing! Value=7Hi, I'm Thread-4 and I'm done processing! Value=1
Hi, I'm Thread-2 and I'm done processing! Value=9
Hi, I'm Thread-3 and I'm done processing! Value=16
Hi, I'm Thread-4 and I'm done processing! Value=0
Hi, I'm Thread-3 and I'm done processing! Value=9

Hi, I'm Thread-4 and I'm done processing! Value=0

Incorrect line breaks while printing to stdout is usually not terribly dire, but failing to synchronize threads during other I/O applications can corrupt data and lead to disastrous results.

Now, as promised, here’s an example that implements a simple lock to synchronize printing among threads:

from __future__ import print_function
import threading
from random import randint

class LockingThread(threading.Thread):
        def __init__(self, lock, *args, **kwargs):
                super(LockingThread, self).__init__(*args, **kwargs)
                self.lock = lock

def do_some_work():
        '''Placeholder for doing some actual, important computation'''
        return randint(0,10)**randint(1000,10000) % 20

def output_results():
        current_thread = threading.current_thread()
        for i in xrange(0,100):
                value = do_some_work()
                with current_thread.lock:
                    print("Hi, I'm {0} and I'm done processing! Value={1}"\
                                        .format(, value))

if __name__=='__main__':
        num_threads = 4
        the_lock = threading.Lock()
        for i in xrange(0, num_threads):
                LockingThread(the_lock, target=output_results).start()

With the lock in place, all of the printing goes smoothly, resulting in nice, clean output like this:

Hi, I'm Thread-2 and I'm done processing! Value=0
Hi, I'm Thread-1 and I'm done processing! Value=5
Hi, I'm Thread-2 and I'm done processing! Value=1
Hi, I'm Thread-2 and I'm done processing! Value=12
Hi, I'm Thread-2 and I'm done processing! Value=4
Hi, I'm Thread-2 and I'm done processing! Value=0
Hi, I'm Thread-2 and I'm done processing! Value=9

A few comments on the code:

  • The basic flow of the program is this:  Spawn a few threads (the part under __main__) and have them do some operation (the  do_some_work() function).
  • The lock’s “with” statement acquires and releases the lock. It’s equivalent to calling lock.acquire() before the print statement, and then lock.release() after the print statement has completed.
  • The program imports print from __future__ because Python 2’s print statement isn’t thread safe, even if you put it inside a lock.  Most of the time everything works fine, but occasionally you’ll get extra line breaks.  The new print function introduced in Python 3 doesn’t seem to have this problem, so as long as you import from __future__ everything works as expected.
If you liked this post, say thanks by sharing it:

Quick Guide: Automatically log in to an SSH session without a password prompt

Brief SSH key setup instructions:

From your local box create a hidden folder, .ssh, in your home directory, and restrict the permissions on this folder.  Then, generate the RSA keys with ssh-keygen:

mkdir ~/.ssh
chmod 700 ~/.ssh
ssh-keygen -t rsa -b 4096

When you run ssh-keygen, the command will ask where you want to save the key. Press Enter to accept the default directory (~/.ssh/id_rsa). ssh-keygen will then ask if you want to enter a passphrase to protect the key. For many applications an empty passphrase is perfectly acceptable, but if you are on a system that demands high security, like a production system, read the note below about passphrases. Otherwise, just press enter twice for an empty passphrase.

Ignore the key fingerprint and the “randomart” image that is printed out after ssh-keygen returns. Finally, intsall the public key on the remote machine:

ssh-copy-id [remote username]@[remote host]

After running this command, you will likely be greeted with a message similar to this one:

RSA key fingerprint is f9:b5:78:da:ad:db:98:2f:b4:0c:1c:2e:ec:6b:ee:ca.
Are you sure you want to continue connecting (yes/no)?

This is an expected warning that is displayed because your local computer does not yet know the RSA key fingerprint of the remote host. Hereafter, no authenticity warnings will be shown when you connect to the remote host. Type in “yes” to continue and enter your password when prompted to complete the RSA key setup process.

Try it out

After setting up SSH keys, try to connect to the remote host:

ssh [remote username]@[remote host]

If the RSA keys were setup successfully, the SSH session to the remote host should be initiated without the need to enter a password.

Why use RSA keys?

Setting up RSA keys for authentication allows you to connect to a remote host without entering a password. Typing in your password when connecting to a remote terminal is a hassle, especially if your administrator has implemented needlessly onerous password restrictions. More importantly, executing tasks or checking conditions on a remote host from an automated test script is much easier when there is no need to interact with a password prompt.

About passphrases

Think of the passphrase like a password to unlock your private key (the contents of the ~/.ssh/id_rsa file). If an attacker steals your private key, and the key doesn’t have a passphrase, the attacker will be able to use the key to access any remote hosts that are authorized with that key. Conversely, an RSA key that is protected with a passphrase will need the passphrase to be entered before the key can be used to connect to the remote host. In this way, using a passphrase provides some extra protection. If at some point you realize that your local computer has been compromised, a passphrase might delay the attacker long enough for you to delete your old key from the remote host.

Note that even if you choose to use a passphrase, entering the passphrase each time the computer is booted or an SSH session is initialized generally is not necessary. Most modern Linux distributions will handle the keychain process automatically. You will only be required to enter the passphrase the first time that you use the key. In this sense, adding a passphrase is almost always a good idea, simply because the overhead of adding a passphrase is so low. The flip side to this argument is that if you are using keys on a system where security is not paramount (in a test environment, for example) and where the probability of attack is relatively low, adding a passphrase to your SSH keys may be unnecessary.


Here’s the whole process in review:

bash-4.1$ ssh-keygen -t rsa -b 4096
Generating public/private rsa key pair.
Enter file in which to save the key (/home/username/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/username/.ssh/id_rsa.
Your public key has been saved in /home/username/.ssh/
The key fingerprint is:
b7:52:99:79:43:3b:d6:33:25:23:86:f1:46:ff:d4:da username@localhost
The key's randomart image is:
+--[ RSA 4096]----+
|   ... .+        |
| .  . oo + +     |
|     +. . o . o .|
|    . E  . . . +.|
|  +     S . . =o.|
|     =   . = = .o|
|          o + .  |
|           . .   |
|                 |
bash-4.1$ ssh-copy-id remoteUsername@remoteHost
The authenticity of host 'remotehost (' can't be established.
RSA key fingerprint is f9:b5:78:da:ad:db:98:2f:b4:0c:1c:2e:ec:6b:ee:ca.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'remotehost100.100.100.100' (RSA) to the list of known hosts.
username@remotehost's password:
Now try logging into the machine, with "ssh 'username@remotehost'", and check in:


to make sure we haven't added extra keys that you weren't expecting.

bash-4.1$ssh remoteUsername@remoteHost
If you liked this post, say thanks by sharing it:

GWT, Selenium WebDriver, and the dreaded StaleElementReferenceException

Automating GWT (Google Web Toolkit) applications with Selenium WebDriver can be challenging.  One hurdle is that as page events fire, GWT applications tend to drop existing DOM elements and replace them with new DOM elements that are identical, or very similar to, elements in the old DOM.  Consequently, even simple commands like driver.findElement("someElementId")).click() will fail sporadically with stale element reference exceptions.

The reason is that in the tiny time gap between when the WebDriver finds the element and when the WebDriver tries to click the element, GWT may drop the existing DOM element that you are trying to click and replace it with a new, identical element.  The WebDriver, which is blissfully unaware that GWT has modified the DOM in this way, tries to click on the old element (which is no longer attached to the DOM), and your test fails with a StaleElementReferenceException.

Luckily, this problem is easy enough to fix.  If you are using Selenium WebDriver on a GWT application, you probably already have built up some kind of framework with a utilities class that has static helper methods that you use often.  Drop the method shown below into this class, and replace your existing click methods with this new one.  This method will try to find and then click on the specified WebElement up to the MAX_STALE_ELEMENT_RETRIES count, after which, the exception is propagated to the calling method.  I’ve found that setting the number of retries to 5 is more than enough.

     * Attempts to click on an element multiple times (to avoid stale element
     * exceptions caused by rapid DOM refreshes)
     * @param d
     *            The WebDriver
     * @param by
     *            By element locator
    public static void dependableClick(WebDriver d, By by)
        final int MAXIMUM_WAIT_TIME = 10;
        final int MAX_STALE_ELEMENT_RETRIES = 5;

        WebDriverWait wait = new WebDriverWait(d, MAXIMUM_WAIT_TIME);
        int retries = 0;
        while (true)

            catch (StaleElementReferenceException e)
                if (retries < MAX_STALE_ELEMENT_RETRIES)
                    throw e;
If you liked this post, say thanks by sharing it: