[FRPythoneers] Are lists threadsafe?

Jim Baker jbaker at zyasoft.com
Mon Mar 8 11:40:37 MST 2004


Yes, but...

>From the Python Cookbook 6.1:

"However, adding threads to a Python program to speed it up is often not a
successful strategy. The reason for this is the Global Interpreter Lock
(GIL), which protects Python's internal data structures. This lock must be
held by a thread before it can safely access Python objects. Without the
lock, even simple operations (such as incrementing an integer) could fail.

Therefore, only the thread with the GIL can manipulate Python objects or
call Python/C API functions. To make life easier for programmers, the
interpreter releases and reacquires the lock every 10 bytecode instructions
(a value that can be changed using sys.setcheckinterval). The lock is also
released and reacquired around I/O operations, such as reading or writing a
file, so that other threads can run while the thread that requests the I/O
is waiting for the I/O operation to complete. However, effective
performance-boosting exploitation of multiple processors from multiple
pure-Python threads of the same process is just not in the cards. Unless the
CPU performance bottlenecks in your Python application are in C-coded
extensions that release the GIL, you will not observe substantial
performance increases by moving your multithreaded application to a
multiprocessor machine."

The GIL noted above will indeed protect reads & writes to lists &
dictionaries, to prevent corruption to the underlying data structures.
Queues are really for distributing work to threads.  However, you will need
to use some sort of locking scheme if you want atomicity for a larger set of
ops.  I would turn to the Python Cookbook for some good recipes there.  The
ReadWriteLock they have is not bad.

Finally, the received wisdom is that concurrent I/O is significantly more
scalable and easier with an async event framework like Twisted.  (You're
probably not doing something compute bound with Python, I would imagine...)
However, we do have to work with threads when working with say mod_python on
Windows and some globals around for database caching.

- Jim


-----Original Message-----
From: frpythoneers-bounces at lists.community.tummy.com
[mailto:frpythoneers-bounces at lists.community.tummy.com]On Behalf Of Jim
Vickroy
Sent: Friday, March 05, 2004 2:41 PM
To: Front Range Python Users -- General Discussion
Subject: RE: [FRPythoneers] Are lists threadsafe?


To the best of my knowledge, the only thread-safe container is a Queue; that
may very-well meet your needs.

-----Original Message-----
From: frpythoneers-bounces at lists.community.tummy.com
[mailto:frpythoneers-bounces at lists.community.tummy.com]On Behalf Of
Dhruva B. Reddy
Sent: Friday, March 05, 2004 2:34 PM
To: FR Pythoneers
Subject: [FRPythoneers] Are lists threadsafe?


I would like to have a list to which elements are appended by multiple
threads.  Would I be responsible for making sure each thread locks the
list, or is that taken care of for me?

Thanks,
Dhruva
_______________________________________________
This message sent by the FRPythoneers mailing list.
Unsubscribe: echo unsubscribe |
FRPythoneers-request at lists.community.tummy.com
URL: http://lists.community.tummy.com/mailman/listinfo/frpythoneers

_______________________________________________
This message sent by the FRPythoneers mailing list.
Unsubscribe: echo unsubscribe |
FRPythoneers-request at lists.community.tummy.com
URL: http://lists.community.tummy.com/mailman/listinfo/frpythoneers




More information about the FRPythoneers mailing list