[TriLUG] Do a lot of you program in Python?

Kevin Hunter Kesling hunteke at earlham.edu
Wed Jan 8 19:37:20 EST 2014


At 7:08pm -0500 Wed, 08 Jan 2014, Greg Brown wrote:
> And since I'm on the topic that thing is forked process management. I
> have a network consisting of ~10,000 devices and in order to run a
> job against all of those devices and have it complete within a
> reasonable amount of time I need to fork about 15 or 20 processes at
> a time, ssh or telnet to each one, do whatever needs to be done to
> the devices then move on to the next one.

I've heard varying opinions on Python's process management capabilities. 
  Personally, I think they're excellent, and built in to the core.  No 
external libraries.  If you need Expect capabilities, check out 
pexpect[1].  I have not used it, other than rudimentarily.  It's not a 
core lib, so YMMV.

I note that you wrote "process management".  In Python, the difference 
between threads and processes is an important detail of which to be 
aware.  Python has what is known as the "GIL" (global interpreter lock), 
which means that though you can use threads from an API standpoint, you 
will only have concurrent activity if you use latent actions (like I/O). 
  For example, threads are useless in my work as the majority of what I 
do is CPU bound.  Processes are completely separate (as per what you'd 
expect from fork()), and so you /can/ have truly concurrent activity, 
but at the expense of needing to figure out communication.  If you're 
only using text and stdio/stderr, it should be straightforward.

Also of note is that you /can/ use os.fork(), like you'd expect from 
your C training, but that you may appreciate the object-oriented 
approach that the Python libraries provide.  And, like all things, I'm a 
fan of "choose the right tool for the job".  Just because Python has the 
object-oriented version, does not necessarily mean it's the best choice. 
  The rest will be a matter of getting into your head the specific 
workflow that Python utilizes in terms of process management, which is a 
lot cleaner than what you might do in C, but also a mind job if you grew 
up in Perl and C (as I did).

Finally, I suggest you take advantage of the 'with' statement with 
context managers.  This is one bit of syntactic sugar that I've found 
can really clean up sections of code.

That all said, I think the available resources are excellent, as is 
freenode's #python if you need help.

Cheers,

Kevin

[1] https://github.com/pexpect/pexpect


More information about the TriLUG mailing list