Python Cookbook, 2^nd Ed.

Recipes from the Python Community

Alex Martelli, Anna Martelli Ravenscroft, David Ascher

Publisher: O'Reilly, 2005 , 807 pages

ISBN: 0-596-00797-3

Synopsis:

Toggle Synopsis

Covers Python 2.3 & 2.4

Python is well-known as a robust and elegant language, with support for both functional and object-oriented programming styles. When you choose to use Python, however, you get more than just a language-you also become part of an incredibly enthusiastic and supportive community of Python programmers who reach out to help each other learn and grow in their use of the language. Python Cookbook, 2nd Edition is just one tangible manifestation of that Python community. In it, you'll find practical solutions to many common Python programming problems. These solutions come from the community, from Python programmers who for many years have contributed to the online Python Cookbook hosted by ActiveState at http://aspn.activestate.com/ASPN/Cookbook/Python. These are battle-tested solutions; contributed, reviewed, enhanced, and now brought together in book form. Here is just a sampling of the many topic areas covered by the more than 300 recipes in this book:

• Searching and sorting
• Manipulating text
• Working with files
• Object-oriented programming
• Multithreaded programming
• System administration
• Web programming
• Network programming
• Processing XML
• Distributed programming
• Debugging and testing
• Metaprogramming

Solutions in this book are organized by topic area. All of the chapters begin with introductions written by well-known Python luminaries such as: Facundo Batista, David Beazley, Donn Cave, Fred L. Drake, Jr., Paul F. Dubois, Mark Hammond, Raymond Hettinger, Jeremy Hylton, Fredrik Lundh, Mark Lutz, Andy McKay, Gustavo Niemeyer, Tim Peters, Paul Prescod, Aaron Watters, and Greg Wilson.

This book is a treasure trove of useful code for all Python programmers, from novices to advanced practitioners. It's a gift, from the community to you. By purchasing this book, you are also giving back to the community: 25 percent of all royalties from this book are donated to the Python Software Foundation.

Table of Contents:

Toggle Table of Contents

1. Text
1. Processing a String One Character at a Time
2. Converting Between Characters and Numeric Codes
3. Testing Whether an Object Is String-like
4. Aligning Strings
5. Trimming Space from the Ends of a String
6. Combining Strings
7. Reversing a String by Words or Characters
8. Checking Whether a String Contains a Set of Characters
9. Simplifying Usage of Strings' translate Method
10. Filtering a String for a Set of Characters
11. Checking Whether a String Is Text or Binary
12. Controlling Case
13. Accessing Substrings
14. Changing the Indentation of a Multiline String
15. Expanding and Compressing Tabs
16. Interpolating Variables in a String
17. Interpolating Variables in a String in Python 2.4
18. Replacing Multiple Patterns in a Single Pass
19. Checking a String for Any of Multiple Endings
20. Handling International Text with Unicode
21. Converting Between Unicode and Plain Strings
22. Printing Unicode Characters to Standard Output
23. Encoding Unicode Data for XML and HTML
24. Making Some Strings Case-Insensitive
25. Converting HTML Documents to Text on a Unix Terminal
2. Files
1. Reading from a File
2. Writing to a File
3. Searching and Replacing Text in a File
4. Reading a Specific Line from a File
5. Counting Lines in a File
6. Processing Every Word in a File
7. Using Random-Access Input/Output
8. Updating a Random-Access File
9. Reading Data from zip Files
10. Handling a zip File Inside a String
11. Archiving a Tree of Files into a Compressed tar File
12. Sending Binary Data to Standard Output Under Windows
13. Using a C++-like iostream Syntax
14. Rewinding an Input File to the Beginning
15. Adapting a File-like Object to a True File Object
16. Walking Directory Trees
17. Swapping One File Extension for Another Throughout a Directory Tree
18. Finding a File Given a Search Path
19. Finding Files Given a Search Path and a Pattern
20. Finding a File on the Python Search Path
21. Dynamically Changing the Python Search Path
22. Computing the Relative Path from One Directory to Another
23. Reading an Unbuffered Character in a Cross-Platform Way
24. Counting Pages of PDF Documents on Mac OS X
25. Changing File Attributes on Windows
26. Extracting Text from OpenOffice.org Documents
27. Extracting Text from Microsoft Word Documents
28. File Locking Using a Cross-Platform API
29. Versioning Filenames
30. Calculating CRC-64 Cyclic Redundancy Checks
3. Time and Money
1. Calculating Yesterday and Tomorrow
2. Finding Last Friday
3. Calculating Time Periods in a Date Range
4. Summing Durations of Songs
5. Calculating the Number of Weekdays Between Two Dates
6. Looking up Holidays Automatically
7. Fuzzy Parsing of Dates
8. Checking Whether Daylight Saving Time Is Currently in Effect
9. Converting Time Zones
10. Running a Command Repeatedly
11. Scheduling Commands
12. Doing Decimal Arithmetic
13. Formatting Decimals as Currency
14. Using Python as a Simple Adding Machine
15. Checking a Credit Card Checksum
16. Watching Foreign Exchange Rates
4. Python Shortcuts
1. Copying an Object
2. Constructing Lists with List Comprehensions
3. Returning an Element of a List If It Exists
4. Looping over Items and Their Indices in a Sequence
5. Creating Lists of Lists Without Sharing References
6. Flattening a Nested Sequence
7. Removing or Reordering Columns in a List of Rows
8. Transposing Two-Dimensional Arrays
9. Getting a Value from a Dictionary
10. Adding an Entry to a Dictionary
11. Building a Dictionary Without Excessive Quoting
12. Building a Dict from a List of Alternating Keys and Values
13. Extracting a Subset of a Dictionary
14. Inverting a Dictionary
15. Associating Multiple Values with Each Key in a Dictionary
16. Using a Dictionary to Dispatch Methods or Functions
17. Finding Unions and Intersections of Dictionaries
18. Collecting a Bunch of Named Items
19. Assigning and Testing with One Statement
20. Using printf in Python
21. Randomly Picking Items with Given Probabilities
22. Handling Exceptions Within an Expression
23. Ensuring a Name Is Defined in a Given Module
5. Searching and Sorting
1. Sorting a Dictionary
2. Sorting a List of Strings Case-Insensitively
3. Sorting a List of Objects by an Attribute of the Objects
4. Sorting Keys or Indices Based on the Corresponding Values
5. Sorting Strings with Embedded Numbers
6. Processing All of a List's Items in Random Order
7. Keeping a Sequence Ordered as Items Are Added
8. Getting the First Few Smallest Items of a Sequence
9. Looking for Items in a Sorted Sequence
10. Selecting the nth Smallest Element of a Sequence
11. Showing off quicksort in Three Lines
12. Performing Frequent Membership Tests on a Sequence
13. Finding Subsequences
14. Enriching the Dictionary Type with Ratings Functionality
15. Sorting Names and Separating Them by Initials
6. Object-Oriented Programming
1. Converting Among Temperature Scales
2. Defining Constants
3. Restricting Attribute Setting
4. Chaining Dictionary Lookups
5. Delegating Automatically as an Alternative to Inheritance
6. Delegating Special Methods in Proxies
7. Implementing Tuples with Named Items
8. Avoiding Boilerplate Accessors for Properties
9. Making a Fast Copy of an Object
10. Keeping References to Bound MethodsWithout Inhibiting Garbage Collection
11. Implementing a Ring Buffer
12. Checking an Instance for Any State Changes
13. Checking Whether an Object Has Necessary Attributes
14. Implementing the State Design Pattern
15. Implementing the "Singleton" Design Pattern
16. Avoiding the "Singleton" Design Pattern with the Borg Idiom
17. Implementing the Null Object Design Pattern
18. Automatically Initializing Instance Variablesfrom _ _init__ Arguments
19. Calling a Superclass _ _init_ _ Method If It Exists
20. Using Cooperative Supercalls Concisely and Safely
7. Persistence and Databases
1. Serializing Data Using the marshal Module
2. Serializing Data Using the pickle and cPickle Modules
3. Using Compression with Pickling
4. Using the cPickle Module on Classes and Instances
5. Holding Bound Methods in a Picklable Way
6. Pickling Code Objects
7. Mutating Objects with shelve
8. Using the Berkeley DB Database
9. Accesssing a MySQL Database
10. Storing a BLOB in a MySQL Database
11. Storing a BLOB in a PostgreSQL Database
12. Storing a BLOB in a SQLite Database
13. Generating a Dictionary Mapping Field Names to Column Numbers
14. Using dtuple for Flexible Accessto Query Results
15. Pretty-Printing the Contents of Database Cursors
16. Using a Single Parameter-Passing StyleAcross Various DB API Modules
17. Using Microsoft Jet via ADO
18. Accessing a JDBC Database from a Jython Servlet
19. Using ODBC to Get Excel Data with Jython
8. Debugging and Testing
1. Disabling Execution of Some Conditionals and Loops
2. Measuring Memory Usage on Linux
3. Debugging the Garbage-Collection Process
4. Trapping and Recording Exceptions
5. Tracing Expressions and Comments in Debug Mode
6. Getting More Information from Tracebacks
7. Starting the Debugger Automatically After an Uncaught Exception
8. Running Unit Tests Most Simply
9. Running Unit Tests Automatically
10. Using doctest with unittest in Python 2.4
11. Checking Values Against Intervals in Unit Testing
9. Processes, Threads, and Synchronization
1. Synchronizing All Methods in an Object
2. Terminating a Thread
3. Using a Queue.Queue as a Priority Queue
4. Working with a Thread Pool
5. Executing a Function in Parallel on Multiple Argument Sets
6. Coordinating Threads by Simple Message Passing
7. Storing Per-Thread Information
8. Multitasking Cooperatively Without Threads
9. Determining Whether Another Instance of a ScriptIs Already Running in Windows
10. Processing Windows Messages Using MsgWaitForMultipleObjects
11. Driving an External Process with popen
12. Capturing the Output and Error Streamsfrom a Unix Shell Command
13. Forking a Daemon Process on Unix
10. System Administration
1. Generating Random Passwords
2. Generating Easily Remembered Somewhat-Random Passwords
3. Authenticating Users by Means of a POP Server
4. Calculating Apache Hits per IP Address
5. Calculating the Rate of Client Cache Hits on Apache
6. Spawning an Editor from a Script
7. Backing Up Files
8. Selectively Copying a Mailbox File
9. Building a Whitelist of Email Addresses From a Mailbox
10. Blocking Duplicate Mails
11. Checking Your Windows Sound System
12. Registering or Unregistering a DLL on Windows
13. Checking and Modifying the Set of Tasks Windows Automatically Runs at Login
14. Creating a Share on Windows
15. Connecting to an Already Running Instance of Internet Explorer
16. Reading Microsoft Outlook Contacts
17. Gathering Detailed System Information on Mac OS X
11. User Interfaces
1. Showing a Progress Indicator on a Text Console
2. Avoiding lambda in Writing Callback Functions
3. Using Default Values and Bounds with tkSimpleDialog Functions
4. Adding Drag and Drop Reordering to a Tkinter Listbox
5. Entering Accented Characters in Tkinter Widgets
6. Embedding Inline GIFs Using Tkinter
7. Converting Among Image Formats
8. Implementing a Stopwatch in Tkinter
9. Combining GUIs and Asynchronous I/O with Threads
10. Using IDLE's Tree Widget in Tkinter
11. Supporting Multiple Values per Row in a Tkinter Listbox
12. Copying Geometry Methods and Options Between Tkinter Widgets
13. Implementing a Tabbed Notebook for Tkinter
14. Using a wxPython Notebook with Panels
15. Implementing an ImageJ Plug-in in Jython
16. Viewing an Image from a URL with Swing and Jython
17. Getting User Input on Mac OS
18. Building a Python Cocoa GUI Programmatically
19. Implementing Fade-in Windows with IronPython
12. Processing XML
1. Checking XML Well-Formedness
2. Counting Tags in a Document
3. Extracting Text from an XML Document
4. Autodetecting XML Encoding
5. Converting an XML Document into a Tree of Python Objects
6. Removing Whitespace-only Text Nodesfrom an XML DOM Node's Subtree
7. Parsing Microsoft Excel's XML
8. Validating XML Documents
9. Filtering Elements and Attributes Belonging to a Given Namespace
10. Merging Continuous Text Events with a SAX Filter
11. Using MSHTML to Parse XML or HTML
13. Network Programming
1. Passing Messages with Socket Datagrams
2. Grabbing a Document from the Web
3. Filtering a List of FTP Sites
4. Getting Time from a Server via the SNTP Protocol
5. Sending HTML Mail
6. Bundling Files in a MIME Message
7. Unpacking a Multipart MIME Message
8. Removing Attachments from an Email Message
9. Fixing Messages Parsed by Python 2.4 email.FeedParser
10. Inspecting a POP3 Mailbox Interactively
11. Detecting Inactive Computers
12. Monitoring a Network with HTTP
13. Forwarding and Redirecting Network Ports
14. Tunneling SSL Through a Proxy
15. Implementing the Dynamic IP Protocol
16. Connecting to IRC and Logging Messages to Disk
17. Accessing LDAP Servers
14. Web Programming
1. Testing Whether CGI Is Working
2. Handling URLs Within a CGI Script
3. Uploading Files with CGI
4. Checking for a Web Page's Existence
5. Checking Content Type via HTTP
6. Resuming the HTTP Download of a File
7. Handling Cookies While Fetching Web Pages
8. Authenticating with a Proxy for HTTPS Navigation
9. Running a Servlet with Jython
10. Finding an Internet Explorer Cookie
11. Generating OPML Files
12. Aggregating RSS Feeds
13. Turning Data into Web Pages Through Templates
14. Rendering Arbitrary Objects with Nevow
15. Distributed Programming
1. Making an XML-RPC Method Call
2. Serving XML-RPC Requests
3. Using XML-RPC with Medusa
4. Enabling an XML-RPC Server to Be Terminated Remotely
5. Implementing SimpleXMLRPCServer Niceties
6. Giving an XML-RPC Server a wxPython GUI
7. Using Twisted Perspective Broker
8. Implementing a CORBA Server and Client
9. Performing Remote Logins Using telnetlib
10. Performing Remote Logins with SSH
11. Authenticating an SSL Client over HTTPS
16. Programs About Programs
1. Verifying Whether a String Represents a Valid Number
2. Importing a Dynamically Generated Module
3. Importing from a Module Whose Name Is Determined at Runtime
4. Associating Parameters with a Function (Currying)
5. Composing Functions
6. Colorizing Python Source Using the Built-in Tokenizer
7. Merging and Splitting Tokens
8. Checking Whether a String Has Balanced Parentheses
9. Simulating Enumerations in Python
10. Referring to a List Comprehension While Building It
11. Automating the py2exe Compilationof Scripts into Windows Executables
12. Binding Main Script and Modules into One Executable on Unix
17. Extending and Embedding
1. Implementing a Simple Extension Type
2. Implementing a Simple Extension Type with Pyrex
3. Exposing a C++ Library to Python
4. Calling Functions from a Windows DLL
5. Using SWIG-Generated Modules in a Multithreaded Environment
6. Translating a Python Sequence into a C Arraywith the PySequence_Fast Protocol
7. Accessing a Python Sequence Item-by-Item with the Iterator Protocol
8. Returning None from a Python-Callable C Function
9. Debugging Dynamically Loaded C Extensions with gdb
10. Debugging Memory Problems
18. Algorithms
1. Removing Duplicates from a Sequence
2. Removing Duplicates from a SequenceWhile Maintaining Sequence Order
3. Generating Random Samples with Replacement
4. Generating Random Samples Without Replacement
5. Memoizing (Caching) the Return Values of Functions
6. Implementing a FIFO Container
7. Caching Objects with a FIFO Pruning Strategy
8. Implementing a Bag (Multiset) Collection Type
9. Simulating the Ternary Operator in Python
10. Computing Prime Numbers
11. Formatting Integers as Binary Strings
12. Formatting Integers as Strings in Arbitrary Bases
13. Converting Numbers to Rationals via Farey Fractions
14. Doing Arithmetic with Error Propagation
15. Summing Numbers with Maximal Accuracy
16. Simulating Floating Point
17. Computing the Convex Hulls and Diameters of 2D Point Sets
19. Iterators and Generators
1. Writing a range-like Function with Float Increments
2. Building a List from Any Iterable
3. Generating the Fibonacci Sequence
4. Unpacking a Few Items in a Multiple Assignment
5. Automatically Unpacking the Needed Number of Items
6. Dividing an Iterable into n Slices of Stride n
7. Looping on a Sequence by Overlapping Windows
8. Looping Through Multiple Iterables in Parallel
9. Looping Through the Cross-Product of Multiple Iterables
10. Reading a Text File by Paragraphs
11. Reading Lines with Continuation Characters
12. Iterating on a Stream of Data Blocks as a Stream of Lines
13. Fetching Large Record Sets from a Database with a Generator
14. Merging Sorted Sequences
15. Generating Permutations, Combinations, and Selections
16. Generating the Partitions of an Integer
17. Duplicating an Iterator
18. Looking Ahead into an Iterator
19. Simplifying Queue-Consumer Threads
20. Running an Iterator in Another Thread
21. Computing a Summary Report with itertools.groupby
20. Descriptors, Decorators, and Metaclasses
1. Getting Fresh Default Values at Each Function Call
2. Coding Properties as Nested Functions
3. Aliasing Attribute Values
4. Caching Attribute Values
5. Using One Method as Accessor for Multiple Attributes
6. Adding Functionality to a Class by Wrapping a Method
7. Adding Functionality to a Class by Enriching All Methods
8. Adding a Method to a Class Instance at Runtime
9. Checking Whether Interfaces Are Implemented
10. Using _ _new_ _ and _ _init_ _ Appropriately in Custom Metaclasses
11. Allowing Chaining of Mutating List Methods
12. Using Cooperative Supercalls with Terser Syntax
13. Initializing Instance Attributes Without Using _ _init_ _
14. Automatic Initialization of Instance Attributes
15. Upgrading Class Instances Automatically on reload
16. Binding Constants at Compile Time
17. Solving Metaclass Conflicts