The Hacker's Guide to Scaling Python

Description

In this course, you will learn :

• About CPU scaling, concurrency, and event loops, all of which are essential for implementing a distributed system.
• About queue-based distribution, lock management, and group memberships, all of which are important concepts for maintaining availability and consistency in distributed applications.
• How, with the right Python implementation, you can write applications that scale horizontally, perform well, and are distributed.

Syllabus :

1. Scaling

• Scaling Across CPUs
• Distributed Systems
• Service-Oriented Architecture

 

2. CPU Scaling

• Using Threads
• Using Processes
• Using Futures
• Advanced Futures Usage
• Daemon Processes
• Mehdi Abaakouk on CPU Scaling

 

3. Event Loops

• Introduction to Event Loops
• Basic Pattern: Blocking Events
• Using Asyncio
• Server And Client
• Naoki Inada on asyncio

 

4. Functional Programming

• Functional Programming Toolkit: Maps, Filters, and More
• Functional Programming Toolkit: "next", "first", and "itertools"

 

5. Queue-Based Distribution

• RQ
• Celery
• Handling Failures
• Chaining Tasks
• Multiple Queues
• Monitoring
• Joshua Harlow on Task Distribution

 

6. Designing for Failure

• Naive Retrying
• Retrying with Tenacity

 

8. Lock Management

• Thread Locks
• Processes Locks
• Processes Locks: Multiprocessing Locks
• Processes Locks: Inter-Processes Locks
• Using etcd for Distributed Locking
• Using Tooz Locking Abstraction

 

9. Group Membership

• Creating, Joining and Leaving Groups
• Using Capabilities
• Using Watchers Callbacks
• Consistent Hash Rings
• Partitioner
• Alexys Jacob-Monier on Cluster Management

 

10. Building REST API

• The WSGI Protocol
• Streaming Data
• Using ETag
• Asynchronous HTTP API
• Fast HTTP Client
• Comparison Between Fast HTTP Clients
• Testing REST API
• Chris Dent on HTTP

 

11. Deploying on PaaS

• Heroku
• Amazon Beanstalk
• Google App Engine
• OpenShift
• Beyond PaaS

 

12. Testing Distributed Systems

• Introduction to Testing Distributed Systems
• Setting Up Environments with tox
• Manage External Services with pifpaf
• Using Fixtures with pifpaf

 

13. Caching

• Local Caching
• Memoization
• Distributed Caching
• Jason Myers on Databases

 

14. Performance

• Introduction to performance optimization
• Memory and CPU Profiling
• Profiling Strategy and a Case
• Zero-Copy
• Disassembling Code
• Victor Stinner on Performance

Show more