Back to: Designing a Web Crawler 0 Previous Lesson Problem Definition & Requirements for a Distributed Web Crawler Next Lesson Scalability, Optimization & Real-World Considerations