The general idea of a producer-consumer architecture is related to building a pipeline of threads, similar to the web spider from Project 2. Each step of the processing will now be done by a specialized threads; whenever one thread is done with its part of the processing, it forwards the data to the next stage of the pipeline, like a factory assembly line. In a producer-consumer system, we guarantee that each stage of the pipeline signals the next stage of the pipeline when it completes its task, and blocks when there's nothing else for it to do, etc.

If we were to implement our web server in a producer-consumer fashion, we could simply create a pair of threads: one for reading the URL requested by the client, and another one for writing the answer back to the network. We could also create yet another intermediate thread, which would ?rst look for the URL in a cache. This intermediate threadmight help the overall average response time of the system, if we can implement the shortest time to completion ?rst policy, i.e. ?nish the easy things ?rst, which leads to the lowest average response time.

As with the thread pool approach, it is not always clear how many threads we need for each of stage of the pipeline; the exact amount depends on the application and is usually ?ne-tuned manually. Since each thread has a specialized task, if the number of threads for each task is not well-tuned, then many threads might be idle, wasting resources.

In general, the producer-consumer approach works well if the producer and the consumer are symmetric, i.e. if they proceed roughly at same rate. On the other hand, if the order of processing doesn't matter, it doesn't make sense to use a producer-consumer architecture, with its ordered pipeline of processing.