Use this pattern when:

The processing required by an application can easily be broken down into a set of independent steps.

The processing steps performed by an application have different scalability requirements.

It's possible to group filters that should scale together in the same process. For more information, see the Compute Resource Consolidation pattern.

Flexibility is required to allow reordering of the processing steps performed by an application, or the capability to add and remove steps.

The system can benefit from distributing the processing for steps across different servers.

A reliable solution is required that minimizes the effects of failure in a step while data is being processed.

This pattern might not be useful when:

The processing steps performed by an application aren't independent, or they have to be performed together as part of the same transaction.

The amount of context or state information required by a step makes this approach inefficient. It might be possible to persist state information to a database instead, but don't use this strategy if the additional load on the database causes excessive contention.

https://learn.microsoft.com/en-us/azure/architecture/patterns/pipes-and-filters