Common transformation patterns in Foundry?

• A. Python for relational operations, SQL for custom logic, Spark for small-scale processing.
• B. Spark for relational operations, SQL for custom logic, Python for large-scale processing.
• C. SQL for large-scale processing, Python for relational operations, Spark for custom logic.
• D. SQL for relational operations, Python for custom logic, Spark for large-scale processing.

Answer: (D)

Foundry’s transformation patterns are about using the right tool for the right kind of work. SQL is ideal for relational operations—joins, filters, aggregations, grouping—because it expresses how data should be related and reshaped in a declarative, optimized way. When the transformation requires procedural logic, complex rules, or operations that are easier to implement with code and libraries, Python fits that niche well as a flexible layer for custom transformations. For processing that involves large data volumes across many machines, Spark provides a distributed engine that scales horizontally to handle big workloads efficiently. Putting these together—SQL for relational reshaping, Python for bespoke logic, and Spark for large-scale processing—matches the strengths of each tool and supports both clarity and scalability. The other options mix these roles in ways that don’t align with how each technology performs best, such as using Python for relational operations or restricting Spark to small-scale tasks.