Testing
Testing Apache Spark Applications with ScalaTest
Apache Spark applications are notoriously hard to test. The framework was designed for distributed execution at scale, but your tests need to run fast and locally. The good news: Spark runs in local mode, and with the right setup, you can have a test suite that gives real confidence without