Skip to content

Commit

Permalink
allspark: added Spark inspec test
Browse files Browse the repository at this point in the history
This will run the Pi Spark test job to see if all is configured
correctly. It also checks that `$SPARK_HOME` is set correctly.

NOTE: Pi Spark example job returns a very approximate value for
Pi. During one run for example it returnes 3.13### so testing
output against `3.1` to avoid/reduce inspec test failures.
  • Loading branch information
xoen committed Dec 16, 2020
1 parent 65cf5c7 commit e6a2c90
Showing 1 changed file with 22 additions and 0 deletions.
22 changes: 22 additions & 0 deletions allspark-notebook/tests/controls/pyspark_spec.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
title 'Working pyspark'

control "pyspark is available" do
impact "high"
title "pyspark should be installed and work"
desc "pyspark installed and can run pyspark jobs"
tag "pyspark"
tag "spark"

describe command("echo $SPARK_HOME") do
its("stdout.strip") { should eq "/usr/local/spark" }
end

# Can run on of the spark examples
describe command("python3 /usr/local/spark/examples/src/main/python/pi.py") do
its("exit_status") { should eq 0 }
# Pi calculated using Spark example job is very
# approximate, once it returned 3.13###
# so checking against 3.1 to avoid random failures
its("stdout") { should match /Pi is roughly 3.1/ }
end
end

0 comments on commit e6a2c90

Please sign in to comment.