chore(spark): Update major/minor versions for 26.7.0 #1502

Part of #1479.

• Change: 4.1.1 to become LTS
• Deprecate: 3.5.8
• Remove: 3.5.7, 4.0.1

Update tasks

• Update boil-config.toml to reflect the agreed upon versions in the spreadsheet (including the removal of old versions).
• Upload new version (see spark/upload_new_spark_version.sh).
• Update boil-config.toml to the latest supported version of JVM (base and devel).
• Update boil-config.toml to the latest supported version of Hadoop. (it was problematic, we stuck to 3.4.2)
• Ensure that the transitive dependencies in boil-config.toml are compatible with the Spark version(s).
• Update other dependencies if applicable (eg: python, jmx_exporter, aws_java_sdk_bundle, etc).
• Check other operators (getting_started / kuttl / supported-versions) for usage of the versions. Add the PR(s) to the list below.
• Update the version in demos. Add the PR(s) to the list below.
• Update the version in the SparkApplication within the stackable-airflow/dags private repository.

Related Pull Requests

• chore(spark): Remove 3.5.7 and 4.0.1 #1525
• chore: Remove support for 3.5.7 and 4.0.1 spark-k8s-operator#695
• chore: Update Spark versions to 4.1.1 demos#408

Acceptance

• Can build image (either locally, or in CI)
• All kuttl tests pass (either locally, or in CI)

# Where x.y.z is the new version added in this PR
boil build spark-k8s=x.y.z --strip-architecture --load

kind load docker-image oci.stackable.tech/sdp/spark-k8s:x.y.z-stackable0.0.0-dev

# Change directory into the spark-k8s-operator repository and update the
# product version in tests/test-definition.yaml
./scripts/run-tests --test-suite smoke-latest # or similar

Please consider updating this template if these instructions are wrong, or
could be made clearer.