Skip to content

k8s launcher python version compatibility (Python 3.7/3.8) #105

@andrijaperovic

Description

@andrijaperovic

Running historical retrieval with os.environ["FEAST_SPARK_LAUNCHER"] = "k8s" breaks due to python compatibility with generated spark python file:

kubectl logs feast-m5mz27p6-driver -n spark-operator                                                                            main@d2a800f ✘
++ id -u
+ myuid=0
++ id -g
+ mygid=0
+ set +e
++ getent passwd 0
+ uidentry=root:x:0:0:root:/root:/bin/bash
+ set -e
+ '[' -z root:x:0:0:root:/root:/bin/bash ']'
+ SPARK_CLASSPATH=':/opt/spark/jars/*'
+ env
+ grep SPARK_JAVA_OPT_
+ sort -t_ -k4 -n
+ sed 's/[^=]*=\(.*\)/\1/g'
+ readarray -t SPARK_EXECUTOR_JAVA_OPTS
+ '[' -n '' ']'
+ '[' '' == 2 ']'
+ '[' '' == 3 ']'
+ '[' -n '' ']'
+ '[' -z ']'
+ case "$1" in
+ shift 1
+ CMD=("$SPARK_HOME/bin/spark-submit" --conf "spark.driver.bindAddress=$SPARK_DRIVER_BIND_ADDRESS" --deploy-mode client "$@")
+ exec /usr/bin/tini -s -- /opt/spark/bin/spark-submit --conf spark.driver.bindAddress=10.244.1.33 --deploy-mode client --properties-file /opt/spark/conf/spark.properties --class org.apache.spark.deploy.PythonRunner wasbs://feasttest@feastdrivingpoc.blob.core.windows.net/artifacts/44d600e498692f5f4d9e183718f12fbd7925c1b97205373c181486ccab33f667.py --feature-tables W3siZmVhdHVyZXMiOiBbeyJuYW1lIjogImF2Z19kYWlseV90cmlwcyIsICJ0eXBlIjogIklOVDMyIn0sIHsibmFtZSI6ICJhY2NfcmF0ZSIsICJ0eXBlIjogIkZMT0FUIn0sIHsibmFtZSI6ICJjb252X3JhdGUiLCAidHlwZSI6ICJGTE9BVCJ9XSwgInByb2plY3QiOiAiZmVhc3Rkcml2aW5ncG9jdGVzdCIsICJuYW1lIjogImRyaXZlcl9zdGF0aXN0aWNzIiwgImVudGl0aWVzIjogW3sibmFtZSI6ICJkcml2ZXJfaWQiLCAidHlwZSI6ICJJTlQ2NCJ9XSwgIm1heF9hZ2UiOiA4NjQwMCwgImxhYmVscyI6IHt9fSwgeyJmZWF0dXJlcyI6IFt7Im5hbWUiOiAidHJpcHNfdG9kYXkiLCAidHlwZSI6ICJJTlQzMiJ9XSwgInByb2plY3QiOiAiZmVhc3Rkcml2aW5ncG9jdGVzdCIsICJuYW1lIjogImRyaXZlcl90cmlwcyIsICJlbnRpdGllcyI6IFt7Im5hbWUiOiAiZHJpdmVyX2lkIiwgInR5cGUiOiAiSU5UNjQifV0sICJtYXhfYWdlIjogODY0MDAsICJsYWJlbHMiOiB7fX1d --feature-tables-sources W3siZmlsZSI6IHsiZmllbGRfbWFwcGluZyI6IHt9LCAiZXZlbnRfdGltZXN0YW1wX2NvbHVtbiI6ICJkYXRldGltZSIsICJjcmVhdGVkX3RpbWVzdGFtcF9jb2x1bW4iOiAiY3JlYXRlZCIsICJkYXRlX3BhcnRpdGlvbl9jb2x1bW4iOiAiZGF0ZSIsICJwYXRoIjogIndhc2JzOi8vZmVhc3R0ZXN0QGZlYXN0ZHJpdmluZ3BvYy5ibG9iLmNvcmUud2luZG93cy5uZXQvZHJpdmVyX3N0YXRpc3RpY3MiLCAiZm9ybWF0IjogeyJqc29uX2NsYXNzIjogIlBhcnF1ZXRGb3JtYXQifX19LCB7ImZpbGUiOiB7ImZpZWxkX21hcHBpbmciOiB7fSwgImV2ZW50X3RpbWVzdGFtcF9jb2x1bW4iOiAiZGF0ZXRpbWUiLCAiY3JlYXRlZF90aW1lc3RhbXBfY29sdW1uIjogImNyZWF0ZWQiLCAiZGF0ZV9wYXJ0aXRpb25fY29sdW1uIjogImRhdGUiLCAicGF0aCI6ICJ3YXNiczovL2ZlYXN0dGVzdEBmZWFzdGRyaXZpbmdwb2MuYmxvYi5jb3JlLndpbmRvd3MubmV0L2RyaXZlcl90cmlwcyIsICJmb3JtYXQiOiB7Impzb25fY2xhc3MiOiAiUGFycXVldEZvcm1hdCJ9fX1d --entity-source eyJmaWxlIjogeyJmaWVsZF9tYXBwaW5nIjoge30sICJldmVudF90aW1lc3RhbXBfY29sdW1uIjogImV2ZW50X3RpbWVzdGFtcCIsICJjcmVhdGVkX3RpbWVzdGFtcF9jb2x1bW4iOiAiIiwgImRhdGVfcGFydGl0aW9uX2NvbHVtbiI6ICIiLCAicGF0aCI6ICJ3YXNiczovL2ZlYXN0dGVzdEBmZWFzdGRyaXZpbmdwb2MuYmxvYi5jb3JlLndpbmRvd3MubmV0L2FydGlmYWN0cy9hODYxMTg0My0yY2QxLTQ1MjktYTliNi1mMTEwNGExYTEzYjMiLCAiZm9ybWF0IjogeyJqc29uX2NsYXNzIjogIlBhcnF1ZXRGb3JtYXQifX19 --destination eyJmb3JtYXQiOiAicGFycXVldCIsICJwYXRoIjogImFiZnNzOi8vZmVhc3Rwb2NAZmVhc3Rwb2MuZGZzLmNvcmUud2luZG93cy5uZXQvZmVhc3Qvb3V0LzQ5ODQ1MzRhLWJiOGUtNDVmZi1hYzc3LTM2YzE3ZjA2YjRhOCJ9
22/01/23 22:43:01 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
22/01/23 22:43:02 WARN MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-azure-file-system.properties,hadoop-metrics2.properties
  File "/tmp/spark-ac4d3d12-c430-4f11-987a-7a193b8fef54/44d600e498692f5f4d9e183718f12fbd7925c1b97205373c181486ccab33f667.py", line 73
    event_timestamp_column: str,
                          ^
SyntaxError: invalid syntax
log4j:WARN No appenders could be found for logger (org.apache.spark.util.ShutdownHookManager).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

Upgrading to python3.8 from source in gcr.io/kf-feast/spark-py:v3.0.1 doesn't appear to fix the issue either.
Using historical features example:
https://github.com/Azure/feast-azure/blob/main/cluster/samples/feature_store_azure.ipynb

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions