The deployment conf does not take into account the libraries: java.lang.ClassNotFoundException

[J1034567]

Hi team,

I get this error when I try to launch a job with dbx launch on pool.

java.lang.ClassNotFoundException: : java.lang.ClassNotFoundException: Failed to find data source: cosmos.oltp. Please find packages at https://spark.apache.org/third-party-projects.html

My deployement.yml file :
`custom:
default-cluster-spec: &default-cluster-spec
spark_version: "11.0.x-cpu-ml-scala2.12"
azure_attributes:
availability: "SPOT_WITH_FALLBACK_AZURE"
first_on_demand: 1
spot_bid_max_price: -1
libraries:
- maven:
coordinates: "com.azure.cosmos.spark:azure-cosmos-spark_3-2_2-12:4.12.2"

dev-cluster-config: &dev-cluster-config
new_cluster:
<<: *default-cluster-spec
num_workers: 1
driver_instance_pool_id: "xxxx-xxxx-xxx-pool-xxxx"
instance_pool_id: "xxxx-xxxx-xxxx-pool-xxxx"`

Thanks in advance for any tips and advise.

[renardeinside]

hi @J1034567 ,

for future, please create such tickets in issues, not in discussions.

As for the error, it's not relevant to dbx, so unfortunately I cannot help much with it.

Most probably you're misspelling the format name, or installing the dependency in a wrong way. Please create a ticket in azure-cosmosdb-spark repo instead.

Also please verify that you specify the libraries section accordingly to the Jobs API specification.