Spark database is double-quoted when it's not a valid identifier

[julijonas]

What happens?

When your spark session database is not a valid SQL identifier, it gets double quoted (both by spark and splink) in database_api.splink_data_store and you cannot construct a linker.

`spark_catalog`.``1111``

To Reproduce

You get an error when spark uses a database name with a leading digit:

spark.sql("CREATE DATABASE IF NOT EXISTS `1111`")
spark.sql("USE DATABASE `1111`")
api = SparkAPI(spark_session=spark)
linker = Linker(df, settings, api)

ParseException: 
[PARSE_SYNTAX_ERROR] Syntax error at or near '1111'.(line 1, pos 35)

== SQL ==
show tables from `spark_catalog`.``1111`` like '__splink__input_table_0'

OS:

Mac, Linux

Splink version:

4.0.6

Have you tried this on the latest master branch?

• I agree

Have you tried the steps to reproduce? Do they include all relevant data and configuration? Does the issue you report still appear there?

• I agree