Skip to content

Commit

Permalink
initialize dbt-fabric
Browse files Browse the repository at this point in the history
  • Loading branch information
prdpsvs committed Aug 22, 2023
1 parent 9546c40 commit c851c66
Show file tree
Hide file tree
Showing 71 changed files with 456 additions and 1,347 deletions.
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
default_language_version:
python: python3.9
python: python3.10.8
repos:
- repo: 'https://github.com/pre-commit/pre-commit-hooks'
rev: v4.4.0
Expand Down
Empty file added 23.1.2
Empty file.
383 changes: 0 additions & 383 deletions CHANGELOG.md

Large diffs are not rendered by default.

6 changes: 3 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ Create a virtual environment, [pyenv](https://github.com/pyenv/pyenv) is used in

```shell
pyenv install 3.10.7
pyenv virtualenv 3.10.7 dbt-sqlserver
pyenv activate dbt-sqlserver
pyenv virtualenv 3.10.7 dbt-fabric
pyenv activate dbt-fabric
```

Install the development dependencies and pre-commit and get information about possible make commands:
Expand Down Expand Up @@ -64,7 +64,7 @@ All CI/CD pipelines are using GitHub Actions. The following pipelines are availa
* `publish-docker`: publishes the image we use in all other pipelines.
* `unit-tests`: runs the unit tests for each supported Python version.
* `integration-tests-azure`: runs the integration tests for Azure SQL Server.
* `integration-tests-sqlserver`: runs the integration tests for SQL Server.
* `integration-tests-fabric`: runs the integration tests for SQL Server.
* `release-version`: publishes the adapter to PyPI.

There is an additional [Pre-commit](https://pre-commit.ci/) pipeline that validates the code style.
Expand Down
30 changes: 14 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,17 @@
# dbt-sqlserver
# dbt-fabric

[dbt](https://www.getdbt.com) adapter for Microsoft SQL Server and Azure SQL services.
[dbt](https://www.getdbt.com) adapter for Microsoft Fabric Synapse Data Warehouse.

The adapter supports dbt-core 0.14 or newer and follows the same versioning scheme.
The adapter supports dbt-core 1.4 or newer and follows the same versioning scheme.
E.g. version 1.1.x of the adapter will be compatible with dbt-core 1.1.x.

## Documentation

We've bundled all documentation on the dbt docs site:

TODO
* [Profile setup & authentication](https://docs.getdbt.com/reference/warehouse-profiles/mssql-profile)
* [Adapter documentation, usage and important notes](https://docs.getdbt.com/reference/resource-configs/mssql-configs)

Join us on the [dbt Slack](https://getdbt.slack.com/archives/CMRMDDQ9W) to ask questions, get help, or to discuss the project.

## Installation

This adapter requires the Microsoft ODBC driver to be installed:
Expand All @@ -33,16 +31,16 @@ sudo apt-get install -y unixodbc-dev
</p>
</details>

Latest version: ![PyPI](https://img.shields.io/pypi/v/dbt-sqlserver?label=latest%20stable&logo=pypi)
Latest version: ![PyPI](https://img.shields.io/pypi/v/dbt-sqlserver?label=latest%20stable&logo=pypi): TODO

```shell
pip install -U dbt-sqlserver
pip install -U dbt-fabric
```

Latest pre-release: ![GitHub tag (latest SemVer pre-release)](https://img.shields.io/github/v/tag/dbt-msft/dbt-sqlserver?include_prereleases&label=latest%20pre-release&logo=pypi)
Latest pre-release: ![GitHub tag (latest SemVer pre-release)](https://img.shields.io/github/v/tag/microsoft/dbt-sqlserver?include_prereleases&label=latest%20pre-release&logo=pypi): TODO

```shell
pip install -U --pre dbt-sqlserver
pip install -U --pre dbt-fabric
```

## Changelog
Expand All @@ -51,18 +49,18 @@ See [the changelog](CHANGELOG.md)

## Contributing

[![Unit tests](https://github.com/dbt-msft/dbt-sqlserver/actions/workflows/unit-tests.yml/badge.svg)](https://github.com/dbt-msft/dbt-sqlserver/actions/workflows/unit-tests.yml)
[![Integration tests on SQL Server](https://github.com/dbt-msft/dbt-sqlserver/actions/workflows/integration-tests-sqlserver.yml/badge.svg)](https://github.com/dbt-msft/dbt-sqlserver/actions/workflows/integration-tests-sqlserver.yml)
[![Integration tests on Azure](https://github.com/dbt-msft/dbt-sqlserver/actions/workflows/integration-tests-azure.yml/badge.svg)](https://github.com/dbt-msft/dbt-sqlserver/actions/workflows/integration-tests-azure.yml)
[![Unit tests](https://github.com/microsoft/dbt-fabric/actions/workflows/unit-tests.yml/badge.svg)](https://github.com/microsoft/dbt-sqlserver/actions/workflows/unit-tests.yml)
[![Integration tests on Microsoft Fabric Synapse Data Warehouse](https://github.com/microsoft/dbt-fabric/actions/workflows/integration-tests-sqlserver.yml/badge.svg)](https://github.com/microsoft/dbt-fabric/actions/workflows/integration-tests-sqlserver.yml)
[![Integration tests on Azure](https://github.com/microsoft/dbt-fabric/actions/workflows/integration-tests-azure.yml/badge.svg)](https://github.com/microsoft/dbt-fabric/actions/workflows/integration-tests-azure.yml)

This adapter is community-maintained.
You are welcome to contribute by creating issues, opening or reviewing pull requests or helping other users in Slack channel.
You are welcome to contribute by creating issues, opening or reviewing pull requests.
If you're unsure how to get started, check out our [contributing guide](CONTRIBUTING.md).

## License

[![PyPI - License](https://img.shields.io/pypi/l/dbt-sqlserver)](https://github.com/dbt-msft/dbt-sqlserver/blob/master/LICENSE)
[![PyPI - License](https://img.shields.io/pypi/l/dbt-fabric)](https://github.com/microsoft/dbt-fabric/blob/master/LICENSE)

## Code of Conduct

This project and everyone involved is expected to follow the [dbt Code of Conduct](https://community.getdbt.com/code-of-conduct).
This project and everyone involved is expected to follow the [Microsoft Code of Conduct](https://community.getdbt.com/code-of-conduct).
23 changes: 23 additions & 0 deletions dbt/adapters/fabric/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
from dbt.adapters.base import AdapterPlugin

from dbt.adapters.fabric.fabric_adapter import FabricAdapter
from dbt.adapters.fabric.fabric_column import FabricColumn
from dbt.adapters.fabric.fabric_configs import FabricConfigs
from dbt.adapters.fabric.fabric_connection_manager import FabricConnectionManager
from dbt.adapters.fabric.fabric_credentials import FabricCredentials
from dbt.include import fabric

Plugin = AdapterPlugin(
adapter=FabricAdapter,
credentials=FabricCredentials,
include_path=fabric.PACKAGE_PATH,
)

__all__ = [
"Plugin",
"FabricConnectionManager",
"FabricColumn",
"FabricAdapter",
"FabricCredentials",
"FabricConfigs",
]
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,15 @@
from dbt.events.functions import fire_event
from dbt.events.types import SchemaCreation

from dbt.adapters.sqlserver.sql_server_column import SQLServerColumn
from dbt.adapters.sqlserver.sql_server_configs import SQLServerConfigs
from dbt.adapters.sqlserver.sql_server_connection_manager import SQLServerConnectionManager
from dbt.adapters.fabric.fabric_column import FabricColumn
from dbt.adapters.fabric.fabric_configs import FabricConfigs
from dbt.adapters.fabric.fabric_connection_manager import FabricConnectionManager


class SQLServerAdapter(SQLAdapter):
ConnectionManager = SQLServerConnectionManager
Column = SQLServerColumn
AdapterSpecificConfigs = SQLServerConfigs
class FabricAdapter(SQLAdapter):
ConnectionManager = FabricConnectionManager
Column = FabricColumn
AdapterSpecificConfigs = FabricConfigs

def create_schema(self, relation: BaseRelation) -> None:
relation = relation.without_identifier()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
from dbt.adapters.base import Column


class SQLServerColumn(Column):
class FabricColumn(Column):
TYPE_LABELS: ClassVar[Dict[str, str]] = {
"STRING": "VARCHAR(MAX)",
"TIMESTAMP": "DATETIMEOFFSET",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,5 @@


@dataclass
class SQLServerConfigs(AdapterConfig):
class FabricConfigs(AdapterConfig):
auto_provision_aad_principals: Optional[bool] = False
Original file line number Diff line number Diff line change
Expand Up @@ -21,14 +21,14 @@
from dbt.contracts.connection import AdapterResponse, Connection, ConnectionState
from dbt.events import AdapterLogger

from dbt.adapters.sqlserver import __version__
from dbt.adapters.sqlserver.sql_server_credentials import SQLServerCredentials
from dbt.adapters.fabric import __version__
from dbt.adapters.fabric.fabric_credentials import FabricCredentials

AZURE_CREDENTIAL_SCOPE = "https://database.windows.net//.default"
_TOKEN: Optional[AccessToken] = None
AZURE_AUTH_FUNCTION_TYPE = Callable[[SQLServerCredentials], AccessToken]
AZURE_AUTH_FUNCTION_TYPE = Callable[[FabricCredentials], AccessToken]

logger = AdapterLogger("SQLServer")
logger = AdapterLogger("fabric")


def convert_bytes_to_mswindows_byte_string(value: bytes) -> bytes:
Expand Down Expand Up @@ -67,7 +67,7 @@ def convert_access_token_to_mswindows_byte_string(token: AccessToken) -> bytes:
return convert_bytes_to_mswindows_byte_string(value)


def get_cli_access_token(credentials: SQLServerCredentials) -> AccessToken:
def get_cli_access_token(credentials: FabricCredentials) -> AccessToken:
"""
Get an Azure access token using the CLI credentials
Expand All @@ -92,7 +92,7 @@ def get_cli_access_token(credentials: SQLServerCredentials) -> AccessToken:
return token


def get_msi_access_token(credentials: SQLServerCredentials) -> AccessToken:
def get_msi_access_token(credentials: FabricCredentials) -> AccessToken:
"""
Get an Azure access token from the system's managed identity
Expand All @@ -110,7 +110,7 @@ def get_msi_access_token(credentials: SQLServerCredentials) -> AccessToken:
return token


def get_auto_access_token(credentials: SQLServerCredentials) -> AccessToken:
def get_auto_access_token(credentials: FabricCredentials) -> AccessToken:
"""
Get an Azure access token automatically through azure-identity
Expand All @@ -128,7 +128,7 @@ def get_auto_access_token(credentials: SQLServerCredentials) -> AccessToken:
return token


def get_environment_access_token(credentials: SQLServerCredentials) -> AccessToken:
def get_environment_access_token(credentials: FabricCredentials) -> AccessToken:
"""
Get an Azure access token by reading environment variables
Expand All @@ -146,7 +146,7 @@ def get_environment_access_token(credentials: SQLServerCredentials) -> AccessTok
return token


def get_sp_access_token(credentials: SQLServerCredentials) -> AccessToken:
def get_sp_access_token(credentials: FabricCredentials) -> AccessToken:
"""
Get an Azure access token using the SP credentials.
Expand Down Expand Up @@ -175,7 +175,7 @@ def get_sp_access_token(credentials: SQLServerCredentials) -> AccessToken:
}


def get_pyodbc_attrs_before(credentials: SQLServerCredentials) -> Dict:
def get_pyodbc_attrs_before(credentials: FabricCredentials) -> Dict:
"""
Get the pyodbc attrs before.
Expand Down Expand Up @@ -268,8 +268,8 @@ def byte_array_to_datetime(value: bytes) -> dt.datetime:
)


class SQLServerConnectionManager(SQLConnectionManager):
TYPE = "sqlserver"
class FabricConnectionManager(SQLConnectionManager):
TYPE = "fabric"

@contextmanager
def exception_handler(self, sql):
Expand Down Expand Up @@ -444,7 +444,7 @@ def add_query(
return connection, cursor

@classmethod
def get_credentials(cls, credentials: SQLServerCredentials) -> SQLServerCredentials:
def get_credentials(cls, credentials: FabricCredentials) -> FabricCredentials:
return credentials

@classmethod
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@


@dataclass
class SQLServerCredentials(Credentials):
class FabricCredentials(Credentials):
driver: str
host: str
database: str
Expand Down Expand Up @@ -41,7 +41,7 @@ class SQLServerCredentials(Credentials):

@property
def type(self):
return "sqlserver"
return "fabric"

def _connection_keys(self):
# return an iterator of keys to pretty-print in 'dbt debug'
Expand Down
23 changes: 0 additions & 23 deletions dbt/adapters/sqlserver/__init__.py

This file was deleted.

File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: dbt_sqlserver
name: dbt_fabric
version: 1.0

config-version: 2
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{% macro sqlserver__get_show_grant_sql(relation) %}
{% macro fabric__get_show_grant_sql(relation) %}
select
GRANTEE as grantee,
PRIVILEGE_TYPE as privilege_type
Expand All @@ -9,7 +9,7 @@
{% endmacro %}


{%- macro sqlserver__get_grant_sql(relation, privilege, grantees) -%}
{%- macro fabric__get_grant_sql(relation, privilege, grantees) -%}
{%- set grantees_safe = [] -%}
{%- for grantee in grantees -%}
{%- set grantee_safe = "[" ~ grantee ~ "]" -%}
Expand All @@ -19,7 +19,7 @@
{%- endmacro -%}


{%- macro sqlserver__get_revoke_sql(relation, privilege, grantees) -%}
{%- macro fabric__get_revoke_sql(relation, privilege, grantees) -%}
{%- set grantees_safe = [] -%}
{%- for grantee in grantees -%}
{%- set grantee_safe = "[" ~ grantee ~ "]" -%}
Expand All @@ -37,7 +37,7 @@
{% endmacro %}


{% macro sqlserver__apply_grants(relation, grant_config, should_revoke=True) %}
{% macro fabric__apply_grants(relation, grant_config, should_revoke=True) %}
{#-- If grant_config is {} or None, this is a no-op --#}
{% if grant_config %}
{% if should_revoke %}
Expand Down
Loading

0 comments on commit c851c66

Please sign in to comment.