Support libsql/Turso

[danawoodman]

Describe the feature

Unless I am mistaken, Gorm does not support a network connected SQLite database, specifically libsql / Turso

I've had success connecting to libsql/Turso with Go's built-in database/sql as well as sqlx, but I get the following error when attempting with Gorm and I see no docs on how to do this, so I'm assuming it isn't supported.

unable to open database file: no such file or directory

Perhaps integration with libsql-client-go, which allows support in the above mentioned tools, could be integrated?

Motivation

The ability to connect to any libsql SQLite database, specifically Turso

Related Issues

I could not find anything in issues or discussions.

[ekristen]

I'll offer to test. I would try and write it but I suspect that @jinzhu will be about 1000% faster if they have the time.

[ekristen]

I actually made this work locally by cloning the sqlite driver and replacing just the imported library under the hood from

_ "github.com/mattn/go-sqlite3"

to

_ "github.com/libsql/libsql-client-go/libsql"

I'll create a fork or repo as an example later today.

[ekristen]

@danawoodman give this a shot -- https://github.com/ekristen/gorm-libsql -- happy to contribute my changes back to go-gorm

Note: DSN must contain the prefix libsql:// or a few different options that libsql library underneath expects.

gorm.Open(libsql.Open(dsn), config)

[jinzhu]

I think we can make it works like this example:

https://gorm.io/docs/connecting_to_the_database.html#Customize-Driver

[ekristen]

If you could work up an example that'd be great.

[danawoodman]

Anyone have some success with this?

[ekristen]

This doesn't seem to work with sqlite.

[ytsruh]

Just wondered if anybody has been working on this? I tried @ekristen library mentioned earlier and updated it to the new Turso driver but I was getting a nil pointer deference whenever I try to access the database. I could get a connection successfully & use AutoMigrate successfully but it seems the connection is being closed after each use.

If anyone can provide some general guidance/direction on this then I'm happy to work on it. I'm no DB or Gorm genious but I'm sure I could have a good stab at it.

[ekristen]

I made it work by creating this https://github.com/ekristen/gorm-libsql, theoretically the customize driver should work but I haven't had any luck. ekristen/gorm-libsql@74475fc...master

[ytsruh]

Hi all, I've had a go at this and you can find the PR here - go-gorm/sqlite#185

You use it the same way as you can with the Postgres one:

db, err = gorm.Open(sqlite.New(sqlite.Config{
		DriverName: "libsql",
		DSN:       <url including authToken>,
	}), &gorm.Config{})

I'm 99.99% sure this will work as expected and I've tried a fair few combinations of CRUD and its working for me. Could somebody test it and let me know please? I'll also post in the Turso Discord as well.

If this PR is successful then I'll also update the docs for Gorm

[ytsruh]

Right this is now working as expected & all tests are passing.

You'll need to add the import of _ "github.com/tursodatabase/libsql-client-go/libsql" with the code example below but this is now fully working with a Turso remote DB :)

[asilvadesigns]

👀

[pMatt1988]

Right this is now working as expected & all tests are passing.

You'll need to add the import of _ "github.com/tursodatabase/libsql-client-go/libsql" with the code example below but this is now fully working with a Turso remote DB :)

@ytsruh
I just tried this out and it works a treat. Thanks!

[ytsruh]

Small update on the PR. Seems it introduces possible corruptions issues with its implementation. I've not had feedback from the maintainers but to prevent a clash of drivers I'm going to break this out into its own package and publish myself.

This will prevent a breaking change to the current SQLite driver but allow us to use it with Turso & libsql

[ytsruh]

Hi all, small update to say that I've published a version/fork of the SQLite driver that supports both embedded replicas and the standard (non cGo) features as well. available here : https://github.com/ytsruh/gorm-libsql

go get -u github.com/ytsruh/gorm-libsql this should work & instructions are in the Readme

[sejoalfaro]

I think it can be closed by the following PR go-gorm/sqlite#185.
That PR allow us to use gorm with turso.
And ty @ytsruh for your contribution 🚀.

[thomas-cabral]

I am able to use github.com/tursodatabase/libsql-client-go/libsql with the changes from #185 but if I try to use NewEmbeddedReplicaConnector from github.com/tursodatabase/go-libsql I get conflicts with mattn/go-sqlite3. I don't know if I am just doing something wrong or if it just won't work as is.

[ytsruh]

I am able to use github.com/tursodatabase/libsql-client-go/libsql with the changes from #185 but if I try to use NewEmbeddedReplicaConnector from github.com/tursodatabase/go-libsql I get conflicts with mattn/go-sqlite3. I don't know if I am just doing something wrong or if it just won't work as is.

Sorry I missed this. I don't really use Gorm anymore but thought I'd try & help out. Did this full feature example not work?
https://github.com/ytsruh/gorm-libsql?tab=readme-ov-file#full-feature-usage