Skip to content

Commit

Permalink
Removing graphql dependency from dataloader
Browse files Browse the repository at this point in the history
  • Loading branch information
@bbakerman
bbakerman committed Sep 9, 2017
1 parent fbb35ef commit 25047ce
Show file tree
Hide file tree
Showing 5 changed files with 0 additions and 319 deletions.
122 changes: 0 additions & 122 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -165,128 +165,6 @@ a list of user ids in one call.

That said, with key caching turn on (the default), it will still be more efficient using `dataloader` than without it.

# Using dataloader in graphql for maximum efficiency


If you are using `graphql`, you are likely to making queries on a graph of data (surprise surprise). `dataloader` will help
you to make this a more efficient process by both caching and batching requests for that graph of data items. If `dataloader`
has previously see a data item before, it will cached the value and will return it without having to ask for it again.

Imagine we have the StarWars query outlined below. It asks us to find a hero and their friend's names and their friend's friend's
names. It is likely that many of these people will be friends in common.



{
hero {
name
friends {
name
friends {
name
}
}
}
}

The result of this query is displayed below. You can see that Han, Leia, Luke and R2-D2 are tight knit bunch of friends and
share many friends in common.


[hero: [name: 'R2-D2', friends: [
[name: 'Luke Skywalker', friends: [
[name: 'Han Solo'], [name: 'Leia Organa'], [name: 'C-3PO'], [name: 'R2-D2']]],
[name: 'Han Solo', friends: [
[name: 'Luke Skywalker'], [name: 'Leia Organa'], [name: 'R2-D2']]],
[name: 'Leia Organa', friends: [
[name: 'Luke Skywalker'], [name: 'Han Solo'], [name: 'C-3PO'], [name: 'R2-D2']]]]]
]

A naive implementation would called a `DataFetcher` to retrieved a person object every time it was invoked.

In this case it would be *15* calls over the network. Even though the group of people have a lot of common friends.
With `dataloader` you can make the `graphql` query much more efficient.

As `graphql` descends each level of the query ( eg as it processes `hero` and then `friends` and then for each their `friends`),
the data loader is called to "promise" to deliver a person object. At each level `dataloader.dispatch()` will be
called to fire off the batch requests for that part of the query. With caching turned on (the default) then
any previously returned person will be returned as is for no cost.

In the above example there are only *5* unique people mentioned but with caching and batching retrieval in place their will be only
*3* calls to the batch loader function. *3* calls over the network or to a database is much better than *15* calls you will agree.

If you use capabilities like `java.util.concurrent.CompletableFuture.supplyAsync()` then you can make it even more efficient by making the
the remote calls asynchronous to the rest of the query. This will make it even more timely since multiple calls can happen at once
if need be.

Here is how you might put this in place:


```java

// a batch loader function that will be called with N or more keys for batch loading
BatchLoader<String, Object> characterBatchLoader = new BatchLoader<String, Object>() {
@Override
public CompletionStage<List<Object>> load(List<String> keys) {
//
// we use supplyAsync() of values here for maximum parellisation
//
return CompletableFuture.supplyAsync(() -> getCharacterDataViaBatchHTTPApi(keys));
}
};

// a data loader for characters that points to the character batch loader
DataLoader characterDataLoader = new DataLoader<String, Object>(characterBatchLoader);

//
// use this data loader in the data fetchers associated with characters and put them into
// the graphql schema (not shown)
//
DataFetcher heroDataFetcher = new DataFetcher() {
@Override
public Object get(DataFetchingEnvironment environment) {
return characterDataLoader.load("2001"); // R2D2
}
};

DataFetcher friendsDataFetcher = new DataFetcher() {
@Override
public Object get(DataFetchingEnvironment environment) {
StarWarsCharacter starWarsCharacter = environment.getSource();
List<String> friendIds = starWarsCharacter.getFriendIds();
return characterDataLoader.loadMany(friendIds);
}
};

//
// DataLoaderRegistry is a place to register all data loaders in that needs to be dispatched together
// in this case there is 1 but you can have many
//
DataLoaderRegistry registry = new DataLoaderRegistry();
registry.register("character", characterDataLoader);

//
// this instrumentation implementation will dispatched all the dataloaders
// as each level fo the graphql query is executed and hence make batched objects
// available to the query and the associated DataFetchers
//
DataLoaderDispatcherInstrumentation dispatcherInstrumentation
= new DataLoaderDispatcherInstrumentation(registry);

//
// now build your graphql object and execute queries on it.
// the data loader will be invoked via the data fetchers on the
// schema fields
//
GraphQL graphQL = GraphQL.newGraphQL(buildSchema())
.instrumentation(dispatcherInstrumentation)
.build();
```

One thing to note is the above only works if you use `DataLoaderDispatcherInstrumentation` which makes sure `dataLoader.dispatch()` is called. If
this was not in place, then all the promises to data will never be dispatched ot the batch loader function and hence nothing would ever resolve.

See below for more details on `dataLoader.dispatch()`

### Error object is not a thing in a type safe Java world

Expand Down
1 change: 0 additions & 1 deletion build.gradle
View file
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,6 @@ task myJavadocs(type: Javadoc) {
}

dependencies {
compile "com.graphql-java:graphql-java:4.0"
testCompile "junit:junit:$junitVersion"
testCompile 'org.awaitility:awaitility:2.0.0'
}
Expand Down
33 changes: 0 additions & 33 deletions src/main/java/org/dataloader/graphql/DataLoaderDispatcherInstrumentation.java
View file

This file was deleted.

82 changes: 0 additions & 82 deletions src/test/java/ReadmeExamples.java
View file
Original file line number Diff line number Diff line change
@@ -1,16 +1,10 @@
import graphql.GraphQL;
import graphql.schema.DataFetcher;
import graphql.schema.DataFetchingEnvironment;
import graphql.schema.GraphQLSchema;
import org.dataloader.BatchLoader;
import org.dataloader.CacheMap;
import org.dataloader.DataLoader;
import org.dataloader.DataLoaderOptions;
import org.dataloader.DataLoaderRegistry;
import org.dataloader.Try;
import org.dataloader.fixtures.User;
import org.dataloader.fixtures.UserManager;
import org.dataloader.graphql.DataLoaderDispatcherInstrumentation;

import java.util.ArrayList;
import java.util.List;
Expand Down Expand Up @@ -81,82 +75,6 @@ public CompletionStage<List<User>> load(List<Long> userIds) {
}


class StarWarsCharacter {
List<String> getFriendIds() {
return null;
}
}

void starWarsExample() {

// a batch loader function that will be called with N or more keys for batch loading
BatchLoader<String, Object> characterBatchLoader = new BatchLoader<String, Object>() {
@Override
public CompletionStage<List<Object>> load(List<String> keys) {
//
// we use supplyAsync() of values here for maximum parellisation
//
return CompletableFuture.supplyAsync(() -> getCharacterDataViaBatchHTTPApi(keys));
}
};

// a data loader for characters that points to the character batch loader
DataLoader characterDataLoader = new DataLoader<String, Object>(characterBatchLoader);

//
// use this data loader in the data fetchers associated with characters and put them into
// the graphql schema (not shown)
//
DataFetcher heroDataFetcher = new DataFetcher() {
@Override
public Object get(DataFetchingEnvironment environment) {
return characterDataLoader.load("2001"); // R2D2
}
};

DataFetcher friendsDataFetcher = new DataFetcher() {
@Override
public Object get(DataFetchingEnvironment environment) {
StarWarsCharacter starWarsCharacter = environment.getSource();
List<String> friendIds = starWarsCharacter.getFriendIds();
return characterDataLoader.loadMany(friendIds);
}
};

//
// DataLoaderRegistry is a place to register all data loaders in that needs to be dispatched together
// in this case there is 1 but you can have many
//
DataLoaderRegistry registry = new DataLoaderRegistry();
registry.register("character", characterDataLoader);

//
// this instrumentation implementation will dispatched all the dataloaders
// as each level fo the graphql query is executed and hence make batched objects
// available to the query and the associated DataFetchers
//
DataLoaderDispatcherInstrumentation dispatcherInstrumentation
= new DataLoaderDispatcherInstrumentation(registry);

//
// now build your graphql object and execute queries on it.
// the data loader will be invoked via the data fetchers on the
// schema fields
//
GraphQL graphQL = GraphQL.newGraphQL(buildSchema())
.instrumentation(dispatcherInstrumentation)
.build();

}

private GraphQLSchema buildSchema() {
return null;
}

private List<Object> getCharacterDataViaBatchHTTPApi(List<String> keys) {
return null;
}


private void tryExample() {
Try<String> tryS = Try.tryCall(() -> {
Expand Down
81 changes: 0 additions & 81 deletions src/test/java/org/dataloader/graphql/DataLoaderDispatcherInstrumentationTest.java
View file

This file was deleted.

0 comments on commit 25047ce

Please sign in to comment.