[Feature Request] Speed up forge queries

[glennsarti]

According to

# Fetching metadata for all releases takes ages (which is weird, since it's mostly static info)
      mod.releases.take(FETCH_LIMIT).each do |rel|
        meta = get_release_metadata(name, rel)
        rel_artifact = @graph.artifact(name, rel.version)
        puts "...Recursively adding requirements for dependency #{name} version #{rel.version}"
        # We don't want to add the requirements to the list of demands for all versions,
        # but we need them in the graph to be able to solve dependencies
        add_reqs_to_graph(rel_artifact, meta, :no_demands)
      end

I had a similar problem when I wrote my Puppetfile Resolver. Ultimately I had to write my own HTTP query-er as the PuppetForge:Modules class is inflexible and is very costly as far as traffic and latency goes. (Doesn't batch query, doesn't reduce HTTP payload)

Here's my version which is purely used to fetch the metadata for all released versions.

https://github.com/lingua-pupuli/puppetfile-resolver/blob/master/lib/puppetfile-resolver/spec_searchers/forge.rb#L38-L73

It batches in lots of 50 and excludes a bunch of text files which are notorious for bloating HTTP payloads (:exclude_fields => 'readme changelog license reference tasks')