Skip to content
This repository has been archived by the owner on Nov 14, 2019. It is now read-only.

Crawler is connecting then disconnecting?? #128

Open
osmanra2 opened this issue Jun 27, 2017 · 1 comment
Open

Crawler is connecting then disconnecting?? #128

osmanra2 opened this issue Jun 27, 2017 · 1 comment
Labels

Comments

@osmanra2
Copy link

Using river web 2.4.0

My es version:

{
name: "LinuxGrants",
cluster_name: "LinuxOER",
version: {
number: "2.4.0",
build_hash: "ce9f0c7394dee074091dd1bc4e9469251181fc55",
build_timestamp: "2016-08-29T09:14:17Z",
build_snapshot: false,
lucene_version: "5.5.2"
},
tagline: "You Know, for Search"
}

Log shows:

2017-06-27 12:18:34,478 [main] INFO Connected to xxx.xxx.xxx.xxx:9300
2017-06-27 12:18:34,712 [Crawler-836317c4-95e9-485a-9c1a-935b2dea7117-1] INFO Crawling URL: http://www.xxxxxxxxxxxx.com/
2017-06-27 12:18:34,747 [Crawler-836317c4-95e9-485a-9c1a-935b2dea7117-1] INFO Checking URL: http://www.xxxxxxxxxxx.com/robots.txt
2017-06-27 12:18:34,809 [Crawler-836317c4-95e9-485a-9c1a-935b2dea7117-1] INFO Redirect to URL: http://www.xxxxxx.com/
2017-06-27 12:19:06,012 [Thread-0] INFO Disconnected to LinuxOER: xxx.xxx.xxx.xxx:9300

I tried changing to https and still the same thing. Any help?

@marevol
Copy link
Contributor

marevol commented Jun 27, 2017

What is the crawl config?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

2 participants