Skip to content
This repository has been archived by the owner on Feb 10, 2023. It is now read-only.

Https proxy support #22

Merged
merged 3 commits into from
Jun 30, 2020

Conversation

Jokser
Copy link

@Jokser Jokser commented Jun 19, 2020

pocoproject#3035
Added possibility to connect to HTTPS proxy.
Not all proxies support CONNECT request, so it's possible to send a request directly to HTTPS proxy without CONNECT request. This behavior is controlled by the proxy tunnel option which is 'true' by default for backward-compatibility.
In the case of HTTP endpoint and HTTP proxy, there is no tunneling despite the tunnel option is true by default. This was done also for backward-compatibility.

@akuzm
Copy link

akuzm commented Jun 19, 2020

We don't really have the resources to maintain a fork of Poco. Last time we (I) had to update it, it took half a year of on-and-off attempts to port all the patches, resolve cryptic conflicts and debug mysterious integration test failures. It sure took its toll on my wellbeing, and the next person asked to do will be probably better off just resigning immediately.

So, some suggestions:

  1. Do you mind sending this upstream?
  2. They will ignore it for a couple of years and will just close your PR.
  3. For your use case, how hard it is just to ditch Poco and use some other library that is actually alive?
  4. Of course we won't be able to invest into the hard work of ditching Poco, because it can massively slow down new features, and the incentive systems we have are not aligned with paying off the technical debt (especially so in the particular department that employs you, I guess). Instead, we are going to sacrifice the sanity of somebody else who will have to maintain it "later".
  5. So, in the light of the above, please keep to a bare minimum of changes -- no fomatting changes, no whitespace changes, no changes in error messages. The more changes you make, the more conflicts we will have porting this to a new version.

On a less apocalyptic note, do you have a PR in the main repo that will make use of this functionality? We sure need an integration test for it. And please add a description to the PR -- you add an option to use CONNECT proxy as opposed to MITM proxy functionality that Poco already implements, right?

@akuzm
Copy link

akuzm commented Jun 19, 2020

From TG (in Russian):

Pavel Kovalenko, [19.06.20 17:45]
Привет.
Я видел ты ответил по поводу PRa про HTTPS proxy в Poco. Я лучше отвечу в личку по всем пунктам.
Эта фича нужна для S3 клиента, чтобы ходить в S3 через прокси в нашем Облаке.
1-2. Issue и PR в главное репе есть
pocoproject#3035
pocoproject#3040
Чувак их главный меня не заигнорил, даже подсказал как правильно это сделать.
3. Это невозможно. Раньше для S3 был curl, но от него отказались, т.к. были с ним проблемы, остался единственный вариант на Poco
4. Я не знаю что там происходит именно с Poco, но другого варианта у меня пока нет.
5. Это минимальные изменения, которые черрипикнулись в upstream с минимальными расхождениями.

Я добавил возможность использовать HTTPS proxy (раньше поддерживался только HTTP) и возможность использовать этот прокси напрямую (не через CONNECT). Тестировал я это дело в облаке с нашими S3 прокси и все прекрасно работает. PR, который это использует я выложу как это будет вмержено.

proxyUri.setHost(getProxyHost());
proxyUri.setPort(getProxyPort());

SharedPtr<HTTPClientSession> proxySession (HTTPSessionFactory::defaultFactory().createClientSession(proxyUri));
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand where and how the suggested separate factory _proxySessionFactory is used (with which the proxies for "http/s" are registered).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Forgot to use it here.

Copy link

@Akazz Akazz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@KochetovNicolai KochetovNicolai merged commit 74c9344 into ClickHouse:clickhouse Jun 30, 2020
azat added a commit to azat/ClickHouse that referenced this pull request Jul 4, 2022
GCS server does not handle requests with port, and simply report an
error:

```xml
    <?xml version="1.0"?>
    <?xml version='1.0' encoding='UTF-8'?>
    <Error>
        <Code>InvalidURI</Code>
        <Message>Couldn't parse the specified URI.</Message>
        <Details>Invalid URL: storage.googleapis.com:443/...</Details>
    </Error>
```

Removing the port fixes the issue. Note that there is port in the Host
header anyway.

Note, this is a problem only for proxy in a tunnel mode, since only it
sends such requests, other sends requests directly via HTTP methods.

Refs: ClickHouse/poco#22 (comment) (cc @Jokser)
Refs: ClickHouse/poco#63
Refs: ClickHouse#38069 (cc @CurtizJ)
Cc: @alesapin @kssenii

Signed-off-by: Azat Khuzhin <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants