reactor\netty\tcp\ProxyProvider.java no HTTPS support?

[bobdeanant]

reactor\netty\tcp\ProxyProvider.java

public enum Proxy 
{ 	
HTTP, SOCKS4, SOCKS5 
} 

Why would you not add HTTPS support for proxy ?

While using netty in spring cloud gateway. I am trying to connect to HTTPS type of proxy (Both with and without authentication). The proxy settings are provided using VM args like -Dhttps.proxyHost=myproxy.com -Dhttps.proxyPort=. I have also created a trust manager and set the same in client but while connecting to destination host like google.com am getting
"Caused by: io.netty.handler.proxy.ProxyConnectException: http, none, myproxy.com /xx.xxx.xxx.xxx:xxxx => www.google.com:443, java.io.IOException: Connection reset by peer"

Could you please point me to what am I doing wrong.

[violetagg]

@bobdeanant Please provide more information how you enable the proxy in Spring Cloud Gateway. Do you use https://projectreactor.io/docs/netty/release/api/reactor/netty/transport/ClientTransport.html#proxyWithSystemProperties-- ?
Please provide information for Spring Cloud Gateway/Reactor Netty version.

[bobdeanant]

@violetagg thanks for a quick response.
Answers: I am using below spring cloud gateway <3.1.1>

<dependencyManagement>
		<dependencies>
			<dependency>
				<groupId>org.springframework.cloud</groupId>
				<artifactId>spring-cloud-dependencies</artifactId>
				<version>2021.0.1</version>
				<type>pom</type>
				<scope>import</scope>
			</dependency>
		</dependencies>
	</dependencyManagement>

<netty.version>4.1.73.Final</netty.version>

2. Yes. I am using https://projectreactor.io/docs/netty/release/api/reactor/netty/transport/ClientTransport.html#proxyWithSystemProperties--
   by setting -D VM args.

Thanks
Anant

[violetagg]

@violetagg thanks for a quick response. Answers: I am using below spring cloud gateway <3.1.1>

<dependencyManagement>
		<dependencies>
			<dependency>
				<groupId>org.springframework.cloud</groupId>
				<artifactId>spring-cloud-dependencies</artifactId>
				<version>2021.0.1</version>
				<type>pom</type>
				<scope>import</scope>
			</dependency>
		</dependencies>
	</dependencyManagement>

<netty.version>4.1.73.Final</netty.version>

2. Yes. I am using https://projectreactor.io/docs/netty/release/api/reactor/netty/transport/ClientTransport.html#proxyWithSystemProperties--
   by setting -D VM args.

Thanks Anant

In addition to -D VM args, can you provide here the snippet how you customise the HttpClient/Spring Cloud Gateway?
Is it some configuration that Spring Cloud Gateway exposes or you directly invoke HttpClient.proxyWithSystemProperties()?

[violetagg]

I see Spring Cloud Gateway has some configuration
https://cloud.spring.io/spring-cloud-gateway/reference/html/appendix.html
spring.cloud.gateway.httpclient.proxy
Do you use this or something else?

[bobdeanant]

@violetagg I tried below things and everytime ending up in "io.netty.handler.proxy.ProxyConnectException: http, none, myproxy.com => www.google.com:443, java.io.IOException: Connection reset by peer"

Approach 1. I used spring cloud gateway configuration and added below in application.properties

spring.cloud.gateway.httpclient.proxy.host=myproxy.com
spring.cloud.gateway.httpclient.proxy.port=3129

Where myproxy.com is my host and is a squid proxy which I set on linux machine and is accessible with https://myproxy.com:3129 on TLS

Approach 2: By creating a custom httpclient

public class AHttpClientFactory extends HttpClientFactory {

    public AHttpClientFactory(HttpClientProperties properties, ServerProperties serverProperties, List<HttpClientCustomizer> customizers) {
        super(properties, serverProperties, customizers);
    }

    protected HttpClient configureProxy(HttpClient httpClient) {
            httpClient = (HttpClient)httpClient.proxyWithSystemProperties();

        return httpClient;
    }
}

Then add this as bean

@Bean
   @ConditionalOnMissingBean({HttpClient.class, HttpClientFactory.class})
   public HttpClientFactory gatewayHttpClientFactory(HttpClientProperties properties, ServerProperties serverProperties,
                                                     List<HttpClientCustomizer> customizers) {
       return new AHttpClientFactory(properties, serverProperties, customizers);
   }

Also add below snippet in application.properties so to create a trustmanager with my proxy certificates
spring.cloud.gateway.httpclient.ssl.trusted-x509-certificates=classpath:myproxy-com.pem

Where myproxy-com.pem is PEM file for my certificate

Thanks
Anant

[bobdeanant]

Hi Violeta,

Thanks for the help. Here are few observations and points I have which could be taken into consideration.

1. We have a requirement for support of both HTTP and HTTPS proxy. I believe present Netty implementation considers HTTP with CONNECT method to secure the channel.
2. In case of a specific HTTPS proxy, the proxy itself needs certificate based auth or a Trusted trust manager, which might need an inclusion of SSL socket factory in the execution pipeline.

Below is my SQUID proxy configuration for HTTP (3128) and HTTPS (3129).

acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
acl localnet src 172.16.0.0/12  # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl localnet src fc00::/7       # RFC 4193 local private network range
acl localnet src fe80::/10      # RFC 4291 link-local (directly plugged) machines
acl SSL_ports port 443
acl Safe_ports port 80    # http
acl Safe_ports port 21    # ftp
acl Safe_ports port 443   # https
acl Safe_ports port 70    # gopher
acl Safe_ports port 210   # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280   # http-mgmt
acl Safe_ports port 488   # gss-http
acl Safe_ports port 591   # filemaker
acl Safe_ports port 777   # multiling http
acl CONNECT method CONNECT
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost manager
http_access deny manager
http_access allow localnet
http_access allow localhost
http_access allow all
http_port 3128 

https_port 3129 intercept ssl-bump \
  cert=/etc/squid/certs/squid-ca-cert-key.pem \
  generate-host-certificates=on dynamic_cert_mem_cache_size=16MB
sslcrtd_program /usr/local/squid/libexec/ssl_crtd -s /var/lib/ssl_db -M 16MB
acl step1 at_step SslBump1
ssl_bump peek step1
ssl_bump bump all
ssl_bump splice all
coredump_dir /var/spool/squid
refresh_pattern ^ftp:   1440  20% 10080
refresh_pattern ^gopher:  1440  0%  1440
refresh_pattern -i (/cgi-bin/|\?) 0 0%  0
refresh_pattern .   0 20% 4320
cache_log /tmp/cache.log
access_log /tmp/access.log
max_filedescriptors 65535
debug_options ALL,1 11,3 20,3

[pderop]

I just got a private meeting with @bobdeanant, and in the end, it turns out that what is needed is just to use usual proxy tunneling over a proxy http port, not https.
So, from squid, the following configuration is just needed, and it's even not needed to configure squid with ssl-bump / splice mode:

http_port 3128

I'm closing this issue for the moment, but you can reopen if needed.
thanks.