如何使用java.net.URLConnection发起和处理HTTP请求

技术背景

在Java开发中，经常需要与网络进行交互，发送HTTP请求并处理响应。java.net.URLConnection 是Java提供的一个用于处理URL连接的抽象类，通过它可以方便地发起和处理HTTP请求。不过，Oracle的官方教程对其介绍较为简略，仅展示了如何发起GET请求并读取响应，对于一些“高级”操作，如POST请求、设置请求头、读取响应头、处理Cookie、提交HTML表单、上传文件等未做详细说明。

实现步骤

Java 11及以上版本

从Java 11开始，除了java.net.URLConnection，还可以使用java.net.http.HttpClient以更简洁的方式处理HTTP请求。以下是具体步骤：

1. 准备工作：确定URL和字符集，参数可选，根据功能需求而定。

String url = "http://example.com";
String charset = "UTF-8";  // Or in Java 7 and later, use the constant: java.nio.charset.StandardCharsets.UTF_8.name()
String param1 = "value1";
String param2 = "value2";
// ...

String query = String.format("param1=%s&param2=%s",
    URLEncoder.encode(param1, charset),
    URLEncoder.encode(param2, charset));

2. 发起HTTP GET请求（可选查询参数）

URLConnection connection = new URL(url + "?" + query).openConnection();
connection.setRequestProperty("Accept-Charset", charset);
InputStream response = connection.getInputStream();
// ...

若不发送查询字符串，可省略Accept-Charset头；若无需设置任何头，可使用URL#openStream()快捷方法。

InputStream response = new URL(url).openStream();
// ...

3. 发起HTTP POST请求（带查询参数）

URLConnection connection = new URL(url).openConnection();
connection.setDoOutput(true); // Triggers POST.
connection.setRequestProperty("Accept-Charset", charset);
connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded;charset=" + charset);

try (OutputStream output = connection.getOutputStream()) {
    output.write(query.getBytes(charset));
}

InputStream response = connection.getInputStream();
// ...

也可将URLConnection转换为HttpURLConnection，并使用HttpURLConnection#setRequestMethod()方法。

HttpURLConnection httpConnection = (HttpURLConnection) new URL(url).openConnection();
httpConnection.setRequestMethod("POST");
// ...

4. 显式发起HTTP请求：可使用URLConnection#connect()显式发起请求，但在获取HTTP响应信息（如使用URLConnection#getInputStream()获取响应体）时，请求会自动发起，因此connect()调用通常是多余的。
5. 设置超时时间

httpConnection.setConnectTimeout(3000); // 3s
httpConnection.setReadTimeout(6000); // 6s

使用基于Sun/Oracle的JRE时，读取超时存在一个问题，它会在抛出超时异常前静默重试读取。可通过以下方式关闭此行为：

System.setProperty("sun.net.http.retryPost", "false")

在Android中，上述方法无效，可使用以下解决方法：

httpConnection.setChunkedStreamingMode(0);

若性能受影响较大，可考虑切换到其他HTTP客户端，如OkHttp。
6. 收集HTTP响应信息
- HTTP响应状态

int status = httpConnection.getResponseCode();

- **HTTP响应头**

for (Entry<String, List<String>> header : connection.getHeaderFields().entrySet()) {
    System.out.println(header.getKey() + "=" + header.getValue());
}

- **HTTP响应编码**

String contentType = connection.getHeaderField("Content-Type");
String charset = null;

for (String param : contentType.replace(" ", "").split(";")) {
    if (param.startsWith("charset=")) {
        charset = param.split("=", 2)[1];
        break;
    }
}

if (charset != null) {
    try (BufferedReader reader = new BufferedReader(new InputStreamReader(response, charset))) {
        for (String line; (line = reader.readLine()) != null;) {
            // ... System.out.println(line)?
        }
    }
} else {
    // It's likely binary content, use InputStream/OutputStream.
}

7. 维护会话：服务器端会话通常由Cookie支持，可使用CookieHandler API维护Cookie。

// First set the default cookie manager.
CookieHandler.setDefault(new CookieManager(null, CookiePolicy.ACCEPT_ALL));

// All the following subsequent URLConnections will use the same cookie manager.
URLConnection connection = new URL(url).openConnection();
// ...

connection = new URL(url).openConnection();
// ...

connection = new URL(url).openConnection();
// ...

若上述方法不起作用，可手动收集和设置Cookie头。

// Gather all cookies on the first request.
URLConnection connection = new URL(url).openConnection();
List<String> cookies = connection.getHeaderFields().get("Set-Cookie");
// ...

// Then use the same cookies on all subsequent requests.
connection = new URL(url).openConnection();
for (String cookie : cookies) {
    connection.addRequestProperty("Cookie", cookie.split(";", 2)[0]);
}
// ...

8. 流式模式：HttpURLConnection默认会在发送请求体之前缓冲整个请求体，可能导致OutOfMemoryExceptions。可使用HttpURLConnection#setFixedLengthStreamingMode()或HttpURLConnection#setChunkedStreamingMode()避免此问题。

httpConnection.setFixedLengthStreamingMode(contentLength);
httpConnection.setChunkedStreamingMode(1024);

9. 设置User-Agent

connection.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36");

10. 错误处理：若HTTP响应码为4xx（客户端错误）或5xx（服务器错误），可读取HttpURLConnection#getErrorStream()获取服务器发送的错误信息。

InputStream error = ((HttpURLConnection) connection).getErrorStream();

若HTTP响应码为 -1，可能是连接和响应处理出现问题，可通过以下方式关闭连接保持：

System.setProperty("http.keepAlive", "false");

11. 上传文件：通常使用multipart/form-data编码处理混合POST内容（二进制和字符数据）。

String param = "value";
File textFile = new File("/path/to/file.txt");
File binaryFile = new File("/path/to/file.bin");
String boundary = Long.toHexString(System.currentTimeMillis()); // Just generate some unique random value.
String CRLF = "\r\n"; // Line separator required by multipart/form-data.
URLConnection connection = new URL(url).openConnection();
connection.setDoOutput(true);
connection.setRequestProperty("Content-Type", "multipart/form-data; boundary=" + boundary);

try (
    OutputStream output = connection.getOutputStream();
    PrintWriter writer = new PrintWriter(new OutputStreamWriter(output, charset), true);
) {
    // Send normal param.
    writer.append("--" + boundary).append(CRLF);
    writer.append("Content-Disposition: form-data; name=\"param\"").append(CRLF);
    writer.append("Content-Type: text/plain; charset=" + charset).append(CRLF);
    writer.append(CRLF).append(param).append(CRLF).flush();

    // Send text file.
    writer.append("--" + boundary).append(CRLF);
    writer.append("Content-Disposition: form-data; name=\"textFile\"; filename=\"" + textFile.getName() + "\"").append(CRLF);
    writer.append("Content-Type: text/plain; charset=" + charset).append(CRLF); // Text file itself must be saved in this charset!
    writer.append(CRLF).flush();
    Files.copy(textFile.toPath(), output);
    output.flush(); // Important before continuing with writer!
    writer.append(CRLF).flush(); // CRLF is important! It indicates end of boundary.

    // Send binary file.
    writer.append("--" + boundary).append(CRLF);
    writer.append("Content-Disposition: form-data; name=\"binaryFile\"; filename=\"" + binaryFile.getName() + "\"").append(CRLF);
    writer.append("Content-Type: " + URLConnection.guessContentTypeFromName(binaryFile.getName())).append(CRLF);
    writer.append("Content-Transfer-Encoding: binary").append(CRLF);
    writer.append(CRLF).flush();
    Files.copy(binaryFile.toPath(), output);
    output.flush(); // Important before continuing with writer!
    writer.append(CRLF).flush(); // CRLF is important! It indicates end of boundary.

    // End of multipart/form-data.
    writer.append("--" + boundary + "--").append(CRLF).flush();
}

12. 处理不受信任或配置错误的HTTPS站点：在某些情况下，连接HTTPS URL时可能会遇到javax.net.ssl.SSLException或java.security.cert.CertificateException等异常。可使用以下静态初始化器使HttpsURLConnection更宽松。

static {
    TrustManager[] trustAllCertificates = new TrustManager[] {
        new X509TrustManager() {
            @Override
            public X509Certificate[] getAcceptedIssuers() {
                return null; // Not relevant.
            }
            @Override
            public void checkClientTrusted(X509Certificate[] certs, String authType) {
                // Do nothing. Just allow them all.
            }
            @Override
            public void checkServerTrusted(X509Certificate[] certs, String authType) {
                // Do nothing. Just allow them all.
            }
        }
    };

    HostnameVerifier trustAllHostnames = new HostnameVerifier() {
        @Override
        public boolean verify(String hostname, SSLSession session) {
            return true; // Just allow them all.
        }
    };

    try {
        System.setProperty("jsse.enableSNIExtension", "false");
        SSLContext sc = SSLContext.getInstance("SSL");
        sc.init(null, trustAllCertificates, new SecureRandom());
        HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
        HttpsURLConnection.setDefaultHostnameVerifier(trustAllHostnames);
    }
    catch (GeneralSecurityException e) {
        throw new ExceptionInInitializerError(e);
    }
}

最佳实践

• 使用Java 11及以上版本的java.net.http.HttpClient：它提供了更简洁、更现代的API，能更方便地处理HTTP请求。
• 设置合理的超时时间：避免因网络问题导致程序长时间阻塞。
• 正确处理异常：在发起HTTP请求时，可能会抛出各种异常，如IOException、MalformedURLException等，应进行适当的异常处理。
• 使用合适的字符集：确保请求和响应的字符编码一致，避免出现乱码问题。

常见问题

1. 使用setDoOutput(true)时，GET请求变为POST请求：若只想发起GET请求，应确保不调用setDoOutput(true)。
2. 读取超时问题：基于Sun/Oracle的JRE在读取超时时有静默重试机制，可通过设置系统属性或使用流式模式解决。
3. 处理Cookie时不起作用：若CookieHandler API无法正常工作，可手动收集和设置Cookie头。
4. 连接HTTPS站点时出现异常：可使用上述静态初始化器使HttpsURLConnection更宽松，但在生产环境中需谨慎使用。